Godot visionOS Simulator Kit
4Develop & test Godot visionOS apps in the Simulator — input, hand-tracking, one-command build switching.
The story
A Godot visionOS app renders an immersive custom-Metal CompositorServices scene — which means the visionOS Simulator's built-in pinch simulation can't reach it. Out of the box you get zero input and zero hands in the sim, forcing a physical-device round-trip for every change. This kit closes the gap: simulated input over UDP, a SwiftUI control panel, canned hand feeds, and an engine module for real articulated MediaPipe hands.
Highlights
- ✦One command builds + runs in Simulator or on device — signing and install differences hidden
- ✦UDP input injection keeps the Simulator focused while actions reach the app
- ✦Webcam → MediaPipe → Godot XRHandTracker articulated hands in the sim
- ✦Extracted from the Cascade Countdown pilot — drops into any Godot visionOS repo
Docs · live from GitHub wiki
Godot visionOS Simulator Kit — Wiki
Tools to develop and test Godot visionOS apps in the visionOS Simulator — working input, hand‑tracking, and a one‑command sim/device build switcher. (The README is the quick reference; these pages go a little deeper.)
Why this exists
A Godot visionOS app renders an immersive, custom‑Metal CompositorServices scene. The Simulator's built‑in pinch/gesture simulation only reaches RealityKit/SwiftUI apps — so a Godot immersive app gets zero input and zero hands in the sim out of the box, and you'd be stuck testing only on a physical Apple Vision Pro. This kit closes that gap so you can iterate in the Simulator, which is far faster than a device round‑trip.
Pages
- [[Build Switcher]] —
build.sh sim | device: one command, the right destination/signing/install for each. - [[Simulator Input and Hands]] — drive your immersive app in the sim: keyboard/GUI input over UDP, and real or canned hand‑tracking.
The pieces (all in the repo)
| Piece | What it does | Engine rebuild? |
|---|---|---|
build.sh | one‑command sim/device build switcher | no |
godot/simulator_input.gd | UDP 127.0.0.1:9999 input autoload (sim‑gated) | no |
tools/SimInputTap.swift | macOS global key‑tap → UDP | no |
tools/SimControlPanel/ | macOS GUI for input + hand feed + live calibration | no |
tools/simhands_canned_sender.swift | canned hand feed over MultipeerConnectivity | no |
engine/simhands_bridge.mm | native bridge: webcam hands → XRHandTracker in the sim | yes (Clancey engine) |
Pairs great with Cascade Countdown
This kit was extracted from Cascade Countdown — a hand‑tracked physics arcade game for Apple Vision Pro (the pilot's wiki is a beginner‑friendly Godot‑on‑AVP walkthrough). Cascade uses this kit to prototype in the Simulator; any Godot visionOS project can do the same.
Built by Alex Coulombe. MIT licensed. Interoperates with VisionOS‑SimHands (Ben Lumen Digital); hand‑tracking engine support rides on Clancey/godot (Marshall Nowak).
Want this level of depth on your own project?
The classes teach the same production techniques these repos are built with — scar tissue included.