Apple Vision Pro + OpenXR

Ongoing engine contributions

The engine-level work behind Unreal on Apple Vision Pro over OpenXR — punch-list fixes today, Lumen and Nanite research pushing further.

Unreal's Vision Pro support runs over OpenXR and is officially Experimental: source-only builds, sparse docs, and a set of sharp edges every team hits in the same order. This is an ongoing effort to fix those at the engine level and push rendering further than Epic has taken it on this platform — Lumen global illumination and Nanite virtualized geometry included. The fixes land through the community fork ecosystem where visionOS UE development actually lives, and because the foundation is OpenXR, most of them aren't Vision-Pro-specific.

Lifecycle & stability

The crown-exit 'purgatory' fix: dismissing the immersive scene via the Digital Crown used to leave a zombie process holding the OS's singleton immersive layer, so the next launch needed a device reboot. And the level-travel fix: OpenLevel and ServerTravel crashed the compositor on every transition — a frame-lifecycle guard (upstreamed as a fork PR) makes multi-level visionOS apps possible at all.

Rendering & passthrough

Translucency over passthrough used to produce blocky grey halos around glass, smoke, and glow — a depth/alpha invariant Apple's compositor enforces and the engine didn't satisfy. The fix lands real glass in mixed reality. Plus: foveation artifact mitigation, and passthrough-safe material patterns documented for everyone.

Lumen, demonstrated on device

Lumen — Unreal's dynamic global illumination and reflections — is confirmed rendering on real Apple Vision Pro hardware: GI and reflections visibly correct, blocky shadow quality the main open complaint. It runs today at roughly 20 fps on M5, GPU-bound by scene complexity rather than a setting you can just turn down — demonstrated, not yet a usable frame rate. Stability work continues alongside it: shader-permutation crashes on the SM5/SM6 paths get root-caused and fixed one at a time as they surface.

Nanite: under investigation, not yet working

Nanite's SM6 compute path hits a hard compiler crash on this fork today, so it ships disabled while the cause gets pinned down. The leading hypothesis, drawn from Epic's own engineering notes on macOS parity: Nanite-on-SM6 depends on Apple's Metal Shader Converter, and this fork's visionOS build has no visionOS slice of that converter — forcing an unsupported fallback path. Plausible, but unverified. Call this active research, not a roadmap item with a date.

The Mac-native toolchain

Lightmass — Unreal's baked-lighting renderer — shipped x86-only and crashed under Rosetta on Apple Silicon (Embree probes for AVX). A native arm64 Lightmass build makes baked lighting possible on the same M-series Mac you deploy from. Add layered parallax app icons, simulator build revival, and headless commandlet pipelines, and the whole loop runs on one machine.

At a glance

✦Crown-exit lifecycle fix — no more reboot-to-recover
✦OpenLevel / ServerTravel compositor fix, upstreamed as a fork PR
✦Passthrough translucency depth/alpha fix — real glass in mixed reality
✦Lumen GI + reflections confirmed rendering on Apple Vision Pro hardware — ~20 fps today, GPU-bound; frame rate and shadow quality are ongoing work, not solved
✦Nanite on visionOS under active investigation — a missing Metal Shader Converter slice is the leading, unverified hypothesis for the current compiler crash
✦Native arm64 Lightmass — baked lighting on Apple Silicon
✦Layered parallax app icons, simulator builds, headless cook pipelines

Updates · 2 videos

Apple Vision Pro can do Lumen ??

Everything I'm Building with Unreal Engine 5.8 in Apple Vision Pro

Want in early?

Apple Vision Pro + OpenXR is in active development. Early-access spots, pilot projects, and collaborations are all on the table.

Also in the lab

Video QA Workbench

Stop scrubbing for proof. Review the whole render with evidence.

Forage

Fab's best friend. An AI-first scout and installer for the Unreal Engine packs you already own.

Project Ion

Your Vision Pro is a projector now. Professional-grade projection-mapping previsualization — honest ambient simulation, real projector specs, and a full creative rig in your room.

UnRealityKit Bridge

Unreal Engine as the simulation brain. RealityKit as the renderer. The hybrid architecture Unity proved out with PolySpatial — open for Unreal.

UE→RenderMan Bridge

One button in the Unreal editor. Film-grade Pixar RenderMan on the other side.

Pinchwork

Universal OpenXR hand tracking. Built in Unreal first — designed for every headset, not just one.

Constellation

Your second brain as a navigable 3D star map. Walk through your own notes on Apple Vision Pro.

Promptbook

Spatial blocking and set previz for theatre on Apple Vision Pro — the stage manager's promptbook, reimagined in 3D.

AgileLensMultiplayer

The shared SharePlay engine every Agile Lens Vision Pro multiplayer app runs on.

Mission Control

A live radar for SharePlay sessions — watch every player, in every app, without joining the call.

xrsim

A simulator for VR apps: boot the build, script the headset, assert on what the app actually believes — no headset, no human in the room.

RoomScanPro

Collaborative room scanning and real-world measurement for Apple Vision Pro and iPhone.

Co-Location Engine

Fiducial-marker-based co-location for Unreal Engine VR — real and virtual players sharing one room, without a shared coordinate system to begin with.

Fable Showcase

I handed a brand-new UE 5.8 project to Claude Fable 5.1 for one night, gave it the Lab's own tools, and walked away.