Co-Location Engine

Private — Agile Lens internal, in active development

Fiducial-marker-based co-location for Unreal Engine VR — real and virtual players sharing one room, without a shared coordinate system to begin with.

Multiplayer XR in a physical space has a chicken-and-egg problem: every headset needs to agree on where the room actually is, and nothing hands that to you for free. This system solves it with printed ArUco/AprilTag/QR markers of known position — a camera-relative detection becomes a rigid-body solve that snaps a VR pawn's tracking origin onto real-world map space, then keeps it there as people walk around.

One rigid solve, not a position-then-yaw hack

Position and rotation are solved jointly — a weighted planar rigid-body fit (Kabsch/Umeyama) across every visible marker at once, not a separate 2-marker yaw formula bolted onto an averaged position. More markers in view means a tighter, self-correcting solve instead of diminishing returns from just picking two.

Corrections you can't feel

A locked-in position doesn't get nudged on every noisy frame. Drift correction is velocity-gated — standing still means never moving the player, walking gets an imperceptible pull toward true, and being grossly wrong (inches, sustained across multiple readings) triggers a full snap instead of a slow, visible slide. The goal is co-location that fades into the background.

Three independent detector families

ArUco, AprilTag, and QR all feed the same localizer, each carrying its own reserved ID range so a printed marker map can mix families without collision. A digital-twin overlay renders the exact expected marker pattern at its known position, so misalignment is visible and diagnosable in headset — no guesswork.

Tested without ever putting a headset on

A three-tier headless harness validates the whole pipeline before it ever touches hardware: pure-math unit tests against synthetic marker geometry, a real render-target camera detecting a live 3D scene, and a fully scripted OpenXR device driving the packaged app end-to-end. The last one already closes the loop on co-location accuracy to sub-centimeter — headless, in CI, no one wearing anything.

At a glance

✦Joint N-marker rigid-body solve — yaw and position from every visible marker at once, not sequential 2-marker hacks
✦Velocity-gated drift correction: still = untouched, walking = imperceptible, grossly wrong = clean snap
✦ArUco + AprilTag + QR detection unified behind one localizer, namespaced marker IDs
✦In-headset digital-twin overlay for instant visual alignment checks
✦Three-tier headless test harness — unit, synthetic-camera, and full scripted-OpenXR-device — no physical headset required

Want in early?

Co-Location Engine is in active development. Early-access spots, pilot projects, and collaborations are all on the table.

Also in the lab

Surreal Minds

Four small Unreal worlds learning to react to photos, wishes, gestures, and keys.

Harborfolk

A little world at human scale, with people at every station.

Persona Live

Your Vision Pro Persona, driving a live MetaHuman.

Nightbloom

Procedural fireworks in your real sky.

SceneAudit

Measure the gap your viewport hides.

Video QA Workbench

Stop scrubbing for proof. Review the whole render with evidence.

Forage

Fab's best friend. An AI-first scout and installer for the Unreal Engine packs you already own.

Project Ion

Your Vision Pro is a projector now. Professional-grade projection-mapping previsualization — honest ambient simulation, real projector specs, and a full creative rig in your room.

UnRealityKit Bridge

Unreal Engine as the simulation brain. RealityKit as the renderer. The hybrid architecture Unity proved out with PolySpatial — open for Unreal.

UE→RenderMan Bridge

One button in the Unreal editor. Film-grade Pixar RenderMan on the other side.

Pinchwork

Universal OpenXR hand tracking. Built in Unreal first — designed for every headset, not just one.

Apple Vision Pro + OpenXR

The engine-level work behind Unreal on Apple Vision Pro over OpenXR — punch-list fixes today, Lumen and Nanite research pushing further.

Constellation

Your second brain as a navigable 3D star map. Walk through your own notes on Apple Vision Pro.

Promptbook

Spatial blocking and set previz for theatre on Apple Vision Pro — the stage manager's promptbook, reimagined in 3D.

AgileLensMultiplayer

The shared SharePlay engine every Agile Lens Vision Pro multiplayer app runs on.

Mission Control

A live radar for SharePlay sessions — watch every player, in every app, without joining the call.

xrsim

A simulator for VR apps: boot the build, script the headset, assert on what the app actually believes — no headset, no human in the room.

Ultimate OpenXR Template

Hands. Controllers. Your next XR project.
See it in action. Then make it yours.

Echo Cathedral

Draw a line in VR.
Then play the room.

RoomScanPro

Collaborative room scanning and real-world measurement for Apple Vision Pro and iPhone.

Fable Showcase

A playable experiment in how reliably AI and a headless Unreal toolchain can build a museum-scale experience with limited handholding.