Co-Location Engine

Private — Agile Lens internal, in active development

Fiducial-marker-based co-location for Unreal Engine VR — real and virtual players sharing one room, without a shared coordinate system to begin with.

Multiplayer XR in a physical space has a chicken-and-egg problem: every headset needs to agree on where the room actually is, and nothing hands that to you for free. This system solves it with printed ArUco/AprilTag/QR markers of known position — a camera-relative detection becomes a rigid-body solve that snaps a VR pawn's tracking origin onto real-world map space, then keeps it there as people walk around.

One rigid solve, not a position-then-yaw hack

Position and rotation are solved jointly — a weighted planar rigid-body fit (Kabsch/Umeyama) across every visible marker at once, not a separate 2-marker yaw formula bolted onto an averaged position. More markers in view means a tighter, self-correcting solve instead of diminishing returns from just picking two.

Corrections you can't feel

A locked-in position doesn't get nudged on every noisy frame. Drift correction is velocity-gated — standing still means never moving the player, walking gets an imperceptible pull toward true, and being grossly wrong (inches, sustained across multiple readings) triggers a full snap instead of a slow, visible slide. The goal is co-location that fades into the background.

Three independent detector families

ArUco, AprilTag, and QR all feed the same localizer, each carrying its own reserved ID range so a printed marker map can mix families without collision. A digital-twin overlay renders the exact expected marker pattern at its known position, so misalignment is visible and diagnosable in headset — no guesswork.

Tested without ever putting a headset on

A three-tier headless harness validates the whole pipeline before it ever touches hardware: pure-math unit tests against synthetic marker geometry, a real render-target camera detecting a live 3D scene, and a fully scripted OpenXR device driving the packaged app end-to-end. The last one already closes the loop on co-location accuracy to sub-centimeter — headless, in CI, no one wearing anything.

At a glance

✦Joint N-marker rigid-body solve — yaw and position from every visible marker at once, not sequential 2-marker hacks
✦Velocity-gated drift correction: still = untouched, walking = imperceptible, grossly wrong = clean snap
✦ArUco + AprilTag + QR detection unified behind one localizer, namespaced marker IDs
✦In-headset digital-twin overlay for instant visual alignment checks
✦Three-tier headless test harness — unit, synthetic-camera, and full scripted-OpenXR-device — no physical headset required

Want in early?

Co-Location Engine is in active development. Early-access spots, pilot projects, and collaborations are all on the table.

Also in the lab

Forage

Fab's best friend. An AI-first scout and installer for the Unreal Engine packs you already own.

Project Ion

Your Vision Pro is a projector now. Professional-grade projection-mapping previsualization — honest ambient simulation, real projector specs, and a full creative rig in your room.

UnRealityKit Bridge

Unreal Engine as the simulation brain. RealityKit as the renderer. The architecture ILM shipped, opened up.

Pinchwork

Universal OpenXR hand tracking. Built in Unreal first — designed for every headset, not just one.

Unreal × visionOS

The punch list for making Unreal Engine on Apple Vision Pro feel production-ready.

Constellation

Your second brain as a navigable 3D star map. Walk through your own notes on Apple Vision Pro.

Promptbook

Spatial blocking and set previz for theatre on Apple Vision Pro — the stage manager's promptbook, reimagined in 3D.

AgileLensMultiplayer

The shared SharePlay engine every Agile Lens Vision Pro multiplayer app runs on.

Mission Control

A live radar for SharePlay sessions — watch every player, in every app, without joining the call.

RoomScanPro

Collaborative room scanning and real-world measurement for Apple Vision Pro and iPhone.