Comment by solSpeedofLight

Toward the UX of spatial computing: beyond controllers

Spatial computing is still in its early days. The hardware is here, but the interaction language is only starting to emerge. We don’t yet have the “mouse” or the “touchscreen gesture” equivalent for this new medium.

In building Gravitas Dark Matter for Apple Vision Pro, I wanted to see what would happen if I pushed past the familiar metaphors of controllers, joysticks, and handhelds — and designed around something simpler: the eye and a pinch.

Eye Pinch Targeting

What I call Eye Pinch Targeting combines two inputs that Vision Pro offers natively:

Eye gaze: natural head + eye motion to aim.

Pinch: a subtle, controller-free gesture to confirm.

The result surprised me. It didn’t feel like a hack or a compromise. It felt precise — almost inevitable. The dexterity of a gentle pinch gave me more confidence than any floating joystick UI could.

It’s not a virtual gun. It’s not a VR controller disguised as a wand. It’s something native to the medium.

Modes of spatial play

The second experiment was seeing how play changes depending on how much of the environment you keep:

Passthrough mode: your actual room remains visible, with targets anchored in physical space.

Apple Immersion: the curated environments Apple ships with Vision Pro.

Galaxy immersion: a full spatial takeover — black holes and celestial targets filling your field of view.

Each mode carried different affordances. Passthrough made play feel like AR. Apple’s environments gave a filmic layer. The galaxy mode created scale and immersion you simply can’t fake with a flat screen.

Hardcore stress testing

Performance was a hurdle. Cranking graphics to High at first made the experience unplayable. But after optimization passes, I was able to run a “hardcore” session: Arena Mode across the entire length of my home — living room, dining room, kitchen, hallway.

It was physical. You tread lightly to avoid throwing off tracking, but in the heat of it I spun quickly, weaving through space. I lost multiple rounds. It felt more like a sport than a demo. And unlike pre-canned VR attractions (like The VOID at Downtown Disney), this wasn’t on rails. It was reactive, competitive, and punishing.

Why this matters for UX

Spatial computing won’t scale if input feels clumsy or bolted-on. Eye Pinch Targeting showed me that precision and intuitiveness don’t need controllers. They can come from inputs already built into the hardware — if we design for them.

The broader takeaway: UX in spatial computing isn’t about recreating old metaphors. It’s about letting interaction feel native to the body and the environment.

Closing thoughts

Gravitas Dark Matter is still experimental. There are hitches, optimizations, and unanswered questions. But the exercise points toward something larger: that the UX of spatial computing may not look like controllers at all. It may look like gaze, pinch, and movement — gestures that feel inevitable once you experience them.