Building My Own Spatial Accessory for visionOS 27

visionOS 27 opens spatial accessory tracking up to anyone. I dug into Apple's spec to find out what's required, what's optional, and what I still have to figure out to build one myself.

Building My Own Spatial Accessory for visionOS 27

WWDC26 quietly opened up one of the more interesting visionOS capabilities: spatial accessories. Last year with visionOS 26, Apple added support for the Logitech Muse and the PSVR2 Sense controller. This year, Apple is opening that up to anyone to build your own accessory, train it, and Vision Pro will track it in real time. They also announced a few companies with accessories coming out (DFRobot's seeMote and MikroE's Spatial Anchor R1) that can be "attached" to other objects, enabling them as tracked items.

Two sessions cover the topic:

Finding the actual spec

Session 283's Resources section points to Apple's Accessory Design Guidelines for Apple Devices, a 401-page PDF that covers every kind of Apple-compatible accessory. Chapter 20, "Spatial Accessories," is where the fun begins.

There is a disclaimer, as visionOS 27 isn't final things could change. But the document appears to be a robust spec with all the usual standards of "shall"/"should"/"may" to know what you HAVE to do versus what you might WANT to do, and a complete reference hardware design with real part numbers.

What's required

Per the spec, a spatial accessory shall:

  • Support Bluetooth Low Energy 5.3 or later
  • Support HID over GATT (HOGP)
  • Integrate a 6-axis IMU (either Bosch BMI323 or TDK ICM-42686)
  • Integrate an LED constellation - many infrared LEDs (recommended 800-870nm), arranged so at least 4 are visible from any viewing angle, with fairly specific geometry rules (blob size, spacing, uniqueness across angles)
  • Have a minimum dimension of 10mm along any axis

That's the floor. Everything else is optional, built on top of that same board.

What's optional

The accessory can also have interactivity:

  • Buttons (binary or analog-depression)
  • Thumbsticks
  • Proximity sensors
  • Force sensors (reported in newtons)
  • Haptics (either an LRA or a plain rotating-weight motor, driven through a defined 5-waveform controller)

What I still need to figure out

This is where the actual project starts. Having the spec is step one; there's a real list of things it doesn't hand you:

  • The firmware. The doc defines the BLE/HID protocol byte-for-byte, but there is plenty of code to figure out and program onto this hardware.
  • The hardware itself. Which parts, from where, at what cost. This is all new to me and probably where I'll quickly run into hurdles.
  • The 3D model + training pipeline. Every LED and the IMU need to be annotated in a specific USDZ format, then trained into a .referenceaccessory file via CreateML. I've done the training, but haven't looked at the annotations.
  • An actual enclosure. Some enclosure to collect this "accessory" within, and matches the 3D model to train on.
  • Verification. Apple documents an actual bring-up/verification procedure that I'll have to run thru and see if I can actually get the device to connect and start working.

That's the shape of the project. Next up: working through the actual parts list.