Introduction
As a way to experiment with audio in augmented reality, I needed an augmented reality headset. Since Magic Leap and Microsoft’s Hololens were a little too expensive, I decided to build Leap Motion’s Project Northstar from scratch.
Assembly
After 3D printing the parts and assembling the electronics and reflectors from Smart Prototyping, I had a working headset. And while it was pretty neat, the 0DOF made it a little limited. So I set myself to designing and building an Arduino solution to bring 3DOF to the headset.
3DOF with Arduino + MPU6050
Built upon the MPU6050 IMU, the Arduino sends serial quaternion data to Unity, where it is decoded and parsed to produce rotation. After getting this to work, it was now time for 6DOF.
6DOF with Intel Realsense T265
This was initially accomplished by using the Intel Realsense T265 sensor mounted on a bracket above the headset. While this worked well to provide reliable tracking, I also wanted the ability to perform computer vision functions like spatial mapping, plane detection, and anchoring.
6DOF + AR with ZED Mini
For this, I switched over to the ZED Mini to be able to use their API. Although originally intended to be mounted on the front of VR headsets and function as video passthrough to the screens, it works really well for this borrowed concept.
