Because of their ability to visually sense the environment, head-mounted mixed reality (MR) systems can detect when a user touches a wall or other surface. These surfaces can then become interactive panels, with the small caveat that they traditionally treat a finger coming within 10mm of the surface as a touch. 

While this leads to sometimes inaccurate readings, researchers at Tsinghua University in Beijing have implemented an inertial measurement unit (IMU) ring apparatus for contact sensing to increase precision from around 85% to just under 99%. 

The experimental setup for this exercise used an Arduino Uno to read accelerometer data, along with a capacitive arrangement to confirm when a touch actually took place. Accelerometer data was compared with readings from a Leap Motion optical sensor, which in addition to greater accuracy, also reduced latency.

As part of a recent Microsoft HoloLens hackathon in San Francisco, Maker Ian Sterling developed a new app that interacts with you smart home via augmented reality. The proof of concept, dubbed “IoTxMR,” allows a user to simply glance at a gadget and control it through gestures.

As you can see in the video below, IoTxMR enables Sterling to connect various Android and Arduino-based devices with the HoloLens to create a customized interdependent network. It also features a mixed reality experience called “virtual zen mode,” complete with calming sounds and light orbs in his surrounding environment.

During a recent interview with Digital Trends, Sterling revealed:

The primary goal of the app is to provide a 3D spatial UI for cross-platform devices — Android Music Player app and Arduino-controlled fan and light — and to interact with them using gaze and gesture control.

The connectivity between Arduino and a mixed reality device is something which holds a huge amount of creative opportunity for developers to create some very exciting applications — be it [Internet of Things], robotics, or other sensor data visualization. Besides this, our app features some fun ways to connect devices. Our demo featured a connection between a music player and a light in order to set a certain mood in your home.

Although just a demo, IoTxMR does highlight the endless possibilities that AR platforms like HoloLens offer in the not-too-distant future.