It’s surprisingly easy to misjudge tips that come into the Hackaday tip line. After filtering out the omnipresent spam, a quick scan of tip titles will often form a quick impression that turns out to be completely wrong. Such was the case with a recent tip that seemed from the subject line to be a flight simulator cockpit. The mental picture I had was of a model cockpit hooked to Flight Simulator or some other off-the-shelf flying game, many of which we’ve seen over the years.

I couldn’t have been more wrong about the project that Grant Hobbs undertook. His cockpit simulator turned out to be so much more than what I thought, and after trading a few emails with him to get all the details, I felt like I had to share the series of hacks that led to the short video below and the story about how he somehow managed to build the set despite having no previous experience with the usual tools of the trade.

A Novel and a Film

Grant has been making short films for a while, mainly in collaboration with John Dwyer, an author of historical novels. Grant’s shorts are used as promos for John’s books, and nicely capture the period and settings of John’s novels. Most of these films required little in the way of special sets, relying instead on stock footage and vintage costumes to achieve their look and feel. John’s latest novel would change all that.

Called Mustang, the novel centers on a hotshot fighter pilot in WWII. Grant’s vision for the short to promote the book was inspired by the recent Christopher Nolan film Dunkirk, which featured intricate sequences filmed in the cockpit of a Spitfire. Granted wanted a similar look, and began arranging to use a real P-51 Mustang for filming. That presented immediate problems. First, there aren’t that many of the vintage aircraft left, and those that are still flying usually have anachronistic instruments in the cockpit, like GPS. Also, Grant wanted the instruments to respond as if the plane were airborne, and to have the shadows cast by the canopy into the cockpit suggest aerial maneuvers. Such an effect would be difficult to achieve with a plane stuck on a runway.

That’s when Grant realized that a gimballed cockpit simulator was needed. It could have a period-accurate dashboard, be positioned outdoors to take advantage of natural daylight and real backgrounds rather than CGI, and could be pitched, rolled and yawed to simulate flight. It would be perfect, and it would save the project. There was just one problem: he had no idea how to build it.

Helping Hands

Wisely, Grant turned to his local hackerspace, Dallas Maker Space, for help. There he found not only the tools he lacked, but kindred spirits with the necessary skills and the willingness to share them. They started working on the cockpit instrument panel, which ended up including a combination of actual flight hardware and mocked-up instruments. The fake instruments used steppers and an Arduino to drive the needles, which were controlled by a custom iPad app that was used to animate them live during filming. The real instruments, like the artificial horizon and turn-and-slip indicator, were powered by a vacuum pump and responded to the movements of the simulator on its gimbals.

Mounting this convincing panel into something was an entirely different undertaking. Grant relied heavily on the experience of DMS members to design a structure strong enough to support the actor and allow for the motion needed to create a convincing effect. The cockpit mockup, made from plasma-cut sheet metal and plywood, is mounted to a heavy-duty three-axis gimbal, including a massive bearing from a pallet jack for the yaw axis.

Grant had originally planned to place the mockup on a mountaintop for shooting, much as the Spitfire mockup from Dunkirk was placed on the edge of a cliff to give an unobstructed horizon to simulate flying over the English Channel. When that proved logistically challenging, he set up on an airport runway and used clever camera blocking to avoid shooting the horizon. Grips manually moved the simulator while Grant manipulated the fake instruments and filmed the results, which I think speak for themselves. If only the budget – and on-set safety – would have supported simulating the massive four-blade Mustang propeller, the illusion would have been complete.

I really enjoyed digging into this project and all the hacks that it entailed. Movie magic is as much about hacking as anything else, at least behind the cameras, and it’s good to see what’s possible with a limited budget. We recently featured a low-budget but high-style sci-fi movie set build, and we’ve gone in-depth with a playback designer for the Netflix series Lost in Space, both in these pages and as a Hack Chat.

When using a virtual reality (VR) system, you may need to flip a switch, touch a button, etc., which can be represented by a carefully coordinated series of pixels in front of your eyes. As a physical alternative — or augmentation — researchers at the National Chiao Tung University in Hsinchu, Taiwan have developed a system of interchangeable physical control panels, called FaceWidgets, that reside on the backside of head-mounted unit itself.

When a wearer places their palm near their face (and headset), this is sensed and an on-screen canvas appears depending on the application. They can then manipulate these widgets both physically and in the virtual world to control the experience. 

Physical interactions are detected with the help of an Arduino Mega and the facial control pad even extends and retracts for optimal usage via a motor shield and stepper motors.

We present FaceWidgets, a device integrated with the backside of a head-mounted display (HMD) that enables tangible interactions using physical controls. To allow for near range-to-eye interactions, our first study suggested displaying the virtual widgets at 20 cm from the eye positions, which is 9 cm from the HMD backside. We propose two novel interactions, widget canvas and palm-facing gesture, that can help users avoid double vision and allow them to access the interface as needed. Our second study showed that displaying a hand reference improved performance of face widgets interactions. We developed two applications of FaceWidgets, a fixed-layout 360 video player and a contextual input for smart home control. Finally, we compared four hand visualizations against the two applications in an exploratory study. Participants considered the transparent hand as the most suitable and responded positively to our system.

As seen here, “Standard controllers for virtual reality (VR) lack sophisticated means to convey realistic, kinesthetic impression on size, resistance or inertia.” To overcome these limitations, André Zenner and Antonio Krüger at the German Research Center for Artificial Intelligence (DFKI) have come up with Drag:on—a haptic feedback device that changes air resistance and weight distribution using a commercially-available hand fan.

Drag:on uses a pair of MG996R servos to actuate the fan, shifting its weight and air resistance as needed to simulate a virtual environment. The assembly is attached to an HTC Vive tracker, and an Arduino Nano provides control and computer interface via a USB serial link.

Drag:on leverages the airflow occurring at the controller during interaction. By dynamically adjusting its surface area, the controller changes the drag and rotational inertia felt by the user. In a user study, we found that Drag:on can provide distinguishable levels of haptic feedback. Our prototype increases the haptic realism in VR compared to standard controllers and when rotated or swung improves the perception of virtual resistance. By this, Drag:on provides haptic feedback suitable for rendering different virtual mechanical resistances, virtual gas streams, and virtual objects differing in scale, material and fill state

More details on the project can be found in the researchers’ paper here.

VR environments are meant to be immersive, but if you’ve ever thought what was missing is being actually pummeled by robotic fists, then James Bruton’s newest project could be just the thing. 

Bruton recently teamed up with students from Portsmouth University to build a robot that works in the real world, and coordinates its movements with a virtual setting displayed on the human’s headset.

The robot itself is controlled by an Arduino Mega, and features a differential (tank) drive with encoders for feedback. Shoulders can tilt from left to right, and the actual punching motion is handled by pneumatic actuators built from modified bicycle pumps. Robo-fists are covered by boxing gloves to keep humans relatively safe, and flesh-based competitors are given a small shield and sword-bat with which to fight back!

I worked on this project with final year degree students in Computer Games Technology at Portsmouth University CCI faculty. The robot hardware is controlled over a serial interface, the team built an VR game which controls the robot, so when you get hit in VR you get hit in real life! The robot is tracked back into VR with Vive trackers so it stays in sync.

While you’ve been hearing about virtual reality for the last 20 years or so, today the hardware required to build such a rig is finally to the point where it’s within the reach of consumers. As seen here, Relativty is a SteamVR-compatible headset that can be made for around $100.

Relativty uses a 3D-printed frame to house its 2560 x 1440 LCD screen, along with a pair of 80mm Fresnel lenses to properly focus the image. Control is accomplished via an Arduino Due and an MPU-6050 accelerometer, which feeds head-tracking info to an external gaming system. 

At this point, the device is clean though fairly basic, and will hopefully be the start of a truly excellent open source project as features are added.

Haptic feedback is something commonly used with handheld controllers and the like. However, in a virtual reality environment, it could also be used with the other interface surface attached to your body: the VR headset itself.

That’s the idea behind FacePush, which employs an Arduino Uno-powered pulley system to place tension on the straps of an HTC Vive headset. A corresponding pushing force is felt by the wearer through the headset in response to this action, creating yet another way to help immerse users in a virtual world. 

Applications tried so far include a boxing game, dive simulator, and 360-degree guidance You can check it out in a short demo below, and read more about it in the full research paper here.

Like a lot of 16-year-olds, [Maxime Coutée] wanted an Oculus Rift. Unlike a lot of 16-year-olds, [Maxime] and friends [Gabriel] and [Jonas] built one themselves for about a hundred bucks and posted it on GitHub. We’ll admit that at 16 we weren’t throwing around words like quaternions and antiderivatives, so we were duly impressed.

Before you assume this is just a box to put a phone in like a Google Cardboard, take a look at the bill of materials: an Arduino Due, a 2K LCD screen, a Fresnel lens, and an accelerometer/gyro. The team notes that the screen is what will push the price unpredictably, but they got by for about a hundred euro. At the current exchange rate, if you add up all the parts, they went a little over $100, but they were still under $150 assuming you have a 3D printer to print the mechanical parts.

The system uses two custom libraries that you could use even if you wanted a slightly different project. FastVR creates 3D virtual reality using Unity and WRMHL allows Unity to communicate with an Arduino. Both of these are on the team’s GitHub page, as well.

There was one other member of the team, their math teacher [Jerome Dieudonne] who they call [Sensei]. According to [Jerome] he is “… the theoretician of the team. I teach them math and I help them solving algorithm issues.” He must be very proud and we always applaud when someone takes the time to share what they know with students.

We don’t know what’s next for this group, but we will be keeping an eye on them to see what’s next. Maybe they’ll work on smell-o-vision.

As reported by the Creative Applications Network, “Tangibles Worlds explores the effects of tactile experience as a catalyst for full immersion in VR.”

The project by Stella Speziali takes the form of three separate boxes, along with an Oculus Rift headset. When a hand is placed in one of these boxes, the user is virtually transported to another dimension of sight and sound, controlled by IR distance sensors, flex sensors, capacitive wire, and several other devices interfaced with an Arduino Mega.

Each box contains an IR distance sensor, which detects when a hand is inserted and display the virtual world attributed to the box. This new virtual world surrounds the user. A sensor is placed on each wall within the boxes, this sensor recognizes the hand and activates an animation inside the virtual world. I tried to map the sensors in the virtual universe so that a little clue is given to the user and will lead him to trigger the animations.

The idea behind this installation is to go beyond “traditional” VR controllers for entirely new level of interaction. The video seen here gives an excellent preview of the strangeness of this type of interface, though using it with a headset and sensors would likely be an altogether different experience!

Whether you choose to control this vehicle with your mind or a joystick, the camera mounted on it will give you a new view of the world.

Maker “Imetomi” was inspired to create a tracked robot after he was able to salvage a camera off of a cheap drone. This became the basis of his FPV setup, which he fitted onto a little tracked vehicle. Although this would have been enough for most people, in addition to building a joystick-based controller, he also made it work with a brainwave headset.

Imetomi now has something that he can drive around virtually, spying on passersby, as long as it stays within the VR transmitter’s 50-meter range. Be sure to check out the video below, where the small bot shows of its impressive all-terrain capabilities, and read his Instructables write-up here.

[Florian] has been putting a lot of work into VR controllers that can be used without interfering with a regular mouse + keyboard combination, and his most recent work has opened the door to successfully emulating a Vive VR controller in Steam VR. He uses Arduino-based custom hardware on the hand, a Leap Motion controller, and fuses the data in software.

We’ve seen [Florian]’s work before in successfully combining a Leap Motion with additional hardware sensors. The idea is to compensate for the fact that the Leap Motion sensor is not very good at detecting some types of movement, such as tilting a fist towards or away from yourself — a movement similar to aiming a gun up or down. At the same time, an important goal is for any added hardware to leave fingers and hands free.

[Florian]’s DIY VR hand controls emulate the HTC Vive controllers in Valve’s Steam VR Tracking with a software chain that works with his custom hardware. His DIY controller doesn’t need to be actively held because by design it grips the hand, leaving fingers free to do other tasks like typing or gesturing.

Last time we saw [Florian]’s work, development was still heavy and there wasn’t any source code shared, but there’s now a git repository for the project with everything you’d need to join the fun. He adds that “I see a lot of people with Wii nunchucks looking to do this. With a few edits to my FreePIE script, they should be easily be able to enable whatever buttons/orientation data they want.”

We have DIY hardware emulating Vive controllers in software, and we’ve seen interfacing to the Vive’s Lighthouse hardware with DIY electronics. There’s a lot of hacking around going on in this area, and it’s exciting to see what comes next.