Programming an Arduino to do simple things like turn on an LED or read a sensor is easy enough via the official IDE. However, think back to your first experiences with this type of hardware. While rewarding, getting everything set up correctly was certainly more of a challenge, requiring research that you now likely take for granted.
To assist with these first steps of a beginner’s hardware journey, researchers at KAIST in South Korea have come up with HeyTeddy, a “conversational test-driven development [tool] for physical computing.”
As seen in the video below, HeyTeddy’s voice input is handled by an Amazon Echo Dot, which passes these commands through the cloud to a Raspberry Pi. The system then interacts with the physical hardware on a breadboard using an Arduino Uno running Firmata firmware, along with a 7” 1024 x 600 LCD touchscreen for the GUI. Once programmed, code can be exported and used on the board by itself.
Those wishing to learn more can check out the entire research paper here.
HeyTeddy is a conversational agent that allows users to program and execute code in real-time on an Arduino device without writing actual code but instead operating it through dialogue. This conversation can either be based on voice or text (through a Web chat). Commands spoken to HeyTeddy are parsed, interpreted, and executed in real-time, resulting in physical changes to the hardware. For example, the “write high” command configures an I/O pin to behave as a digital output with its internal state set to high (e.g., a 5V logic level), making driving an LED possible. Hence, the user does not need to write any code, compile it, deal with errors, and manually upload it on the hardware.
Furthermore, HeyTeddy supervises the user’s choices, preventing incorrect logic (e.g., writing an analog value to a digital pin), guiding the user through each step needed to assemble the circuit, and providing an opportunity to test individual components through separate unit tests without interrupting the workflow (i.e., TDD functionalities). Finally, the user has the option of exporting the issued commands as a written code for Arduino (i.e., an Arduino sketch in C++, ready for upload).