At Arduino Day, I talked about a project I and my collaborators have been working on to bring machine learning to the maker community. Machine learning is a technique for teaching software to recognize patterns using data, e.g. for recognizing spam emails or recommending related products. Our ESP (Example-based Sensor Predictions) software recognizes patterns in real-time sensor data, like gestures made with an accelerometer or sounds recorded by a microphone. The machine learning algorithms that power this pattern recognition are specified in Arduino-like code, while the recording and tuning of example sensor data is done in an interactive graphical interface. We’re working on building up a library of code examples for different applications so that Arduino users can easily apply machine learning to a broad range of problems.

The project is a part of my research at the University of California, Berkeley and is being done in collaboration with Ben Zhang, Audrey Leung, and my advisor Björn Hartmann. We’re building on the Gesture Recognition Toolkit (GRT) and openFrameworks. The software is still rough (and Mac only for now) but we’d welcome your feedback. Installations instructions are on our GitHub project page. Please report issues on GitHub.

Our project is part of a broader wave of projects aimed at helping electronics hobbyists make more sophisticated use of sensors in their interactive projects. Also building on the GRT is ml-lib, a machine learning toolkit for Max and Pure Data. Another project in a similar vein is the Wekinator, which is featured in a free online course on machine learning for musicians and artists. Rebecca Fiebrink, the creator of Wekinator, recently participated in a panel on machine learning in the arts and taught a workshop (with Phoenix Perry) at Resonate ’16. For non-real time applications, many people use scikit-learn, a set of Python tools. There’s also a wide range of related research from the academic community, which we survey on our project wiki.

For a high-level overview, check out this visual introduction to machine learning. For a thorough introduction, there are courses on machine learning from coursera and from udacity, among others. If you’re interested in a more arts- and design-focused approach, check out alt-AI, happening in NYC next month.

If you’d like to start experimenting with machine learning and sensors, an excellent place to get started is the built-in accelerometer and gyroscope on the Arduino or Genuino 101. With our ESP system, you can use these sensors to detect gestures and incorporate them into your interactive projects!

This unique electronic instrument combines a chopped up guitar and a hacked apart glockenspiel with an Arduino. [Aaron]‘s Glockentar consists of guitar hardware and glockenspiel keys mounted to a wood body. Solenoids placed above the keys actuate metal rods to play a note.

Under the hood, an Arduino connects the pieces. The conductive pick closes a circuit, which is a digital input into the Arduino. This actuates the corresponding solenoid to play the glockenspiel key, and sends a character to a computer over serial.

On the computer, an openFrameworks based program creates lighting that is projected onto each string. MadMapper is used for projection mapping, which maps the openFrameworks output to each string. Video is passed between applications using the Syphon framework.

[Aaron] has provided a write up that goes into details, including the Arduino and openFrameworks source for the project. There’s also a video overview and demo of the Glockentar after the break.

Makers need to familiarize themselves with the core concepts and the theory involved in creating applications such as Motion Sensing and Face Tracking. As the technology is churning out new hardware day and night, DIYers need to work hard to keep up and always be in touch with the latest technology around them.

-->-->

For example, anyone working with Accelerometers/ Gyroscopes or Inertial Measurement Units needs to understand the theory of Vectors, Force, Gravity and be able to work out complex mathematical problems. They may easily get an Arduino Board and an Accelerometer Breakout or an IMU Board and use a library instead of writing their own code but to truly understand the theory behind it; how the device actually works, is not for the faint of heart.

One such problem is the Face Tracking Application. Unless you know the real theory behind how the Algorithm actually works, you can only wonder about that robot which follows its master. Greg Borenstein had an idea of creating a website dedicated to this issue. Makematics – Math for Makers.

In an introductory post, Greg writes:

” I hope to show that a normal programmer with no special academic training can grapple with these areas of research and find a way in to understanding them. And as I go I aim to create material that will help others do the same. If I can do it, there’s no reason you can’t.”

More and more people should step forward and create or compile a good amount of research data to help fellow makers and DIYers in solving complex mathematical problems.

Remember this?

A good tutorial by Sparkfun will help us make the above display. Before we begin let us take a few questions on Open Frameworks.

Q> What is Open Frameworks?

openFrameworks is a very handy software library written in C++ that is written for the sole purpose of reducing the software development overhead faced by designers and artists that would like to create pieces that use various media (graphics, sound, video, electronics, etc.).

Q> Why Open Frameworks when there is Processing?

Because there are some things that oF is well suited for that Processing just can’t handle. oF is much better at creating projects that use a lot of 3D Graphics, computer vision libraries like OpenCV or projects that involve the real-time manipulation of video. Also, while Processing requires a Java backend, oF is simply a set of C++ libraries, meaning that developers comfortable with C++ will be right at home.

Q> Where can I find more data and examples on Open Frameworks?

You can start by reading this article.

Try it, it works!

Via:[sparkfun]