An SO-101 robot arm, originally a desktop toy, now picks and places rubber ducks by reasoning with a vision-language-action model. Dmitry Maslov’s project swaps the original brain for an Arduino VENTUNO Q board, which runs Hugging Face’s SmolVLA model locally. The result shows that robotics tasks combining vision and language, once confined to costly labs, now fit on a 299-dollar board.

Hardware: two processors on one board

The Arduino VENTUNO Q board has a dual nature. On one side sits an SBC system with a Qualcomm Dragonwing IQ8 processor, equipped with an Adreno 623 GPU and a Hexagon Tensor NPU, built for AI processing. On the other side is an STM32H5F5 microcontroller with an Arm Cortex-M33 core running at 250 MHz, which handles real-time servo control. The board offers 16 GB of LPDDR5 RAM and 64 GB of integrated eMMC storage.

The original SO-101 arm carries gearmotors on every joint and two cameras: one fixed overhead and one on the gripper. The images and joint position data go to the Dragonwing IQ8 processor, where the SmolVLA model interprets them. The model was trained with 50 demonstrations of the task of grabbing and moving the rubber ducks.

The SmolVLA model and the system’s numbers

SmolVLA is an open source vision-language-action model from Hugging Face. It processes the camera images and joint data, then produces the commands for the servo motors. All the processing happens locally on the VENTUNO Q board, with no connection to external services. The board costs 299 dollars, against 399 dollars for the Jetson Orin Nano Super Developer Kit, cited as a competitor.

Dmitry Maslov’s system points the way to accessible robotics. What’s more, the fact that the model runs on a commercial board costing less than 300 dollars opens up new scenarios for makers. For anyone who wants to see the project in action, Dmitry Maslov’s video shows the arm grabbing and moving the rubber ducks.

• Arduino VENTUNO Q board with Qualcomm Dragonwing IQ8 processor and STM32H5F5
• Hugging Face SmolVLA model running locally
• 50 demonstrations to train the pick-and-place task
• Two cameras: one overhead and one on the gripper
• 16 GB of LPDDR5 RAM and 64 GB of eMMC storage

For anyone wanting to rebuild the project, the trickiest part is the integration between the SBC side and the microcontroller side of the board. The Dragonwing IQ8 processor runs the model, while the STM32H5F5 controls the servos. Communication between the two sides is essential to synchronise vision with movement.

Source: https://youtu.be/T0frBNASr3M?si=Wf1VPI8SoJ6tr18p

The post SO-101 Robot Arm Meets Arduino VENTUNO Q: Local-AI Pick-and-Place appeared first on Open Electronics.