Project Round-Up

ml5.js
3 min readNov 18, 2020

--

Every few months we want to highlight creative projects from people in our community—that means you! If you have a project you’d like to submit to the round-up, submit it through this form. This month we’re featuring work from Yining Shi’s Machine Learning for the Web class.

Machine Learning for the Web is a 14-week class at ITP/NYU with a focus on building creative and interactive projects in the browser with machine learning techniques. Tools like Teachable Machine, RunwayML, ml5.js and TensorFlow.js are really making the creating process easy and beginner friendly. I’m so happy and proud to see so many creative projects made by students for every week’s homework. I’d love to share some of the projects with you and discuss creative applications of Machine Learning with you. — Yining

A gif of a person waving their arms up and down. Algorithmically generated colorful dots trail behind their hands.

Bubble World by Sihan Zhang

Bubble World includes three mini games in which you can interact with bubbles in fun ways through your webcam. You can move the bubbles with your hands, bounce the bubbles with your head, or use your voice to control the bubbles. (Video | Documentation | Live Example)

A cardboard box dispenses candy with voice control.

Voice Candy Dispenser by Philip Cadoux

This project uses Google’s Teachable Machine Audio Classifier to identify when a user says “candy.” Upon hearing the keyword, a servo motor moves 90 degrees, allowing m&ms to fall on to a ramp and to the user. (Video | Documentation | Live Example)

A person uses their face to eat moon cakes on a screen.

Mooncake Hunter by KJ Ha

Mooncake Hunter is an interactive game using ML5.js PoseNet and P5.js. In this game, PoseNet is used to track the user’s nose position. The user’s nose turns into a fork, which serves as a tool to hunt for mooncakes. (Video | Documentation | Live Example)

A person uses their nose to play a digital guitar on the screen.

Nose Guitar by Michelle Sun

Move your head to make some noises! A virtual guitar that will be triggered by the physical movements of the user. (Documentation | Live Example)

Dancing Pumpkin Character by Julie Lizardo

In the holiday spirit, I decided to recreate my favorite halloween gif: the dancing Halloween man. This project allows users to become their own dancing pumpkin character by using ml5.js Posenet model to detect your body from your webcam and move the character on the canvas. (Video | Documentation | Live Example)

--

--

ml5.js

Friendly Machine Learning for the Web. ml5.js aims to make machine learning approachable for a broad audience of artists, creative coders, and students.