Happy New Year! We’re back and excited to kick off the new year with a project round up. Below, we’ll take a look at some amazing projects incorporating ml5.js from students at NYU’s Intro to ML for the Arts class! From machine learning-powered art installations to music generation algorithms, these projects showcase the boundless potential of incorporating artificial intelligence into the creative process.
If you have a project you’d like to submit to be featured on our socials, submit it through this form.
Disclaimer: This post contains content that may not be suitable for minors or younger audiences. Reader discretion is advised.
Intro to ML for the Arts is a course in the ITP department at NYU Tisch where students can learn to create interactive web-based projects with the help of beginner friendly machine learning tools, taught by Yining Shi.
‘Quick, Madlibs!’ by Hannah Zhao and Carolina Herrera
Quick, Madlibs is a game that uses Google’s Quick, Draw! to fill in the blanks of various stories, Mad Libs-style
Tools Used: ml5js, p5.js
(Video | Documentation | Live Example)
‘NN Playground’ by Ziyuan Lin
Inspired by Teachable Machine. NN Playground is an in-browser graphical interface that provides both an intuitive and comprehensive approach to training machine learning models. It will provide us with a shortcut to our dream application or project.
Tools Used: ml5js, React
(Video | Live Example)
‘Performer — Canon remix’ by Jiayi Tan
Users use hand pose and facial features to interact with the elements, and perform the “Canon In D Major” that belongs to you.
Tools Used: ml5.js
(Video | Documentation | Live Example)
‘Choose your best Nike Shoe’ by Jasper Zhao
You describe features of a running shoe you want, and AI helps you choose which Nike running shoe is best for you.
Tools Used: ml5.js, Teachable Machine, RunwayML, p5.js
(Video | Documentation | Live Example)
‘AI Artist’ by Mingren Fu
AI Artist is a tool that will transform users’ imaginations into photorealistic images. In terms of how it works, each color on the canvas will correspond to a specific type of element (such as mountains, rivers, trees, etc.) in nature. Based on the shape and color of the doodles that users draw on the canvas, AI Artist will identify the associated nature elements and generate an output as a photorealistic image. Users can control and interact with my project by either clicking the buttons or making verbal commands such as “mountain” or “generate” through speech recognition.
Tools Used: RunwayML, p5.js
(Video | Documentation | Live Example)
‘MagentaMuse’ by Anzhelika Nastashchuk
MagentaMuse is a web-based tool that allows users to generate, manipulate, upload, and save MIDI files. This application was built using Magenta.js, a JavaScript library for creating music using machine learning algorithms. Two different models from the Magenta library were implemented: a MusicVAE and MusicRNN. This project was done specifically for music producers or composers who would use this application in case of writer’s block or for inspiration.
Tools Used: Magenta
(Video | Documentation | Live Example)
‘A drawing of a full body beautiful girl, young, hyper-realistic, very detailed, intricate, very sexy pose, unreal engine, damatic lighting, 8k, detailed, black and white.’ by Jiwon Yu
A mixed media project that explores interaction with a traditional media: the goal of the project is to protray the suffocating gazes that women recieve in the digital world. The audio of women breathing, will stop when a face is detected in front of the painting. Then, the audio resumes when the user gets out of the frame.
Tools Used: Posenet, RunwayML, p5.js
(Video | Documentation | Live Example)
We look forward to showcasing more incredible work from you all in the near future. Don’t hesitate to submit your project using the form above, we can’t wait to see what you create!
