報告者：王新仁
報告PPT：https://reurl.cc/qDveOD
影片：https://vimeo.com/259129367
文獻題目：AI DJ PROJECT — A DIALOG BETWEEN HUMAN AND AI THROUGH MUSIC
文獻作者：Shoya Dozono, Qosmo, Nao Tokui
文獻來源：http://archive.aec.at/prix/showmode/61564/
摘要：
The system of AI DJ consists of the following three features:
1. Music selection
We trained three different neural networks for inferring genres, musical instruments and drum machines used in the track from spectrogram images. AI DJ “listens” to what human DJ plays and extracts auditory features using those networks. The extracted features are compared with those of all tracks in our pre-selected record box, so that the system can select the closest one, which presumably has similar musical tone/mood.
2. Beatmatching
It is also a task for AI DJ to control the pitch (speed) of the turntable to match the beat. We used “reinforcement learning” (RL) to teach the model how to speed up/down, nudge/pull the turntable to align downbeats through trials and errors. For this purpose, we built an OSC-compatible custom turntable and robot fingers to manipulate.
3. Crowd-reading
A good DJ should pay attention to the energy of the audience. We utilize a deep learning-based motion tracking technique to quantify how much people in the audience dance to the music AI plays for future music selection.
We have performed several times in different locations in Japan and Europe. AI’s slight unpredictability always brought amusing tension into the performance and gave new ideas to human DJs on what/how to play music as a DJ. AI is not a replacement for the human DJ. Instead, it is a partner that can think and play alongside its human counterpart, bringing forth a wider perspective of our relationship to contemporary technologies.
