We are building an intelligent musical partner that plays duets with you and teaches music. This research teaches us how to design more intimate human-AI symbiotic systems, innovate on machine learning models and architectures, as well as add a "spark of life" to automata. It evolves an initial corpus as you use it. This research bridges the gap between generative machine learning models for music that are unresponsive to real time input and real time interactive music systems that typically struggle to generate high-fidelity phrases that span a larger coherent movement or gesture. Music is a test case of the more general issue of how we explore ideas and make decisions in partnership with a machine that is increasingly viewed as “creative.”
Try it out here: https://www.neural-notes.ai/
Paper:
Mike Hao Jiang and Andrew Lippman. 2026. Neural Notes: An Online Platform for Real-Time Structured Musical Improvisation Supporting Human-Human, Human-AI, and AI-AI Interaction. In Companion Proceedings of the 31st International Conference on Intelligent User Interfaces (IUI Companion ’26), March 23–26, 2026, Paphos, Cyprus. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3742414.3794766