Blanchard, L., Naseck, P., Liang, K., Tan, J., Lei, H., Huang, C.-Z. A., & Paradiso, J. (2026). Enhancing Expressive Musical Conversation in the jam_bot. Proceedings of the International Conference on New Interfaces for Musical Expression, 621--626. https://doi.org/10.5281/zenodo.20784228

Abstract

Previous work introduced the jam_bot, a real-time system that embeds live music language models capable of generating symbolic music sequences coherent with a performer’s input. The system supports multiple interaction strategies that have been demonstrated in several public performances. However, these strategies limit expressive musical conversation by constraining tempo, form, or musical roles. We extend the jam_bot to support more expressive, open-ended interaction through four key improvements: (1) modeling velocity, a key dimension of expression in symbolic music; (2) increasing model throughput via a ggml implementation–required to accommodate the longer sequences induced by velocity modeling; (3) developing a new training modality that enables free-form call-and-response interaction across varying tempi; and (4) compensating for external MIDI output latency to ensure rhythmic coherence with the performer. We quantitatively evaluate the model throughput improvement and our latency compensation strategy, and offer generated MIDI samples online.1 Together, these enhancements enable the jam_bot to engage in natural, expressive musical conversation, eliminating key musical limitations to enable the development of future performances and installations.