Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models

By Ryan Louie, Andy Coenen, Cheng-Zhi Anna Huang, Michael Terry, Carrie J. Cai. 2020

326 citations recorded by the source corpus.

While generative deep neural networks (DNNs) have demonstrated their capacity for creating novel musical compositions, less attention has been paid to the challenges and potential of co-creating with these musical AIs, especially for novices. In a needfinding study with a widely used, interactive musical AI, we found that the AI can overwhelm users with the amount of musical content it generates, and frustrate them with its non-deterministic output. To better match co-creation needs, we developed AI-steering tools, consisting of Voice Lanes that restrict content generation to particular voices; Example-Based Sliders to control the similarity of generated content to an existing example; Semantic Sliders to nudge music generation in high-level directions (happy/sad, conventional/surprising); and Multiple Alternatives of generated content to audition and choose from. In a summative study (N=21), we discovered the tools not only increased users' trust, control, comprehension, and sense of collaboration with the AI, but also contributed to a greater sense of self-efficacy and ownership of the composition relative to the AI.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition, LLM Agents & Tool Use, Generative Models, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

  1. Monica Dinculescu
  2. Michael J. Muller
  3. Mary Lou Maher
  4. Carrie J. Cai
  5. Jeba Rezwana
  6. Lydia B. Chilton
  7. Justin D. Weisz
  8. Vivian Liu

In Computer Vision and Pattern Recognition

  1. Monica Dinculescu
  2. Carrie J. Cai
  3. Vivian Lai
  4. Andy Coenen
  5. Ed Newton-Rex
  6. Michael Terry
  7. Michael J. Muller
  8. Lydia B. Chilton

In LLM Agents & Tool Use

  1. Carrie J. Cai
  2. Daphne Ippolito
  3. Swaroop Mishra
  4. Mihir Parmar
  5. Chitta Baral
  6. Man Luo

In Generative Models

  1. Monica Dinculescu
  2. Andy Coenen
  3. Carrie J. Cai
  4. Ann Yuan
  5. Chenhao Tan
  6. Cheng-Zhi Anna Huang
  7. Daphne Ippolito

In Human-Computer Interaction

  1. Michael Terry
  2. Tongshuang Sherry Wu
  3. Vivian Lai
  4. Michael J. Muller
  5. Carrie J. Cai
  6. Lydia B. Chilton
  7. Justin D. Weisz
  8. Han Liu

Citations in the corpus