Harmony Generation Driven by a Perceptually Motivated Tonal Interval Space

Abstract

We present D'accord, a generative music system for creating harmonically compatible accompaniments of symbolic and musical audio inputs with any number of voices, instrumentation, and complexity. The main novelty of our approach centers on offering multiple ranked solutions between a database of pitch configurations and a given musical input based on tonal pitch relatedness and consonance indicators computed in a perceptually motivated Tonal Interval Space. Furthermore, we detail a method to estimate the key of symbolic and musical audio inputs based on attributes of the space, which underpins the generation of key-related pitch configurations. The system is controlled via an adaptive interface implemented for Ableton Live, MAX, and Pure Data, which facilitates music creation for users regardless of music expertise and simultaneously serves as a performance, entertainment, and learning tool. We perform a threefold evaluation of D'accord, which assesses the level of accuracy of our key-finding algorithm, the user enjoyment of generated harmonic accompaniments, and the usability and learnability of the system.

References

1. Reinhold Behringer and John Elliot. 2009. Linking physical space with the Riemann Tonnetz for exploration of Western tonality. In Music Education, João Hermida and Mariana Ferrero (Eds.). Nova Science Publishers. Inc., Hauppauge, NY.
2. Emmanouil Benetos, Simon Dixon, Dimitrios Giannoulis, Holger Kirchhoff, and Anssi Klapuri. 2013. Automatic music transcription: Challenges and future directions. Journal of Intelligent Information Systems 41, 3, 407--434.
3. Gilberto Bernardes, Diogo Cocharro, Carlos Guedes, and Matthew Davies. 2016. A multi-level tonal interval space for modelling pitch relatedness and musical consonance. Journal of New Music Research 45, 4, 281--294. .
4. Louis Bigo, Jérémie Garcia, Antoine Spicher, and Wendy E. Mackay. 2012. Papertonnetz: Music composition with interactive paper. In Proceedings of the 9^th Sound and Music Computing Conference (Copenhagen, Denmark). 219--225.
5. Axel Bruns. 2007. Produsage: Towards a broader framework for user-led content creation. In Proceedings of the 6th ACM SIGCHI Conference on Creativity and Cognition. ACM, New York, 99--106.
6. Chris Cannam, Michael Jewell, Christopher Rhodes, Mark Sandler, and Mark d'Inverno. 2010. Linked data and you: Bringing music research software into the semantic web. Journal of New Music Research 39, 4, 313--325.
7. Elaine Chew. 2000. Towards a mathematical model of tonality. PhD dissertation, MIT.
8. Richard Cohn. 1997. Neo-Riemannian operations, parsimonious trichords, and their “tonnetz” representations. Journal of Music Theory 41, 1--66.
9. Matthew E. P. Davies, Mark D. Plumbley, and Douglas Eck. 2009. Towards a musical beat emphasis function. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. 61--64.
10. Rui Dias and Carlos Guedes. 2012. Gimmedablues app for iOS: Overview and ongoing developments. In Proceedings of the SIGGRAPH Asia 2012 Symposium on Apps (SA'12). ACM, New York, Article 2, 1 page.
11. Simon Dixon, Matthias Mauch, and Amélie Anglade. 2010. Probabilistic and logic-based modelling of harmony. In Proceedings of the 7th International Conference on Exploring Music Contents, Sølvi Ystad, Mitsuko Aramaki, Richard Kronland-Martinet, and Kristoffer Jensen (Eds.). Springer-Verlag, Berlin, 1--19.
12. Arne Eigenfeldt and Philippe Pasquier. 2010. Realtime generation of harmonic progressions using controlled Markov selection. In Proceedings of the ICCC-X-Computational Creativity Conference. 16--25.
13. Morwaread M. Farbood, Gary Marcus, and David Poeppel. 2013. Temporal dynamics and the identification of musical key. Journal of Experimental Psychology: Human Perception and Performance 39, 4, 911--918.
14. Dan Gang, Daniel Lehmann, and Naftali Wagner. 1997. Harmonizing melodies in real-time: The connectionist approach. In Proceedings of the International Computer Music Association. 27--31.
15. Emilia Gómez. 2006. Tonal description of polyphonic audio for music content processing. INFORMS Journal on Computing 18, 3, 294--304.
16. Garth Griffin and Robert Jacob. 2013. Priming creativity through improvisation on an adaptive musical instrument. In Proceedings of the 9th ACM Conference on Creativity and Cognition, Ellen Yi-Luen Do, Steven Dow, Jack Ox, Steve Smith, Kazushi Nishimoto, and Chek Tien Tan (Eds.). ACM, New York, 146--155.
17. Christopher Harte and Mark Sandler. 2005. Automatic chord identification using a quantised chromagram. Audio Engineering Society Convention. 118, 1--6.
18. Christopher Harte, Mark Sandler, and Martin Gasser. 2006. Detecting harmonic change in musical audio. In Proceedings of the 1st ACM Workshop on Audio and Music Computing Multimedia. ACM, New York, 21--26.
19. Christopher Harte. 2010. Towards automatic extraction of harmony information from music signals. Doctoral dissertation, Queen Mary, University of London.
20. David Huron. 1994. Interval-class content in equally tempered pitch-class sets: Common scales exhibit optimum tonal consonance. Music Perception: An Interdisciplinary Journal 11, 3, 289--305.
21. David Huron. 2001. Tone and voice: A derivation of the rules of voice-leading from perceptual principles. Music Perception 19, 1, 1--64.
22. William Hutchinson and Leon Knopoff. 1978. The acoustic component of western consonance. Interface -- Journal of New Music Research 7, 1, 1--29.
23. Özgür Izmirli. 2006. Audio key finding using low-dimensional spaces. In Proceedings of the International Society for Music Information Retrieval Conference. 127--132.
24. Akio Kameoka and Mamoru Kuriyagawa. 1969. Consonance theory. Part I: Consonance of dyads. Journal of Acoustical Society of America 45, 6, 1451--1459.
25. C. L. Krumhansl and E. J. Kessler. 1982. Tracing the dynamic changes in perceived tonal organization in a spatial map of musical keys. Psychological Review 89, 334--368.
26. Carol L. Krumhansl. 1990. Cognitive Foundations of Musical Pitch. Oxford University Press, New York.
27. Carol L. Krumhansl and Mark A. Schmuckler. 1986. Key finding in music: An algorithm based on pattern matching to tonal hierarchies. Paper presented at the 19th annual Mathematical Psychology meeting, Cambridge, MA.
28. Fred Lerdahl. 2001. Tonal Pitch Space. Oxford University Press.
29. David Lewin. 1987. Generalized Musical Intervals and Transformations. Yale University Press, New Haven, CT.
30. Ofir Lindenbaum, Arie Yeredor, and Israel Cohen. 2015. Musical key extraction using diffusion maps. Signal Processing 117, C, 198--207.
31. Hugh C. Longuet Higginsz. 1962. Two letters to a musical friend. Music Review 23, 244--248, 271--280.
32. Constantine F. Malmberg. 1918. The perception of consonance and dissonance. Psychological Monographs 25, 2, 93--133.
33. Bill Manaris, David Johnson, and Yiorgos Vassilandonakis. 2013. Harmonic navigator: A gesture-driven, corpus-based approach to music analysis, composition, and performance. In Proceedings of the 9th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. 67--74.
34. Matthias Mauch and Simon Dixon. 2010. Approximate note transcription for the improved identification of difficult chords. In Proceedings of the 11th International Society Music Information Retrieval Conference. 135--140.
35. Katy Noland and Mark B. Sandler. 2006. Key estimation using a hidden Markov model. In Proceedings of the International Society for Music Information Retrieval Conference. 121--126.
36. François Pachet and Pierre Roy. 2001. Musical harmonization with constraints: A survey. Constraints Journal 6, 7, 7--19.
37. François Pachet and Pierre Roy. 1998. Formulating constraint satisfaction problems on part-whole relations: The case of automatic musical harmonization. In Proceedings of the ECAI Workshop on Constraints for Artistic Applications. 1--11.
38. François Pachet. 1994. The muses system: An environment for experimenting with knowledge representation techniques in tonal harmony. In Proceedings of the 1st Brazilian Symposium on Computer Music. 95--201.
39. Hélène Papadopoulos and George Tzanetakis. 2012. Modeling chord and key structure with Markov logic. In Proceedings of International Society for Music Information Retrieval Conference. 121--126.
40. Somnuk Phon-Amnuaisuk, Andrew Tuson, and Geraint Wiggins. 1999. Evolving musical harmonisation. Artificial Neural Nets and Genetic Algorithms. Springer, Vienna, 229--234.
41. Hugo Riemann. 1893. Vereinfachte harmonielehre, oder die lehre von den tonalen funktionen der akkorde. 1896. Tr. H. Bewerunge, Augener. London.
42. Thomas Rocher, Matthias Robine, Pierre Hanna, and Laurent Oudre. 2010. Concurrent estimation of chords and keys from audio. In Proceedings of the International Society for Music Information Retrieval Conference. 141--146.
43. Helmut Schaffrath and David Huron. 1995. The Essen Folksong Collection in the Humdrum Kern Format. Center Computer Assisted Research in the Humanities, Menlo Park, CA..
44. Marc Schonbrun. 2011. The everything music theory: Take your understanding of music to the next level. Adams Media.
45. Roger Shepard. 1982. Structural representations of musical pitch. In The Psychology of Music, Diana Deutsch (Ed.). Academic Press, 335--353.
46. Karen Smith and Lola Cuddy. 1986. The pleasingness of melodic sequences: Contrasting effects of repetition and rule-familiarity. Psychology of Music 14, 1, 17--32.
47. Mark Steedman. 1999. Categorical grammar. In The MIT Encyclopedia of Cognitive Sciences. MIT Press.
48. David Temperley. 1999. What's key for key? The Krumhansl-Schmuckler key-finding algorithm reconsidered. Music Perception: An Interdisciplinary Journal 17, 1, 65--100.
49. Geraint Wiggins. 1999. Automated generation of musical harmony: What's missing. In Proceedings of the International Joint Conference in Artificial Intelligence.

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Music retrieval