Onomato planets: physical computing of Japanese onomatopoeia

ABSTRACT

We proposed a design idea of an interactive educational toy for children based on Japanese onomatopoeia. Onomatopoeia "is a word or a grouping of words that imitates the sound it is describing, such as animal noises like "oink" or "meow", or suggesting its source object". Thus, onomatopoeia words have interesting sound and rhythm. There are a large number of Japanese onomatopoeia words and in the Japanese language in particular, they are also used to describe phenomena apart from the purely auditive.

With "Onomato Planets" we can create various Japanese onomatopoeia words and the corresponding sounds, through touch. The rules, by which these are created, are defined by the actual meaning of these words. Japanese onomatopoeia words contain many elements that are based on cultural background and experiences (called "Nonverbal elements"). For instance, sound intensity and gesture (movement, force) of the phenomenon being described are nonverbal elements of Japanese onomatopoeia. Through the interface we identify the strength and type of the impact and the gesture of the hand on a multi touch display, and then map this information to Japanese Onomatopoeia words characters and sound.

Finally, we aim to provide an exciting experience of sounds and characters to the children who play "Onomato Planets". There is no previous work that introduces mappings between characters or words, to vocal and motion parameters. Therefore this project expands the expression of vocal to motion interactive art by relating gestures, sounds and characters (words).

References

1. Golan Levin and Zachary Lieberman: "In-situspeech visualization in real-time interactive installation and performance", NPAR '04, pp. 7--14 (2004).
2. Paradiso, J, Leo, C. K., Checka, N. & Hsiao, K. "Passive Acoustic Knock Tracking for Interactive Windows", ACM CHI '02, pp. 732--733 (2002)