Pooja Mathur, Tony Bergstrom, Karrie Karahalios

The number of face to face conversations is consistently decreasing as new technologies increasingly available. We attempt to add back some of the nonverbal information lost with the addition of visual cues. These cues are in the form of visualized auditory data. That is, one sees what one hears. We will look at evolution of the conversation mapping program in this thesis. Using four visualizations we asked users to describe to us how the visualizations aided or disrupted their conversations. With that feedback, we try to discover any support the program gives users. We conclude with our results on how to continue adding back nonverbal cues in conversations.

With online forms of communication evolving rapidly, sometimes, people feel the real essence of communicating is getting lost in the process. After emails, there was instant messaging. After instant messaging, came voice over IP. People have tried to find ways to bring more meaning to these conversations that are not held face to face. In instant messaging, emoticons were used to express facial expressions. That is just one form of nonverbal cues. However, with voice conversations, emoticons do not make as much sense. This is why we are looking at ways to visualize audio. We are creating visualizations to represent what one hears, just like one emoticons were used to represent what one sees.

To test our visualizations impact on voice over IP conversations, we performed numerous user studies. In these studies, two users were first asked to have a normal conversation. No visualizations were used. Each person was in a separate room. There was no way of seeing each other. After this initial conversation, which is used as a control, the users were then introduced to the program and its visualizations. Users had the basics of program explained, but it was left up to the users to find more meaning what was being displayed. After this conversation, users were asked to fill out a short survey, answering questions on how they felt the visualizations helped or hindered the conversation.