Social Spaces Group
people | projects | publications | classes | hci | videos
Plexlines and EnGaze: Visualizing autism pre-screening sessions

We present a system to visualize the annotations of videos of children engaged in a 3-5 minute interactive protocol that assesses communicative behavior, the Rapid ABC. Rapid ABC session videos are annotated to capture the child's gestures, gaze, and vocalization -- important developmental indicators for a young child. From these annotations, we create a graphical fingerprint for each session. In doing so, we introduce Plexlines and EnGaze, two webtools for visualizing multiple types of information layered as timelines.
Ha-Kyung Hidy Kong
Accelerating Data Exploration via Visual Query Systems

Through a year-long collaboration with scientists in astronomy, genetics, and material science, we study the impact of various features within visual query systems that can aid rapid visual data analysis, and how visual query systems fit into a scientists’ analysis workflow.
Doris Lee, Tarique Siddiqui
Characterizing Scalability Issues in Spreadsheet Software using Online Forums

We seek out complaints from users to better understand the usability issues that develop in spreadsheets as data increases in order to inform the creation of better spreadsheet software for large quantities of data.
Kelly Mack and John Lee
process communication
Communicating Algorithmic Process in Online Behavioral Advertising

Advertisers develop algorithms to select the most relevant advertisements for users. However, the opacity of these algorithms, along with their potential for violating user privacy, has decreased user trust and preference in behavioral advertising. To mitigate this, advertisers have started to communicate algorithmic processes in behavioral advertising. However, how revealing parts of the algorithmic process affects users' perceptions towards ads and platforms is still an open question. To investigate this, we exposed 32 users to why an ad is shown to them, what advertising algorithms infer about them, and how advertisers use this information. Users preferred interpretable, non-creepy explanations about why an ad is presented, along with a recognizable link to their identity. We further found that exposing users to their algorithmically-derived attributes led to algorithm disillusionment-users found that advertising algorithms they thought were perfect were far from it.
Motahhare Eslami
Effects of Socially Stigmatized Crowdfunding Campaigns in Shaping Opinions

Donation-based crowdfunding platforms have an increasing number of campaigns on socially stigmatized topics. Little research has been done to investigate whether these campaigns impact individuals' opinions and how. We conducted an experiment to explore how an attitude-inconsistent campaign on fairness and equality for LGBTIQ people influenced participants' opinion on this topic. Although all the participants changed their perceived opinions after reading the support for the campaigns, participants opposing equality were less inclined to change their attitude, less sensitive to the level of social support, and wanted to donate significantly more money to anti-equality campaigns compared to those who supported equality.
Sanorita Dey
Understanding Biased Algorithms and Users' Behavior around Them in Rating Platforms

Awareness of bias in algorithms is growing among scholars and users of algorithmic systems. But what can we observe about how users discover and behave around such biases? We used a cross-platform audit technique that analyzed online ratings of 803 hotels across three hotel rating platforms and found that one site's algorithmic rating system biased ratings, particularly low-to-medium quality hotels, significantly higher than others (up to 37%). Analyzing reviews of 162 users who independently discovered this bias, we seek to understand if, how, and in what ways users perceive and manage this bias. Users changed the typical ways they used a review on a hotel rating platform to instead discuss the rating system itself and raise other users' awareness of the rating bias. This raising of awareness included practices like efforts to reverseengineer the rating algorithm, efforts to correct the bias, and demonstrations of broken trust.
Motahhare Eslami
Parental Perceptions, Experiences, and Desires of Music Therapy

In this work, we explore the landscape of music therapy and development of ways to overcome the challenges that lead to the underuse of music therapy. Contrary to the public perception that music therapy only addresses emotional needs, 47 out of 59 participants reported seeing improvements in other areas including behavioral, cognitive, linguistic, and social changes. We further found that the inaccessibility and the cost were major limitations surrounding music therapy adoption. Based on these results, we explore technological implications such as teletherapy and online information organization.
Ha-Kyung Hidy Kong
The Invisible Side of Medical Crowdfunding

A growing number of people use online medical crowdfunding sites to solicit financial support for their medical expenses. In medical crowdfunding campaigns, the community support plays a critical role in promoting a campaign webpage to a wider audience and persuading many people to donate money. Despite the importance of the community support behaviors, they are often invisible on medical crowdfunding sites. The goal of this project is to investigate the invisible support behaviors and integrate them into existing medical crowdfunding sites.
Jennifer G. Kim
The Art and Science of Persuasion: Not All Crowdfunding Campaign Videos Are The Same

Campaign videos play an important role in communicating ideas or products effectively to the potential backers of crowdfunding campaigns. To better understand how videos affect campaign outcomes, we analyzed videos from 210 Kickstarter campaigns across three different project categories. Our analysis revealed product related and video related factors that were predictive of the final outcome of campaigns over and above the static project representation features identified in previous studies. Both the qualitative and quantitative analysis showed that videos influenced perception differently for projects in different categories, and the differential perception was important for predicting successes of the projects.
Sanorita Dey
Designing around Algorithm Sensemaking in Social Media

Our daily digital life is full of algorithmically selected content such as social media feeds, recommendations and personalized search results. These algorithms have great power to shape users' experiences, yet users are often unaware of their presence. Whether it is useful to give users insight into these algorithms' existence or functionality and how such insight might affect their experience are open questions. To address them, we conducted a user study with 40 Facebook users to examine their perceptions of the Facebook News Feed curation algorithm. In this study, we developed a system, FeedVis, to reveal the difference between the algorithmically curated and an unadulterated News Feed to users, and used it to study how users perceive this difference
Motahhare Eslami
Culture, Imagined Audience, and Language Choices

We explored the language choice of multilingual Chinese and Korean students in the United States on Facebook. Results show that multilingual users use language for dividing and filtering their imagined audience. Culture played two contrasting roles; users wanted to share their culture in English but share their emotions in their native language.
Ha-Kyung Hidy Kong
Designing the Future of Personal Fashion

Consumers and industry are highly interested in the potential of personalized fashion. In this work, we both develop models to automatically capture the elements of fashion style and develop tools to expose consumer pain points and opportunities for tech interventions in fashion.
Kristen Vaccaro
Evaluation of Automated Friend Grouping in Online Social Networks

Managing friendship relationships is challenging due to the growing number of people in online social networks (OSNs). While grouping friends sometimes mitigates this challenge, the burden of manual grouping still prevents OSNs users to create groups widely for privacy control, selective sharing and filtering. In this paper, we present an automated friend grouping tool which utilizes three different clustering algorithms to create groups from Facebook friendship networks. By conducting 18 semi-structured interviews, we investigated the advantages and disadvantages of automated friend grouping in OSNs.
Motahhare Eslami
CrowdBand: Crowdsourced Sound Composition
CrowdBand: Crowdsourced Sound Composition

Crowdband is an automated, crowdsourced sound composition system. Requestors use the CrowdBand interface to describe the desired result, and CrowdBand automatically partitions the job into one of three kinds of mechanical turk tasks: 1) fundamental sound provision, 2) sound assembly, and 3) sound evaluation. The crowd workers' responses demonstrated that workers who are non-musicians can design sound and create novel sound compositions through our system. We also conclucded that our system facilitates the use of common compositional techniques for musically untrained workers.
Mary Pietrowicz
SonicShapes: Visualizing Vocal Expression
SonicShapes: Visualizing Vocal Expression

SonicShapes is a real-time tool for the analysis and visualization of vocal expression. Many researchers have investigated speech and speaker recognition; and others focused on lie detection. Yet, the analysis and visualization of vocal expression remain relatively unexplored. Given the impact that semantic literacy had on society, what could the impact of nonsemantic literacy be? We begin to explore this question, and the potential impact that real time analysis and visualization of speech could have on other domains, such as speech therapy, medicine, and the creative arts.
Mary Pietrowicz
BEDA: Visual Analysis of Multimodal data for Longitudinal Studies
BEDA: Visual Analysis of Multimodal data for Longitudinal Studies

BEDA is a visual analytic tool that helps users explore many sessions of multimodal data that are collected from different sources (e.g., video data from a video-recorder, audio data from audio-recorder, temperature and heart rate data from sensors etc.). The tool supports visualization, synchronization, annotation, and analysis of multimodal data.
Jennifer Kim
VocSyl: Real-Time Multisyllabic Speech Visualization
VocSyl: Real-Time Multisyllabic Speech Visualization

VocSyl is a real-time audio visualization system for use within a clinical speech-therapy setting. To teach multisyllabic speech production, we visualize changes in syllables, timing (speed), tone (pitch) and emphasis (volume).
Joshua Hailpern
ACES Aphasia Emulation Software
ACES: Aphasia Emulation Software

We introduce a novel system and model, called Aphasia Characteristics Emulation Software (ACES), that enables users to experience the speech-distorting effects of aphasia. This model adheres to the wealth of literature in language distortions resulting from aphasia.
Joshua Hailpern
YouPivot: finding your digital car keys

Have you ever thought, "Where is that PDF or Website I was looking at the other day? I don't remember it's name, but I was listening to AC/DC while I was reading it." Until YouPivot, you cannot search your computer this way. YouPivot is a new search technique, grounded in cognitive psychology, that lets you search on contextual cues!
Joshua Hailpern
The Clotho Project
The Clotho Project: Predicting Application Utility

When using the computer, each user has some notion that "these applications are important" at a given point in time. Existing techniques fail to identify at least 57% of these applications. Our work directly associates measurable computer interaction (CPU consumption, window area, etc.) with the user’s perceived application utility/importance without identifying task.
Joshua Hailpern
espresso machine
Understanding Deja Reviewers

People who review products on the web invest considerable time in what they write. So why would someone write a review that restates earlier ones? This work is a mixed-method study of deja reviewers, latecomers who echo what other people said.
Eric Gilbert
visualization of photo collection
Photo Khipu

Photo Khipu provides a "metaphotograph" of digital photo collections by visualizing album, date, and social information. This work serves as an evocative display for the home.
Kora Bongen
tie strength model
Predicting Tie Strength

Social media treats all users the same: trusted friend or total stranger. In reality, relationships fall everywhere along this spectrum. Our work bridges this gap between theory and practice by building a model that maps social media data to tie strength.
Eric Gilbert
Visualization of Voice Space Example
Conversation Clusters

Conversation Clusters demonstrates means to capture topics from live conversation. Combined with human input, this technique creates a bridge between human understanding and computational techinques of modeling language.
Tony Bergstrom
Visualization of Voice Space Example

Visualizing audio content from remote conversations can be used as an artifact of the conversation. VoiceSpace explores user reactions to visualizing audio for archival and memory.
Pooja Mathur
A3: A Coding Guidline
A3: A Coding Guideline for HCI+Autism Research using Video Annotation.

A3 (pronounced A-Cubed) or Annotation for ASD Analysis, define and operationalize a set of dependent variables which are ideal for the examination and evaluation (through Video Annotation) of the behavior of subjects with ASD who interact with technology.
Joshua Hailpern
Visualization of Remote Conversation Example
Visual Remote Conversations

The number of face to face conversations is consistently decreasing as new technologies increasingly available. We attempt to add back some of the nonverbal information lost with the addition of visual cues.
Pooja Mathur, Tony Bergstrom
A Spoken Impact
Spoken Impact Project: Audio and Visual Feedback for Children with Autistic Spectrum Disorder.

We explore how audio and visual feedback can encourage speech and vocalization in children with Autistic Spectrum Disorder (ASD).
Joshua Hailpern
Better Energy Consumption Indicator
Designing a Better Energy Consumption Indicator

Rethinking how we can visualize energy consumption to homeowners using multiple levels of disaggregated goal-based feedback, including real-time, historical, and community comparisons
Erik Hinterbichler
network in the garden
The Network in the Garden: Social Media in Rural Life

Empirical analysis of behavioral differences between over 3,000 rural and urban social media users.
Eric Gilbert
VCode & VData
VCode and VData

VCode and VData are two new free tools to facilitate annotation and agreement calculations for Video. They are optimized for the annotation workflow, thus helping to provide accuracy and reliability.
Joshua Hailpern & Joey Hagedorn
conversation votes
Conversation Votes

Conversation Votes allows non-speaking participants an opportunity to augment and annotate a social mirror of conversation. This table visualization gives a voice to quiet comments.
Tony Bergstrom

CodeSaw is a social visualization of distributed software development. It visualizes both code and the conversation surrounding the code in a single space.
Eric Gilbert

Isochords is a visualization of music focused upon the consonat intervals. Visually, it emphasizes chord structures and modulation to aid the understanding of chord quality, chord progressions, modulations, etc.
Tony Bergstrom


Ginger extended the forms of Chit Chat Club to explore abstracted visual interaction to communicate social cues. Ginger provides a physical avatar to allowing people to interact in mixed virtual and physical environments.
Tony Bergstrom and Matthew Yapchaian

conversation clock
Conversation Clock

Conversation Clock seeks to augment developing dialog by displaying a representation of conversation to all present. The table demonstrates turn-taking, domination, interruption and activity throughout a conversation.
Tony Bergstrom
portal frame
Portal Frame

Portal Frame provides a low maintanence means to foster social bonds. Based upon context aware devices, Portal Frame creates a constantly updating image indicative of the user's activity.
Tony Bergstrom
buddy squares

An application to visualize instant messenger contact lists on the desktop in real time. The visualization aims to compute and display presence data in an ambient manor.
Matthew Critchlow
online queues
Online Queues

An applet that visualizes online queues and encourages interaction between those stuck in line.
Matthew Critchlow


text tone

A versatile and flexible way of expressing tone and emotion in online chat.
Ankur Kalra