-
Proceedings of the 14th Annual Workshop on the Intersection of HCI and PL (PLATEAU 2024) • 2024Conference Paper
Scientists and Code: Programming as a Tool ↗
Click to read abstract
Many scientists use programming to analyze their data. In this paper, we explore the computational ecosystem of scientists and their socio-technical system of computing through a human-centered approach. By employing contextual inquiry techniques with nine scientists drawn from fields such as theoretical physics, biomedical science, and entomology, we learned that programming is a tool for scientists, and as such the output is more important than the code itself. We found that during analysis, scientists often write code to create plots, and then compare these plots to assess the match of output to their expectation. Participants used ChatGPT while coding. We also found that scientists' programming tools and practices often limit their analysis. Finally, based on a combined human-computer interaction and programming language analysis, we identify drivers and blockers of scientists' work. Our findings uncover opportunities for the design of programming tools and languages.
-
doi The Reality of the Situation: A Survey of Situated Analytics ↗
Click to read abstract
The advent of low-cost, accessible, and high-performance augmented reality (AR) has shed light on a situated form of analytics where in-situ visualizations embedded in the real world can facilitate sensemaking based on the user's physical location. In this work, we identify prior literature in this emerging field with a focus on situated analytics. After collecting 47 relevant situated analytics systems, we classify them using a taxonomy of three dimensions: situating triggers, view situatedness, and data depiction. We then identify four archetypical patterns in our classification using an ensemble cluster analysis. We also assess the level which these systems support the sensemaking process. Finally, we discuss insights and design guidelines that we learned from our analysis.
-
doi Towards Visualization Thumbnail Designs That Entice Reading Data-Driven Articles ↗
Click to read abstract
As online news increasingly include data journalism, there is a corresponding increase in the incorporation of visualization in article thumbnail images. However, little research exists on the design rationale for visualization thumbnails, such as resizing, cropping, simplifying, and embellishing charts that appear within the body of the associated article. Therefore, in this paper we aim to understand these design choices and determine what makes a visualization thumbnail inviting and interpretable. To this end, we first survey visualization thumbnails collected online and discuss visualization thumbnail practices with data journalists and news graphics designers. Based on the survey and discussion results, we then define a design space for visualization thumbnails and conduct a user study with four types of visualization thumbnails derived from the design space. The study results indicate that different chart components play different roles in attracting reader attention and enhancing reader understandability of the visualization thumbnails. We also find various thumbnail design strategies for effectively combining the charts' components, such as a data summary with highlights and data labels, and a visual legend with text labels and Human Recognizable Objects (HROs), into thumbnails. Ultimately, we distill our findings into design implications that allow effective visualization thumbnail designs for data-rich news articles. Our work can thus be seen as a first step toward providing structured guidance on how to design compelling thumbnails for data stories.
-
doi Dataopsy: Scalable and Fluid Visual Exploration using Aggregate Query Sculpting ↗
Click to read abstract
We present aggregate query sculpting (AQS), a faceted visual query technique for large-scale multidimensional data. As a 'born scalable' query technique, AQS starts visualization with a single visual mark representing an aggregation of the entire dataset. The user can then progressively explore the dataset through a sequence of operations abbreviated as P6: pivot (facet an aggregate based on an attribute), partition (lay out a facet in space), peek (see inside a subset using an aggregate visual representation), pile (merge two or more subsets), project (extracting a subset into a new substrate), and prune (discard an aggregate not currently of interest). We validate AQS with Dataopsy, a prototype implementation of AQS that has been designed for fluid interaction on desktop and touch-based mobile devices. We demonstrate AQS and Dataopsy using two case studies and three application examples.
-
doi Lodestar: Supporting Rapid Prototyping of Data Science Workflows through Data-Driven Analysis Recommendations ↗
Click to read abstract
Keeping abreast of current trends, technologies, and best practices in visualization and data analysis is becoming increasingly difficult, especially for fledgling data scientists. In this paper, we propose lodestar, an interactive computational notebook that allows users to quickly explore and construct new data science workflows by selecting from a list of automated analysis recommendations. We derive our recommendations from directed graphs of known analysis states, with two input sources: one manually curated from online data science tutorials, and another extracted through semi-automatic analysis of a corpus of over 6000 Jupyter notebooks. We validated Lodestar through three separate user studies: first a formative evaluation involving novices learning data science using the tool. We used the feedback from this study to improve the tool. This was followed by a summative study involving both new and returning participants from the formative evaluation to test the efficacy of our improvements. We also engaged professional data scientists in an expert review assessing the utility of the different recommendations. Overall, our results suggest that both novice and professional users find Lodestar useful for rapidly creating data science workflows.
-
doi Riverside: A Design Study on Visualization for Situation Awareness in Cybersecurity ↗
Click to read abstract
Real-time situation awareness is a key challenge of cybersecurity defense. Visual analytics has been utilized for this purpose, but existing tools tend to require detailed knowledge about the network, which can be challenging in large-scale, production networks. We conducted an interview study involving 24 security professionals to gather requirements for the design, development, and evaluation of visualization to aid situation awareness in cybersecurity. Using these findings, we designed a visualization tool -- called RIVERSIDE -- for providing a real-time view of the dynamically changing computer network to support situation awareness. We evaluated Riverside in a user study involving 10 participants. Participants were placed in an incident response scenario that tasked them to identify malicious activity on a network. 20\% of the users identified all attack component, while an additional 40\% only missed one component.
-
doi TactualPlot: Spatializing Data as Sound Using Sensory Substitution for Touchscreen Accessibility ↗
Click to read abstract
Tactile graphics are one of the best ways for a blind person to perceive a chart using touch, but their fabrication is often costly, time-consuming, and does not lend itself to dynamic exploration. Refreshable haptic displays tend to be expensive and thus unavailable to most blind individuals. We propose TactualPlot, an approach to sensory substitution where touch interaction yields auditory (sonified) feedback. The technique relies on embodied cognition for spatial awareness - i.e., individuals can perceive 2D touch locations of their fingers with reference to other 2D locations such as the relative locations of other fingers or chart characteristics that are visualized on touchscreens. Combining touch and sound in this way yields a scalable data exploration method for scatterplots where the data density under the user's fingertips is sampled. The sample regions can optionally be scaled based on how quickly the user moves their hand. Our development of TactualPlot was informed by formative design sessions with a blind collaborator, whose practice while using tactile scatterplots caused us to expand the technique for multiple fingers. We present results from an evaluation comparing our TactualPlot interaction technique to tactile graphics printed on swell touch paper.
-
doi Visualization According to Statisticians: An Interview Study on the Role of Visualization for Inferential Statistics ↗
Click to read abstract
Statisticians are not only one of the earliest professional adopters of data visualization, but also some of its most prolific users. Understanding how these professionals utilize visual representations in their analytic process may shed light on best practices for visual sensemaking. We present results from an interview study involving 18 professional statisticians (19.7 years average in the profession) on three aspects: (1) their use of visualization in their daily analytic work; (2) their mental models of inferential statistical processes; and (3) their design recommendations for how to best represent statistical inferences. Interview sessions consisted of discussing inferential statistics, eliciting participant sketches of suitable visual designs, and finally, a design intervention with our proposed visual designs. We analyzed interview transcripts using thematic analysis and open coding, deriving thematic codes on statistical mindset, analytic process, and analytic toolkit. The key findings for each aspect are as follows: (1) statisticians make extensive use of visualization during all phases of their work (and not just when reporting results); (2) their mental models of inferential methods tend to be mostly visually based; and (3) many statisticians abhor dichotomous thinking. The latter suggests that a multi-faceted visual display of inferential statistics that includes a visual indicator of analytically important effect sizes may help to balance the attributed epistemic power of traditional statistical testing with an awareness of the uncertainty of sensemaking.
-
doi Wizualization: A 'Hard Magic' Visualization System for Immersive and Ubiquitous Analytics ↗
Click to read abstract
What if magic could be used as an effective metaphor to perform data visualization and analysis using speech and gestures while mobile and on-the-go? In this paper, we introduce Wizualization, a visual analytics system for eXtended Reality (XR) that enables an analyst to author and interact with visualizations using such a magic system through gestures, speech commands, and touch interaction. Wizualization is a rendering system for current XR headsets that comprises several components: a cross-device (or Arcane Focuses) infrastructure for signalling and view control (Weave), a code notebook (Spellbook), and a grammar of graphics for XR (Optomancy). The system offers users three modes of input: gestures, spoken commands, and materials. We demonstrate Wizualization and its components using a motivating scenario on collaborative data analysis of pandemic data across time and space.
-
doi Is Native Naive? Comparing Native Game Engines and WebXR as Immersive Analytics Development Platforms ↗
Click to read abstract
Native game engines have long been the 3-D development platform of choice for research in mixed and augmented reality. For this reason, they have also been adopted in many immersive visualization and immersive analytics systems and toolkits. However, with the rapid improvements of WebXR and related open technologies, this choice may not always be optimal for future visualization research. In this article, we investigate common assumptions about native game engines versus WebXR and find that while native engines still have an advantage in many areas, WebXR is rapidly catching up and is superior for many immersive analytics applications.
-
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems • 2024Conference Paper
doi The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive Visualization ↗
Click to read abstract
The use of Large Language Models (LLMs) for writing has sparked controversy both among readers and writers. On one hand, writers are concerned that LLMs will deprive them of agency and ownership, and readers are concerned about spending their time on text generated by soulless machines. On the other hand, AI-assistance can improve writing as long as writers can conform to publisher policies, and as long as readers can be assured that a text has been verifed by a human. We argue that a system that captures the provenance of interaction with an LLM can help writers retain their agency, conform to policies, and communicate their use of AI to publishers and readers transparently. Thus we propose HaLLMark, a tool for visualizing the writer's interaction with the LLM. We evaluated HaLLMark with 13 creative writers, and found that it helped them retain a sense of control and ownership of the text.
-
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems • 2024Conference Paper
doi VisTorch: Interacting with Situated Visualizations using Handheld Projectors ↗
Click to read abstract
Spatial data is best analyzed in situ, but existing mixed reality technologies can be bulky, expensive, or unsuitable for collaboration. We present VisTorch: a handheld device for projected situated analytics consisting of a pico-projector, a multi-spectrum camera, and a touch surface. VisTorch enables viewing charts situated in physical space by simply pointing the device at a surface to reveal visualizations in that location. We evaluated the approach using both a user study and an expert review. In the former, we asked 20 participants to first organize charts in space and then refer to these charts to answer questions. We observed three spatial and one temporal pattern in participant analyses. In the latter, four experts---a museum designer, a statistical software developer, a theater stage designer, and an environmental educator---utilized VisTorch to derive practical usage scenarios. Results from our study showcase the utility of situated visualizations for memory and recall.
-
doi uxSense: Supporting User Experience Analysis with Visualization and Computer Vision ↗
Click to read abstract
Analyzing user behavior from usability evaluation can be a challenging and time-consuming task, especially as the number of participants and the scale and complexity of the evaluation grows. We propose UXSENSE, a visual analytics system using machine learning methods to extract user behavior from audio and video recordings as parallel time-stamped data streams. Our implementation draws on pattern recognition, computer vision, natural language processing, and machine learning to extract user sentiment, actions, posture, spoken words, and other features from such recordings. These streams are visualized as parallel timelines in a web-based front-end, enabling the researcher to search, filter, and annotate data across time and space. We present the results of a user study involving professional UX researchers evaluating user data using uxSense. In fact, we used uxSense itself to evaluate their sessions.
-
Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility • 2024Conference Paper
doi Understanding the Visualization and Analytics Needs of Blind and Low-Vision Professionals ↗
Click to read abstract
Inclusivity for blind and low vision (BLV) professionals in data science and analytics is limited by a gap in understanding their unique data analysis needs. We contribute to the literature by reporting on a two-step online survey delving into the experiences and challenges faced by BLV individuals engaged in data-related roles. Our fndings highlight that despite expertise in programming and GUI-based analysis tools, BLV professionals faced accessibility issues at various points in the data analysis pipeline---issues ranging from data loading and transformation, availability and compatibility of data tools with assistive technology, and visualization authoring. The prevalent use of tools such as Excel, Python, and SAS alongside heavy reliance on assistive technologies highlights persistent accessibility challenges. Furthermore, frequent collaboration with sighted colleagues indicates compromised independence. These results underscore the urgent need for "born accessible" tools that ensure the inclusivity and autonomy of BLV professionals in the feld of data science.
-
doi ``Wichita 1-1, Fox Three''---The Role of 3D Telemetry Analysis in Combat Flight Simulation ↗
Click to read abstract
Analyzing 3D telemetry data collected from competitive video games on the Internet can support players in improving performance as well as spectators in viewing data-driven narratives of the gameplay. In this paper, we conduct an in-depth qualitative study on the use of telemetry analysis by embedding over several weeks in a virtual F-14A Tomcat squadron in the multiplayer combat flight simulator DCS World (DCS) (2008). Based on formative interviews with DCS pilots, we design a web-based game analytics framework for rendering 3D telemetry from the flight simulator in a live 3D player, incorporating many of the data displays and visualizations requested by the participants. We then evaluate the framework with real mission data from several air-to-air engagements involving the virtual squadron. Our findings highlight the key role of 3D telemetry playback in competitive multiplayer gaming.
-
doi Visualizing Multilayer Spatiotemporal Epidemiological Data with Animated Geocircles ↗
Click to read abstract
Objective: The COVID-19 pandemic emphasized the value of geospatial visual analytics for both epidemiologists and the general public. However, systems struggled to encode temporal and geospatial trends of multiple, potentially interacting variables, such as active cases, deaths, and vaccinations. We sought to ask (1) how epidemiologists interact with visual analytics tools, (2) how multiple, time-varying, geospatial variables can be conveyed in a unified view, and (3) how complex spatiotemporal encodings affect utility for both experts and non-experts. Materials and Methods: We propose encoding variables with animated, concentric, hollow circles, allowing multiple variables via color encoding and avoiding occlusion problems, and we implement this method in a browser-based tool called CoronaViz. We conduct task-based evaluations with non-experts, as well as in-depth interviews and observational sessions with epidemiologists, covering a range of tools and encodings. Results: Sessions with epidemiologists confirmed the importance of multivariate, spatiotemporal queries and the utility of CoronaViz for answering them, while providing direction for future development. Non-experts tasked with performing spatiotemporal queries unanimously preferred animation to multi-view dashboards. Discussion: We find that conveying complex, multivariate data necessarily involves trade-offs. Yet, our studies suggest the importance of complementary visualization strategies, with our animated multivariate spatiotemporal encoding filling important needs for exploration and presentation. Conclusion: CoronaViz's unique ability to convey multiple, time-varying, geospatial variables makes it both a valuable addition to interactive COVID-19 dashboards and a platform for empowering experts and the public during future disease outbreaks. CoronaViz is open-source and a live instance is freely hosted at http://coronaviz.umiacs.io.