Tennison, J.L. – VEMI Lab

Bridging the Gap of Graphical Information Accessibility in Education With Multimodal Touchscreens Among Students With Blindness and Low Vision

aforbes — Fri, 15 Nov 2024 17:16:33 +0000

Informational graphics and data representations (e.g., charts and figures) are
critical for accessing educational content. Novel technologies, such as the multimodal
touchscreen which displays audio, haptic, and visual information, are promising for being
platforms of diverse means to access digital content. This work evaluated educational graphics rendered on a touchscreen compared to the current standard for accessing graphical content.

Citation:
Tennison, J. L., Goswami, S., Hairston, J. R., Merlin Drews, P., Smith, D. W., Giudice, N. A., Stefik, A., & Gorlewicz, J. L. (2023). Bridging the Gap of Graphical Information Accessibility in Education With Multimodal Touchscreens Among Students With Blindness and Low Vision. Journal of Visual Impairment & Blindness, 117(6), 453-466. https://doi.org/10.1177/0145482X231217496

Download PDF

Spatial audio-enhanced multimodal graph rendering for efficient data trend learning on touchscreen devices

ederosby — Sat, 11 May 2024 15:53:32 +0000

Touchscreen-based rendering of graphics using vibrations, sonification, and text-to-speech is a promising approach for nonvisual access to graphical information, but extracting trends from complex data representations nonvisually is challenging. This work presents the design of a multimodal feedback scheme with integrated spatial audio for the exploration of histograms and scatter plots on touchscreens. We detail the hardware employed and the algorithms used to control vibrations and sonification adjustments through the change of pitch and directional stereo output. We conducted formative testing with 5 blind or visually impaired participants, and results illustrate that spatial audio has the potential to increase the identification of trends in the data, at the expense of a skewed mental representation of the graph. This design work and pilot study are critical to the iterative, human-centered approach of rendering multimodal graphics on touchscreens and contribute a new scheme for efficiently capturing data trends in complex data representations.

�侱�ٲ��پ��ǲ�:��

Robinson-Moore, W. Kalal, M., Tennison, J.L., Giudice, N.A., & Gorlewicz, J.L. (2024). Spatial audio-enhanced multimodal graph rendering for efficient data trend learning on touchscreen devices. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). DOI: 10.1145/3613904.3641959.

Establishing Vibration-based Tactile Line Profiles for Use in Multimodal Graphics

eblackwood — Thu, 27 Aug 2020 20:49:30 +0000

Vibration plays a significant role in the way users interact with touchscreens. For many users, vibration affords tactile alerts and other enhancements. For eyes-free users and users with visual impairments, vibration can also serve a more primary role in the user interface, such as indicating streets on maps, conveying information about graphs, or even specifying basic graphics. However, vibration is rarely used in current user interfaces beyond basic cuing. Furthermore, designers and developers who do actually use vibration more extensively are often unable to determine the exact properties of the vibration signals they are implementing, due to out-of-the-box software and hardware limitations. We make two contributions in this work. First, we investigate the contextual properties of touchscreen vibrations and how vibrations can be used to effectively convey traditional, embossed elements, such as dashes and dots. To do so, we developed an open source, Android-based library to generate vibrations that are perceptually salient and intuitive, improving upon existing vibration libraries. Second, we conducted a user study with 26 blind or visually impaired users to evaluate and categorize the effects with respect to traditional tactile line profiles. We have established a range of vibration effects that can be reliably generated by our haptic library and are perceptible and distinguishable by users.

Citation:

Tennison, J.L., Uesbeck, P.M., Giudice, N.A., Stefik, A., Smith, D.W., & Gorlewicz, J.L. (2020). Establishing Vibration-based Tactile Line Profiles for Use in Multimodal Graphics. Transactions on Applied Perception, 17(2), 1-14.

Download PDF

Design Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics

eblackwood — Mon, 24 Aug 2020 20:20:31 +0000

With content rapidly moving to the electronic space, access to graphics for individuals with visual impairments is a growing concern. Recent research has demonstrated the potential for representing basic graphical content on touchscreens using vibrations and sounds, yet few guidelines or processes exist to guide the design of multimodal, touchscreen-based graphics. In this work, we seek to address this gap by synergizing our collective research efforts over the past eight years and implementing our findings into a compilation of recommendations, which we validate through an iterative design process and user study. We start by reviewing previous work and then collate findings into a set of design guidelines for generating basic elements of touchscreen-based multimodal graphics. We then use these guidelines to generate exemplary graphics in mathematics, specifically bar charts and geometry concepts. We discuss the iterative design process of moving from guidelines to actual graphics and highlight challenges. We then present a formal user study with 22 participants with visual impairments, comparing learning performance on using touchscreen-rendered graphics to embossed graphics.We conclude with qualitative feedback from participants on the touchscreen based approach and offer areas of future investigation as these recommendation are expanded to include more complex graphical concepts.

Citation:

Gorlewicz, J.L., Tennison, J.L., Uesbeck, P.M., Richard, M.E., Palani, H.P., Stefik, A., Smith, D.W., & Giudice, N.A. (2020). Design Guidelines and Recommendations for Multimodal, Touchscreen-Based Graphics. ACM Transactions on Accessible Computing (TACCESS), 13(3), 1-30.

Download PDF

The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways Forward

eblackwood — Fri, 03 May 2019 16:19:07 +0000

Graphical access is one of the most pressing challenges for individuals who are blind or visually impaired. This chapter discusses some of the factors underlying the graphics access challenge, reviews prior approaches to addressing this long-standing information access barrier, and describes some promising new solutions. We specifically focus on touchscreen-based smart devices, a relatively new class of
information access technologies, which our group believes represent an exemplary model of user-centered, needs-based design. We highlight both the challenges and the vast potential of these technologies for alleviating the graphics accessibility gap and share the latest results in this line of research. We close with recommendations on ideological shifts in mindset about how we approach solving this vexing access problem, which will complement both technological and perceptual advancements that are rapidly being uncovered through a growing research community in this domain.

Keywords: haptics, touchscreen-based accessibility, vibrotactile displays, multimodal interfaces, information-access technologies

Citation: Gorlewicz, J. L., Tennison, J. L., Palani, H. P., & Giudice, N. A. (2018). The Graphical Access Challenge for People with Visual Impairments: Positions and Pathways Forward. In Interactive Multimedia. IntechOpen (online publication ahead of print).

Download PDF

Comparing Haptic Pattern Matching on Tablets and Phones: Large Screens Are Not Necessarily Better

eblackwood — Fri, 07 Sep 2018 14:47:51 +0000

SIGNIFICANCE: Touchscreen-based, multimodal graphics represent an area of increasing research in digital access for individuals with blindness or visual impairments; yet, little empirical research on the effects of screen size on graphical exploration exists. This work probes if and when more screen area is necessary in supporting a pattern-matching task.

PURPOSE: Larger touchscreens are thought to have distinct benefit over smaller touchscreens for the amount of space available to convey graphical information nonvisually. The current study investigates two questions: (1) Do screen size and grid density impact a user’s accuracy on pattern-matching tasks? (2) Do screen size and grid density impact a user’s time on task?

METHODS: Fourteen blind and visually impaired individuals were given a pattern-matching task to complete on either a 10.5-in tablet or a 5.1-in phone. The patterns consisted of five vibrating targets imposed on sonified grids that varied in density (higher density = more grid squares). At test, participants compared the touchscreen pattern with a group of physical, embossed patterns and selected the matching pattern. Participants were evaluated on time exploring the pattern on the device and their pattern-matching accuracy. Multiple and logistic regressions were performed on the data.

RESULTS: Device size, grid density, and age had no statistically significant effects on the model of pattern-matching accuracy. However, device size, grid density, and age had significant effects on the model for grid exploration. Using the phone, exploring low-density grids, and being older were indicative of faster exploration time.

CONCLUSIONS: A trade-off of time and accuracy exists between devices that seems to be task dependent. Users may find a tablet most useful in situations where the accuracy of graphic interpretation is important and is not limited by time. Smaller screen sizes afforded comparable accuracy performance to tablets and were faster to explore overall.

Citation: Tennison, J.L., Carril, Z.S., Giudice, N.A., and Gorlewicz, J.L. (2018). Comparing Graphical Pattern Matching on Tablets and Phones: Large Screens are Not Necessarily Better. Optometry and Vision Science. 95(9), 720-726.

Download PDF

Touchscreen-based Haptic Information Access for assisting Blind and Visually-Impaired Users: Perceptual Parameters and Design Guidelines

eblackwood — Mon, 09 Jul 2018 21:05:58 +0000

Touchscreen-based smart devices, such as smartphones and tablets, offer great promise for providing blind and visually-impaired (BVI) users with a means for accessing graphics non-visually. However, they also offer novel challenges as they were primarily developed for use as a visual interface. This paper studies key usability parameters governing accurate rendering of haptically-perceivable graphical materials. Three psychophysically-motivated usability studies, incorporating 46 BVI participants, were conducted that identified three key parameters for accurate rendering of vibrotactile lines. Results suggested that the best performance and greatest perceptual salience is obtained with vibrotactile feedback based on: (1) a minimum width of 1mm for detecting lines, (2) a minimum gap of 4mm for discriminating lines rendered parallel to each other, and (3) a minimum angular separation (i.e., cord length) of 4mm for discriminating oriented lines. Findings provide foundational guidelines for converting/rendering visual graphical materials on touchscreen-based interfaces for supporting haptic/vibrotactile information access.

Keywords: Assistive Technology, Haptic information access, Haptic interaction, Multimodal interface, Design Guidelines

Citation:

Palani, H.P., Tennison, J.L., Giudice, G.B., & Giudice, N.A. (2018). Touchscreen-based haptic information access for assisting blind and visually-impaired users: Perceptual parameters and design guidelines. In: Ahram T., Falcão C. (eds.) Advances in Usability, User Experience and Assistive Technology, part of the International Conference on Applied Human Factors and Ergonomics (AHFE’18). Advances in Intelligent Systems and Computing, vol 798, (Pp. 837-847). Springer, Cham.

Download PDF