Smith, D.W. – VEMI Lab

Toward Accessible Parsons Problems on Mobile Platforms

ederosby — Tue, 17 Feb 2026 17:38:58 +0000

Parsons problems have become a mainstay of computer science education. They are heavily used among students, especially in K-12 and provide a small puzzle-like experience for students to practice their skills. Today, while prior work has explored complex issues with accessibility and block languages in general, the 2024 changes to accessibility regulations by the U.S. Department of Justice includes new rules around mobile platforms. These rules are ill-defined and in need of evaluation. In this work, we make several contributions. First, we conducted an evaluation of existing blocks with respect to their regulatory compliance and discuss a new blocks technology that we developed that meets these new mobile guidelines. Second, we conducted three empirical studies using Parsons problems to evaluate the usability of the technology with teachers of the visually impaired (n = 32), high-school students with diverse disabilities (n = 28), and high-school students with blindness or low vision (n = 13).

Citation:

Kluthe, T., Contreras, G., Allee, W., Robinson-Moore, W.J., Roy, N., Williams, H., Hoffman, A., Smith, D.W., Blaser, B., Gorlewicz, J.L., Giudice, N.A., & Stefik, A. (2026). Toward Accessible Parsons Problems on Mobile Platforms. In the Proceedings of the 57th ACM Technical Symposium on Computer Science Education (SIGCSETS’26). Vol 1, 589-596. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3770762.3772646.

Bridging the Gap of Graphical Information Accessibility in Education With Multimodal Touchscreens Among Students With Blindness and Low Vision

aforbes — Fri, 15 Nov 2024 17:16:33 +0000

Informational graphics and data representations (e.g., charts and figures) are
critical for accessing educational content. Novel technologies, such as the multimodal
touchscreen which displays audio, haptic, and visual information, are promising for being
platforms of diverse means to access digital content. This work evaluated educational graphics rendered on a touchscreen compared to the current standard for accessing graphical content.

Citation:
Tennison, J. L., Goswami, S., Hairston, J. R., Merlin Drews, P., Smith, D. W., Giudice, N. A., Stefik, A., & Gorlewicz, J. L. (2023). Bridging the Gap of Graphical Information Accessibility in Education With Multimodal Touchscreens Among Students With Blindness and Low Vision. Journal of Visual Impairment & Blindness, 117(6), 453-466. https://doi.org/10.1177/0145482X231217496

Download PDF

Establishing Vibration-based Tactile Line Profiles for Use in Multimodal Graphics

eblackwood — Thu, 27 Aug 2020 20:49:30 +0000

Vibration plays a significant role in the way users interact with touchscreens. For many users, vibration affords tactile alerts and other enhancements. For eyes-free users and users with visual impairments, vibration can also serve a more primary role in the user interface, such as indicating streets on maps, conveying information about graphs, or even specifying basic graphics. However, vibration is rarely used in current user interfaces beyond basic cuing. Furthermore, designers and developers who do actually use vibration more extensively are often unable to determine the exact properties of the vibration signals they are implementing, due to out-of-the-box software and hardware limitations. We make two contributions in this work. First, we investigate the contextual properties of touchscreen vibrations and how vibrations can be used to effectively convey traditional, embossed elements, such as dashes and dots. To do so, we developed an open source, Android-based library to generate vibrations that are perceptually salient and intuitive, improving upon existing vibration libraries. Second, we conducted a user study with 26 blind or visually impaired users to evaluate and categorize the effects with respect to traditional tactile line profiles. We have established a range of vibration effects that can be reliably generated by our haptic library and are perceptible and distinguishable by users.

Citation:

Tennison, J.L., Uesbeck, P.M., Giudice, N.A., Stefik, A., Smith, D.W., & Gorlewicz, J.L. (2020). Establishing Vibration-based Tactile Line Profiles for Use in Multimodal Graphics. Transactions on Applied Perception, 17(2), 1-14.

Download PDF

Design Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics

eblackwood — Mon, 24 Aug 2020 20:20:31 +0000

With content rapidly moving to the electronic space, access to graphics for individuals with visual impairments is a growing concern. Recent research has demonstrated the potential for representing basic graphical content on touchscreens using vibrations and sounds, yet few guidelines or processes exist to guide the design of multimodal, touchscreen-based graphics. In this work, we seek to address this gap by synergizing our collective research efforts over the past eight years and implementing our findings into a compilation of recommendations, which we validate through an iterative design process and user study. We start by reviewing previous work and then collate findings into a set of design guidelines for generating basic elements of touchscreen-based multimodal graphics. We then use these guidelines to generate exemplary graphics in mathematics, specifically bar charts and geometry concepts. We discuss the iterative design process of moving from guidelines to actual graphics and highlight challenges. We then present a formal user study with 22 participants with visual impairments, comparing learning performance on using touchscreen-rendered graphics to embossed graphics.We conclude with qualitative feedback from participants on the touchscreen based approach and offer areas of future investigation as these recommendation are expanded to include more complex graphical concepts.

Citation:

Gorlewicz, J.L., Tennison, J.L., Uesbeck, P.M., Richard, M.E., Palani, H.P., Stefik, A., Smith, D.W., & Giudice, N.A. (2020). Design Guidelines and Recommendations for Multimodal, Touchscreen-Based Graphics. ACM Transactions on Accessible Computing (TACCESS), 13(3), 1-30.

Download PDF