This dissertation research investigated navigation related spatio-cognitive changes associated with normal and healthy lifespan development. Age-related changes can negatively impact navigation ability, including increased safety concerns, greater risk of getting lost, and reduced navigation confidence for older adults. Experiment 1 explored the interaction between spatial memory and age. Participants learned 1, 3, or 6 target locations and were then asked to walk to one of these locations either directly or indirectly. Results showed that age had a significant impact on spatial memory limitations, as older adults were significantly less precise and accurate with responses when 3 to 6 targets were learned. Experiment 2 studied navigation behavior during driving between older and younger adults using an immersive virtual reality simulator. Participants were monitored as they drove a course that contained specific driving events known a priori to cause problems for older adults during real-world driving, such as intersection awareness and lane drifting. Results reflect clear disadvantages for older adult drivers as compared to younger drivers, but show that the same concerns in real world driving arise in the virtual simulation.

Experiments 3 and 4 characterized how stored cognitive maps, a critical mental structure needed to support navigation, change over time as a function of age. Participants learned outdoor (experiment 3) and indoor (experiment 4) environments, and then maintained those mental representations over the course of 2 weeks. Results demonstrated that the process of cognitive map decay follows a logarithmic trend over this temporal delay and occurs at greater magnitude for older adults as compared to their younger counterparts. The fifth study in this dissertation tested mitigation of cognitive map decay in older adults through the use of compensatory augmentations (spatial knowledge aids). The augmentations focused on online learning through enhanced visual access to landmarks and memory reconsolidation to support offline memory retention. Results show that the compensatory augmentations developed and tested were successful in reducing cognitive map decay. Contributions from this dissertation extend known theories of spatial aging literature to characterize a process of cognitive map decay for older adults and develop / evaluate solutions to reduce this decay.

Background: Blind and Visually Impaired (BVI) spatial learning research is mostly focused on indoor learning and cityscape learning. Little significant research has been conducted regarding the ideal method for conveying topographical information to BVI individuals. Filling this gap, this research aims to investigate several approaches to convey this important information and to empirically assess which is the best suited and the most accurate method to present the rendered information in an understandable manner. Methods: 30 adult participants (ages 18-40) were broken into two groups. Each performed several matching tasks, with one group given a tactile map to match to a 3D printed model and the other group given a 3D printed model to match to a tactile map. Learning Time, Selection Time, and accuracy of selection were evaluated.                 Results: All selection measures showed obvious learning effects for both conditions. Accuracy for selection was well above random chance. Having a pre-disposition to map-reading had a positive correlation with both selection speed and accuracy performance. Conclusion: Results suggest that the tactile contour maps were as successful in conveying topographic information to the participants as were the 3D printed models. The findings also suggest that after being trained with this type of tactile map, it is possible to recognize a map that resembles a 3D printed model.

People often become disoriented when navigating in complex, multi-level buildings. To efficiently find destinations located on different floors, navigators must refer to a globally coherent mental representation of the multi-level environment, which is termed a multi-level cognitive map. However, there is a surprising dearth of research into underlying theories of why integrating multi-level spatial knowledge into a multi-level cognitive map is so challenging and error-prone for humans. This overarching problem is the core motivation of this dissertation.

We address this vexing problem in a two-pronged approach combining study of both basic and applied research questions. Of theoretical interest, we investigate questions about how multi-level built environments are learned and structured in memory. The concept of multi-level cognitive maps and a framework of multi-level cognitive map development are provided. We then conducted a set of empirical experiments to evaluate the effects of several environmental factors on users’ development of multi-level cognitive maps. The findings of these studies provide important design guidelines that can be used by architects and help to better understand the research question of why people get lost in buildings. Related to application, we investigate questions about how to design user-friendly visualization interfaces that augment users’ capability to form multi-level cognitive maps. An important finding of this dissertation is that increasing visual access with an X-ray-like visualization interface is effective for overcoming the disadvantage of limited visual access in built environments and assists the development of multi-level cognitive maps. These findings provide important human-computer interaction (HCI) guidelines for visualization techniques to be used in future indoor navigation systems.

In sum, this dissertation adopts an interdisciplinary approach, combining theories from the fields of spatial cognition, information visualization, and HCI, addressing a long-standing and ubiquitous problem faced by anyone who navigates indoors: why do people get lost inside multi-level buildings. Results provide both theoretical and applied levels of knowledge generation and explanation, as well as contribute to the growing field of real-time indoor navigation systems.

This dissertation investigates schematisation of computer-generated tactile orientation maps that support mediation of spatial knowledge of unknown urban environments. Computer-generated tactile orientation maps are designed to provide the blind with an overall impression of their surroundings. Their details are displayed by means of elevated features that are created by embossers and can be distinguished by touch.

The initial observation of this dissertation states that only very little information is actu-
ally transported through tactile maps owing to the coarse resolution of tactual senses and the cognitive effort involved in the serial exploration of tactile maps. However, the differences between computer-generated, embossed tactile maps and manufactured, deep-drawn tactile maps are significant. Therefore the possibilities and confines of communicating information through tactile maps produced with embossers is a primary area of research. This dissertation has been able to demonstrate that the quality of embossed prints is an almost equal alternative to traditionally manufactured deep-drawn maps. Their great advantage is fast and individual production and (apart from the initial procurement costs for the printer) low price, accessibility and easy understanding without the need of prior time-consuming training.

Simplification of tactile maps is essential, even more so than in other maps. It can be achieved by selecting a limited number from all map elements available. Qualitative simpli fication through schematisation may present an additional option to simplification through quantitative selection. In this context schematisation is understood as cognitively motivated simplification of geometry and synchronised maintenance of topology. Rather than further reducing the number of displayed objects, the investigation concentrates on how the presentation of different forms of streets (natural vs. straightened) and junctions (natural vs. prototypical) affects the transfer of knowledge. In a second area of research, a thesis establishes that qualitative simplification of tactile orientation maps through schematisation can enhance their usability and make them easier to understand than maps that have not been schematised. The dissertation shows that simplifying street forms and limiting them to prototypical junctions does not only accelerate map exploration but also has a beneficial influence on retention performance. The majority of participants that took part in the investigation selected a combination of both as their preferred display option.

Tactile maps that have to be tediously explored through touch, uncovering every detail, complicate attaining a first impression or an overall perception. A third area of research is examined, establishing which means could facilitate map readers’ options to discover certain objects on the map quickly and without possessing a complete overview. Three types of aids are examined: guiding lines leading from the frame of the map to the object, position indicators represented by position markers at the frame of the map and coordinate specifications found within a grid on the map. The dissertation shows that all three varieties can be realized by embossers. Although a guiding line proves to be fast in size A4 tactile maps containing only one target object and few distracting objects, it also impedes further exploration of the map (similar to the grid). In the following, advantages and drawbacks of the various aids in this and other applications are discussed.

In conclusion the dissertation elaborates on the linking points of all three examinations. They connect and it is argued that cognitively motivated simplification should be a principle of construction for embossed tactile orientation maps in order to support their use and comprehension. A summary establishes the recommendations that result from this dissertation regarding construction of tactile orientation maps considering the limitations through embosser constraints. Then I deliberate how to adapt schematisation of other maps contingent to intended function, previous knowledge of the map reader, and the relation between the time in which knowledge is acquired and the time it is employed. Closing the dissertation, I provide an insight into its confines and deductions and finish with a prospective view to possible transfers of the findings to other applications, e.g. multimedia or interactive maps on pin-matrix displays and devices.

The existing research addressing non-visual indoor navigation is limited to route guidance between locations (i.e., the corridor network). This focus ignores many critical regions contained within indoor spaces (e.g., rooms, lobbies, etc.), locations which are often as challenging to learn and navigate without vision as are the routes connecting them. To address this challenge, this thesis investigates the use of natural language (NL) descriptions as a non-visual medium for providing access to indoor scenes, including room structure, furniture placement, and location of salient landmarks. The work is part of a larger project to develop a system, called the Describer for Indoor Scenes (DISc) that uses automatically generated NL descriptions to represent indoor scenes based on photos taken in real-time from mobile devices. In order to develop cognitively comprehensible NL descriptions of indoor scenes, it is critical to first understand how humans describe and interpret the scene in order to support spatial behavior. To this end, six behavioral experiments were conducted to characterize scene descriptions generated by human observers and to optimize these descriptions based on cognitive constraints and the structure of linguistic information to be included to best support non-visual learning, representation, and navigation.

The visual information that can be captured about a scene from photographs is potentially limited, both in quality and quantity, compared to the information apprehended from real time scene perception. Importantly for the DISc system, results from experiments 1, 2, and 3 converge to demonstrate that photographic observations are functionally equivalent to real time observations of indoor scenes in supporting spatial behavior and show that photographs can be used as information source in DISc. The data collected in these experiments showed that humans adopted different scene description strategies. To understand how the description strategy (i.e., order of objects) affected scene learning and reconstruction, a 4th behavioral experiment was conducted. Results from this experiment suggest that following a cyclic path while describing an indoor scene (called a “Round-91福利 strategy”) was the most efficient approach for acquiring and representing spatial knowledge.

The results from the first four experiments elucidated that people used two different angular units (clock face and degree measurements) to describe directional information. However, it was not clear from the extant literature how angular units affect spatial apprehension of the listener or which measure yields the most accurate performance. As directional information is critical for specifying the location of objects in a scene, this question was addressed in a fifth experiment, with results demonstrating that the most accurate performance manifested when angular directions were given as clock face units rather than degree measurements (i.e., 1:00 versus 30 degrees). Results also demonstrated that participants were equally accurate at producing angular values of 15 degrees or half hour increments (e.g., 1:30), which is meaningful as this is a 100% increase in precision from the standard clock face units employed in previous studies.

The sixth and final behavioral experiment was conducted to investigate whether the optimized NL scene descriptions support non-visual navigation of indoor scenes and if performance differs when using static or updated descriptions, meaning that they either were given from a fixed user perspective in the scene (as was done in the earlier experiments) or that the perspective changed based on the user’s position and orientation. Results showed a clear advantage for updated NL descriptions on navigation accuracy, indicating that to be maximally effective, DISc should implement descriptions based on the user’s real-time position and orientation as they move. Taken together, the results of six human experiments extend earlier research with route navigation by showing that optimizing NL indoor scene descriptions based on perceptual and cognitive factors led to efficient spatial learning, representation, and navigation. These empirical results provide the much needed proof of concept for the efficacy of future development of DISc as a fully automated NL scene description system.

Accessing graphical material such as graphs, figures, maps, and images is a major challenge for blind and visually impaired people. The traditional approaches that have addressed this issue have been plagued with various shortcomings (such as use of unintuitive sensory translation rules, prohibitive costs and limited portability), all hindering progress in reaching the blind and visually-impaired users. This thesis addresses aspects of these shortcomings, by designing and experimentally evaluating an intuitive approach —called a vibro-audio interface— for non-visual access to graphical material. The approach is based on commercially available touch-based devices (such as smartphones and tablets) where hand and finger movements over the display provide position and orientation cues by synchronously triggering vibration patterns, speech output and auditory cues, whenever an on-screen visual element is touched. Three human behavioral studies (Exp 1, 2, and 3) assessed usability of the vibro-audio interface by investigating whether its use leads to development of an accurate spatial representation of the graphical information being conveyed. Results demonstrated efficacy of the interface and importantly, showed that performance was functionally equivalent with that found using traditional hardcopy tactile graphics, which are the gold standard of non-visual graphical learning.

One limitation of this approach is the limited screen real estate of commercial touch-
screen devices. This means large and deep format graphics (e.g., maps) will not fit within the screen. Panning and zooming operations are traditional techniques to deal with this challenge but, performing these operations without vision (i.e., using touch) represents several computational challenges relating both to cognitive constraints of the user and technological constraints of the interface. To address these issues, two human behavioral experiments were conducted, that assessed the influence of panning (Exp 4) and zooming (Exp 5) operations in non-visual learning of graphical material and its related human factors. Results from experiments 4 and 5 indicated that the incorporation of panning and zooming operations enhances the non-visual learning process and leads to development of more accurate spatial representation. Together, this thesis demonstrates that the proposed approach —using a vibro-audio interface is a viable multimodal solution for presenting dynamic graphical information to blind and visually-impaired persons and supporting development of accurate spatial representations of otherwise inaccessible graphical materials.

Recent advancements in the field of indoor positioning and mobile computing promise development of smart phone based indoor navigation systems. Currently, the preliminary implementations of such systems only use visual interfaces—meaning that they are inaccessible to blind and low vision users. According to the World Health Organization, about 39 million people in the world are blind. This necessitates the need for development and evaluation of non-visual interfaces for indoor navigation systems that support safe and efficient spatial learning and navigation behavior.

This thesis research has empirically evaluated several different approaches through which spatial information about the environment can be conveyed through audio. In the first experiment, blindfolded participants standing at an origin in a lab learned the distance and azimuth of target objects that were specified by four audio modes. The first three modes were perceptual interfaces and did not require cognitive mediation on the part of the user. The fourth mode was a non-perceptual mode where object descriptions were given via spatial language using clockface angles. After learning the targets through the four modes, the participants spatially updated the position of the targets and localized them by walking to each of them from two indirect waypoints. The results also indicate hand motion triggered mode to be better than the head motion triggered mode and comparable to auditory snapshot.

In the second experiment, blindfolded participants learned target object arrays with two spatial audio modes and a visual mode. In the first mode, head tracking was enabled, whereas in the second mode hand tracking was enabled. In the third mode, serving as a control, the participants were allowed to learn the targets visually. We again compared spatial updating performance with these modes and found no significant performance differences between modes. These results indicate that we can develop 3D audio interfaces on sensor rich off the shelf smartphone devices, without the need of expensive head tracking hardware.

Finally, a third study, evaluated room layout learning performance by blindfolded participants with an android smartphone. Three perceptual and one non-perceptual mode were tested for cognitive map development. As expected the perceptual interfaces performed significantly better than the non-perceptual language based mode in an allocentric pointing judgment and in overall subjective rating.

In sum, the perceptual interfaces led to better spatial learning performance and higher user ratings. Also there is no significant difference in a cognitive map developed through spatial audio based on tracking user’s head or hand. These results have important implications as they support development of accessible perceptually driven interfaces for smartphones.

Thesis Advisor: Dr. Nicholas A. Giudice

An Abstract of the Thesis Presented in Partial Fulfillment of the Requirements for the Degree of Master of Science (in Spatial Information Science and Engineering) December, 2011

Maps are an important source of spatial knowledge; but most maps are visual in nature and not accessible by visually impaired users. Although there is significant research on accessible maps based on audio and tactile cues, these non-visual maps have various shortcomings. For instance, most of these map displays are non-refreshable displays and require substantial time, cost, and effort to create and update. Available refreshable displays are very expensive which make them cost exclusive for the majority of visually impaired users, especially those living in low-income countries (which represent the largest percentage of the visually impaired). Most of these displays are bulky and cumbersome to carry around, often requiring a fixed setup and desktop computer. To overcome these shortcomings in accessible maps, this thesis research work has designed and tested a novel non-visual interface for conveying indoor spatial information layouts using vibro-tactile and audio cues called “Vibro-Audio Map”. This interface is implemented on highly portable, comparably inexpensive, off-the-shelf smartphones. The thesis describes the conceptualization and development of this new interface, and presents the results of an initial usability experiment to show that the Vibro-Audio Map is equivalent in spatial path learning performance when compared with a hardcopy tactile map conveying the same information. Map panning is inevitable given that most maps are bigger than the available device screen size. Non-visual map panning presents a unique set of challenges which includes both cognitive and interface level challenges. An easy-to-use map panning method called “Button-based-Pan” method was developed as part of this thesis research in order to facilitate non-visual exploration of large maps. This panning method though easy and intuitive, still presents the map in a manner that requires high cognitive demands to process by a non-visual user. One solution to this problem, investigated in this thesis, is to reduce the amount of map panning required using a device by extending its effective screen space. This requirement led to the “Extended-Display” concept where the smartphone becomes the information window (or an information lens) to a virtual map projected on a flat table surface. The Extended-Display system increases the effective accessible search space for large map exploration. The Extended-Display is a generic concept and can be used for many purposes and scenarios including large visual map exploration. This experimental work presents a functional proof-of-concept of an Extended-Display based on an optical target identification system using a Kinect camera, and demonstrates its efficacy in displaying Vibro-Audio Maps. The thesis then presents experimental results, which compare indoor layout learning performance among three interface conditions. These three conditions are: (1) Vibro-Audio Map with pan mode, (2) Vibro-Audio Map with Extended-Display mode, and (3) Hardcopy tactile map with audio information. The results provide clear evidence that the Vibro-Audio Map is equivalent in spatial learning performance when compared against the traditional hardcopy tactile map condition, which is currently the most accepted mode of non-visual map learning. Finally, the thesis suggests future research directions on Vibro-Audio Map and Extended-Display technology.

The efficiency of virtual spatialized audio has been studied for route guidance, but its utility for presenting information that is integral to spatial learning is still unclear. In these two studies, we evaluate the efficacy of virtual spatialized sound for use with auditory displays by comparing the ability of participants to perform basic spatial tasks when auditory targets were presented through headphones (virtual audio) versus speakers (external audio). Our findings show more accurate orientation toward auditory targets, and equivalent recall of auditory target azimuths, using virtual audio as compared to external audio. Therefore, our findings demonstrate that virtual audio is efficient for providing information that assists in the representation and recall of object locations. These results have important implications for the development of auditory displays for navigation systems.

The purpose of this research was to investigate how well individuals learn novel indoor environments by means of verbal descriptions. Seven experiments were conducted addressing the following four questions:

1) Does spatial learning performance differ based on whether a building layout is learned using verbal descriptions or visual input?

2) Is learning performance influenced by the amount of environmental detail available to the navigator?

3) Does learning performance differ for exploration of real vs. virtual environments?

4) Does learning performance differ based on visual status? (Experiments included sighted, blindfolded sighted and blind participants)

The studies incorporated two experimental phases. During the training phase, participants freely explored a layout and were instructed to use the verbal/visual information to learn the complete network of corridors as well as to find four hidden target locations (indicated by an auditory cue). During the testing phase, knowledge of the training environment was evaluated by several measures, including straight-line distance estimation between target locations, map reproduction and route navigation.

These experiments represent the first known work to study whether spatial verbal descriptions support wayfinding behavior in large-scale layouts and to investigate whether learning with a virtual verbal display in simulated environments transfers to accurate navigation performance in the corresponding real environment. The results from these experiments provide clear evidence that verbal descriptions are an effective non-visual mode of environmental access. Performance during the training period was very similar between modalities, environments and participant groups, suggesting that verbal information is sufficient to support accurate learning, wayfinding and cognitive map development across a range of factors. In general, test performance was also quite good, although transfer to real-world navigation after virtual verbal learning was somewhat less accurate than after real-world learning or learning in visually rendered environments. The level of description provided did not reliably affect performance for verbal learning, suggesting that a minimal message describing local geometry is sufficient. In contrast, the same minimal message led to significantly degraded performance in the visual conditions.