From third-person to first-person cartographies with immersive virtual environments

Recent cognitive research indicates that immersive virtual reality (VR) systems can increase the impact of visualization products through the formation of spatial presence, defined as a sense of “being there” in a virtual place. These findings make VR a highly interesting tool for cartography, but challenge the subject’s self-conception in different regards. The present article aims at highlighting the fundamental characteristics of geovisualization with immersive environments. We will approach the challenge of 1:1 representation with a typology borrowed from video game theory, where players can experience games from a first-person or third-person perspective. These two categories provide a useful framework to describe the basic difference between non-/lowand high-immersive geovisualization. In order to project the firstvs. third-person metaphor from a gaming to a cartographic mapping context, we will try to semiotically express the general process of map use in form of a triadic sequence, where the representation mediates between users and geospatial phenomena. Compared with common cartographic products, this mediation process is fundamentally different in VR systems, as immersive applications merge map user and map space. A set of future research questions and further considerations on first-person cartography will close the text. These considerations on firstvs. third-person visualization shall facilitate a conceptually better integration of IVE into current cartographic theory and practice.


Definitions
Virtual reality (VR) is yet an ambiguous concept that has been defined in many different ways (Hruby et al., 2019). Still there are two key notions of VR that seem to be widely accepted within the scientific community: immersion and interactivity (Chalmers, 2017). Both notions can be explained as follows.

Immersion
Immersion can be defined as a technical characteristic of VR systems, i.e. " […] the extent to which the computer displays are capable of delivering an inclusive, extensive, surrounding and vivid illusion of reality" (Slater & Wilbur, 1997, p. 3).
Currently, stereoscopic VR headsets are typical devices to provide this kind of illusion (figure 1).
Immersion is considered to have a strong impact on spatial presence: that is, the sense of really being there in a virtual environment (cf. section 4). In turn, feeling present in VR space can strengthen the user's involvement with the issues mapped (Bailey et al., 2016). This makes immersive VR systems promising communication tools, also for cartographic matters.

Interactivity
In an interactive VR system, user input decides on the elements of the virtual environment to be rendered. Common input devices are handheld controllers, but also head-and body-tracking tools that translate the user's (e.g. head-) movement in physical space into corresponding movement and viewing directions in virtual space.

High-immersive vs. low-immersive VR
Analyzing possible combinations between immersion and/or interactivity, high-and lowimmersive VR systems can be distinguished: A user exploring interactively a realistically modelled VR space with a headset tracked on all six degrees of freedom (DoF) will probably experience high-immersive VR. Low-immersive VR, on the contrary, will be desktop based or lacking interactivity (e.g. in the case of 360° VR video tours), to name just a two examples. Figure 1. VR environment on high-(above: rendered on an Oculus Rift VR headset) and low-immersive (below: same scene rendered on a desktop computer screen) computer displays. VR environment available: https://www.biodiversidad.gob.mx /v_ingles/region/geoviz.html In this paper, we will confine the concept of VR to high-immersive applications, i.e. immersive virtual environments (IVE) explored interactively through a VR headset, where the user can develop a strong feeling of being there in virtual space.

Cartosemiotic considerations
Cartographic communication can be structured along three components: (1) the user, (2) a cartographic model and (3) a real-world phenomenon. These three components can be represented semiotically in a triangle of reference, where maps (in the broadest sense) and map signs mediate between user and geospatial phenomena. Within this triadic model, cartographers (and probably also semioticians) have dedicated most of their attention to semantics, i.e. the relation between map (signs) and the things they stand for. Since Charles S. Peirce's pioneering work, icon, index and symbol have been the most prominent classification of signs that describe this relation depending on whether a sign conveys meaning through similarity (icon), causal connection (index) or convention (symbol). For further readings on map-semiotics also cf. MacEachren (2004) and Schlichtmann (2009).
As already stated by Peirce (1998), "the icon is very perfect in respect to signification, bringing its interpreter face to face with the very character signified". Empirical research in a wide range of disciplines (Haeberling et al., 2008;Papadopoulou, 2009; cf. also Korpi & Ahonen-Rainio, 2015) has strengthened this argument through observations that iconic signs are interpreted and remembered more easily than conventional signs (i.e. symbols). Consequently, iconicity can be considered to be a fundamental quality of map design (Hruby, 2009).
As argued in section 1, IVE generate a strong illusion of reality and facilitate the experience of spatial presence. In semiotic terms they can thus be considered as those signs with the probably highest possibly degree of iconicity (cf. Barricelli et al., 2016).

Scale
Reduction, selection, symbolization and generalization have been principles of mapmaking for centuries. However, these principles seem to fall short in an IVE, where the users perceive a virtual representation of a real place at a level of detail as they would do being physically there.
We propose to label this relationship between virtual and real space a 1:1 scale-level.
1:1 representation has been neither reflected in detail in the realm of cartography, nor is this issue within the narrower scope of the present paper (for further reading cf. Hruby & Ressl, forthcoming). However, it is worth mentioning that the 1:1 scale level of IVE not necessarily will result in nongeneralized representation. For instance, in a VR coral reef (cf. figure 1) the 3D model of a given seaturtle species not necessarily will have to resemble any really existing sea-turtle. Rather, the 3D model needs to replicate the typical features of this species at a level of detail that is high enough to recognize the species and that is, in more general terms, true enough to be accepted as part of a realistic and congruent VR environment, so that spatial presence can be experienced (Wirth et al., 2007).
Theories on human categorization (Lakoff, 1987;Rosch, 1983) or selective realism (Pötzsch, 2017) are promising candidates to explore relevant degrees of abstraction in IVE against the criteria of spatial presence.

From third-person to first-person cartography
In section 2 we argued that iconicity and 1:1 scale are two basic characteristics of immersive IVE following from the technological premises we discussed in section 1. However, we feel that these two attributes describe the distinctive features of IVE (compared with low-and non-immersive geovisualization products) just partially.

By way of example
The following exemplary scenario A (taken from Laakso & Tiina Sarjakoski, 2010) shall explain our concerns. In a digital, interactive 2D hiking map, topographic features are visualized both through graphical signs and a series of play-button symbols, giving access to georeferenced soundscapes recorded on site, e.g., singing birds in a forest.
At first view, scenario A does not at all seem to represent a VR product, but rather a common desktop-based, interactive (web-)mapping application. However, regarding the aural dimension, the map provides iconic 1:1 sound features, i.e. the basic characteristics of IVE analyzed in section 2. For instance, clicking on the accordant button you can hear birdsongs as you would do being really there.
Hence, we need to describe IVE in further detail in order to distill a defining criterion that applies not just for visual, but also for multimedia cartographic VR products.
To derive such a criterion, we can translate, in a first step, scenario A into a VR-scenario B: In such an IVE, users would experience not just sound but also topographic features tridimensionally at 1:1 scale. Navigating through the virtual environment, they would perceive land-and soundscapes within visual and hearing range.

"Everything-from-nowhere" vs. "Something-from-everywhere
Two comparative scenarios A and B at hand, we can now ask, how each scenario could be interpreted.
In scenario A, users perceive a clear difference between map space and their own position in physical space. This difference is typically being expressed by spatial scales of 1:x (with x > 1), where viewers have to bring themselves into a relationship with the information displayed, a process probably best illustrated by so-called youare-here (YAH) maps (Montello, 2010). Separating user from map space is the sine qua non for almost all important map use scenarios, e.g. orientation, navigation, analysis or the interpretation of spatial patterns.
In scenario B, users experience just little difference between VR map space and their own position in physical space. Rather, virtual space is perceived at a high iconicity and a scale level of 1:1, where the relationship with the information displayed is predetermined in terms of immersion and spatial presence.
This being said it becomes obvious, that scenario A and B complement each other: In scenario A, the viewer takes what some authors have called a divine perspective, allowing to observe "everything from nowhere". However, the price of this perspective is accuracy: Neither the satellite image with the highest resolution nor the most detailed map of, for instance, a mangrove forest can tell you, how it looks, sounds and smells like.
By contrast, IVE can: Changing the "divine" perspective for a "human" one, users can perceive virtual representations of real places as they would do being physically there. This means, they will experience virtual space within the constraints of human perception, but in high detail allowing them to experience, how geographical environments look, sound and smell like. Rather than seeing "everything from nowhere", they can experience "something from everywhere".

First-vs. Third-person cartographies
In the previous sections, we argued that the fundamental distinctive feature between non-/lowimmersive and high-immersive geovisualization is the relationship between map space and the user's position in physical space. While both are clearly separated in non-immersive applications, they (partially or completely) coincide in high-immersive products. 1:1 scale and iconicity are characteristics we can deduce from this coincidence.
While non-vs. high-immersive visualization has not been high priority in mainstream cartography so far, other disciplines discussed this issue already in further detail. We propose, thus, to adopt technical terms from these disciplines in order to distinguish non-/low-immersive third-person cartographies from high-immersive first-person cartographies.
During the last decades, computer game theory and design have coined the concept of first-person vs. third-person perspective significantly, as player point of view (POV) is a fundamental decision in every game designing process: "First person POV allows the player to perceive the game through the eyes of the character, observing the world around them up close, giving a clear view of the scenery in front of them. This perspective is believed to provide the most immersive feel for the player. Alternatively, a third person POV allows the player to observe the main character in action, without giving the player the sense that they actually are the character." (Denisova & Cairns, 2015, p. 145) Game developers are not to only ones to make use of the dichotomy between first-and third-person perspective. Researchers interested in communication issues from a wide range of disciplines have paid attention to the consequences of first-person vs. third-person perception. Applying a compatible vocabulary also for geovisualization matters could facilitate crosslinks with this field of multidisciplinary research. Possible intersections shall be suggested in section 5 by way of example.
To sum up, we can define the core concepts discussed in this section as follows:

Spatial Presence
Throughout this text, we have mentioned the idea of a spatial presence already several times. Since spatial presence is a central argument regarding the consideration of first-person visualization for cartographic concerns, some key aspects of this concepts shall be discussed subsequently.

By way of argument
Research on highly immersive virtual environments is built on the following logical argument: (1) IVE facilitate the formation of spatial presence.
The generation of understanding and insight from geospatial data through representation has always been a main purpose of cartography, so that the relevance of the conclusion (3) requires no further justification. However, this conclusion will be only as valid as the premises are. Let us analyze these premises individually.

Spatial presence through IVE
Spatial presence is the user's feeling of being there and interacting in a mediated spatial environment (Hartmann et al., 2015). The two dimensions of spatial presence (i.e. self-location and action possibilities) were formalized by Wirth et al. (2007) through a two-step model: As a function of both medium-related (e.g. spatial cues, consistency) and user-related (interest, spatial ability) factors, users can construct, in a first step, a spatial mental model of the environment provided. If this mental model is convincing enough, the users will prefer, in a second step, the mediated environment over their physical environment as the primary frame of spatial reference. That is, step 1 decides on whether the mediated environment represents a plausible space, while step 2 decides on whether the user accepts being actually situated in this place.
Although spatial presence can even result from low-immersive media types (e.g. reading a book; Schubert & Crusius, 2002), a strong body of research indicates an outstanding impact of IVE on the formation of spatial presence (cf. Cummings & Bailenson, 2016).

Understanding through presence
Spatial presence in IVE is not just an interesting topic of research on its own right. It has been also argued that effectiveness of IVE (e.g. as learning tools) increases when user experience spatial presence. Ahn et al. (2016), for instance, showed in a series of experiments how users of VR-systems produce higher levels of concern and engagement with the phenomena being visualized, compared to when the subjects simply watch a video on a common computer screen.
To measure the interconnectedness between (the user's) self and (virtually represented) nature, Schultz´s (2001), inclusion of Nature in Self (INS) scales proved suitable (Ahn et al. 2016).

Conclusion and Outlook
In this paper we argued that providing the user with a first-person perspective is the fundamental characteristics of geovisualization with IVE. Applying immersive technology to geovisualization leads to first-person cartographies, where map user and map space coincide so that users perceive the mapped environment as they would do being physically there. 1:1 scale and iconicity are further qualities of IVE that can be deduced from a firstperson experience.
First-person cartographies raise several questions of both practical and theoretical nature: Starting from an epistemological point of view one might wonder whether virtual objects are still signs (that represent a real object) or already real objects on their own, so that insight gained from IVE is equivalent with real world experience (Dilworth, 2010;Clark & Chalmers, 1998).
Another interesting aspect of first-person cartography is embodiment, where users can experience IVE from the perspective of a human or non-human avatar (Ahn et al., 2016). This allows map users to perceive space not just from different spatial positions but also from different social positions (Groom et al., 2009).
First-person is not just a mode of gaming but of storytelling as well (Ruthrof, 2016). Due to the growing interest in the relationship between geovisualization and narratives (Caquard, 2010) IVE provide a new framework for geospatial storytelling from a first-rather than third-person perspective, e.g. in terms of an immersive journalism (De la Peña et al., 2019) Compatibility between current geographic information systems (GIS) and VR output devices is still limited. Game engines are a typical middleware to make GIS data available within GeoIVE (Edler et al. 2018;Hruby et al., 2019). From software designed for game development to gamified geovisualization IVE it is only a small step. Hence, IVE could also provide a link between cartography and gamification (Freina & Canessa, 2015) approaches.
From a more genuine cartographic viewpoint, it will be challenging to mirror traditional principles of mapmaking (e.g. selection, symbolization and generalization) against highly-iconic visualizations at a 1:1 scale level. As argued in section 2, these principles not necessarily become invalid in IVE, but may require reformulation and interdisciplinary reflection.
Current geospatial research is not just challenged by causal connections in time and space, but also by psychological distance (Trope & Liberman,2010), impeding for instance an effective communication of past and future scenarios like climate change. As stated elsewhere, IVE could "[…] help to reduce distance among laymen, academia, and the geospatial challenges of the twenty-first century." (Hruby et al., 2019).
Finally, also ethical concerns need to be considered regarding the usage of IVE (cf. Brey 2014).

Final Note
Due to the format of publication chosen for this paper, we approached first-person cartographies only in a non-immersive manner. In addition to the bibliography below, interested readers are referred to the following IVE application mapping a coral reef in the Mexican Caribbean: http://www.biodiversidad.gob.mx/region/descarga s/L_2017jl25.7z. This application may serve not only as an example with which to experience the feeling of "being there" but also as a test material for studies on spatial presence with GeoIVE. Updated versions of this application are being published continuously via: https://www.biodiversidad.gob.mx/v_ingles/ region/geoviz.html