An Archaeology of Perception in the Metaverse: Seeing a World Within a World Through the Artist’s Eye

Abstract

This chapter proposes a 3D Virtual Reality (VR) metaverse in the shape of a world within a world, to be used for an experiential archaeological approach, to facilitate understanding the sensoriality of past individuals. Virtual reconstructions of the archaeological remains have so far failed to recreate a realistic experiential environment. For this purpose, we selected as study site a house of artists from the past (1930) from the town of Balchik, around which we built the proposed metaverse, with the objects of study being the house and artistic work of the painter Cecilia Cuţescu-Stork. The metaverse is designed to provide rich sensory experiences in order to allow a better understanding of the artworks and the context in which they were made. The VR metaverse is comprised two connected 3D virtual spaces that users can traverse, including with VR headsets, each space contributing to a cognitive augmentation through the sensory qualities of the constructed virtual space. The metaverse reconstruction will allow access to both specialists and the public, as the metaverse is implemented on a free VR platform (Mozilla Hubs).

1 Introduction. Archaeology and Metaverse

This chapter discusses the metaverse’s ability to merge different types of knowledge, from the perception of space in a world with a realistic mirror reconstruction of the one that existed previously, to a visual artist’s mind. The chapter is therefore a work of archaeology in the metaverse of an artist’s mind, presented as an augmentation of the reality of the architectural context in which she lived and created.

Archaeology is not only a material recovery of the past, but also an attempt to approach the humanity of past people. Cognitive archaeology tries to understand how the mind works when people deal with material culture, and experimental archaeology tries to identify the chaînes-opératoires within the process of the making of objects. But both subdisciplines fail to approach “the sensory dimensions of human activities”, this being now the role of phenomenological archaeology that “describes the embodied experience of archaeologists” (Skeates 2010, p. 6), i.e. their experientiality. The study of experientiality in archaeology can therefore approach the human sensorium, from the embodied visual perception (Eve 2014) to archaeoacoustics (Scarre and Lawson 2006).

Historical heritage and archaeological research (Sequeira and Morgado 2013; Gheorghiu and Ştefan 2019) have accepted in recent decades Virtual Reality as a working tool, the reconstructions made in this environment being “an impressive, immersive, interactive and involving product”, which offers a “sense of personal presence of the user in its environment”, as stated by the European Commission report (2001) on “New technologies for Cultural and Scientific Heritage” (see also Monod and Klein 2005). This type of approach allows for both experimental and experiential archaeological research. Metaverse communities have also been investigated as methods of research, documentation, and opening up to the public (Zhang et al. 2022). Research in this area focused on “reconstruction of the archaeological sites in the Roman period” (Forte et al. 2010), “virtual musealization of a Roman city” (Forte et al. 2010), cyber-archaeology (Forte and Kurilo 2010), Virtual Environment in architecture education (Gaafar 2021), “developing temporary and permanent exhibitions in cultural heritage through “phygital”, blended, and cross-reality (XR) experiences” (Borsotti 2022), “opportunities and challenges for museums, in pursuing their edutainment goals, as it allows visitors to fully experience artworks and archaeological artifacts” (Longo and Faraci 2023), or “Archaeological Simulation as a collective virtual shared space” (Silva 2022). Metaverses for archaeology or cultural heritage are still at an early stage, being used especially for visual facilities, which allow archaeological research and interpretation.

Niccolucci et al. (2022) propose a “Heritage Digital Twin (HDT) of a Heritage Asset (HA)” as a “digital representation of the complex of knowledge about such heritage asset, organised according to a specific ontology”.

In an attempt to reveal to the public, the sensoriality of past individuals, we proposed an archaeological experiential approach made in a 3D Virtual Reality (VR) metaverse, due to the facilities of virtual worlds to highlight a part of human sensoriality, such as visual and auditory reception. For this purpose, we chose as study site a villa from the early-twentieth century built for a family of artists in the town of Balchik on the Black Sea coast. There we focused on the study in VR of two different worlds: the first a historically correct reconstruction of the architecture of a villa built in the early-twentieth century, and the second, the mental world of a visual artist. These two virtual worlds, respectively, a 3D reconstruction of the loggia space of Villa Storck, representing a mirror world of the real one and a subjective visualization of the perceptual space of the owner of the house, painter Cecilia Cuţescu-Storck were interconnected.

Part of the artist’s sensory world thus became an augmentation of the “real” world of architecture, helping the visitor to access [a fragment of] the artist's cultural and perceptual context. The visitor’s experience in both worlds has therefore been augmented by combining visual and auditory experience.

By creating a metaverse called “World-within-World”, this chapter attempts to detail the following topics: (a) what metaverse characteristics would fit an experiential, i.e. subjective/sensory approach; (b) how an artist perceived the world and art in a particular cultural context (in this case the Storck villa); (c) the concepts and technologies to be used to implement a metaverse that includes both a digital world, represented as a mirror image of the real one, and a world that digitally represents a palimpsest of the artist’s thought and creation; (d) an evaluation of the experiment from an archaeological, artistic, and educational point of view; (e) a discussion of the limitations of this phenomenological approach in the metaverse space in comparison with what could be achieved similarly in the real world, as well as in relation to traditional 3D virtual worlds; (f) research conclusions.

2 Background: Place, Artist, House, and Artistic Imagination

2.1 The Place

We will start with the presentation of Balchik, which was the place of a crucial artistic phenomenon in Romanian visual art between the two world wars (Bucuţa 1931) (Fig. 10.1).

Fig. 10.1

Palace of Queen Mary of Romania, an architectural synthesis of the old village of Balchik. Photo by D. Gheorghiu

Balchik in the early-twentieth century was a Turkish-Bulgarian village (Murgoci 1913), built on the ruins of a Greek colony (Tafrali 1928), on the Black Sea coast (Fig. 10.2).

Fig. 10.2

Street in Balchik. Photo by M. Moţăianu

In the 1920s Balchik became the favourite gathering place of artists who were attracted by the location’s strong oriental exoticism, creating the so-called “Balchik Art School” (Păuleanu 2007), to which the visual artist Cecilia Cuţescu-Storck adhered from the very beginning (Fig. 10.3).

Fig. 10.3

Two paintings of Balchik by Cecilia Cuţescu-Storck in the house-museum in Bucharest. Photo by C. Moţăianu

2.2 The Artist

Cecilia Cuţescu-Storck (1879–1969) was a famous painter (Thevenin 1932), educated in Munich and Paris who created decorative arts and monumental murals. In 1916, she became the first female professor at a state art university in Europe. Her art was influenced by the symbolist art of the late-nineteenth century, but also by modernity, working in different techniques, from fresco to oil on canvas and pastel (Fig. 10.4).

Fig. 10.4

Paintings and decorative panels by Cecilia Cuţescu-Storck in the house-museum in Bucharest. Photo by C. Moţăianu

2.3 The House

In the 1920s, the artist built a villa in Florentine style, which she used as a creative workshop, a building that still exists today in an advanced state of ruin, but which has begun to undergo restoration (Fig. 10.5).

Fig. 10.5

Villa Storck in Balchik. Photo by D. Gheorghiu

Both the ruined villa, the stone terraces facing the sea (Fig. 10.6), and [part of] Cecilia Cuţescu-Storck’s work, still exist today, but they are not perceived as a whole, or in their true splendour. This was the motivation of the present work, which aims to reveal to the virtual visitor an image of the real space of the villa from which one can escape to another space, that of the artist’s mind, in which to imagine her sensory and emotional experiences (see also Cuţescu-Storck 1943) in relation to her work and to the city of Balchik.

Fig. 10.6

Stone garden and stairs, Villa Storck in Balchik. Photo by D. Gheorghiu

This manner of presenting the related real and imaginary, in the form of “worlds-within-worlds” (Smith 2019, p. 198), allows one to experience or interact with other visitors in the two different virtual worlds in the metaverse. Also, these participatory capabilities allow access to cultural memory in a very efficient manner, which is the ultimate educational goal of this experiment.

2.4 The Artist’s Mind

In order to create the space to visualize the elements that were the basis of the creation, i.e. the artist’s “mind”, the idea of “mind–body interaction in virtual reality” (Chalmers 2022, p. 263) was started. Movement in the VR space of the loggia of the villa Storck is one in which the experience of space is closer to that of reality (this experience in a digital representation “in the mirror” being an exercise of phenomenological archaeology of the built space), while movement in the space of the mind is an imaginary movement in a surreal space, representing a subjective, experiential action that does not respect the laws of physics and normal perception.

Therefore, in the second case, some elements of the artist's life experience were visualized in the form of “realistic” memories (for example, the narrowness of the streets of Balchik, or some sources of inspiration such as Paul Gauguin’s paintings, or her important paintings), which were mixed with other data collected from her work, from the house-museum in Bucharest and from the experientiality of the authors in contact with the artist’s work and with the city of Balchik.

All this space is like a three-dimensional mental map in which geometric spaces in the real world and in the “mirrored” virtual world have been replaced by spaces formed by occasionally palimpsest-like overlapping plans, containing different information about personal artistic production, works of artists who influenced the painter’s work, texts from her memoirs, or landscapes from Balchik.

In creating this “space of mind” of the artist, the following ideas were taken into account: the classical concepts of representational properties of the mind (e.g. Aristotle De Anima 1984), namely that “all representational states have their content in virtue of their sensory phenomenal features” (Pitt 2020), the theory of pictorial representations or images (Kosslyn 1980), and the discursive theory (Pylyshyn 2003), the last two being combined, since they are allowed by the affordances of cyberspace.

The discursive affordance of the cyberspace urged the authors to metaphorically use¹ the idea of a road, of a way to go, along which to position the most representative pictorial representations of the artist's work. Therefore, amodal symbols such as texts from Cecilia Cuţescu-Strock’s opera were put together with modal symbols, or in Barsalou’s (1999). Barsalou et al. (2003) terminology perceptual symbols (see also Shapiro and Spaulding 2021), in the form of iconic images positioned on a multitude of planes, similar to paintings or drawings of the artist, in some cases overlapping planes increasing the meaning of each of the two overlapping images. The overlapping and juxtaposition of these images are not a collage but an augmentation of meaning.

Furthermore, augmentations of textures or colours of various materials were made. The sounds introduced were the natural ones of the marine context and melodies of the turn of the century and their role was to augment the feeling of an outworldly reality.

The result of these operations of coherently mixing important theories about mental representation produced a digital 3D mental map that could be visualized and experienced in virtual space.

At the same time, this result represents a digital artwork, which when visited generates a multisensory experience. An advantage of this structuring of the virtual space is that when experimenting it as an educational space, it will also generate an embodied imagination (Joy and Sherry 2003). For example, in the two virtual spaces, one can hold art history courses and organize exhibitions.

3 Why the Metaverse? The Choice of Environments and the Experiment in the Metaverse

The virtual spaces described form two virtual worlds that have been populated with various stimuli identified as specific to the context studied and allowing sensory immersion. The metaverse is the perfect environment to host this kind of experiment.

3.1 Benefits of the Metaverse

Why use a metaverse? Because it allows us to build a series of virtual worlds, connected to each other, that users can traverse, including with VR headsets, each space contributing to cognitive augmentation through sensory qualities of the constructed virtual space. The metaverse allows us to populate it with both archaeological, scientific, and subject/artistic reconstructions, which function as visual metaphors, which trigger the viewer’s imagination and allow the augmentation of information, such as access to the mind of the artist who lived in the house, Cecilia Cuţescu-Storck.

4 Metaverse: State-of-the-Art

4.1 Definitions of Metaverse

The idea of metaverse is not new or recent, it appeared with the first online VR platforms, such as SecondLife or OpenSim, or with games like Minecraft, and evolved as a desideratum to create a universe of virtual worlds, allowing users to travel from one virtual world to another.

In recent years, the advancement of 3D, VR, AR technologies, including devices, as well as blockchain technologies, has opened a new stage for the realization of the metaverse. Although there are several platforms for creating metaverses and virtual worlds, they are still unconnected, or only used by specific user groups, i.e. gamers, social media, or blockchain users.

In the ideal definition, the metaverse represents a universe of all 3D virtual worlds that models the real world, in a more realistic way, so that objects, goods, characters, and human activities are transferred from the real world to the virtual one. Currently, the metaverse, in its entirety, has not been realized, with only partial “metaverses” implemented in certain areas, especially financial, industrial, and with a rapid evolution towards social, cultural, and educational fields.

The term metaverse was first used by Neal Stephenson in his novel Snow Crash and described as a “computer-generated universe […] this imaginary place is known as Metaverse” (Stephenson 2011, p. 22). Technically Stephenson’s Metaverse is a “computer-graphics protocol […] pieces of software, made available to the public over the worldwide fiber-optics network” (Stephenson 2011, p. 23), in other words, which represents a “virtual reality-based internet” (Papagiannidis and Bourlakis 2010).

Currently, there is still no generally agreed definition and the existing ones refer mainly to technological aspects (Dwivedi et al. 2022) or user perception and interaction (Park and Kim 2022; Cantaluppi and Ceccon 2022).

The technical definition given by Ball (2022, p. 53) describes the metaverse as “a massively scaled and interoperable network of real-time rendered 3D virtual worlds and environments which can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments”. Also, Stackpole (2022, p. 146) offers a technical perspective of the metaverse in its ideal and complete form: “many metaverses, or digital spaces, which typically are decentralized, incorporate augmented and virtual reality, store information on blockchain, and allow users to own digital goods”.

The metaverse, unlike traditional virtual spaces, focuses on shared virtual spaces and user interaction (Forte et al. 2010; Cantaluppi and Ceccon 2022) in the form of “parallel, virtual universe that uses ambient intelligence to enhance physical spaces, products and services” (Buhalis and Karatay 2022, p. 16). Dwivedi et al. (2022, p. 30) state that the metaverse “is not the same as current immersive technologies […] is a network of always-on three-dimensional virtual environments […] and by using the metaphor of the real world but without its physical limitations”. Papagiannidis and Bourlakis (2010, p. 4) consider that metaverses and the activities within them are not isolated from the rest of the electronic space or the real world itself”.

In another sense, the metaverse is associated with blockchain-based metaverse platforms and defined as “[…] one of a new generation of virtual world, popularly referred to as metaverses, built on the blockchain” (Dowling 2021, p. 1), but these platforms target uses in areas where advanced implementations of identity and ownership are required, such as cryptocurrency-based transactions. These platforms are used “as a target”, for various implementations, e.g. Decentraland, Illuvium, DeFi, or The Sandbox (Dwivedi et al. 2022).

In conclusion, a complete explanation of what the Metaverse could be, as described by Stackpole (2022, p. 147), citing van Rijmenam (2022), is that it will allow users to move directly from one virtual world to another, “to easily move communities and digital goods from platform to platform […] take a Facebook group to Roblox, and then transfer a piece of art made there over to Fortnite”.

4.2 History and Concepts of the Metaverse

Historically, there has been a quest to create virtual worlds. Schell (2008, p. 301) mentions the term transmedia coined by Henry Jenkins inspired by the Star Wars movie, to refer to a fantasy world that can be entered through many different media—video, animation, games, or other devices. The search for immersion, 3D visualizations, and experiences related or parallel to the real world has a long history, facilitated by the development of optical devices such as “magic lanterns, stereoscopic, photography, panoramas, maréoramas”, so the metaverse constitutes a continuation of this research (Quintana et al. 2022).

Concepts of the Metaverse such as infrastructure and usage are found in Neal Stephenson’s novel “Snow Crash” (1992). Later, in 2011, following the release of the movie “Ready, Player One” based on the novel of the same name by Ernest Cline (Terry and Keeney 2022), a fictional metaverse called “Oasis” is described, as a universe containing different virtual worlds and accessible only with VR devices. Both novels, although published at a great distance from the current evolution and explosion of virtual worlds, also draw attention to some negative effects of the metaverse.

Social virtual worlds based on Second Life, launched in 2003, Roblox, a gaming platform, in 2006, Minecraft in 2011, VRChat (VRChat 2023), launched in 2014, Fortnite an online game developed by Epic Games in 2017 are referred to as precursors of the metaverse, illustrating the “potential socio-economic impact of a fully functional persistent cross platform metaverse” (Dwivedi et al. 2022; Gent 2022). Nevertheless, these platforms proved limited in functionality and interconnectivity with other platforms (Dwivedi et al. 2022; Ştefan and Moţăianu 2022).

In 2015, development began on Decentraland (2023), a blockchain-based metaverse (Terry and Keeney 2022), surpassing social platforms, e.g. Facebook/Meta and video games. A notable implementation that uses AR is the mobile game Pokémon GO, launched in 2016, in which AR overlapped a virtual world with the real one, respectively, virtual Pokémons that can be hunted with smartphones.

In 2018, Mozilla launched Mozilla Hub, a virtual collaborative multi-user MR and social platform (Mozilla 2023).

In 2019, the Facebook Company launched Horizon Worlds and in 2021 has changed its name to Meta, to dedicate itself to the concept of metaverse as a social platform (Meta 2023). Launched as a free Virtual Reality, online video game platform, it is described by Mark Zuckerberg as “an integrated immersive ecosystem where the barriers between the virtual and real worlds are seamless to users, allowing the use of avatars and holograms to work, interact and socialise via simulated shared experiences” (Meta 2023). In 2021, Microsoft launched the Mesh platform (Mesh 2023) that enables the creation of avatars for the Teams collaboration and communication platform.

In manufacturing and automotive, Nvidia’s Omniverse cloud services platform enables digital twins, 3D avatars, and 3D scenes in OpenUSD standardized format for virtual worlds (Omniverse 2023).

Digital twins are used to populate the virtual worlds of the metaverse. These are real-time 3D simulations of real objects and environments that have properties and display behaviour similar to those in the real world. The notion appeared in the industrial, automotive, healthcare, manufacturing, and logistics environments (Liu et al. 2021) but was also taken up in the cultural environment for making digital replicas of cultural artefacts “as the real-time digital counterpart of a physical object” (Niccolucci et al. 2022). Shared, these digital twins “would ensure a biunivocal link between virtual and real CH items” (Cantaluppi and Ceccon 2022, p. 1).

An important concept related to the metaverse is the avatar or 3D hologram (Dwivedi et al. 2022), “the user’s alter ego” (Park and Kim 2022), which acts within the metaverse through XR, in the sense of connecting the avatar with the real world (Park and Kim 2022).

Extended Reality (XR) is an umbrella term for AR, VR, and MR, which introduces a continuum from VR, AR to MR, with the benefit that users “will no longer be able to distinguish between virtual elements and real objects and will use them both interchangeably” (Dwivedi et al. 2022). VR allows a telepresence continuum to be established between “atomistic virtual reality” and “holistic virtual reality”, while AR allows a continuum between “assisted reality” and “mixed reality” (Dwivedi et al. 2022).

The modern, future concept of the metaverse is related to a new generation of Internet, called Web 3.0 by Ethereum (Ethereum 2023). This represents in Ethereum’s vision a decentralized network, based on blockchain technologies and principles, payments with crypto-currencies, and the use of non-fungible tokens (NFTs) for access, security, and ownership.

The COVID-19 pandemic has contributed to the expansion and use of immersive 3D virtual worlds VR, AR, and metaverse communities for educational purposes (Gaafar 2021; LaJevic 2021; Ştefan et al. 2022a, 2022b), cultural (Gheorghiu and Ştefan 2019; Gheorghiu et al. 2021; Moţăianu 2021; Longo and Faraci 2023; Kunstmatrix 2023), entertainment (Minecraft 2023; Fortnite 2023).

4.3 Metaverse Tools and Implementations

The technologies underlying the current metaverse concept are: blockchain decentralized platforms and tools, such as crypto-currencies, digital assets, digital identities, smart contracts or non-fungible tokens (NFTs); AR/VR technologies under the term of Extended Reality (XR); Machine Learning (ML), and Artificial Intelligence (AI); real-time 3D simulations technologies; cloud computing; Internet of Things (IoT) such as devices and real-time communication protocols. The sense of immersion is provided by head-mounted displays (HMD) and other physical devices (Park and Kim 2022).

Mozilla Hubs is an online virtual multi-platform, i.e. allowing access from different devices, including VR headsets. Mozilla Hubs is customizable for different purposes, and at the current stage of development, it is a free platform. Mozilla Hubs provides pre-defined 3D scenes for creating virtual spaces called “rooms” within its virtual spatial “hub”, which can be accessed individually via a link and can also be interconnected within the hub, via links, to facilitate the navigation (Ştefan and Moţăianu 2022).

Meta Live Studio (2023) is the metaverse platform from Meta that allows user interactions, social activities, or the hosting of events within dedicated virtual worlds, e.g. Venues. It can be used with the Oculus Meta Quest 2 VR headset, made in collaboration with Oculus.

Ethereum (2023) is a public open-source blockchain launched in 2015, which provides smart contract and decentralized applications’ (DApps) functionalities (Holotescu et al. 2018), and uses ether as the platform’s cryptocurrency.

To implement the current metaverse, service platforms or development tools are used (Park and Kim 2022; Terry and Keeney 2022). The most popular metaverse platforms are generally dedicated to online gaming, social life, remote workforce collaboration, virtual communities such as Roblox, Meta’s Horizon Worlds, Microsoft Mesh or Mozilla Hubs; virtual real estate business, such as Decentraland; business and financial operations, such as Blockchain as a Service (BaaS) or Ethereum Blockchain as a Service (Park and Kim 2022; Dwivedi et al. 2022).

Terry and Keeney (2022, pp. 91–117) propose three categories of metaverse: (a) Sandbox; (b) Gaming; (c) Miscellaneous or custom metaverses. For custom metaverse implementations as VR or AR environments, the most commonly used tools are Unity (Unity 2023), a cross-platform game engine developed by Unity Technologies, and Unreal Engine (Unreal 2023), a series of 3D computer graphics game engines developed by Epic Games. The Unity game engine was used in Pokémon GO the mobile game and in numerous other games, AR/VR applications, and non-game applications, while Unreal Engine was used for the development of Fortnite, a very successful online game (Bradshaw and Kruppa 2020). A feature of these tools is their interoperability and the fact that they can be deployed on a multitude of devices, e.g. smartphones, PC, VR headsets, as a “translation layer for the metaverse”.

A secondary class of tools is represented by software tools for 3D design, with complementary features, such as Blender, AutoCAD, AutoDesk, Cinema4D, or ZBrush.

Mixed reality (MR) involves the use of specialized hardware devices, such as Google Glass, Oculus Meta Quest 2, Microsoft HoloLens, MagicLeap, to “integrate realistic-looking 3D content into the user’s physical environment” (Rauschnabel 2021).

For our research, we chose the Mozilla Hubs platform, which can be used to implement the World-within-World metaverse concept and explore it within an online community with diverse experiences, including embodied VR.

5 Interacting with the Metaverse

An important component of 3D virtual environments and consequently of the metaverse is VR headsets, which facilitate their exploration, offering immersive experiences and advanced ways of interacting with the environment. VR headsets are not dedicated solely to gaming, but increasingly have uses in other fields, including education, science, and medicine.

The use of VR headsets is closely related to essential concepts of 3D virtual environments, namely immersion and presence. VR immersion refers to the objective degree to which a VR environment realistically presents a real-world setting. Presence, on the other hand, is the subjective sensation of being “mentally transported” into the VR world.

Research by Smith and Mulligan (2021) highlights the distinction between immersion and presence, noting that their effects on memory and experience can vary. They found that various aspects of immersion, such as visual field, audio-visual elements, and light realism, can influence memory and presence in various ways, without a clear mediating effect of presence on memory performance (Smith and Mulligan 2021).

5.1 Headset Types and Usage Modes

VR headsets and paired controllers offer various modes of interaction that bridge the digital and physical worlds.

VR headsets can be classified into three types: connected, standalone, and smartphone based. Connected headphones, such as the Oculus Rift and HTC Vive, deliver high-quality images and require a connection to a powerful computer. Standalone headsets such as the Oculus Quest provide a wireless experience with all the necessary components built into the headset. Smartphone-based headsets such as the Samsung Gear VR use a smartphone as a display and processing unit.

VR headsets work by displaying two slightly different angles of the same scene to each eye, creating a stereoscopic 3D image. This illusion of depth, combined with head-tracking technology, allows users to feel as if they are truly immersed in the 3D digital world. Recent advances in VR technology have introduced systems for tracking motion in three dimensions, enhancing the realism of the experience (Rosi et al. 2021).

Interaction in VR environments is facilitated by controllers, which range in complexity from simple button-based devices to advanced controllers with motion tracking and haptic feedback. Livatino et al. (2022) compare head and joystick rotational movements in VR, highlighting how different control mechanisms can affect user immersion and comfort.

Moreover, emerging technologies integrate emotion recognition into VR experiences. Suhaimi et al. (2022) explore the use of EEG headsets in conjunction with VR to recognize and respond to users’ emotional states, paving the way for more intuitive and responsive VR environments.

The mode of interaction in VR can range from passive observation to whole-body involvement. Some systems use manual controller for navigation and interaction, while others use full-body motion recognition for a more immersive experience. Kritikos et al. (2020) demonstrate that whole-body motion recognition cameras in VR therapy for acrophobia provide a more realistic and immersive experience compared to traditional hand controllers.

5.2 Methods of Locomotion

Teleportation is a common method by which users move instantaneously, i.e. with a click, from one virtual world to another, or to a location inside the virtual world in which they are a part. This method helps reduce motion sickness but can interrupt immersion (Boletsis 2017). Walking-In-Place simulates walking by detecting users’ physical movements. It is more immersive but can be physically demanding and require larger physical spaces (Ruddle and Lessels 2009). The movement of the joystick/controller allows movement in virtual space. It is easy to use, but it can cause motion sickness in some users.

5.3 User Interface (UI) and Interaction Design

The VR UI, including menus and maps, must be intuitive and easily accessible, allowing users to navigate without interrupting immersion. Interaction design should facilitate natural and seamless navigation, such as gesture commands or gaze-based navigation (Bowman et al. 2007). VR environments should be designed to promote easy navigation and spatial awareness, with spatial cues and cues, such as lighting and sound that can guide users and improve their sense of direction (Riecke and McNamara 2007).

5.4 Real-World Space Constraints

Navigation techniques must take into account the physical space available to the user, adapting virtual movements to avoid real-world obstacles (Cirio et al. 2011).

6 The “World-Within-World” Metaverse

For the research presented in this chapter, the authors selected Mozilla Hubs as an online 3D platform, for its capabilities to be a hub of several 3D virtual spaces in VR, including by using VR headsets, to interconnect the spaces (rooms in Mozilla Hub’s terminology) and also to its collaboration and social communication features.

6.1 Design and Implementation

The “World-within-World” Metaverse was designed to allow users to immerse and explore two interconnected virtual worlds. The first one is a realistic 3D reconstruction representing the loggia space of Storck villa and is named “Villa Storck”. The second one is a D reconstruction representing a visualization of the perceptual space of the owner’s house, i.e. the painter Cecilia Cuţescu-Storck, and is named “Cecilia Storck”.

The two worlds are closely interconnected, a user being able to teleport (“jump”) from one world to another, just by one click from defined hotspots, without perceiving any discontinuities.

The 3D virtual spaces were designed with Mozilla Spoke (Spoke 2023), an online editor for 3D scenes, using photos from Villa Storck and its surroundings in Balchik, images from frescoes and paintings, objects imported from Sketchfab (Sketchfab 2023), free drawing tools and audio files, to create the inner atmosphere. Two types of ambient light were used, i.e. directional and spotlights.

In addition to performing 3D modelling, Mozilla Spoke was also used for configuring the physical characteristics of the virtual spaces, i.e. settings for users’ orientation, movement, teleportation, and interaction within the 3D spaces.

For the implementation of the teleportation from a virtual world to another, a specialized component from the Spoke editor was used, i.e. the “spawn point”, which indicates the exact placement of the user in the virtual space where he/she teleports. The teleporting “door” is indicated by images, in our case, a fresco and painting, on whose properties were set the web link of the virtual world to teleport to.

The authors also created teleporting points inside the second virtual space “Cecilia Storck”, as this space is more difficult to be navigated. Specialized Spoke components were used, i.e. “link”, for users’ fast movement and guidance to locations inside this space.

Finally, in order to be used as a virtual space within Mozilla Hubs, the 3D spaces were published, i.e. exported, on the Mozilla Hubs platform, which resulted in the generation of two web links.

For a user to enter the “World-within-World” Metaverse, it has to use the web link corresponding to “Villa Storck” virtual world: https://hubs.mozilla.com/ZccyPMm/world-within-world-villa-storck/. The second web link corresponding to “Cecilia Storck” virtual world, https://hubs.mozilla.com/LHxRkjB/world-within-world-cecilia-storck, will be necessary only internally, for the teleportation.

6.2 Explanation of the Navigation Within the Loggia Space

The visitor initially accesses Villa Storck through the loggia of the building, a space that was used by the painter to house a series of large frescoes with symbolist subjects. A phenomenological description of the embodied experience of those who have accessed this space is basically the following: the visitor turns first towards the sea, which is brightly illuminated by the sun and from where one hears the noise of waves and the cries of seagulls (Fig. 10.7).

Fig. 10.7

VR reconstruction of the loggia of the Villa Storck. Image by M. Hodea

This synesthetic experience provides a sense of “reality” immersion. Oriental carpets give it a credible supporting surface and enhance the feeling of reality (Fig. 10.8).

Fig. 10.8

VR reconstruction of the loggia of the Villa Storck. Image by M. Hodea

Then, the visitor returns to look at the works of art (Fig. 10.9).

Fig. 10.9

VR reconstruction of the loggia of the Villa Storck. Image by M. Hodea

The loggia, which is a parallelepiped space, analogous to the real one and therefore not creating problems of spatial orientation in immersion, is extremely scarcely furnished, just an antique chair from which you can admire the landscape of the seacoast, oriental carpets, and easels with paintings in progress (Fig. 10.10).

Fig. 10.10

VR reconstruction of the loggia of the Villa Storck. Image by M. Hodea

Once teleported to this space, the visitor will perceive these points of interest: the sea, frescoes, and paintings and will approach the last two. When relatively close to the frescoes, a button with the text “visit room” containing the teleportation link is activated. By approaching these frescoes, the visitor is teleported to another virtual world, that of the artist’s mind, i.e. “World-within-World-Cecilia-Storck”.

6.3 Explanation of the Navigation Within Cecilia’s Mind Space

The virtual space of the artist’s memory is a metaphor of the human mind made in the form of a construction formed by various memories. We considered that memories can be visually represented in the form of planes with overlapping or juxtaposed images, augmented by different sounds and intensities of textures and colours (Fig. 10.11).

Fig. 10.11

Visual metaphor of the painter’s mind in Mozilla Hub, made in the form of a construction formed by various memories and augmented with sounds and intense colours. Image by C. Moţăianu

This synaesthesia effect was achieved by combining the surrealist imaginary with different sounds, natural or orchestral from various important eras in the artist's life, trying to create an outworldly effect totally different from the one achieved in the loggia space. The archaeological action of recomposing the artist’s mind was achieved by associating the iconography of the work with different auditory sensations and with the embodied experience of the visitor of this virtual world, who is forced to go through routes analogous to those travelled by the artist on the beaches, or on the oriental streets of Balchik (Fig. 10.12).

Fig. 10.12

Visual metaphor of the painter’s mind in Mozilla Hubs, made in the form of a construction formed by various memories and augmented with sounds and intense colours. Image by C. Moţăianu

Moving around in this environment is done with directional keys/arrows for visitors without a headset and with Oculus Quest 2 controllers and analogic sticks for those with a headset.

The visitor’s movement is different from that in the loggia of the house, because the twisted shape of the roads offers at every step a new and unexpected perspective. The juxtaposition of artistic works with fragments of Balchik’s landscape suggests the influence of context on the genesis of the work, as well as on the artist’s memory. In her youth, Cecilia Cuţescu-Storck was influenced by the work of Paul Gauguin; for her, as for a lot of Romanian artists, Balchik was a place with a strong exoticism, which reminded her of Gauguin’s work.

The visitor’s movement is different from that in the loggia of the house, because the twisted shape of the roads offers at every step a new and unexpected perspective. The exit from this space is done by clicking on the message “visit room” that appears on an easel located in an area with objects from the artist’s patrimony. It can also be quickly navigated through several sensitive spotlights, which ensures the user’s quick “jump” to different areas of this more complex space.

6.4 Representing Users in the Metaverse

Users accessing the metaverse without VR headsets will be represented by an avatar, selected from those offered by Mozilla Hubs, which can be used as such, or customized from the platform’s menu, the “Avatar” option. Depending on the avatar and the capabilities of the platform, various aspects of the avatar can be customized, such as colours, clothing, and accessories. Some avatars offer more customization options than others.

Another option is to upload custom avatars to other environments. This requires creating a 3D model in a compatible format (such as GLB) and uploading it to the platform. This option is more advanced and may require knowledge of 3D modelling software.

Any avatar changes must be saved to be retrieved the next time you access the environment. When changing the appearance of an avatar, performance must be considered: highly detailed or complex avatars can affect performance, especially in spaces with many participants.

6.5 Metaverse Exploration from a Desktop

In the desktop experiment (without headphones), users logged in from their browsers—Chrome or Firefox, based on a URL link generated by Mozilla Hubs and https://hubs.mozilla.com/ZccyPMm/world=within-world-villa-storck/, respectively.

Through a login window, the user chooses the “Join Room” button, in which he can choose a name and an avatar, as well as accept the use of the microphone and headphones.

6.6 Metaverse Exploration from a VR Headset

Mozilla Hubs, being a browser-based VR social platform, allows users to easily create and explore 3D spaces. The authors used Oculus Quest 2 headsets, a standalone VR headset that made the experience more intuitive and immersive.

To enter Mozilla Hubs on the Oculus Quest 2, users must put on their Oculus Quest 2 headset and open the Oculus or Firefox Reality browser. By navigating to the Mozilla Hubs link, https://hubs.mozilla.com/ZccyPMm/world=within-world-villa-stork/, they can enter a virtual room. A simple click of the “Enter VR” button takes the user from viewing the 2D browser to a full 3D VR experience (Fig. 10.13).

Fig. 10.13

Entering in the virtual room of the reconstruction of the Villa Storck using Oculus Quest 2 headset, performer M. Moţăianu. Photo by D. Gheorghiu

Once in VR mode, navigation is primarily done through the Oculus Quest 2 controllers. Analog sticks allow smooth movement in virtual space. Users can walk or fly through the environment, depending on the camera settings. The headset’s six degrees of freedom (6DoF) tracking allows users to move naturally, enhancing their sense of presence in the virtual world.

Interacting in Mozilla hubs through Oculus Quest 2 offers several possibilities. Users can achieve:

• 1. Communication with other users: voice chat is enabled by default, allowing for a natural conversation with other participants. Hand gestures and avatar expressions further enrich communication.

• 2. Object manipulation: Users can grab, move, or interact with various objects in the environment using controllers.

• 3. Avatar customization: Avatar customization is possible, providing a unique identity in cyberspace.

• 4. Content creation and sharing: Mozilla Hubs allows users to import images, videos, and 3D models, which can be shared and discussed in real-time.

• 5. Teleportation and scene transition: Teleporting to Mozilla Hubs is simple. Users point to the desired location or portal with their controller and use the shutter-release button to teleport. This feature is particularly useful for moving around in large spaces or accessing different areas in the virtual environment.

As an observation, during navigation and interactions in VR, the surrounding physical space must be considered. Oculus Quest 2’s Guardian system helps users stay in a safe area, preventing real-world collisions.

6.7 Metaverse Hybrid Exploration

In the hybrid exploration, some users entered from the desktop, Chrome or Firefox browsers, others from an Android smartphone (Fig. 10.14), from the Chrome mobile browser, and still others used the Oculus Quest 2 headset and opened the Oculus or Firefox Reality browser.

Fig. 10.14

Hybrid exploration of the loggia of the Villa Storck, using Android smartphone and an Oculus Quest 2 headset. Performers M. Moţăianu and M. Hodea. Photo by C. Moţăianu

6.8 Socialization and Learning Within the Metaverse

The socialization of a group of avatars in virtual space is possible in Mozilla Hubs. In the case of the present research, an experiment was carried out that orally described the experience of virtual space in front of an audience consisting of students from the National University of Arts in Bucharest and visitors to the exhibition “Storck House in Balchik”, curators Mihaela Moţăianu and Cornelia Moţăianu, from the Museum of Romanian Literature in Bucharest. An experimentalist equipped with Oculus headsets entered cyberspace, which was projected in front of the public, and began to describe problems of orientation and perception of spatial organization (Fig. 10.15), which were being watched on the screen by the audience.

Fig. 10.15

Presentation of the orientation and perception in the reconstructed spaces in front of the public, at the Museum of Romanian Literature in Bucharest, November 2023. Photo by C. Moţăianu

This first experiment helped us to recompose the virtual space of the artist’s mind, for a better reading and then, after a new documentation at the Storck Museum, to better define the elements that define her visual memory.

The resulting virtual space was immersed with two avatars, as teacher and student (Fig. 10.16), to re-verify the historical correctness of juxtapositions and overlaps and to define a series of questions to be asked of students and visitors who will access this space. Another experiment was drawing and writing in virtual space with digital pencils, also visible to the public (Fig. 10.17), explanatory schemes to help understand the artist’s mind. Two methods of environment interaction are available: a) by pressing the joystick button and performing gestures to highlight selected paragraphs and b) by using the interface to select tools and change the line colour (Taehyun et al. 2023, p. 7).

Fig. 10.16

Teaching exercise with two avatars (a professor and a student). Performers are L. Ştefan and C. Moţăianu. Image by C. Moţăianu

Fig. 10.17

Marking a text of Cecilia Cuţescu-Storck memoirs in virtual space with a digital pencil, in front of the public. Performer is C. Moţăianu. Photo by M. Moţăianu

Moreover, Taehyun et al. (2023, p. 6) state that the “intersection of sketched knowledge representations and spatially complex learning content that VR-based (tracked headset and controllers) drawing programs have, has the potential to allow a learner to fluidly engage with spatial dimensions as they create their drawings” and that “drawing in VR further unleashes the distributed cognition benefits of drawing to learn, as drawing elements can now be placed in space around the user creating a more embodied distributed model of their knowledge”.

Our educational intention is to put the subject (who enters this space from the loggia) in a state of immersion that excites his imagination and creativity, two key elements of the knowledge production of the new experiential archaeology, and to which he contributes, besides images and sounds chosen according to the different aspects of the created context.

We recommend as feedback from the users exploring the World-within-World Metaverse to also visit the Storck Museum and to re-document on the Balchik Art School. We consider that such a post-experience would be one of the major educational gains of the experiment.

7 Discussion: Benefits, Limits, and Challenges of the Metaverse

The experiences with virtual worlds built on traditional 3D online platforms, of which Second Life is the most representative, have proved efficient for testing new concepts without consuming resources, promoting new business models, for conducting online educational activities or for other activities and events that could be difficult to be carried out on a larger scale in reality (Stackpole 2022). In the case of the metaverse, although it is not a new concept, it will take to another level all the characteristics of traditional virtual worlds, and consequently their benefits and dangers, especially those related to “humanity and ethics” (Ryufath 2021).

More than in the traditional simple virtual worlds, the metaverse must ensure trust, privacy, application of law, combat bias, and disinformation and consider “aspects linked to addiction and impact on vulnerable people” (Dwivedi et al. 2022).

For the metaverse to reach the level theoretically described, namely a network of virtual worlds, decentralized but interconnected with each other, it is necessary either to mimic as well as possible the models and concepts of the real world, e.g. mirror-worlds or the integration of digital twins, or to provide new experiences, usually augmented, that attract users to find an alternative or complementarity to the real world.

Significant benefits, but also limitations and challenges are signalled in the case of using the metaverse in education. Traditional virtual worlds have demonstrated definite benefits but were only more widely used during the Covid pandemic. In Dwivedi et al. (2022), it is considered that “virtual worlds in the service of education have not been able to mimic the face-to-face learning experience as they cannot transmit its cognitive and emotional experience arising by gestures, body language, co-presence, social interaction, and engagement”. But the use of the metaverse is expected to “elevate the capabilities of online learning experiences (Dwivedi et al. 2022). The metaverse features that can be exploited in education are the capabilities to create very close virtual worlds or mirror of the real-world learning environments, so as to allow reproducible or impossible activities in the physical world, as well as new forms of training which in turn will require another pedagogical approach, course curricula and digital literacy with the new equipment, as well as new methods for assessing learning outcomes (Dwivedi et al. 2022).

As a special case, art education in the metaverse can make optimal use of the benefits of the virtual worlds, i.e. the creation of virtual spaces in mirror with the real ones or reconstructed, the implementation of digital art concepts with rich multimedia content, stimulation of creativity and imagination, and also the perception of abstract concepts and complex works of art. The ability to create in-world or import 3D content, social capabilities, e.g. chat, immersion, and interaction with the environment, especially through VR and AR equipment, are among the most important features of 3D virtual worlds.

The possibilities of movement and navigation ensure perception from various angles (Ştefan et al. 2022b). In the case of interconnection with other virtual worlds, within an extended metaverse, aspects of intellectual property, privacy, and security must be considered.

Barthélemy and Ondrus (2023) believe that the metaverse will be provided by the major platforms currently being developed, e.g. Microsoft Mesh and Meta Horizon Worlds, but until then, commercial metaverse platforms dedicated to activities can be used, e.g. games on Robox and Fortnite, virtual worlds on Sandbox and Decentraland, as well as “miscellaneous” virtual worlds (Terry and Keeney 2022), created for research and educational purposes, as proposed in this chapter.

8 Conclusions

Due to the Metaverse concepts, the “world-within-world” (Smith 2019, p. 198) approach allowed two different worlds to join together from the point of view of archaeology. If the virtual space of the loggia is a mirror that faithfully copies reality and offers the visitor an objective image of heritage and is correct from the perspective of the current archaeological paradigm and the European Commission report on “New technologies for Cultural and Scientific Heritage”, that of the artist’s mind is a space of interpretation, much more focused on embodiment, reflexive experience and escape from the constraints of ordinary perception.

This chapter was intended to be a plea for the possibility of expanding objective research into areas of subjectivity, and in this respect, the Metaverse is the tool that can easily achieve this goal.

It can be concluded that the originality of the act of creating a metaverse consisting of a world within a world also consists of perceptual differences between spaces, which oblige the visitor to an intellectual involvement, offering him, besides an exercise of experientiality, knowledge in the field of heritage and visual arts.

We tested on a mixed audience of artists and writers the cultural recovery value of such an endeavour and were pleased to see its enthusiastic reaction. The phenomenological experiment of virtual archaeology, carried out in the case of the reconstruction of the loggia of Villa Storck, was continued, by contrast, with the exercise of imagining a human mind, and the latter went beyond the limits of experientiality, as it is perceived today, becoming a work of art inspired by historical data, collected subjectively.

The result was the creation of a subjective world, a metaphor for the mind of an artist a century ago, which augmented the perception of the real world.

The metaverse allowed the visitor to teleport into each of these two worlds and a sense of immersion, with a number of small shortcomings, especially in terms of total sensory experience, lacking tactility, and the possibility of “natural” movement, due to obstacles that required small teleportation jumps. Despite this, the participants’ phenomenological experience allowed them behaviours analogous to those in the real world and a fluency of communication and learning.

We can affirm that a metaverse consisting of an augmentation of the real world with different worlds can create a complex phenomenological experience and become a tool for augmenting the scientific and artistic imagination.