Spatial Praxis in the Staging of Material Evidence
by Bernadette Flynn
A common starting point for digital applications of cultural heritage is the visualization of the artifact as the material evidence of historical inquiry. Since the early visualization systems of the 1980s, the verisimilitude of digitally recreated artifacts has operated as the hallmark of authenticity. Such a quest for seamless constructions of the real has resulted in developments in documentation that often elide the kinaesthetic and situated dimension of embodied human presence.
New research in media practice and archaeology, coupled with the potential of responsive and interactive environments, forms the background to rethinking the role of documentation for user agency. This also has relevance for the project of documentary in the reconfiguration of actuality as a participant-based domain of interactive exploration. Previous reliance on the digital surrogate as the primary evidence of the material past has more recently been challenged by others seeking more equivocal presentations of materiality.[i] Augmented by digital technology, new expressions of tangible and intangible documentation are being developed. In these developments the presentation of, for instance, records, maps, sites, landscapes, and objects, seeks to move beyond a transmission model to incorporate the situation and experience of the human body.
Fig 1: Screen shot of ‘Spaces of Mnajdra’ showing virtual shadow of the user. ‘Spaces of Mnajdra’ is a digital heritage project about the Mnajdra Temples in South Malta © Bernadette Flynn
A phenomenological turn within both archaeology and new media has highlighted interpretation as an embodied practice. Phenomenology, as a particular branch of philosophy developed in the 19th century, explores the relationship between embodied action and meaning, thus co-joining the experience of being in the world with the practices of interpretation. Recent approaches to phenomenology by archaeologists Christopher Tilley (drawing on Merleau-Ponty) and Julian Thomas (drawing on Heidegger) emphasize the tight coupling between perception and action.[ii] Phenomenological approaches suggest that knowledge about the past arises from the material conditions of the body, in the sense that interpretation is an embodied practice made manifest as a condition of the sensorial world. Rather than an adherence to a fixed notion of the pastness of the past, the artifact is embedded in social networks and bound up with the parameters of human situation and agency. As Tilley argues, interpretation is “not so much an attempt at a hermeneutic recovery of original meaning, but more a form of critical production.”[iii] Through an examination of the remains of past actions—including the traces of gestures and movement patterns, evident at different times of day and environmental conditions—meanings about the past emerge. Further, through the social performance of spatiality, knowledge of the past is received and interpretations about past practices are articulated.
Reconfiguration of the spatial as embodied user experience has been enabled through the potential of mixed reality, response, and motion-tracking systems. Large-scale responsive environments in particular allow for new ways of deforming, reinterpreting, and restaging interpretative practice and the processes of interactive documentary production. The interplay between modes of digital and organic spatial practice is evidenced in projects emerging from research centers such as ZKM, Center for Art and Media in Karlsruhe; iCinema, Center for Interactive Cinema Research in Sydney; and ALiVE Applied Laboratory of Interactive Visualization and Embodiment in Hong Kong. Such research into interaction design has facilitated the shift from traditional presentation modes to represent the digital encounter as an expression of the performance of spatiality, in which the organic and the digital co-create the multi-layered aspects of embodied user presence.
In the technological platforms of CAVE (Cave Automatic Virtual Environment) and AVIE (Advanced Visualisation and Interaction Environment), the body is afforded movement in physical space, is sensorially embodied, and has digital presence in the display. As Brian Massumi argues, presence, or the sense of being situated within an environment, is as much a function of proprioceptive and kinaesthetic engagement as it is of vision.[iv] Examples of interaction strategies that incorporate body movement, drawn from new media art practice, include Myron Krueger’s Videospace, Simon Penny’s Fugitive 2, Jeffrey Shaw’s The Legible City and David Rokeby’s Very Nervous System.[v] As postulated by Mark Hansen, these events or acts of dynamic coupling via bodily motility restage materiality in the digital realm.[vi] Examples of the restaging of materiality pertinent to the documentary framework include Dennis Del Favero’s Conversations, Pentimento, and Scenario.
Krueger’s Videospace exhibited at TISEA (Third International Symposium on Electronic Arts) in 1992 encourages user movement through a playful exchange between user and screen characters.[vii] Video capture detects participants’ movements around the room to create live silhouettes on the screen that corresponds to their movements. Watching the silhouette on screen, users lose all sense of self-consciousness in their efforts to enact a playful encounter with body outlines. Extended applications of the system allow users to alter the laws of cause and effect and to play with, for instance, the differential gravitational parameters of digital objects or the animated response of screen characters such as “the critter.” Penny’s Fugitive 2 (2000-2004) constructs an interaction paradigm that reflects the user’s own bodily dynamics through an image window. Using infrared tracking and a wrap-around 360 degree screen users’ bodily gestures, such as directedness, hesitancy and wandering are interpreted through pans, zooms, and circling in the image window. In Shaw’s The Legible City (1989), navigation through a cityscape made up of letters is activated by riding a bicycle. As the user cycles, the letters are animated forming words and sentences. In two versions of The Legible City (Amsterdam 1990, Karlsruhe 1991) the scale of the letters corresponds to the actual properties of the buildings and the texts are assembled from archival documents describing historical events.[viii] The action of peddling and moving the handles of the bicycle activates a spatial documentary of the city. Shaw draws attention to the kinaesthetic impact of these bodily operations: “one experiences one’s whole body as the causal power of the entire journey.”[ix] Rokeby’s Very Nervous System (1986-1990) dispenses with the screen-based interface altogether to develop an interface based on the configuration of sound through full body movement. Semantic gestures along with the involuntary dynamic of the gesture are translated into sound and music in real time, encouraging full spatial participation without the seduction of the image. David Rokeby argues that within a limited template of interactive simulations, perception is changed to reinforce a raw sense of connection with the surroundings.[x] As such, these responsive environments make present and perceivable the results of an execution and, in so doing, complete the loop between proprioceptive awareness and machine exchange.
Considered here as a responsive documentary, Del Favero's Pentimento (2002) is staged as an interactive video installation where the user interrogates a layered narrative surrounding the discovery of a decomposed and unidentified body on the outskirts of Sydney. Using motion-tracking technology, user movement triggers different and contested layers of narrative to present the varying viewpoints within the hybrid space of representation. Investigation of opposing versions of the past is navigated via modes of movement or stillness to enable a personalized matrix of interpretative events where memory and recollection of the past is placed under scrutiny.[xi]
Scenario (2010), a more recent project produced by Del Favero and collaborators, is a “co-evolutionary” narrative. Users enter an immersive mixed reality environment that is populated with digital characters to unfold a narrative of events in co-operation with these characters.[xii] It is part of a larger research agenda concerned with the evolution or emergence of narrative based on the interaction between the activity of a human user and a digital agent. The narrative “backstory” of Scenario is based on the infamous Josef Fritzl case, in which an Austrian father imprisoned and impregnated his daughter. Scenario extends the category of documentary by replaying a different ending through user affect. With the help of user participants, the on-screen characters can discover a way out of their imprisonment. Other on-screen characters function as guards to block their escape. Past events are reorganized through the parameters of the interpretative gesture in a series of co-evolving performative acts. This may constitute interference or co-operation, argument or discussion as embodied attitude between human and on-screen agents.
In the projects described above, interpretation proceeds by way of emergent spatial journeys constructed through the surrounding environmental topography and aural spatiality. The recontextualization of material evidence as spatial praxis suggests different experiential registers of embodied affect. The interplay between these registers of embodiment is central to co-creating interpretative formations of the past. These embodiments are not considered as separate categories but as spheres of influence that overlap and bifurcate depending on the creative intent and the particular situation of the user. The interplay of sensate bodies, the digital environment, the embedded landscape, and narrative context recalibrate material documentation as a dynamic site of lived experience and social agency. Such an embodied and collaborative encounter with the digital environment elicits particular ways of looking, kinaesthetic formulations, and sensory perceptions. Visualization strategies invite ways of looking and enable new understandings of the morphology of the environmental matrix. The stimulation of movement processes provokes particular movement patterns for conveying the presence of the past. The interpretative gesture further invites the possibility of re-enactments.
The extended performative encounter activated in collaboration with others (both digital and organic) reconfigures digital documentation and the project of interactive documentary as both a spatial praxis and a refinement of somatic presence. In this reframing of the relationship between our embodied selves as sensate beings, and our mirrored selves within digital space, new knowledge spaces are generated. The capturing of architectonic space and individual subjectivity is realigned with the landscape of embodied perception. Sensorial affects are reorganized between the physical and the digital, reinvigorating material evidence from the past through present day performance to include body memories and the cultural situation of present embodiments.
When spatial praxis is taken as the primary method of embodied analysis, user interaction arouses and contests prior claims to visual replication as the basis for mediated authenticity. Responsive and interactive systems deployed in museums, galleries and across new media practice extend representation to visitor movement and environmental contextuality relevant to the restaging of historical accounts and the documentary. In a crafted responsive environment, spatial praxis operates as a method of embodied analysis inviting differing types of bodily expression and enabling new forms of knowledge production. Movement patterns, along with the social expression of collaborative gestures, thus enhance the act of interpretation as a contingent localized practice and a kinaesthetic description of somatic knowledge. In the production and experience of this localized embodied practice claims to authenticity previously associated with the verisimilitude of the image are realigned with the somatic processes of user engagement.
Notes
[i] There is a developing focus within digital cultural heritage on the relationship between material culture and interpretation. For some examples on these developments see: Erik Champion, Playing With The Past, HCI series (London: Springer-Verlag 2010); Bernadette Flynn, "The Morphology of Space in Virtual Heritage," in Theorizing Digital Cultural Heritage: A Critical Discourse, ed. Fiona Cameron and Sarah Kenderdine (Cambridge, Massachusetts: The MIT Press, 2007); Mark Gillings, "Virtual Archaeologies and The Hyper-Real or, What Does It Mean to Describe Something as Virtually-Real," in Virtual Reality in Geography, ed. P Fisher and D Unwin (London and New York: Taylor and Francis, 2002); Sarah Kenderdine et al., "Place-Hampi: Co-evolutionary narrative and augmented stereographic panoramas, Vijayanagar, India" (paper presented at the New Heritage Beyond Verisimilitude, University of Hong Kong, 2006).
[ii] Julian Thomas, "The Great Dark Book: Archaeology, Experience, and Interpretation," in A Companion to Archaeology, ed. Timothy Earle and Christopher S. Peebles (Blackwell Publishing, 2004); Christopher Tilley, A Phenomenology of Landscape: Places, Paths and Monuments (Oxford: Berg, 1994); Christopher Tilley, The Materiality of Stone: Explorations in Landscape Phenomenology (Oxford: Berg, 2004).
[iii] Christopher Tilley, "Meta Archaeology Project, Christopher Tilley On Modernity and Archaeological Discourse," http://archaeology.kiev.ua/meta/tilley.html
[iv] Brian Massumi, Parables for the Virtual: Movement, Affect, Sensation (Durham & London: Duke University Press, 2002).
[v] Oliver Grau, ed., Media Art Histories (Cambridge, MA: The MIT Press, 2007).
[vi] Mark Hansen, New Philosophy for New Media (Cambridge, MA: The MIT Press, 2004).
[vii] Myron Krueger, Artificial Reality 2 (Reading, MA: Addison-Wesley Professional, 1991).
[viii] Christiane Paul, Digital Art (London: Thames and Hudson, 2008).
[ix] Mark Hansen, New Philosophy for New Media (Cambridge, MA: The MIT Press, 2004).
[x] David Rokeby, "The Construction of Experience: Interface as Content," in Digital Illusion: Entertaining the Future with High Technology, ed. Clark Dodsworth Jr. (Reading, MA: Addison-Wesley Publishing Company, 1998).
[xi] Jill Bennett, Dennis Del Favero: Fantasmi (Sydney and Hanover: UNSW Press and Sprengel Museum, 2004).
[xii] Dennis Del Favero and Timothy S. Barker, "Scenario: Co-Evolution, Shared Autonomy and Mixed Reality" (paper presented at the IEEE International Symposium on Mixed and Augmented Reality, Seoul, 2010).
Bernadette Flynn is a researcher in interactive media for exhibition. Her research interests include interaction design and theories of spatiality across computer games, documentary and digital cultural heritage. Formerly a lecturer in screen media production at The Griffith Film School, Griffith University she is currently completing a PhD on applying phenomenological archaeology for digitally augmented experiences of cultural heritage. Her thesis considers the spatial dynamics of the Mnajdra temple of Malta and is staged in a responsive panoramic visualization environment at The iCinema Centre for Interactive Cinema Research, at The University of NSW, Australia. Recent publications include chapters in Theorizing Digital Cultural Heritage: a critical discourse, Cambridge: MIT Press, 2007 and New Technologies in the Archaeognostic Sciences, Gutenberg, 2008.
Media Fields Journal