Abstract

Sonic Sculptural Staircase in Highly Immersive and World Aware Augmented Reality

Yichen Wang

In ACMC2021: Proceedings of the Australasian Computer Music Conference 2021, Sydney / Melbourne, Australian Institute of Music, p. 26. Australasian Computer Music Association.

Abstract

Modern head-mounted augmented reality technology has now advanced to the level that it can be said to enable a Highly Immersive and World Aware (HIWAAR) experience. Critical to providing such an experience is the ability of some modern systems, like the Microsoft HoloLens, to capture video information of the surrounding physical environment in real time and to build models of that environment which enable a user to be “world aware” in the augmented world. At the same time, such systems can provide graphical images and audio that surround and augment the user experience in a highly immersive manner. Works that utilize such HIWAAR technology for the creation of augmented multi-sensory experience have been claimed to be effective in activating users’ engagement and enhancing art experience. However, few works have specifically considered how augmented art experiences should be designed and fewer have provided rigorous evaluations of the effectiveness of such experiences. My work explores the affordances of HIWAAR technology with the respect to sonic-tactile interaction in providing an augmented sculptural staircase appreciation experience. I argue that such powerful use of technology prompts a more active, embodied, and emotionally engaging experience to people in discovering, interpreting and acquiring knowledge. I demonstrate this by developing and evaluating an HIWAAR installation, Sonic Sculptural Staircase (SSS). My work commences using interviews with art professionals seeking for design inspirations for the HIWAAR augmented sonic sculpture (or sculptural elements) experience. Drawing on results from these interviews, the SSS was designed and developed. Two different experience conditions, the informational experience and the experiential experience, are introduced in SSS to explore ways in which sound can be integrated with visuals and interaction for people to understand and appreciate a sculptural staircase. Results from user evaluation show that the SSS presented a pleasant augmented sculptural staircase appreciation experience along with enhanced awareness of the physical piece, on both experience conditions over time. Furthermore, preliminary evidence from follow-up studies appear to show that the experiential experience, which utilized real-time generated electronic music and field recordings mapping related context, has a more lasting impact than the informational experience on the way that participants view the sculptural staircase in their daily lives. Based on my work, I conclude the aggregated sonic, visual and tactile HIWAAR experience is far more perceptually immersive and effective in communicating compared to a simple visual experience. Positive results revealed from user evaluations preliminarily support the effective-ness of utilizing such technology for the design of other augmented multi-sensory experiences for relevant communities.

Printed in the proceedings, p. 26 (PDF)

All copyright remains with the authors.

How to cite

Yichen Wang. 2021. “Sonic Sculptural Staircase in Highly Immersive and World Aware Augmented Reality.” In ACMC2021: Proceedings of the Australasian Computer Music Conference 2021, edited by Vincent Giles, Shally Pais, Anna Del Rosario, and Meghann O’Neill, 26. Sydney / Melbourne, Australian Institute of Music: Australasian Computer Music Association. https://computermusic.org.au/proceedings/ACMC2021-proceedings/wang-sonic-sculptural-staircase.html

BibTeX
@inproceedings{wang2021sonic,
  author = {Yichen Wang},
  title = {{Sonic Sculptural Staircase in Highly Immersive and World Aware Augmented Reality}},
  booktitle = {ACMC2021: Proceedings of the Australasian Computer Music Conference 2021},
  editor = {Vincent Giles and Shally Pais and Anna Del Rosario and Meghann O’Neill},
  pages = {26},
  year = {2021},
  publisher = {Australasian Computer Music Association},
  address = {Sydney / Melbourne, Australian Institute of Music},
  url = {https://computermusic.org.au/proceedings/ACMC2021-proceedings/wang-sonic-sculptural-staircase.html}
}

← All of ACMC 2021