DadaochengARToVR

---

Content

• 01 Background information
• 02 Study purpose
• 03 Literature review
• 04 Methodology
• 05 Study results
• 06 Conclusion and Suggestions

Background information

• In recent years, Virtual Reality(VR) and Augmented Reality(AR) play an important role in heritage preservation and guidance. (Hung, 2016; Vlahakis, 2002)
• The human-computer interaction(HCI) of VR are controller in recent years. It may be inconsistent with the way of interactions in daily life and decrease the immersion in VR.
• The combination of modern technology and past historical connotation will help us to understand the history and background of the historic district.

Study purpose

• Based on mobile device and Gear VR HMD as the development platform. Propose the approach to integrate Leap Motion on it.
• Engaging Virtual Interaction Guidance System. Propose the way of virtual transition and virtual exploration to tell historic story about the historic districts of Dadaocheng.

Literature review

VR Conception

• VR uses the technique of 3D computer graphic, spatial audio and perceptual interface to simulate user's physical presence in a virtual world. (Sutherland, 1965；Lanier, 1979；Goldberg, 1998；Steuer, 1992；Krueger, 1991；Coates, 1992；Greebaum, 1992)
• VR participants’ experiences:

1. Interaction 2. Imagination 3. Immersion(Visual & Audio)

• VR construction:

1. Image-based VR 2. Geometry-based VR 3. Hybrid VR

AR Conception

• AR allows user to see virtual elements overlapped upon real world in real time. Three characteristics of AR as follows: (Azuma, 1997)

1. Combine real and virtual 2. Interactive in real time 3. Registered in 3D

• The following technologies are essential for AR experience. (Milgram et al., 1994; Vallino, 1998 ; Bimber & Rasker, 2005; Cawood & Fiala, 2008; Clouth, 2013; Amin & Govilkar, 2015)

1. Display : HMD , Non-HMD 2. Recognition & Tracking : Features , SLAM, GPS

Methodology

• Unity version 2017.3.0.
• Vuforia SDK is built in Unity version 2017.2.0

Study results

• Virtual Transition
  Vuforia Image Target: Painter Kuo’s painting《南街殷賑》.
  Participants firstly see the painting on the wall in real world, and then be sent to the painting’s virtual world.
  https://www.youtube.com/watch?v=tP2K6Ihjbmg&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2&index=7
  https://www.youtube.com/watch?v=StD53ZaEZEo&index=1&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2
• Virtual Exploration VRMap In the last part of virtual transition, we go through clouds and see the Dadaocheng historic map.
  Right now, we end the part of virtual transition and start the part of virtual exploration.
  https://www.youtube.com/watch?v=vxVp9aAjZfo&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2&index=2
• Virtual Exploration VRHarvest Before middle term Qing Dynasty, a large number of Han Chinese and Pinpu ethnic groups lived and cultivated rice in this area. And the name of Dadaocheng also came from the drying rice space.
  https://www.youtube.com/watch?v=nK0Y7VqlC7A&index=3&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2
• Virtual Exploration VRBow In 1853, the conflict between immigrants (Sanyi&Tongan) over land and religion was raised, called the Dingxiajiao Conflict in history.
  Tongan people introduce many bows to train archers in that time, so participants play an role of Tonan’s archer in the stuty.
  https://www.youtube.com/watch?v=uFC-0mVQ_kM&index=5&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2
• Virtual Exploration VRBoat In 1860, Qing Dynasty opened Tanshui as a trading port, so participants can row the boat around the riverside and see the old-time Dadaocheng.
  https://www.youtube.com/watch?v=aviwRxkkMS4&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2&index=6
• Virtual Exploration VRBrush In 1921, Wei-Shui Chiang published Clinical Notes and pointed out various diseases about Taiwan from doctor’s point of view. Therefore, participants in the research can read and write the Clinical Notes in VR.
  https://www.youtube.com/watch?v=nPv9Dp2WRHI&index=4&list=PL2NXpePkCqV_QITu_WtlYieBNmsazoKZ2

Conclusion and Suggestions

Conclusion

• Virtual Transition Propose virtual transition. Participants see the painting on the wall through camera of mobile device and Gear VR HMD, and then are sent to the virtual painting world.
• Virtual Exploration Propose virtual exploration. Participants use both hands interact and explore with virtual world based on historic story.
• Socket Transmit Hand Data Propose the approach that Gear VR HMD and mobile device combined with Leap Motion.

Suggestions

• Try to analyze the immersion of this guidance system.
• Invite professionals involved in the historiography and humanities to participate.
• Try to increase tactile feedback(Arduino Modules).