Reality Composer, an AR creation tool by Apple, was applied in this project. Artists had to learn by themselves how to use the tool to create augmented reality experiences. Kingsley Ng recalled that the creative team spent a lot of time testing the app continuously. Reality Composer gave users an intuitive interface with multiple functions such as fixed-point shooting, triggering actions when moving objects close to the camera, or playing sound.

Virtuality is not necessarily a marvellous and exciting spectacle. The creators presented works with the theme “resilience” capitalising on the characteristics of various media and the resulting works were succinct and implicit. Aza Kwok’s work Goodbye Heaviness was a scene with twelve virtual stones forming a circle. When the user tapped the screen, the stones would slowly rise to the sky. Aza explained that she lived near Wong Shek Pier where she found many stones of special shapes. She then scanned the stones in 3D figures to produce an animation.

Her work carried her feelings during the pandemic and also showed a form of resilient mind with AR. “I felt trapped in the past year. In addition I had some emotions and always fancied about escaping from the present situation. My father told me that Wong Shek Pier had always been a spot on the ‘getaway’ route. I felt there was a great power. Even the stones there were magical,” she said. “So I made a stone circle which obviously should sink but instead flew upwards against gravity.”

Besides in the pier, the charm of resilience can be everywhere, which is the chemical reaction combining AR and arts. Users can watch the works in different places, take photos to share, and so on. Kingsley explained that site-specific creation was based upon the characteristics of the site or its cultural-historical development. This experiment was user-dominant, who created his or her own site-specific works anytime, anywhere and at his or her own pace. Other works of the project such as Absentmindfulness – the Earth and Exhalation in which plants grew in any environment; Sundown invited everybody to pause and enjoy the sunset.

“Letting Go” allows people to use their digital devices to play with the AR artworks, adding colours to their lives anywhere and anytime. (Photos provided by interviewee)

Public Space Turns into Stage at the Fingertips

Arts in the aiR was another project dealing with the relationship between AR and space. Five virtual performances including Chinese music, Western music, Indian-Pakistan folk songs, dance and drama were set up in installations in public housing estates. Curator Tung Tsz-ching was originally an Erhu performer. The pandemic had forced her to put off her performance, so she started to look into developing arts in a virtual direction.

Growing up in a public housing estate, she wanted to bring performances to public spaces in the housing estates and blend the beauty of the two. “Many buildings in public housing estates are beautiful, especially the older estates,” she said. “These estates also take humanistic care into consideration and provide a lot of public space so that people could gather and interact. Now these spaces have become the ‘stages’ for AR performances.”

With the support from Hong Kong Housing Society, Tung Tsz-ching had launched the Arts in the aiR in five estates under the Housing Society, namely Kwun Tong Garden Estate, Yue Kwong Chuen, Ming Wah Dai Ha, Lok Man Sun Chuen and Chun Seen Mei Chuen, and later would gradually extend to other housing estates. The project invited a group of artists to design respective performances based on the characteristics of the housing estates in order to enhance the connection and locality of the two. For instance, the theatre performance was inspired by the Kwun Tong Garden Building, which told the history of Kwun Tong as a salt farm in the Northern Song Dynasty.

Curator of Arts in the aiR Tung Tsz-ching is an Erhu player. She hopes to bring different performing arts to public housing estates with this project. (Photo by Cafe Tong)

Arts in the aiR also took a bold step forward to put portrait videos in AR mode. Tung Tsz-ching explained that this approach was quite new in Hong Kong. “Most common app in AR usually display animations, such as figures or special effects,” she said. “Now that we filmed physical performances done by people and we had to remove the background. This was very difficult for details like motions and hair.”

The project set up “AR Hotspots” in different housing estates, and users could enjoy the performances by scanning the hotspots with relevant mobile app.

“AR Hotspots” are placed in different public housing estates. Users can scan the hotpots with mobile application “ArtsInTheAiR” to view the performances at the spot. (Photo by Cafe Tong)

Users can choose five different performances in the mobile application, namely Chinese and Western music, Indian-Pakistan folk songs, dance and drama. Each performance is about four minutes. (Photo by Cafe Tong)

“Some uncles and aunties found out that performers would ‘pop up’ on the screen when they take pictures or videos of the physical environment with mobile phones at the spot. They were amazed and would call their children, friends and neighbours to join the game.” Tung Tsz-ching said, adding that the creative team had set up street counters in these housing estates to introduce the project and the response was heartening.

She continued, “Many foreign domestic helpers enjoyed the performances of Indian-Pakistan folk songs very much but most of them were Indonesians. I wondered why they loved Indian folk songs. I found out that it was because Bollywood movies were very popular in their social circles and they liked to listen to those songs, similarly as many Asians liked watching Hong Kong movies. Through these works, people had more opportunities to connect with each other.”

Experiencing Optional Fragmented Reality through VR Theatre

Could we really connect or even get into other people’ world through VR? The work A Mind Apart made a further attempt to apply VR technology to make a virtual reality theatre experience. The audience can “enter” the world of a couple and witness the disintegration of their relationship.

The so-called “theatre” combined multiple medium and took place in a hotel room instead of on the theatre stage. After putting on the VR headset, the audience would see the couple’s home and watch the video segment by segment.

Jantzen Tse was responsible for the filming, editing and soundscape design for the project. “We took reference from works such as Black Mirror that let viewers choose from different plots. There were in total 12 storylines so what you might see could be completely different from mine,” he said. Curator, director and co-scriptwriter Chan Ka Wai, Kiwi explained that the story described the inner anxiety of the male and female protagonists, and the fragmented storyline made it impossible for us to see the whole reality but reflected on how judgments and misunderstandings caused harm.

Creative team members of A Mind Apart: Jantzen Tse (left) and Kiwi Chan. (Photo by Cafe Tong)

A Mind Apart is staged in a hotel room. Audience view the performance with a VR headset. (Photo provided by the interviewee)