Interactive Volumetric Live Performance Between Sydney and Tokyo

January 23, 2024

Keio University Graduate School of Media Design (KMD)


For the first time ever, a volumetric video-based live musical performance viewable from any angle was performed to an audience located 7,800km away across the ocean, co-creating improvisational music between the performer in Tokyo and the audience in Sydney. A stable and high-speed communication network, which is required to transfer a high volume of information that cannot be achieved with ordinary lines, was supported by a collaboration of research and education networks in the Asia Pacific.

Keio University Graduate School of Media Design (KMD), Canon Inc., Bascule Inc., and  Mori Building Co., Ltd. have collaborated to deliver a real-time performance using the Research and Education Internet link between TOKYO NODE and International Convention Centre Sydney for a closing session of SIGGRAPH Asia 2023 “Super-Speed Connectivity for Digital Twins” on December 15, 2023.

The audience in Sydney enjoy the performance from Tokyo with point-cloud volumetric video.

Walking around the performer to move the viewpoint.

The audience interacts with the performer in Tokyo.

The violin is performed from the Volumetric Video Studio in TOKYO NODE.

Violin is performed from Tokyo and improvised responding to the audience’s action in Sydney, to co-create the music as participatory music. The Audience in Sydney is able to freely move the viewpoint such as front, side, back, or moving close to the violin.

The Performance was streamed in real-time from the Volumetric Video Studio in TOKYO NODE with 57 cameras in the immersive green screen environment. The captured video was converted to a point-cloud using the proprietary cutting-edge volumetric video technology. 

The SuperSpeed Research and Education connectivity between TOKYO NODE and International Convention Centre Sydney was provided by AARNet, Australia’s Academic and Research Network, ARENA-PAC, Arterial Research and Educational Network in the Asia Pacific operated by WIDE Project, and GOREX, The Guam Open Research and Education eXchange.


Volumetric Video Technology Team

Atsushi Date , Shuntaro Aratani (Canon.Inc)  

Masayoshi Boku, Sakiko Osawa (Bascule) 

Ou Sugiyama , Kazuki Motani (Mori Building) 

Shinji Aoki (NiTRo)


Super Speed Network Team

Steve Maddocks (Director International, AARNet)

Luc Betbeder-Matibet (Director, Research Technology Services, UNSW)

Chris Zane (University of Hawaii / GOREX)

Hirochika Asai, Takashi Tomine, Takumi Ishihara, Mariko Kobayashi (ARENA-PAC / WIDE Project)

Daiya Kato, Shun Arima (KMD)

Keiko Okawa (KMD / WIDE Project)


Performance Team

Yuehui Yang (KMD)

Daichi Ito (IBM)

Yamen Saraiji (SONY AI / ACM SIGGRAPH Hybrid Society adhoc Committee)

Masa Inakage (KMD / ACM SIGGRAPH Hybrid Society adhoc Committee)


    Brain Body Jockey Project

    Updated on 2023.08.18

    Sorry, this entry is only available in Japanese.

    Cybernetic Avatar Experiment in AVATAR ROBOT CAFE DAWN ver.β

    Updated on 2023.04.28

    Three experimental café services will be developed at the  " AVATAR ROBOT CAFE DAWN ver.β". "Virtual avatar service," in which customers can move betw...

    NTT docomo’s New TV Commercial Featuring “FEEL TECH™", Hapti...

    Updated on 2023.02.21

    NTT docomo’s new TV commercial featuring “FEEL TECH™", haptic sharing technology which has been developed in collaboration with NTT docomo, KMD Embodi...

    New Initiatives to Promote a Content+Technology Creation Hub...

    Updated on 2023.01.30

    PoliPro, Keio University Graduate School of Media Design, is pleased to announce the launch of the "Social Creation" project: a creative solution to s...

    DOCOMO Announces World’s First Technology that Utilizes Huma...

    Updated on 2023.01.25

    TOKYO, JAPAN, January 25, 2023 — NTT DOCOMO, INC., the Embodied Media Project led by Professor Kouta Minamizawa at Keio University Graduate School of ...

    Starting a Collaboration Project for the Creation of New Val...

    Updated on 2022.12.05

    Research groups including Professor Kouta Minamizawa (KMD), Professor Yoshihiro Tanaka (NITech), and the Japan Craftsmen's Association are starting a ...

    Collaborative Avatar Experiment in AVATAR ROBOT CAFE DAWN ve...

    Updated on 2022.10.14

    We are developing collaborative avatar experiment through cybernetic avatars in Moonshot Research & Development program. Moonshot Research & D...

    Collaboration Agreement Signed with the National Strategic S...

    Updated on 2022.08.10

    Summary: Keio University Graduate School of Media Design (KMD) and the National Strategic Special Zone of Yabu City (Mayor: Sakae Hirose) in Hyogo Pr...