Nocturnal Fugue + EchoVision

Jiabao Li, Matt McCorkle, Botao Hu, 2024

Inspired by Thomas Nagel's "What Is it Like to Be a Bat?", "Nocturnal Fugue" immerses audiences in the intricate language of bats through AI-discovered social calls and digital habitat simulations, using spatial sound, projection mapping, and haptic transducers. "EchoVision" complements this with a mixed reality experience simulating bat echolocation.

Download EchoVision from Apple App Store

Nocturnal Fugue

With the help of AI, researchers found bats have more complex language than previously thought, with individual names, the ability to argue over food, and vocal learning. Just like human baby talk, mother bats lower their pitch when talking to babies. What would a baby bat lullaby sound like? Would a song made of bat mating calls be a love song?

Enter into Nocturnal Fugue, where you experience an immersive world of bats. Their social vocalizations transform into evocative music set to imaginative digital recreations of their natural habitats. Each digital habitat crafts a different sense of scale and time, creating a mood centric to the social vocalizations that are being heard and felt. Explore and feel the identity of bats through their isolation, sleeping, kissing, mating, feeding, grooming, and fighting vocalizations. As their sounds are transmogrified into imaginative soundscapes you are transported into the unique sonic world of bats through spatial sound, immersive projections, and haptic vibrations.

Inspired by Thomas Nagel's philosophical inquiry in "What Is it Like to Be a Bat?", this project delves into the enigma of truly comprehending the experience of another species. While we may grasp the superficial metadata of bat behavior, the true content of their communication remains elusive. Can we ever fully understand the language of bats, even with the aid of AI? We might not be able to find many words from the bat world in human’s dictionary.

EchoVision

EchoVision is an immersive art installation that allows participants to experience the world of bats using sound visualization and mixed reality technology. With a custom-designed, bat-shaped headset from the open-source HoloKit project, users can simulate echolocation, the natural navigation system bats use in the dark. They do this by using their voices and interpreting the returned echoes.

The exhibit adjusts visual feedback based on the pitch and tone of the user's voice, offering a dynamic and interactive depiction of how bats perceive their environment. This installation combines scientific learning with empathetic engagement, encouraging an ecocentric design perspective and understanding between species. "Echoing Bat" educates and inspires a deeper appreciation for the unique ways non-human creatures interact with their ecosystems.

Appeared on Xinhua News 新华社

在人工智能的助翼下，研究人员发现蝙蝠的语言比我们所知的更加复杂，它们有名字，会争论，还能进行语言学习。就像我们和婴儿说话时会用可爱的声调，蝙蝠妈妈在和蝙蝠宝宝说话时会降低声调。想象蝙蝠的摇篮曲，会是怎样的旋律？蝙蝠求偶之声，又能谱出怎样的情歌？

步入“夜曲赋格”，沉浸于蝙蝠的幻境世界。蝙蝠的语言谱出情感丰沛的乐章，与虚拟的自然栖所相辉映。

每一处栖所，皆有别具一格的规模与时光感，营造出与所闻所感的蝙蝠的语言相应的情境氛围。探索蝙蝠的独特身份，体会它们在孤独、沉睡、亲吻、交配、觅食、梳理及争斗时的各种声音。

本项目灵感源自托马斯·内格尔的哲学探究《成为一只蝙蝠会是什么样？》，深入探讨真正理解另一物种体验的奥秘。我们或许能触及蝙蝠行为的表层元数据，但它们交流的真谛，仍遥不可及。即便借助人工智能，我们真的可以完全解读蝙蝠的语言吗？我们的词典里可能找不到很多数蝙蝠用到的语汇。

At Sheffield DocFest, Alternative Realities, 2024, UK

Introduction

The exhibit adjusts visual feedback based on the pitch and tone of the user's voice, offering a dynamic and interactive depiction of how bats perceive their environment. This installation combines scientific learning with empathetic engagement, encouraging an ecocentric design perspective and understanding between species. EchoVision educates and inspires a deeper appreciation for the unique ways non-human creatures interact with their ecosystems.

"What Is it Like to Be a Bat?"

In "What Is it Like to Be a Bat?" Thomas Nagel explores the unique, subjective experience of consciousness, using bats as a primary example due to their use of echolocation, a sensory perception vastly different from human experiences. He argues that understanding what it is like to be another creature is inherently limited by our inability to fully grasp their subjective experiences. This concept directly relates to EchoVision that allows participants to simulate the echolocation of bats using sound visualization and mixed reality technology. By adjusting visual feedback based on the user's voice, EchoVision bridges the gap between human and bat perception, offering a deeper, empathetic understanding of non-human sensory worlds and emphasizing the profound differences in how various species interact with their environments.

Misconception of Bats

People often harbor misconceptions about bats, viewing them as dangerous, dirty, or associated with negative superstitions. Common myths include the belief that bats are blind, that they commonly get tangled in human hair, or that they are primarily carriers of diseases (especially during the COVID-19 pandemic). These misconceptions contribute to a general fear and misunderstanding of bats, overshadowing their ecological importance and unique biological features.

Bats are ecologically vital, providing essential benefits such as pest control by consuming large quantities of insects, pollination of numerous plants, and seed dispersal that aids in forest regeneration. Their guano enriches soil fertility, contributing to nutrient cycling. These roles help maintain ecosystem balance and biodiversity.

EchoVision aims to challenge and transform these misconceptions by providing participants with an immersive, empathetic experience of the world through a bat's perspective. By simulating echolocation using sound visualization and mixed reality technology, EchoVision allows users to understand how bats navigate and perceive their environment with remarkable precision. This interactive experience highlights the sophistication and elegance of bat echolocation, fostering a greater appreciation for their role in ecosystems, such as pest control and pollination.

Through EchoVision, participants can see bats not as menacing creatures but as fascinating and integral parts of the natural world. The installation encourages scientific learning and empathetic engagement, promoting an ecocentric design perspective that values the diverse ways non-human creatures interact with their surroundings. By offering a new perspective, EchoVision helps dispel myths, reduces unfounded fears, and inspires a deeper connection to and respect for bats and their vital ecological contributions.

Echolocation

Bat echolocation is a sophisticated biological process that allows bats to navigate and hunt in the dark by emitting high-frequency sound waves and listening to the echoes that bounce back from objects in their environment. When a bat emits a sound, it travels through the air and reflects off surfaces, returning to the bat as an echo. By analyzing the time delay and changes in the frequency of the returning echoes, bats can determine the distance, size, shape, and texture of objects, as well as the speed and direction of moving prey.

In "An Immense World," Ed Yong describes how bats' echolocation abilities are finely tuned and highly specialized. He explains that bats can emit sounds at frequencies higher than the human ear can detect, often using calls that range from 20 to 200 kHz. The bat's brain processes these echoes with remarkable speed and accuracy, allowing it to create a detailed auditory map of its surroundings. This process is so efficient that bats can detect tiny insects in complete darkness and avoid obstacles with precision. Yong also highlights the diversity among bat species in their echolocation strategies, noting that different bats have evolved various adaptations to suit their ecological niches. Some bats have highly directional calls for pinpointing prey, while others use more omnidirectional calls to navigate cluttered environments like dense forests. These adaptations demonstrate the incredible versatility and effectiveness of echolocation as a sensory tool for survival.

In addition to bats, several other animals use echolocation to navigate and hunt, including dolphins, porpoises, whales (such as sperm and beluga whales), certain species of shrews, and swiftlets. One notable human who uses echolocation is Daniel Kish. Blind since infancy, Kish developed the ability to navigate his surroundings by making clicking sounds with his tongue and interpreting the echoes that bounce back from objects around him. This technique, known as human echolocation, allows him to "see" the world through sound, enabling him to walk, ride a bike, and engage in various activities independently. Kish has also founded the organization World Access for the Blind to teach echolocation to other visually impaired individuals.

LiDAR Sensor VS Echolocation

The relationship between LiDAR sensors and bat's echolocation lies in the fundamental principles of how they both detect and map their surroundings through the reflection of emitted signals.

Both LiDAR sensors and bats emit a signal that interacts with objects in the environment and then capture the reflected signal to gather information—LiDAR emits laser pulses, while bats emit ultrasonic sound waves. The time it takes for the signals to return provides data about the distance and shape of objects. Both technologies use the returned signals to create a map or image of their surroundings; LiDAR creates detailed 3D models of environments by calculating the distance between the sensor and objects, while bats interpret the time delay and intensity of the echoes to construct a mental map of their surroundings, aiding in navigation and hunting in the dark. Both methods allow for high accuracy and precision in detecting objects and their positions, with LiDAR measuring distances with great accuracy and producing high-resolution 3D models, and bats detecting tiny insects and avoiding obstacles in complete darkness.

EchoVision leverages this similarity by using sound visualization through LiDAR sensor’s reconstruction of the environment to simulate the echolocation experience, drawing a parallel between human-made technology and natural biological processes.

The Contemporary Austin - Host: Fusebox Program

We premiered EchoVision at The Contemporary Austin as part of the Fusebox Festival. The Congress Avenue Bridge in Austin, Texas, famously hosts the largest urban bat colony in the world, with roughly 1.5 million Mexican free-tailed bats residing beneath it every summer. Our performance started under the bat bridge with our collaborator, the bat conservationist Merlin Tuttle, who saved the 1.5 million bats under the Austin Bridge from extermination. About 500 people came. They listened to Merlin’s captivating stories with bats, watched the huge number of bats flying out of the bat bridge after sunset, and experienced EchoVision to be like a bat and to have a rare glimpse into the bat's perceptual universe.

Then, the adventure progressed with a walk straight down Congress Avenue to The Contemporary Austin Museum, where Jiabao Li and Matt McCorkle performed a live bat-inspired soundscape event called "Nocturnal Fugue." Nocturnal Fugue uses bat vocalizations—from social calls to mating rituals—to artistically create a mesmerizing sonic experience. This soundscape is combined with visual projections that represent the environment in which the specific bat call is being made or in response to. At the rooftop, we served Tequila. Bats are agave pollinators. They are part of the key process that turns agave flowers into tequila.