Artist Uses iPhone ARKit to Visualize Sound in 3D Space

Under a minute read
| Cool Stuff Found

Zach Lieberman is an artist who is exploring how to create art using iPhone ARKit. His latest creation? Recording audio in space. In the demo video, Zach makes sounds like “woosh, psh, ah, click.” After each sound, a white blob bursts into the air, and as Zach walks backward, each blob is linked to the other blobs like a audio timeline. When he walks forward again through the trail, you hear each sound playing in reverse. Zach, who helps run the School For Poetic Computation in New York City, built the demo using Simultaneous Localization And Mapping (SLAM). It uses the iPhone’s sensors and camera to create a low-res map of the room. The app records sound with the microphone, processes and visualizes it, and then maps each sound blob to a location within the room.

Quick test recording audio in space and playing back -- (video has audio !) #openframeworks

A post shared by zach lieberman (@zach.lieberman) on

2 Comments Add a comment

  1. Jamie

    That’s cool, but again I have to ask myself what the practical use might be. So far, just as with VR and outside of perhaps specialized applications, once the novelty has worn off, most of the stuff that’s being touted pretty much looks like a tech demo destined for entertainment purposes. Though I enjoy entertainment, I do not live my life for it. Remember when tech actually served a noble purpose in our society? I’m filing this one right alongside Car Karaoke.




    0
    • Andrew Orr

      The way I look at it, right now all of these demos are just showing the possibilities of AR, but we won’t see the real stuff for another year or so, such as a “killer AR app.”




      0
Add a Comment

Log in to comment (TMO, Twitter, Facebook) or Register for a TMO Account