ICM Final Project
A project that sonifies the streets of cities all over the world. Sumi Lee and Chloe Choi have created different visualizations based on the generated sounds through text art and geometric shapes using P5.JS
Visualize with generated sounds
Initial Proposal
As Sumi and I are collaborating from opposite sides of the world, we wanted to create a program that would sonify the people on the streets and observe the variety of music it would create.
Our inspiration comes from these two projects on how they created sounds with people:
Sidewalk Orchestra
Pose Music
Our project will also have a infographic component to visually compare people’s behaviors across different cities
Progress
Prototyping Mechanics
We started off creating a sketch that would create notes when colliding with certain spaces using PoseNet. We were thinking of either a row or grid structure but chose the row when we realized many people seldom moved up and down.
Fleshing it out
We moved the project to a local file when we added the videos due to size limits.
We also added text to label each location and implemented Craig Reynold’s Steering Behavior on the text whenever a person walks by.
Our biggest problem was getting each video to load with their respective PoseNet working for multiple people.
User Feedback
Many people were wondering if the balls were creating the sound. An informative and visual effect ended up confusing the audience but we thought it would be ok since the text and video were not scaled and aligned properly at the time.
The project feels like you’re looking out the window
Should each city have different sounds? For example, using different scale or different instruments.
Future Endeavors
The feedback that our project feels like looking out the window really stuck with us. Due to everyone having to stay home, we thought it would be lovely if we can create a public website where anyone can use their camera to record the streets and hear music real-time. We can also allow people to look into other people’s currently active streams to look out of numerous windows.