A high-speed camera records the tiny vibrations sound produces in a thin surface. A seemingly still bag of chips becomes a visual record of sound.
Article
A camera looks at a bag of chips through soundproof glass. Someone speaks beside the bag, but the camera records no audio. Analyzing its images nevertheless recovers intelligible speech. Researchers from MIT, Microsoft, and Adobe presented this “visual microphone” experiment in 2014. They measured movement in the bag's surface rather than hiding a microphone inside it.
Sound travels as changes in air pressure. When those changes reach an object, its surface moves slightly. Thin, light materials can vibrate with sound even when the movement is too small to see. The researchers reasoned that movement captured on video could provide a way back to the sound that caused it. They tested plant leaves and aluminum foil as well as chip bags.
The method does not depend on a conspicuous movement. It compares the same locations across frames and combines subtle changes. Motion smaller than a pixel can still affect recorded color and brightness. Changes around an edge provide clues to the surface's movement.
The researchers arranged the recovered vibrations over time and converted them into an audio signal. The speech-recovery experiment behind soundproof glass used a high-speed camera, because rapid sound vibrations need sufficiently frequent measurements. The result does not mean conversation can be extracted from any ordinary video.
Material affects the outcome. A thin, easily moved surface responds differently from a hard, heavy one. Distance, lighting, frame rate, and other nearby motion also matter. Recovered sound may contain noise and distortion or reveal only part of the original signal. The team therefore tested a range of objects.
The experiment measures a physical trace left by sound on an object. The published demonstration compares original and recovered audio. Wrinkles in a bag that seemed motionless had registered tiny vibrations in the camera's images.
Related Video: The Visual Microphone: Passive Recovery of Sound from Video
Produced by: Abe Davis's Research
https://www.youtube.com/watch?v=FKXOucXB4a8
Image description: MIT illustration of research that recovers sound from tiny vibrations of a potato-chip bag
Image source: Christine Daniloff / MIT · Research illustration
https://news.mit.edu/2014/algorithm-recovers-speech-from-vibrations-0804
Sources and editorial notes
Abe Davis's Research · The Visual Microphone: Passive Recovery of Sound from Video
Video channel · Abe Davis's Research
Editorial note: Factual statements are based on the sources above; interpretations reflect MUADAILY's editorial perspective. Image: Christine Daniloff / MIT · Research illustration.
Photograph / image source
Christine Daniloff / MIT · Research illustration
Image description: MIT illustration of research that recovers sound from tiny vibrations of a potato-chip bag
Usage terms: https://creativecommons.org/licenses/by-nc-nd/3.0/
Original aspect ratio preserved; resized and reformatted for web display
Machine-translated from Korean. Read the original.
