Can You Recover Sound From Images?
Is it possible to reconstruct sound from high-speed video images?
Part of this video was sponsored by LastPass: http://bit.ly/2SmRQkk
Special thanks to Dr. Abe Davis for revisiting his research with me: http://abedavis.com
This video was based on research by Dr. Abe Davis and colleagues. I found out about this work years ago and was fascinated by the way he was able to capture vibration information in image-only video. I always imagined the motions of objects would be visible as when recording a tuning fork in slow motion - so deriving sound from high speed images seemed a feasible task. But the reality is much more difficult.
Sound vibrations only cause objects to wiggle by about a micrometer. This is much smaller than a pixel, so the algorithm must understand the characteristics of the image. A move in one direction should cause some pixels to lighten slightly, while others darken - and this behavior is correlated along the edges of the image. So noise can be reduced because it's random over the image and there are enough places to sample that you can get it to cancel out.
Something I'm wondering now is - would it be possible to capture sound in a single image? I'm thinking it would have to be an image of a large object or space because the wavelengths of typical sounds are quite long. Maybe a high frequency sound could be imaged in a suitable medium...
Animations by Alan Chamberlain
Music from http://epidemicsound.com "Seaweed"
Видео Can You Recover Sound From Images? канала Veritasium
Part of this video was sponsored by LastPass: http://bit.ly/2SmRQkk
Special thanks to Dr. Abe Davis for revisiting his research with me: http://abedavis.com
This video was based on research by Dr. Abe Davis and colleagues. I found out about this work years ago and was fascinated by the way he was able to capture vibration information in image-only video. I always imagined the motions of objects would be visible as when recording a tuning fork in slow motion - so deriving sound from high speed images seemed a feasible task. But the reality is much more difficult.
Sound vibrations only cause objects to wiggle by about a micrometer. This is much smaller than a pixel, so the algorithm must understand the characteristics of the image. A move in one direction should cause some pixels to lighten slightly, while others darken - and this behavior is correlated along the edges of the image. So noise can be reduced because it's random over the image and there are enough places to sample that you can get it to cancel out.
Something I'm wondering now is - would it be possible to capture sound in a single image? I'm thinking it would have to be an image of a large object or space because the wavelengths of typical sounds are quite long. Maybe a high frequency sound could be imaged in a suitable medium...
Animations by Alan Chamberlain
Music from http://epidemicsound.com "Seaweed"
Видео Can You Recover Sound From Images? канала Veritasium
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Math Has a Fatal FlawThe World in UVThese are the asteroids to worry aboutThese Pools Help Support Half The People On EarthThis Unstoppable Robot Could Save Your LifeMaking Liquid Nitrogen From Scratch!The Illusion Only Some People Can SeeWhy Einstein Thought Nuclear Weapons ImpossibleCan Silence Actually Drive You Crazy?This is why we can't have nice thingsWhat Exactly is the Present?The Bizarre Behavior of Rotating BodiesThis Toy Can Open Any GarageHow to Slow Aging (and even reverse it)Why Machines That Bend Are Better3 Perplexing Physics ProblemsWhy Metals Spontaneously Fuse Together In SpaceThis Particle Breaks Time SymmetryMy Video Went Viral. Here's WhyCan Humans Sense Magnetic Fields?