Video, Canvas, Worker thread - A movement tracker
By Paul Rouget on Friday 20 February 2009, 19:23 - shared - Permalink
About User Experience, Firefox 3.1 is going to be a minor release. But about standards and performance, it's a great step forward.
Check out the new features you can use in your web page: Firefox 3.1 for developers.
I've created a little demo using the video element, canvas, DOM Workers (threads in JS) and a few Javascript 1.7/1.8 tips. It's a movement tracker. It's not perfect, but it's fun :)
Movement tracker: http://people.mozilla.com/~prouget/..., click on the Go button (Firefox3.1 needed).
How does it work ?
For each frame:
- First, I compute the difference between the frame n and the frame n - 1 (greyscale);
- then (in the Worker), I do a matrix convolution to detect pixel moving (you can choose between 3 kind of matrix: Sobel, Harris and Kirsch);
- From the result, I compute the bounding box containing moving pixels (rects displayed in the video) and the centroid of the moving pixels (crosses inside the rects);
- There is a bounding box and a centroid for the x movements (color red) and for y movements (color green);
- The evolution of the weight of each centroid is displayed in the graph;
The arguments:
- Threshold: when a moving pixel should be ignored;
- Work factor: we don't need to compute the entire frame. We can reduce it;
- Zoom factor: the resulting video is not a <video/>, but a canvas. Define the ratio between the canvas size and the original video size;
- Sensibility: when a global movement should be ignored. Size of the bounding box;
- Display video: display the original <video/> element;
- Display diff: display the diff frames;
It's a really quick&dirty implementation, if I have time (or you, if you have some time), I think a good enhancement would be to implement the camshift algorithm to track objects.
Comments
Paul - this is really cool!
I love this demo!
I love too; impressive!
This is really nice, but I'd be careful touting features like <canvas> and <video> as standards compliance as they are not (yet) parts of actually somewhat final and published standards.
The progress in standards compliance can be seen with e.g. Acid3, which still shows impressive progress.
The stuff used here is more "experimental HTML features" than standards compliance - which doesn't make them less interesting or that movement tracker less impressive though :)
I test the demo with Firefox 3.1 beta released by Mozilla (the link you give), and I can't redo that we could watch in the Chris Blizzard's video.
Just like ReLuc I'm unable to see anything using Firefox 3.1b2 (EN/US)
The link doesn't seem to be working anymore. Do you have the code mirrored somewhere I could look at?
Wow! Paul Its really revolution...
Thanks for sharing this nice article and obviously Source Code too.