All posts tagged javascript

SnowCam: A WebRTC experiment with getUserMedia

I’d heard about WebRTC and navigator.getUserMedia as a way to access webcams and microphones, so after I somehow ended up here the other day, I figured it should be possible to do the same in JS and HTML5.

The basic idea was to take a webcam feed, apply a falling snow effect to it, and let the snow settle on any horizontal edges in the video. Just show me already!

Connecting it all up

Getting the webcam connected up to a <;video>; tag was pretty easy, just make sure to use the right vendor prefix and take note of the slight differences between the supported browsers (Firefox Nightly, Chrome, Opera). There is a shim available called The gUM Shield, but I found that after I’d put something together.

Next step was to apply some snow, so I placed a <;canvas>; tag over the top of the <;video>; tag, found simple example of particle generation, and had some snow.

Getting the snow to settle was a bit trickier. A bit of searching revealed that I would want to use a sobel convolution filter (Nice tut here) to find the edges in the current frame, and rather than combine the results from the 2 computations (horizontal and vertical) and place onto a canvas, just take the horizontal edges and put into a hash that I can check when placing the snow flake.

My first attempt at this ran pretty damn slow, the snow effect was stuttering the whole way through. That kind of made sense since I was trying to update ~300 flakes constantly on 1 canvas, and applying a fairly heavy computation to each frame of the webcam on another. This seemed to me to be the ideal time for a WebWorker. I moved the edge detection into the worker, and then just post the resulting hash back which gets injected into the snow object, this way as the snow is falling I can just check if there is a ledge to settle on by checking this.ledges[x][y] and if it exists we can ‘stick’ to the current x,y position, otherwise assume that it is falling – this lets us wipe away the snow from ledges as well.

This works pretty well, but I’m sure it can be optimised a fair bit more – there are some extra elements I probably don’t need to use, and some processing that could be removed – but for the moment I think it’s quite cool being able to see the various stages. I’ve tested this on Firefox Nightly, and the latest version of Chrome. It does run on Opera, but very slowly – I should probably look into that…

There are a couple of enhancements I want to add at some point:

  • Snow piling up when it lands on an existing flake
  • Connect to the microphone to allow the snow to be blown away

Give it a whirl

See it in action: http://ben.periton.co.uk/exp-SnowCam
Get the code: https://github.com/benperiton/SnowCam

If you are using Firefox nightly, you will need to enable the feature:

  • Type “about:config” in the address bar and say yes that you want to make changes
  • Find the “media.navigator.enabled” entry and set it to true

y