Category Archives: directshow

CamMuxer and Tokyo Demo Fest 2013

For the last two weeks I had been working with Zavie on bringing CamMuxer to next level of perfection by making it slightly more compatible with various hardware configurations, not just with my box which – quite accidentally – used to play a role of the streaming server last WeCan 2012 🙂 When ran on configuration that meets all the weird requirements I enforced in Emerald engine for various reasons (including, but not limited to: laziness, time management efficiency, NViDiA fanboyism – pick any or all options of your interest 😉 ), the tool is – how unhumbly of me! – quite stable, due to the fact that no client-side/video memory allocations are done within the rendering pipeline (excluding “camera plugged in” event handlers, but let’s not go too hardcore). The tool is also quite fast, since it has been written with multi-threading in mind from the very beginning – each camera gets its own rendering context, updates texture objects in its own time slice, and is completely separated from the main rendering thread (which refreshes with 60 FPS frame-rate).

Zavie contacted me a couple of weeks ago and asked me if I could give him a hand with setting up a web-stream for his Tokyo Demo Fest 2013 demo-scene party, which is happening next week-end in Japan. We started off with the tool crashing hard when used with nearly all cameras Zavie had at hand (and the time zone differences were not helping either, as Japan is 8 hours ahead of Poland!). After a few days it started to be clear that if you are ever to work with web-cameras and DirectShow, you *need* to have access to various models as there’s close-to-none information as to what to expect from the cameras out there on the market. Here’s a couple of sample assumptions I made.. which turned out to absolutely *wrong*:

Axiom: All web cameras out there obviously expose RGB24 feeds;
Truth:

Some of the web cameras we had a chance to work with only exposed YUY2 feeds. While YUV is not particularly difficult to tackle (as in: to convert to RGB format), this assumption was probably the most deadly of them all, as it soon proved to have been but a tip of an iceberg. As we continued to heat the ice tile, it didn’t take much time to find out there were some pretty nasty bugs down there, which were related to switching between feed formats, and which had to take some time in order to be remotely debugged and fixed.

Axiom: All feeds that web cameras expose can be freely launched;
Truth: 

One of the web cameras we had the pleasure to work with happily informed of a twice Full-HD resolution feed, available at 30 FPS. Considering the bandwidth of USB2.0 bus and amount of traffic the feed would cost per second, this seemed rather immoral. And, well, guess what – it wasn’t real at all. In fact, if you requested for it, the driver instantly reported an error. Still, that feed must have made quite a couple of good screen-shots (marketing-wise 😉 )

Axiom: Switching between media formats is a piece of cake;
Truth:

Switching between different resolutions for the same format does work flawlessly. However, if you ask Media Control interface to switch to a feed using different data-type, things take a different turn. What you *usually* get is an error – can’t do, nope. The only action you can take from this point is to re-instantiate the whole DirectShow graph.. and that’s slippy from the stability point of view 🙁

Axiom: No need to care about native feed’s data format – I can always ask DirectShow to create a graph that will feed the sample grabber with RGBxx data;
Truth:

 

I agree that it is usually worth to ask as many questions as you can (assuming they make the picture clearer, that is) but DirectShow does not have a magic wand at its hands. Some of the cameras we had access to exposed H.264 feed which is quite unfunny to work with if you need to quickly create a GPU-specific solution. These feeds could not have been converted in software to RGB24 – perhaps we missed some filters. But since the rule of a thumb is to always assume the worst-case scenario, I quickly dropped support for the feed format.

All in all, CamMuxer v2 is now finished and it’s very likely going to be used for streaming purposes at the party. I’m starting to seriously consider publishing the tool for all the party organizers out there who would like to see a multi-camera feed broadcasted in the network but don’t have the necessary tools of the trade (or monies to buy the licenses) to do so. If you happen to be looking for a similar solution, drop me a line – the only reason I haven’t put it up on the web-site yet is that I’m afraid I wouldn’t be able to handle all the bug reports that might start to flow in the moment more than just one or two folks start to play with it. Looking at the number of problems we had with just a number of cameras, it’s.. inevitable to happen 🙂

While trying to get the software to run, some new ideas on how to expand the tool so that it’s even more useful to party organisers came to my mind, so you can expect some more posts about CamMuxer in the next months 🙂

Bug-fixing: DirectShow and Flash Player integration

Trying to integrate your DirectShow input filter with Adobe Flash player and can’t get it work? Chances are you started out with Vivek’s sample implementation which features a not-so-trivial omission which causes Chrome to simply refuse playback of the video stream, even though no error is shown. What’s worse is that other DirectShow-dependent applications (Adobe Live Encoder, Skype, VirtualDub, etc.) will continue to work – it seems as if there was something wrong with Flash.. but then other drivers work – how come?

The problem is – as usual – banal 🙂 It’s one of those issues that are difficult to spot but trivial to fix, once you know where to look. In our case, we need to focus at:

HRESULT STDMETHODCALLTYPE CCamMuxerStream::SetFormat(AM_MEDIA_TYPE* media_type_ptr)
{
   IPin* pin_ptr = NULL;

   m_mt = *media_type_ptr;
   
(...)

Easy peasy – the original example does not do a null-pointer check! If you return S_OK whenever media_type_ptr is NULL, things go extra fine! Just like in the gif animation below:

1357589524021