Okay, now what is this all about?
For the last couple of years, I have been busy working on some, perhaps unusual for some, demo applications and tools that might be of interest for the readers. All applications presented below were all created in an over-night manner, with no special commercial purpose. They helped me (and continue to do so 🙂 ) keep my gray matter active in the field of 3D graphics and GPGPU that I happen to be genuinely interested in, but not necessarily deal with at my current workplace.
(If you happen to believe that some of the tools presented here might come in handy for your needs, please drop me a line and we’ll see if something can be arranged)
Demoscene, which is nothing but a large group of computer freaks who are sociable enough to gather in teams in order to compete their combined art & IT skills against those of other groups’, has been fuelling my passion for programming since ever. It takes quite a lot of effort to create a single production from scratch (using free-ware frameworks that many Indie companies rely on or (doh!) Java or even C# is not keenly accepted on the demoscene) but it definitely pays off. Not only are the hands clapping after your demo finishes playing during the competition (or boos if you are unlucky and your production maliciously crashes 🙂 ) an unforgettable experience, but, after the dust settles, you walk away with a lot of experience in:
- coordinating work of a group of people who have their better and worse days
- speed programming (you never forget the joy of bug-fixing 5 minutes before the dead-line, trust me on that)
- various fields related to multimedia programming.
From the very beginning, I had been the ambitious type of a guy who tried to pack all functionalities needed to create a real-time demonstration in a framework (or, more precisely, a single DLL, for those who care). In hindsight, this was probably one of the biggest mistakes I’ve ever done when it comes to programming, which has severely slowed me down on many occasions. ..but then, you live and learn 🙂
Was it not until 2009, when I managed to release my first demo which consisted of something more than “Yet Another Spinning Cube ©” effect at Riverwash 2009 party.
You can view the demo by clicking the screenshot – it will take you directly to a Youtube link.
I’ll never know whether it was the drinks or fatigue after a few weeks marathon, but I slept through the whole demo competition 🙂 The demo landed second and, boy, was I proud and happy that one of my life-time dreams finally made its way!
From a programmer’s point of view, the demo doesn’t leave much to discuss. It relied on NViDiA’s CG library (to simplify shader management), made use of frame-buffers and DirectShow (for streaming multimedia content). Simplistic from the start, you could say, but its purpose was mainly to give me an idea about what fields I should focus on broadening my knowledge in in order to get better.
As I mentioned before, “Three Fourth” relied on a certain framework I wrote specifically for the purpose of simplifying the process of creating demos. The framework, by itself, is something I discussed thoroughly in my Master’s Thesis. Long story short, its fundamental assumption is that demo content is predefined, hence rendering pipeline can be described by an acyclic directional graph, where each node accepts certain amount of inputs and provides at least one output pin. That, in itself, would be limiting, if there was no concept of segments available. The programmer is welcome to put as many video segments on the time-line as he/she wishes, provided that there is only one video segment at a given time. A bell or two might have just rung for those of you who have ever worked with video cutting.
The framework takes care of all the ugly details every OpenGL application has to deal with such as communication with the system, window creation, OpenGL context(s) initialization, multi-dimensional curves that support various interpolators, and so on. What matters is that the programmer is expected to be left to focus on the demo itself, rather than struggling with all the beauties of multi-media programming that many programmers have to deal with on a daily basis. ..or so I thought. 🙂
“Three Fourth” was the one and only demo that I wrote “by hand”. What this means is that every single cut required a separate segment implementation, not to mention synchronising with audio, which took another large time share on its own. Both of these turned out to be monstrously time-consuming, so by the time I’ve finished working on TF, I knew something had to be done or else I’d have to find myself another hobby – quickly!
Not being much of a skilled person, when it comes to gardening or cooking, I started seeking for a solution. What I came up with was a large piece of software called “Tomb” which allows to:
- add/remove audio and video segments on timeline ;
- configure curves, add/remove curve segments, move segment nodes;
- create/modify/delete graphs, as well as to manipulate node properties;
- see effects of your actions in real-time (this also includes texture viewer, which saved an unmeasurable amount of time spent on shader debugging!)
- store and load project files
And a handful of other things that are better to demonstrate with pictures than by words. So, here they are:
Tomb has been in constant development for nearly three years. It laid ground for two other demos (presented below) I have created with folks from Elude group and it saved a *lot* of typing work that would otherwise had to be done. However, as with all large projects, at a certain moment it started reminding me of work I usually do in the office: there simply were too many features to keep track of, and that was something important to consider when extending the architecture. Things stopped being fun, and with demoscene supposed to be a hobby, not a nightmare, I decided to throw in the towel and start all over again, this time with much different approach.
As we are about to leave the demo sector, let me just present you a few pictures from the other two demos I’ve mentioned above (for Youtube links, just click the screen-shot):
How about something.. err, practical, for a change?
All well and good but there happen times when you just have to roll up your sleeves and do something useful for the world. This chapter will serve as a placeholder for a quick show-reel of some of the applications I wrote for non-commercial purposes.
CamMuxer (written for WeCan 2012)
WARNING: The description below is outdated and is to be updated in a few days’ time.
One of the interesting observations I made while preparing software for real-time audio/video web stream for Riverwash 2011 was that it was dreadfully difficult to find free software, that would allow you to mix video streams from more than just a single web-camera (not to mention over-the-top needs like embedding a logo on top of the whole video stream!). Believe it or not, it took no less than a few days of googling and browsing through some of the darkest 😉 corners of the Internet, before I managed to find an application that was freeware for non-commercial purposes. It installed itself as a video output filter that you could use as an input for Adobe’s Live Encoder (which is quite a nice piece of software, once you learn all its nuts and bolts, and if you can excuse its rather slow software-only implementation!) and allowed you to configure contents of a few screens the application switched between every predefined amount of time.
However, it had a few drawbacks:
- It was CPU-only, hence it was *SLOW*. No, really, for 4 web cameras connected to USB (which really matters, because USB cameras require CPU intervention in order to push data through the bus, and you need to see that happening at least 25-30 times a second for an acceptable experience) we had at Riverwash 2011, we had a constant 100% utilisation of 3 cores (out of 4) of an Intel Core2 Quad CPU. With absolutely no hyper-threading involved, one can see that we’re talking about some serious data-pumping business here, not to mention the usual time-consuming image processing routines as well!
- It wasn’t stable. We were getting random camera lock ups every now and then. In order to get it back, you had to restart the whole application and – since it was acting as a sink for the Encoder – it wasn’t really a blink of an eye kind of thing. While striving to get things back on track ASAP, you literally saw these fingers tapping on the keyboards as the stream had to go down and back up every couple of hours.
With WeCan 2012 getting closer and closer, I came up with an idea to write my own piece of software, that would serve exactly the same purpose and – what perhaps is just as much as important – would be extendable. After a few weeks of having fun with DirectShow, WinAPI and OpenGL 3.3, it finally arrived. 🙂 Here’s a quick sum-up of what the application can do:
- Supports up to 4 video inputs, handled in completely multi-threaded manner. Video inputs can be dynamically reconfigured.
- Frames from input RGB24 streams are uploaded to VRAM and then used for creating per-frame output by means of a few fragment shaders.
- Each input stream can have its brightness/contrast/gamma changed, and each feed can be shown with user-defined alpha factor.
- Outputs I420-encoded frames (conversion to YUV color space is fully performed on GPU) for direct Adobe Live Encoder’s usage.
- Includes a separate capture filter that is required for Adobe Live Encoder cooperation. Uses a file mapping for IPC purposes.
- Performance? With same hardware, CamMuxer has an average 25-30% CPU utilisation (mainly due to memory transfers that webcams’ video input filters do and that you can’t really work around).
Here’s a screen-shot or two, presenting the application in action.
Other mini-projects worth mentioning:
– Parallelized OpenCL Kd-tree-ray intersection algorithm implementation (used for SH coefficient calculations)