The quest for a new video codec in Flash 8

Here and there I sometimes see disappointed comments about the fact that we did not pick H.264/AVC as our next generation codec. This in their opinion would have provided the ultimate video quality based on a widely adopted industry standard. Every time I feel compelled to explain the long process we went through to find a better video codec. Quality and standards are just two criterias we used and while they are important others were much more important. Let me put together a (incomplete) list here. I can't talk about all the gory details obviously, that would get me into trouble :-) (especially how I think each codec did on these points)

1. Quality. This is the first thing we looked at and our target was to eat least cut the bandwidth in half while keeping the same visual quality.


2. Code size. You wouldn't believe the difference we saw here. Everything from a few kilobytes to a megabyte. Download size is one of the strengths of Flash and we had to keep the code as small as possible.


3. Portability. We do not only need to support Intel, but also PowerPC, ARM, MIPS and many others. Recompiling for a new platform had to be painless and essentially require no code changes. Optional availability of specialized code was a plus too, although we could have done some of that work ourselves.


4. Stability. This does not only mean crashers, but an ever changing source code base or file format are real problems when creating SDKs.


5. Legacy hardware support. It's nice to have a new shiny video codec, but if it does not run on an older Macintosh what's the point? Flash is about ubiquity, not forcing people to upgrade hardware or even require specialized hardware. Our target was a Pentium III 500Mhz and a Mac G3 running at 800Mhz.


6. Hardware support. Looks like it's a conflict with the previous item, but it's not. We were looking for a codec which could benefit from standard graphics hardware in the future, things like iDCT, YUV conversion/display, motion compensation etc. A lot of experimental codecs failed miserably here.


7. Performance. When your CPU usage doubles during complex scenes I consider a codec to have a serious performance problem. In some cases the video codecs we tested were dropping frames on a 3.4Ghz Pentium 4!


8. Completeness. This mostly affects standards based codecs. If only half of the specification is implemented why even claiming to be compliant? We went this route before with Sorenson Spark which is an incomplete implementation of H.263 and it bit us badly when trying to implement certain solutions. Many codecs failed on this one.


9. Strong support. We were looking for a codec which had excellent support from the vendor, including the ability to come up with customs solutions very quickly, both on the client and deployment side. They also had to have the ability to support not only us, but any 3rd party interested in Flash Video. A vendor which saw us as just another potential to dump their prefabricated closed solution on us was simply not interesting. Our goal is to create a complete ecosystem around Flash Video with as many players as possible.


10. Good encoding tools. Another lesson we learned is that good encoding tools are essential for customers. If the vendor is able to provide alternatives to ours, even better.


11. Risks for Macromedia. We had to know exactly what we were getting into. A codec with an open ended license agreement which has to be renegociated every few years simply bear incalculable risks for a company the size of Macromedia.


12. Risks for customers. Same as the previous, but some codecs required to make a difference between the player and the video streams served by customers.


13. Costs for customers. If you have to pay a fee for streaming your video over the web it can be a real problem. I understand that this model works well for dedicated hardware and I support it. But how do you keep track of this on the web? This is Flash. It would be like asking for money everytime you use a certain HTML tag on your web pages.


14. ROI for customers. This was probably the most important of them all. Flash Video had to be cheaper and easier to deploy than any other solution out there.

All in all the On2 VP6 codec stood out on most of these points. We did not drop H.264/AVC because Macromedia is an evil company who likes everything to be proprietary, on the contrary. We were looking for the best balance on all of the above points. If the choice we made was the right one remains to be seen, but overall I am extremly happy, the video quality is really outstanding. Give it a try.