Saturday, August 13, 2005

The quest for a new video codec in Flash 8

Here and there I sometimes see disappointed comments about the fact that we did not pick H.264/AVC as our next generation codec. This in their opinion would have provided the ultimate video quality based on a widely adopted industry standard. Every time I feel compelled to explain the long process we went through to find a better video codec. Quality and standards are just two criterias we used and while they are important others were much more important. Let me put together a (incomplete) list here. I can't talk about all the gory details obviously, that would get me into trouble :-) (especially how I think each codec did on these points)

  1. Quality. This is the first thing we looked at and our target was to eat least cut the bandwidth in half while keeping the same visual quality.

  2. Code size. You wouldn't believe the difference we saw here. Everything from a few kilobytes to a megabyte. Download size is one of the strengths of Flash and we had to keep the code as small as possible.

  3. Portability. We do not only need to support Intel, but also PowerPC, ARM, MIPS and many others. Recompiling for a new platform had to be painless and essentially require no code changes. Optional availability of specialized code was a plus too, although we could have done some of that work ourselves.

  4. Stability. This does not only mean crashers, but an ever changing source code base or file format are real problems when creating SDKs.

  5. Legacy hardware support. It's nice to have a new shiny video codec, but if it does not run on an older Macintosh what's the point? Flash is about ubiquity, not forcing people to upgrade hardware or even require specialized hardware. Our target was a Pentium III 500Mhz and a Mac G3 running at 800Mhz.

  6. Hardware support. Looks like it's a conflict with the previous item, but it's not. We were looking for a codec which could benefit from standard graphics hardware in the future, things like iDCT, YUV conversion/display, motion compensation etc. A lot of experimental codecs failed miserably here.

  7. Performance. When your CPU usage doubles during complex scenes I consider a codec to have a serious performance problem. In some cases the video codecs we tested were dropping frames on a 3.4Ghz Pentium 4!

  8. Completeness. This mostly affects standards based codecs. If only half of the specification is implemented why even claiming to be compliant? We went this route before with Sorenson Spark which is an incomplete implementation of H.263 and it bit us badly when trying to implement certain solutions. Many codecs failed on this one.

  9. Strong support. We were looking for a codec which had excellent support from the vendor, including the ability to come up with customs solutions very quickly, both on the client and deployment side. They also had to have the ability to support not only us, but any 3rd party interested in Flash Video. A vendor which saw us as just another potential to dump their prefabricated closed solution on us was simply not interesting. Our goal is to create a complete ecosystem around Flash Video with as many players as possible.

  10. Good encoding tools. Another lesson we learned is that good encoding tools are essential for customers. If the vendor is able to provide alternatives to ours, even better.

  11. Risks for Macromedia. We had to know exactly what we were getting into. A codec with an open ended license agreement which has to be renegociated every few years simply bear incalculable risks for a company the size of Macromedia.

  12. Risks for customers. Same as the previous, but some codecs required to make a difference between the player and the video streams served by customers.

  13. Costs for customers. If you have to pay a fee for streaming your video over the web it can be a real problem. I understand that this model works well for dedicated hardware and I support it. But how do you keep track of this on the web? This is Flash. It would be like asking for money everytime you use a certain HTML tag on your web pages.

  14. ROI for customers. This was probably the most important of them all. Flash Video had to be cheaper and easier to deploy than any other solution out there.


All in all the On2 VP6 codec stood out on most of these points. We did not drop H.264/AVC because Macromedia is an evil company who likes everything to be proprietary, on the contrary. We were looking for the best balance on all of the above points. If the choice we made was the right one remains to be seen, but overall I am extremly happy, the video quality is really outstanding. Give it a try.

26 Comments:

Blogger No one said...

So why was On2 VP7 not used? Seems like an even better solution, no?

Monday, August 15, 2005 7:00:00 AM  
Blogger Alain Duchesneau said...

Portability?

If Macromedia got the VP6 working on PowerPC, how come that on2 didn't?

Idleness?

Friday, August 19, 2005 7:05:00 AM  
Anonymous Anonymous said...

Word on the street is that VP7 was a big increase in CPU usage over VP6, with a rather modest increase in quality.

Thursday, September 01, 2005 3:14:00 AM  
Blogger Compressionist said...

H.264 verses VP6
VP6 was a bad choice over H.264.
- Quality
- Compare MainConepts H.264, Moonlights H.264, or Sorenson's H.264 to VP6. They are all better looking than VP6

- System resources for encode
- VP6 brings my system to a crawling halt and no other work can be done and the encode takes A LOT longer than H.264. Twice as long in some cases.

- System resources on decode
- Large H.264 files play smoothly when VP6 files with the same encoding settings stutters or halts in high action scenes.

- unpredictable encoding results
- Use andy VP6 encoder for Flash that you want. Change the keyframe rate. The data rate is affected. Somtimes drastically. Why? I have never seen this in any other codec. The data rate should not be affected by the amount of desired keyframes. The quality should be affected.

- Devices
- I want to see Flash everywhere. Cell phones, DVD players, and other devices are going to support H.264 and Flash could have taped into that. Those devices are not going to support VP6.

- Conclusion
VP6, although pretty good, is not as good as H.264. Quality is lesser, system resources are taxed, encoding results are inconsistant, and VP6 does not take Flash into other markets like H.264.

Thursday, September 15, 2005 3:02:00 PM  
Anonymous Anonymous said...

CNET Review of Flash w/ VP6

From CNET's review of Flash 8:

To better compete in the streaming-video arena, Macromedia renovated the Flash Player from the ground up, using On2 Technologies' VP6 audio and video compression codec. We experienced faster and smoother animation than with Flash MX 2004; our Flash 8.0 animations were on a par with or better than what QuickTime, RealPlayer, and Windows Media Player deliver. The improved optimization of Shockwave Flash (SWF) files gives the Flash 8.0 Player an advantage over such rivals.

Scoop: Macromedia Flash 8.0 review

Friday, September 16, 2005 9:00:00 AM  
Anonymous Ryan said...

How important was Audio-Video synch? Obviously it may be too early to tell, but an interview I read in DV Magazine said that one of the things people complained about with MX was that Flash video drifted in and out of synch. At On2's site they have posted some new Flash 8 samples:
http://www.on2.com/video_samples/flix-video-samples/

And, if you're a discerning viewer, you can tell that the audio lags the video in places (Starsky and Taking Lives trailers), even with the new codec. This is a pretty important issue, perhaps more so to me (as an editor), but if you compare the trailers to their Quicktime counterparts on Apple's site, QT clearly wins in THAT area.

Could this be the audio format the On2 used, or is already a weakness of Flash 8?

Friday, October 07, 2005 10:46:00 PM  
Anonymous Anonymous said...

Tinic,

In the hope you might be reading this I was wondering if you could shed some light on a couple of questions regarding the video encoder in Flash 8 Pro.

I have been doing some tests between On2, Sorenson Media Compression Suite and Flash Encoder. All with filters turned off. Source file was a DV AVI 25 fps and a WMF of the same clip at 25 fps @ 3.5 Mbit/sec. .

All encoded to flv on6VP at 25 fps. 700 kbit/sec.

Here my observations:

1) Sorenson Media (onV6 demo plugin):
Regardless of what settings or data rate I use I don't seem to be able to get the quality up. In addition encoding times are can be upto 15 or 20 times the clip length. in some cases much more Eg. encoding at 700 kbit/sec took 1 1/2 hrs.
Also, there are options in the encoder interface not offered by Flix or Flash 8 encoder.
Nice user interface though.

2) Flix 8 (demo):
Encoding speed is better than Sorenson Media but still quite high. AT 700kbit/sec. the encoding speed is almost on par with the Flash 8 encoder. eg. 50 minutes for a 5 minute clip at 700kbit/sec. 1pass VBR.

The interface is pretty ugly and not very userfriendly. Filter options are not available with batch encoding.

3) Flash 8 Encoder:
Now I am puzzled. Flash 8 encoder is the speediest of all three. the clip at 700 kbit/sec took 35 minutes to encode. User interface is simple put effective especially if you pre-process the video in an editing suite anyway.

As for quality I find it difficult to see a huge difference between what Flash 8 Encoder produces and what Flix 8 does in 2-pass encoding.

The main difference between the two seems to be that Flix applies some level of sharpening to the image while Flash 8 Encoder applies some sort of blur on image.

Unfortunatelly, Sorenson Squeeze plus on2VP6 plugin doesn't seem to get close as far as quality or speed is concerned.

Also, it seems that going past 1000 kbit/sec doesn't increase the quality with any of the applications. At 640x480 the sweet spot seems to be 700 kbit/sec.

Now my questions:

1) What sort of filters does Flash 8 Encoder automatically apply? Blurring/ etc. .

2) Can you second my observations regarding the datarates.

3) In a miniDV to FLV workflow it seems to be possible to encode from AVI to WMF to on2VP6 with the same end results as going from AVI to WMF. Probably because on2VP6 favours the way WMF encodes video vs. using the DV codec in an AVI. Can you second that?

To anyone reading this. Please do your own tests. I didn't do these with strict scientific intend. More like a real world approach. I think the difference in quality between the three is in the way each company has implemented the VP6 SDK into their product. Perhaps the good results Flash 8 Encoder delivers are due to the abilities of Macromedia's software engineers. Or maybe they just know the Flash format better.

Either way I am quite surprised and puzzled at the same time...

Thomas

Thursday, October 20, 2005 10:09:00 PM  
Anonymous Anonymous said...

Hi,
How do you calculate the compression ratio of a codec? (what's the formula for that).
Plus what's the compression ratio of
on2VP6?

Thanks
me:))

Sunday, October 30, 2005 8:07:00 PM  
Anonymous Anonymous said...

And , What about Linux support?

There is no player in Linux that play VP6 , how the Flash Player 8 for Linux would play this video files?

Thursday, December 01, 2005 11:02:00 AM  
Anonymous Anonymous said...

I did some encoding tests today for a client and wanted to share them.

Long story short, Flash 8 is half QT (.mp4) size, identical quality. WMV is the big winner in the "let's-see-how-big-we-can-get" competition.

http://joelcardinal.com/test/movie/movie.html

I probably won't keep this posted for very long so get while the gettins good.

Monday, December 05, 2005 7:37:00 PM  
Anonymous Anonymous said...

All in all Flash finaly has a decent video codec. Not the best maybe, but a still a decent one.
So why on earth did they stick to mp3 which is an antiquated codec when it comes to internet streaming ?
The solution should have been AAC+ in my opinion, or at least a good Vorbis implementation.
Here we have a state of the art video codec with an audio counter part that is at least two generations behind.
MP3 needs at least 128kbps too have decent 44Khz quality.
AAC+ could scrub that down to around 48kbps, more or less.
When you have a 512kbs video, you don't want the audio part taking one fifth of the total bitrate.

Sorry, but i'll stick to real for the time being for my live transmissions.

Thursday, January 05, 2006 3:25:00 AM  
Anonymous Anonymous said...

Could the reason that H.264 didn't make it in to flash due to the licensing fee that comes with it?

For a freely distributed plugin such as flash to have to cover a license charge isn't practical, but lets be aware that Apples Quicktime ships with MP4 capablities, and though WMP doesn't support standard MP4 at this point in time it does support the old MPEG2 which also comes with a licensing fee, so when other freely available media players are willing to impliment these codecs the 'not financial practical' point would seem a bit weak.

In hardware terms most modern mobiles support the MP4 (all be it with a different extension), but also H.264 is going to be included in the both the next generation of disc players: HD DVD, and Blue-ray.

The use of MP3 by flash (in its defense) still stands its ground today, solely for the reason that it is a widely accepted standard, but this point

At the end of the day it doesn't hurt to have another option for an online video standard, as it can be only a good thing to have options, and not all the features of MPEG4 standard are needed in a platform such as flash (MPEG4 covers more than just a video codec)

However imagine the possiblities of a flash based website that would allow the upload of video directly from a phone - a video version of Flickr. Doens't that kind of easy to impliment video solution sound like a dream?

Still I love improved quality that comes with the current codec, maybe we can see H.264 implimented sometime in the future??? Even with the addition of this codec it wouldn't get anywhere near the size of the current crop of Media Playing software that offers H.264.

Sunday, May 07, 2006 5:46:00 AM  
Anonymous James said...

Boo.

Why just one? Why not MP4/h264 as well? Why not choice?

And because of this choice you've made a developer's life - and the media-creating user's life - a little more of a burden. But that's expected right?

What you all have chosen is harder to implement over easier to implement. Shouldn't making your developer base and even your multimedia-creating end-user base the highest of priorites?

The conversion of video to this format is not easy: from finding reasonably priced conversion software options to incorporating a new proceses for creating, converting and publishing multiple formats which are required of video publishers, to the frustration of knowing that this too shall pass - and all the development time and money will soon be for moot when Adobe decides that the limitations of this codec, like the previous one, don't serve its base.

Boo.

Friday, May 12, 2006 3:23:00 PM  
Blogger m88runner said...

I need to do video conferencing (1 on 1) but the potential users could reach over 1 million. This cost, using flash media server, would be too much to be profitable on my end. Is there another solution without reverting to an html based pop-up or another page? I want to keep it all in my flash swf file.

Any ideas??
webstage@yahoo.com

Wednesday, May 31, 2006 7:47:00 PM  
Anonymous brad said...

For Adobe customers an important part of ROI that seems to have been ignored is codec/patent licensing.

MP3 is currently the only viable audio format for many situations, and it requires an encoder license for commercial products.

The player has H.263 for video, which is not the best quality, but free. Why not support Ogg Vorbis for audio so that MP3 licensing can be avoided? The SDK for Vorbis is under a BSD style license and the specification is in the public domain.

Monday, July 31, 2006 7:46:00 PM  
Anonymous encodeflashvideo said...

A terrific overview, thank you. I learned a bit more about this through some recent work I did, so put together a site with more information for those who want to create Flash video, particularly the better-quality and lower-bandwidth Flash 8 video that's highlighted here (and incredibly has already reached something like 90% of computers worldwide now):

http://www.encodeflashvideo.com

Thursday, October 26, 2006 8:27:00 PM  
Blogger Leandro GuimarĂ£es Faria Corcete DUTRA said...

It does not matter if you do like everything to be proprietary or not; you are evil because you chose proprietary when you could have chosen open, without enough justification.

All in all, it may be for the better: the free software world still has H.264/AVC, and we have SVG, and Theora and so on; your steps will just eventually bring you down from your position of total domination to one of open competition against superior, and free, software.

Saturday, March 24, 2007 11:26:00 AM  
Anonymous Anonymous said...

As of September 2006, an open-source implementation of the decoder is part of the libavcodec project.

-from Wikipedia

Tuesday, May 15, 2007 2:19:00 PM  
Anonymous Anonymous said...

As of September 2006, an open-source implementation of the decoder is part of the libavcodec project.

-from wikipedia

Tuesday, May 15, 2007 2:21:00 PM  
Blogger tmarbois said...

i wonder if adobe cares to revisit this issue - now that the iPhone is out and flash is not to be found yet...

It would be greeeeat if flash used h.264 - and let us take advantage of hardware acceleration on the iPhone...instead of forcing us to encode quicktime again....back to quicktime again...aarg.

Thursday, July 05, 2007 12:34:00 PM  
Blogger James said...

I have written a long article on my blog regarding H.264 and Flash Player. And how I feel H.264 looks like it is in the future of Flash from reading between the lines.
Feel free to disagree. I would like to hear your comments.

See it at
http://www.crafted.com.au/blog/?p=37


It is Called: One codec to rule them all, Part2 Web Video

Monday, July 09, 2007 7:57:00 AM  
Blogger Chantha said...

The NEW BETA Flash now supports H264+AAC :)

http://labs.adobe.com/downloads/flashplayer9.html

Sunday, September 09, 2007 6:17:00 PM  
Anonymous Eric said...

Interesting to compare current environment to the original blog post, 2 years on:

1) VP6 was considered better with regards to potential Hardware Support. As far as I know, it never supported hardware. Ah well, you can't get them all right.

2) VP6 was chosen for portability to other processors - ARM was specifically mentioned. But I've now heard that the iPhone didn't ship with Flash Player because of problems getting FP to work on the ARM processor... Maybe it was other flash functionality that was the problem.

3) Code Size - well, the new H264 beta FP is only about 100k larger, if that, so I guess that wasn't a limiting factor? Neither was quality or encoding tools. So what was the limiting factor for H264 back then?

Sunday, September 23, 2007 9:46:00 PM  
Anonymous Anonymous said...

While there is no question he is an expert programmer, I can not accept the quality vs technology agreement.

This decision will isolate Macromedia and force many uses to use the old format from Turbine and FFMPEG (using backwards compatibility support from FLASH). The question is not if FLIX is good the issue is can a small business afford the large price especially for server on demand processing which is now $3750 a year (compared to $2500 one time fee from last year).

In my opinion this was a pure $$$$$$$$$$$$ decision with FLIX and Macromedia to increase ROI with the boom in video.
I guarantee it will backfire with the INDY'S and small businesses that drive sales.

Friday, October 05, 2007 9:35:00 AM  
Blogger derekcfoley said...

I do love the new H.264 capability, but it needs integrating better into the dev tools. For example - I'd be more impressed if you could actually test the H264 streaming inside the flash IDE using the "test movie" functionality. Its impossible inside the IDE without exporting to a web-page or opening with the non IDE flash player.

I work on a lot of multimedia "kiosk" style fullscreen applications that work as "projector" exe files rather than deployed via the web. This aspect of supporting exe workflows (which I've used now for 15 years for screensavers, event displays etc - seems to be getting forgotten about by the Flash development team.

At the moment I have to export.... go to the folder with the app and double click it to test the video streaming. when you're working on a big presentation its a major pain during testing.

Saturday, May 31, 2008 3:11:00 AM  
Anonymous Anonymous said...

I think you need to be fired for choosing the propritery 0n2 v6 codec. Most of the arguments you listed have proved very WRONG. As a principal engineer if you cannot see what is going to be relvent in future, you are not fit for the job.

Saturday, November 22, 2008 9:28:00 AM  

Post a Comment

<< Home