Thursday, May 25, 2006

Linux Flash Player update and new blog

Update (10/31/06): I still see tons of people reading this old entry. Please read this for up to date information. A beta is now available.

Quick note: Emmy Huang has a small update on the Linux Flash Player. Also, we have a new blog from Mike M.. He is the engineer in charge of of the Linux version, so you might want to drop him a comment or two. Maybe this will put me out of the line of fire for a while... ;-) But we actually read and take seriously all feedback from users, no matter how negative or positive it might be.

Sunday, May 07, 2006

Frame rates in the Flash Player

There have been numerous posts about frame rate issues in Flash over the years, sometimes with quite inconsistent tips and workarounds. As we are approaching the final phase of development for Flash Player 9 at Adobe our bug database is filling up with duplicate bugs concerning old known issues. What is frustrating to designers is that they perceive the Flash Player changing its behavior over the past few releases, although it has not. Well, that might actually be the problem.

Flash uses a relative timing model, meaning it does not really care about a global frame rate, but will instead try to enforce frame intervals as best as it can. Say you have f.ex. set your frame rate to 30 frames/sec. That means that the Flash Player will try to wait for 33 milliseconds before trying to display the next frame (excluding the time it takes to render the frame). This loose timing causes all kinds of problems. First, the Flash Player depends on high level OS events to deliver timing messages. In the worst case this means the use of WM_TIMER, dependence on the NetScape plugin API or in the best case we use multimedia timers provided by a special Internet Explorer API. Second, we round frame intervals to milliseconds since Windows and MacOS can't support fractional time intervals. Third, the OS, the browser and the Flash Player will add overhead to the code executed on each frame, meaning that in the end the actual frame rate will sway between -10 to +5 frames/sec from the actual selected frame rate, depending in what environment you play it in. In Flash Player 8 and Flash Player 9 new overhead is originating mostly from the GC, something for which there is no workaround. As I said we do not calculate frame rates on a global basis so we can't correct it actively.

Lets talk about maximum frame rates. In Internet Explorer this is 100 frames/sec. Why? Because the minimum time slice Windows timers can provide is 10 milliseconds. What about FireFox? FireFox does not use special timers and made a decision to limit the maximum frame rate for plugins. Why? The thinking is that users constantly complain that plugins take too much CPU time. A valid complaint I think and every designer who puts online ads out there at higher than 8-12 frames/sec and more that 2 or 3% CPU usage should be ashamed. While a single ad will not be a problem, most pages easily serve 2 or 3 ads on a single page.

The Mozilla team also decided that plugins would get no time when they are on a hidden tab so it would not render the browser unresponsive or less responsive by adding new tabs. So do not be surprised if your SWFs and FLVs do no play on hidden tabs. Apple went even a step further in Safari: If the browser is not active, plugins will only get about 4 frames/sec, mainly to save battery and avoid the dreaded noise of the fans. Try it, go to Google Video, play a video and then switch to another application. The frame rate will drop to about 4 frames/sec. While we could drive our own background thread and work around this, there is a reason they decided to take these steps. We would be ill advised to just hack around it.

What does this mean? Well, the frame rate you select does not really mean too much and you should not depend on it in a way to be accurate to the millisecond. This especially goes for ANY sort of synchronization. If you need synchronization your only choice is placing code in ActionScript which will 'correct' your timing or workarounds like placing a streaming sound on your main time line (In which case we use the audio device to report time correctly to the nanosecond. Due to bugs this does not work correctly on Linux right now, which is the reason audio and video are out of sync, even for FLVs :-( )

What does the future hold? As I explained in a earlier post we will likely add synchronization primitives into the player to allow SMILE like global timing at some point. But there is also a good chance we'll limit what users can actually do when it comes to frame rates and overall CPU usage. There are various ways we could enforce low CPU usage. SWF files originating from a different domain (speaking advertisement) could get a lower priority and have a frame rate cap which would be user selectable. Secondly, with the advent of GPU support in the OS there will be a time when we finally add VBL wait, meaning tearing free drawing. In most cases this means the maximum frame rate will be 60 frames/sec. On high CPU load we might actually cut this into half, e.g. 30 frames/sec. OS X already does this in certain conditions.