Friday, May 16, 2008

What does GPU acceleration mean?

The just released Adobe® Flash® Player 10 beta version includes two new window modes (wmode) which control how the Flash Player pushes its graphics to the screen.

Traditionally there have been 3 modes:

normal: In this mode we are using plain bitmap drawing functions to get our rasterized images to the screen. On Windows that means using BitBlt to get the image to the screen on OSX we are using CopyBits or Quartz2D if the browser supports it.

transparent: This mode tries to do alpha blending on top of the HTML page, i.e. whatever is below the SWF will show through. The alpha blending is usually fairly expensive CPU resource wise so it is advised not to use this mode in normal cases. In Internet Explorer this code path does actually not going through BitBlt, it is using a DirectDraw context provided by the browser into which we composite the SWF.

opaque: Somewhat esoteric, but it is essentially like transparent, i.e. it is using DirectDraw in Internet Explorer. But instead of compositing the Flash Player just overwrites whatever is in the background. This mode behaves like normal on OSX and Linux.

Now to the new modes:

direct: This mode tries to use the fastest path to screen, or direct path if you will. In most cases it will ignore whatever the browser would want to do to have things like overlapping HTML menus or such work. A typical use case for this mode is video playback. On Windows this mode is using DirectDraw or Direct3D on Vista, on OSX and Linux we are using OpenGL. Fidelity should not be affected when you use this mode.

gpu: This is fully fledged compositing (+some extras) using some functionality of the graphics card. Think of it being similar to what OSX and Vista do for their desktop managers, the content of windows (in flash language that means movie clips) is still rendered using software, but the result is composited using hardware. When possible we also scale video natively in the card. More and more parts of our software rasterizer might move to the GPU over the next few Flash Player versions, this is just a start. On Windows this mode uses Direct3D, on OSX and Linux we are using OpenGL.

Now to the tricky part, things which will cause endless confusion if not explained:

1. Just because the Flash Player is using the video card for rendering does not mean it will be faster. In the majority of cases your content will become slower.

...

Confused yet? Good, that means you have the same understanding what GPU support means that everyone else has.

Content has to be specifically designed to work well with GPU functionality. The software rasterizer in the Flash Player can optimize a lot of cases the GPU cannot optimize, you as the designer will have to be aware of what a GPU does and adapt your content accordingly. I realize this statement is useless unless we can provide guidance, something we can hopefully achieve in the not to distant future.

2. The hardware requirements for the GPU mode are stiff. You will need at least a DirectX 9 class card. We essentially have the exact same hardware requirements as Windows Vista with Aero Glass enabled. Aero Glass uses exact same hardware functionality we do. So if Aero Glass does not work well on your machine the Flash Player will likely not be able to run well either in GPU mode (but to clarify, you do NOT need Aero Glash for the GPU mode to work in the Flash Player, I am merely talking about hardware requirements here).

3. Pixel fidelity is not guaranteed when you use the GPU mode. You have to expect that content will look different on different machines, even colors might not match perfectly. This includes video. Future Flash Players will change the look of your content in this mode. We will try our best to limit the pain but please bear in mind that in many cases we have no control on this.

Here is an example, left shows it running using the new gpu mode, right using the normal mode. This a video which is 320x240 pixels large showing red text and as you notice the gpu mode arguably looks better as the hardware does UV blending:



The downside in this specific case is that the edges of the text are not as crisp anymore.

4. Frames rates will max out at the screen refresh rate. So whatever frame rate you set in your Flash movie is meaningless if it is higher than 60. This is the case for both the 'direct' and 'gpu' mode. In most cases you should end up at a frame rate of around 50-55 due to dropped frames which occur from time to time for various reasons.

5. Please do not blindly enable either new mode (gpu or direct) in your content. Creating a GPU based context in the browser is very expensive and will drain memory and CPU resources to the point where the browser will become unresponsive. It is usually best practice to limit yourself to one SWF per HTML page using these modes. The target should be content taking over most of the page and doing full frame changes like video. Never ever, ever enable this for banners. Plain Flex applications should not use these modes either if they are not doing full screen refreshes.

6. GPU functionality ties us together with the video card manufacturers and their drivers. Given that you can expect that a significant amount of customers will not be able to view your content if you enable this mode due to driver incompabilities, and various defects in the software stack.

Finally, this beta version of the Flash Player is not yet tuned for maximum performance in the gpu mode. We are making progress, but all the above points will still apply in the long term.

The 'direct' mode should never make your content slower, except in respect to point 4 I made. It should either not change anything or lower CPU consumption somewhat with very large content, i.e. something larger than 1024x768 pixels.

What we'd ask you to do though is to give this a test drive. This is completely new land for us and we expect to encounter lots of obstacles, meaning bugs.

18 Comments:

Blogger nuiman said...

I am excited to see what how this will run in Flash 10 with GPU

http://gizmodo.com/391103/full+screen-multitouch-mac-os-x-is-here

Friday, May 16, 2008 6:06:00 PM  
Blogger nuiman said...

I am excited to see what how this will run in Flash 10 with GPU

http://gizmodo.com/391103/full+screen-multitouch-mac-os-x-is-here

Friday, May 16, 2008 6:06:00 PM  
Blogger nuiman said...

I'm horribly sorry to double post... but how does these new GPU changes effect the stand alone player?

For example how can you enable GPU mode... when runing as swf to exe projector.

Thanks

Friday, May 16, 2008 6:13:00 PM  
Blogger ultraky said...

Thanks for the info

Friday, May 16, 2008 6:30:00 PM  
Blogger Sascha said...

Interesting details! It would be good to know if it's possible (and how) to utilize these modes in the standalone player and in the AIR runtime (if this is later supported by AIR).

Friday, May 16, 2008 9:38:00 PM  
Blogger Bryan Gale said...

To clarify, is the only disadvantage to 'direct' over 'window' is that it's new and hence a bit buggy? Can we eventually expect 'direct' to obsolete 'window'?

Friday, May 16, 2008 11:38:00 PM  
Blogger Bryan Gale said...

Also, will the visual fidelity of objects drawn to a BitmapData object rather than the screen be affected by 'gpu' mode?

Saturday, May 17, 2008 4:50:00 AM  
Blogger glenn said...

Hi mate,

thanks for the info.

couple of questions.

which version of direct X is the player targeting?

will we ever be able to send gpu code ourselves.

what level of the gpu are you exposing or are you only ever going to do this hidden from the developer.

there's so much we could do with gpu power over and above just graphics, it would be so great be be able to write gpu code and have it sent to the card through a connection in actionScript. I know of course this goes against the main Player push - the whole write once thing. it would just be good if as developers we'd be given the option of hitting a specific target machine or going cross platform. I always think its a shame to give up some of the things we could do if we wanted to be platform specific. Now flash is being used for a lot more than just work inside a browser this is even more the case. for example we have developed several intranet applications for companies in the UK where the target machines are a set platform.

oops, iu was starting to go on a bit there, only meant to say - thanks for the info.

cheers

glenn williams
tinylion development UK
flex-ria.com
tinylion.co.uk

Saturday, May 17, 2008 4:52:00 AM  
Blogger Christian Giordano said...

I look forward to see which kind of tips we will have to follow to get the best from OpenGL and DirectX, I hope they will be the same! :)

Saturday, May 17, 2008 6:41:00 AM  
Blogger news said...

Since the GPU Mode is linked to hardware requirements, will there be hook in the Flash API to see if our user has the proper hardware to run GPU mode? And is the WMODE dynamically configurable in the API (without using a js-externalInterface workaround), so that requirements can be automatically 'downscaled' if the user does not have the right hardware?

Sunday, May 18, 2008 2:13:00 PM  
Blogger tomsamson said...

What is really ugly to me is that Tinic says a DirectX 9 class card will be required.
I strongly hope they implement a propper solution soon which supports older gen cards,too, cause going for requiring the user to have such a recent card and yet not having propper access to pretty much any features of that card is a bad joke.
I´ve always been begging and preaching for gpu support but there i was hoping Macromedia/Adobe would implement it in propper way where one can benefit with it even with an earlier gen gpu.
There were test players made by several others which ran flash content hardware accelerated and they ran quite nicely on my ages old 3D card (well,ages ago).
It makes absolutely no sense to require high end hardware for getting low end features in return which should work fine with 10 year old cards,too.
Also only supportng rather new cards means there would be a massive difference of hardware requirements between going for non gpu mode and gpu mode,what about supporting all those (majority of users) with any card between say rather new ones and all those made say in the last 10+ years?
Please elaborate if you intend to add support for older gen 3d cards as it is your "gpu support" sounds crazy (sadly in negative way).

Tuesday, May 20, 2008 2:11:00 AM  
Blogger tomsamson said...

What is really ugly to me is that Tinic says a DirectX 9 class card will be required.
I strongly hope they implement a propper solution soon which supports older gen cards,too, cause going for requiring the user to have such a recent card and yet not having propper access to pretty much any features of that card is a bad joke.
I´ve always been begging and preaching for gpu support but there i was hoping Macromedia/Adobe would implement it in propper way where one can benefit with it even with an earlier gen gpu.
There were test players made by several others which ran flash content hardware accelerated and they ran quite nicely on my ages old 3D card (well,ages ago).
It makes absolutely no sense to require high end hardware for getting low end features in return which should work fine with 10 year old cards,too.
Also only supportng rather new cards means there would be a massive difference of hardware requirements between going for non gpu mode and gpu mode,what about supporting all those (majority of users) with any card between say rather new ones and all those made say in the last 10+ years?
Please elaborate if you intend to add support for older gen 3d cards as it is your "gpu support" sounds crazy.

Tuesday, May 20, 2008 2:12:00 AM  
Blogger glenn said...

you must keep in mind that this is the first version with gpu access. As such it'll be the most basic implementation. Future versions will take more advantage of this feature i'm sure - well that is if it proves to be useful in the real world.

Glenn Williams
tinylion Development UK
flex-ria.com
tinylion.co.uk

Tuesday, May 20, 2008 4:04:00 AM  
Blogger tomsamson said...

well,i´d agree if it was "basic implementation" as in supporting/using few features of the gpu and therefore only requiring older cards. Like let´s say only having gpu rendering side support and in return running on way older gen cards. Then one could stepwise enhance which features of gpus can be used and accessed by the flash content developer and then also require newer gen cards if one uses features only those have. And no, i don´t think its a good idea to have just the rendering side done in a way that only shader 2.0 cards can run it.
That is a fatal mistake.
There is no serious argument left for using flash when you still can´t make use of any features of the 3D card and at the same time require directx9 shader 2.0 grade cards which reduces your reachable audience a lot.
Any other 3D technology makes way more sense than flash then because the others are way better scalable and also run on way older cards (unless its about some very specific feature of a newer gen card you really want to use as deveoper,which well,with flash you can´t do at all anyway).
A newer gen card should only be required if one as flash content developer gets access to using more of the card´s features and then decides one wants to use features only available with newer card gens.
It shouldn´t be the case that just for basic rendering support one needs one of the latest card gens.

Tuesday, May 20, 2008 3:23:00 PM  
Blogger Si said...

@tomsamson

The GPU features in FP10 have nothing to do with anything 3D, FP10 doesn't support true 3D and probably never will. The whole point of Flash being able to use the GPU is to get raster content to the screen as quickly as possible, the main use of which will probably be video playback.

We should be thankful that Adobe has taken the time to hook the Flash Player up to the GPU, even if it does require a DirectX 9 card.

If you want true 3D then you should be looking at Director not Flash.

Tuesday, May 27, 2008 11:41:00 PM  
Blogger Julian said...

GPU mode is interesting actually.. as FLV has gone the MP4/H.264 way do you think it would be possible to use the Hardware decoders nowadays GPUs have (like using DXVA on Windows) - this would speed up rendering on supported CPUs enormously!

Saturday, May 31, 2008 2:58:00 PM  
Blogger Henrique said...

Why not throw away Lingo (from Director) and put ActionScript in his place?

Friday, June 13, 2008 6:52:00 PM  
Anonymous Anonymous said...

How can you enable gpu/direct-targeting within the mxmlc/fcsh compiler ? Comparing a flash generated "normal" movie with a "direct/Gpu", the binary bytes are different... I cannot find any documentation about it!

Thursday, November 20, 2008 10:02:00 AM  

Post a Comment

<< Home