This is not directly grapihcs related but about interesting enablers for modern UI's anyway...
XInput2/MPX was introduced already in XServer1.7 and the Linux Kernel has had the multitouch support since 2.6.30. So we should have case closed and a working multitouch solution existing for Lixux environment. Right?
Well, what we currently have are couple of reference implementation done to show of the Linux multitouch capabilities, but the full pipeline coming directly from upstream is still missing. When we are discussing about X we should be capable to support all the different use cases from simple gestures with pinch, all the way to cases where we have multiple user interacting simultaneously with a big multitouch device. This makes things complicated.
From the current reference implementations people at ENAC have been working quite actively around the multitouch solutions - directly on top of Linux + they have also patched the existing evdev to propagate multiple pointers through MPX. Another reference implementation of multitouch X driver is done by Henrik Rydberg - found from here. Still either of these solutions have not made it to official XServer.
Now there has been an discussion ongoing in the xorg-devel mailinglist that there would be now an clear first step to include support multitouch features for X. This first step would be limited to have a single focus / input point with auxiliary information attach to the event. Most importantly X would propagate these events to the right clients.
The solution would be mainly for indirect touch devices (like touchpads), but would also cover single-application input cases. From mobile devices point of view, where we would have a simple window manager with top most application fullscreen, this solution would already be able to cover all your favourite two finger pinch gestures - making this basically a full fledged solution for these kinds of devices.
Wednesday, 7 April 2010
Thursday, 11 March 2010
DRI2 driver for Mesa EGL
Couple of weeks ago there was an announcement that Wayland would be moving to Mesa EGL and drop its dependencies to its own Wayland specific Eagle EGL stack. The Eagle is a non-conformant EGL loader for the DRI drivers originally develped by Kristian Høgsberg as an "side product" of his DRI2 work. Eagle as an lightweight and with its way of integrating EGL directly to KMS/DRM, was the solution for Wayland - this at least until recently.
Major reason to allow Wayland to jump to Mesa EGL is Kristian's work for the work behind the DRI2 driver for EGL and to get the Eagle EGL parts as officially to Mesa. This work - EGL on DRI2 is firstly aiming to implement this support through X, but the longer term goal seems to be that the implementation would be standalone - working directly with KMS/DRM and making it suitable for Wayland purposes.
Current plan is that the DRI2 parts for the Mesa EGL would be integrated to the upcoming Mesa 7.8 - planned to be release during this month. This is good news especially for Wayland. Along the Mesa integration the Wayland enablers are now even more mainlined. This move will most definitely make Wayland it self more attractive to outside world.
Major reason to allow Wayland to jump to Mesa EGL is Kristian's work for the work behind the DRI2 driver for EGL and to get the Eagle EGL parts as officially to Mesa. This work - EGL on DRI2 is firstly aiming to implement this support through X, but the longer term goal seems to be that the implementation would be standalone - working directly with KMS/DRM and making it suitable for Wayland purposes.
Current plan is that the DRI2 parts for the Mesa EGL would be integrated to the upcoming Mesa 7.8 - planned to be release during this month. This is good news especially for Wayland. Along the Mesa integration the Wayland enablers are now even more mainlined. This move will most definitely make Wayland it self more attractive to outside world.
Monday, 15 February 2010
Challenges in 2D acceleration - how about the future?
To achieve great 2D graphics performance from Linux Graphics stack has most definitely been an challenge for years. Trying to get the XRender extension more usable with XAA/EXA/UXA (see my previous blog post), utilising all-software image backends, utilise GL directly to accelerate the 2D and so on. The best solution really has been the CPU rasterization backend based on the fact that the CPU simply runs with higher clock speed and quite many of these 2D operations require raw processing power.
So are we stuck just to continue the everlasting optimisation of the existing solutions? Maybe not.
Now it seems that this there might be some light in the tunnel. Cairo has been getting some quite impressive performance figures with its cairo-drm backend. Instead of using XServer to do the rendering or build an abstraction of XRender with GL backend (what glitz was doing) it has the capabilities to use directly the GEM buffers and issues the rendering commands directly to HW. This will allow direct access to GPU and will give quite impressive performance figures.
The thing is still that cairo-drm is an experimental backend and that the backend is limited to Intel HW (i915 to be specific).
With the performance figures - what the cairo-drm can really do, the goal for the guys is now to get this same performance out from the GL interface directly. This would 1st of all remove the limitations mentioned above and 2nd, it would also allow all the rendering to happen under one driver - OpenGL. This of course would mean that there would be only one graphics stack to tune and maintain and would allow CPU to do other stuff. The cairo-gl project aims to do exactly this.
The cairo-gl backend would also make the solution usable with other than Intel specific HW. The questions of course is that how well the solution performs in non-Intel HW.
Time will show how the cairo-gl backend will perform and whether it is capable to beat CPU rasterization backends.
More info about the progress from:
1. http://anholt.livejournal.com/
2. http://ickle.wordpress.com/
So are we stuck just to continue the everlasting optimisation of the existing solutions? Maybe not.
Now it seems that this there might be some light in the tunnel. Cairo has been getting some quite impressive performance figures with its cairo-drm backend. Instead of using XServer to do the rendering or build an abstraction of XRender with GL backend (what glitz was doing) it has the capabilities to use directly the GEM buffers and issues the rendering commands directly to HW. This will allow direct access to GPU and will give quite impressive performance figures.
The thing is still that cairo-drm is an experimental backend and that the backend is limited to Intel HW (i915 to be specific).
With the performance figures - what the cairo-drm can really do, the goal for the guys is now to get this same performance out from the GL interface directly. This would 1st of all remove the limitations mentioned above and 2nd, it would also allow all the rendering to happen under one driver - OpenGL. This of course would mean that there would be only one graphics stack to tune and maintain and would allow CPU to do other stuff. The cairo-gl project aims to do exactly this.
The cairo-gl backend would also make the solution usable with other than Intel specific HW. The questions of course is that how well the solution performs in non-Intel HW.
Time will show how the cairo-gl backend will perform and whether it is capable to beat CPU rasterization backends.
More info about the progress from:
1. http://anholt.livejournal.com/
2. http://ickle.wordpress.com/
Saturday, 23 January 2010
N900/Maemo5 and different texture formats
Garage.maemo.org is providing an nice tool for benchmarking the different texture formats that the meamo5, SGX based device supports. The tool GLMemPerf is meant for measuring the texture memory bandwidth performance of an OpenGL ES 2.0 implementation with a set of different kinds of precompressed textures. As an output it blit the different textures from its /data folder to the screen and provides an nice set of information - how the different formats and texture processing methods effect to the overall performance.
From the results the compressed textures formats (IMG specific PVRTC and ETC1) are clearly giving an performance benefit - as one could expect. The IMG PVR Texture Compression format has the 2-bits-per-pixel and 4-bits-per-pixel options create PVRTC data that encodes source pixels into 2 or 4 bits per pixel. This meaning that each texel is encoded into fewer bits than the equivalent uncompressed texture, which of course is meaning less load for memory bandwidth.
The downside with the compression is that the overall quality of the render will suffer. The quality hit is still depending from the compressed content so might be worth of checking how 2 vs 4 bits per pixel will effect.
The other thing to note is how the textures are stored and accessed from the memory.
SGX as many other GPUs benefit greatly from the usage of using power-of-two sized textures - which improves the implementation of the texture mapping when converting the texture coordinates to texel coordinates. It might be worth of rounding up the textures to the nearest power-of-two to get the benefit. Note also that the PVRTC formats require that the textures are in power-of-two size.
The other thing related accessing the memory is the texture twiddling. The twiddling is by default done for all the SGX uploaded textures. What the twiddling does is that it re-arranges the texture samples to optimize the memory access for different operations. One thing to note is that with pixmaps you will not get this benefit as in a case of pixmaps you don't upload the textures to the video memory but instead source the pixels directly.
How about generating and processing your textures? For this IMGTec is providing an nice tool called PVRTexTool. The tool has an version for command line usage and also an GUI based tool for seeing in more detailed the output of your changes - for ex when wanting to do comparison between the different compression methods. The PVRTexTool supports your normal RGB/ARGB formats + the texture compression format PVRTC and ETC.
From the results the compressed textures formats (IMG specific PVRTC and ETC1) are clearly giving an performance benefit - as one could expect. The IMG PVR Texture Compression format has the 2-bits-per-pixel and 4-bits-per-pixel options create PVRTC data that encodes source pixels into 2 or 4 bits per pixel. This meaning that each texel is encoded into fewer bits than the equivalent uncompressed texture, which of course is meaning less load for memory bandwidth.
The downside with the compression is that the overall quality of the render will suffer. The quality hit is still depending from the compressed content so might be worth of checking how 2 vs 4 bits per pixel will effect.
The other thing to note is how the textures are stored and accessed from the memory.
SGX as many other GPUs benefit greatly from the usage of using power-of-two sized textures - which improves the implementation of the texture mapping when converting the texture coordinates to texel coordinates. It might be worth of rounding up the textures to the nearest power-of-two to get the benefit. Note also that the PVRTC formats require that the textures are in power-of-two size.
The other thing related accessing the memory is the texture twiddling. The twiddling is by default done for all the SGX uploaded textures. What the twiddling does is that it re-arranges the texture samples to optimize the memory access for different operations. One thing to note is that with pixmaps you will not get this benefit as in a case of pixmaps you don't upload the textures to the video memory but instead source the pixels directly.
How about generating and processing your textures? For this IMGTec is providing an nice tool called PVRTexTool. The tool has an version for command line usage and also an GUI based tool for seeing in more detailed the output of your changes - for ex when wanting to do comparison between the different compression methods. The PVRTexTool supports your normal RGB/ARGB formats + the texture compression format PVRTC and ETC.
Tuesday, 8 September 2009
Challenges in 2D acceleration
GPU based 2D acceleration has been an hot topic in Linux already for years. With hot topic meaning should or shouldn't it be utilizing GPU for rendering and if so - what should be accelerated. The first attempts with XAA didn't give basically any speed advantage and accelerated only some core rendering elements to the graphics HW. Along the X Render Extension providing some nice functionality for modern applications like alpha blending, drop shadows and translucency the questions of whether GPU should be used needed to be revisited. EXA enabled this opportunity to HW accelerate render extension requests and therefor improving the X.Org Server 2D performance.
The latest twist in this field is UXA which is based on the EXA code base and utilizing GEM Memory Manager. With this latest combination by moving the memory manager to the Kernel side (with GEM) it seems that the equation of accelerating X is finally getting in shape but there still seems to be challenges especially with text/glyph rendering - more from this in Carl Worth's web pages.
But how about these acceleration architectures in action? As I was anyway about to reinstall my Intel 82852/855GM Graphics Device accelerated laptop (old and integrated, I know...), I decided to put these architectures in test.
As an distro I selected the latest Ubuntu 9.04. The actual benchmarking was done with mx11perf - this mainly as it had nice benchmark script already existing which covered quite nice variety of X11 operations. Some of the most interesting results from this run can be found from below.
Rendering of basic rect:
Then a Copy:
Composite:
(with this one it is interesting to see UXA with 16-bit depth. Why so? Also with 24- and 32-bit UXA seemed to perform really nicely with special sized 512x256 rect's)
and finally text:
(Showing that rendering small areas and text is something where the CPU can still outperform GPU)
At the end the mx11mark gives also an total score. From these figures EXA and actually XAA gets the highets scores which is basically the same ~80 total score. XAA mainly getting these scores as it gets really high figures with pure rect rendering. UXA and NoAccel are also even with ~20 total score - NoAccel performing with small are renders and UXA with 512x256 rect's.
OK, my selection from these was the default one - EXA. Seems that at least with out_of_the_box UXA there is still quite much work to be done and I would definitely be interested to see how UXA for ex performs in tuned Intel 965 graphics device.
The latest twist in this field is UXA which is based on the EXA code base and utilizing GEM Memory Manager. With this latest combination by moving the memory manager to the Kernel side (with GEM) it seems that the equation of accelerating X is finally getting in shape but there still seems to be challenges especially with text/glyph rendering - more from this in Carl Worth's web pages.
But how about these acceleration architectures in action? As I was anyway about to reinstall my Intel 82852/855GM Graphics Device accelerated laptop (old and integrated, I know...), I decided to put these architectures in test.
As an distro I selected the latest Ubuntu 9.04. The actual benchmarking was done with mx11perf - this mainly as it had nice benchmark script already existing which covered quite nice variety of X11 operations. Some of the most interesting results from this run can be found from below.
Rendering of basic rect:
| EXA | UXA | XAA | NoAccel | |
| Rect 8x8 Src | 10410/sec (0.67 Mpix/s) | 7598/sec (0.49 Mpix/s) | 5527/sec (0.35 Mpix/s) | 263154/sec (16.84 Mpix/s) |
| Rect 32x32 Src | 9316/sec (9.54 Mpix/s) | 6701/sec (6.86 Mpix/s) | 10050/sec (10.29 Mpix/s) | 33070/sec (33.86 Mpix/s) |
| Rect 512x256 Src | 3950/sec (517.77 Mpix/s) | 3862/sec (506.19 Mpix/s) | 6245/sec (818.65 Mpix/s) | 299/sec (39.25 Mpix/s) |
Then a Copy:
| EXA | UXA | XAA | NoAccel | |
| Copy (Render) 32x32 | 8884/sec (9.10 Mpix/s) | 6362/sec (6.51 Mpix/s) | 6320/sec (6.47 Mpix/s) | 7467/sec (7.65 Mpix/s) |
| Copy (Render) 128x128 | 10909/sec (178.74 Mpix/s) | 10845/sec (177.69 Mpix/s) | 12141/sec (198.92 Mpix/s) | 530/sec (8.69 Mpix/s) |
Composite:
| EXA | UXA | XAA | NoAccel | |
| Composite (Src 16) 32x32 | 5612/sec (5.75 Mpix/s) | 1254/sec (1.28 Mpix/s) | 5165/sec (5.29 Mpix/s) | 14802/sec (15.16 Mpix/s) |
| Composite (Src 16) 512x256 | 147/sec (19.32 Mpix/s) | 35/sec (4.70 Mpix/s) | 307/sec (40.30 Mpix/s) | 127/sec (16.76 Mpix/s) |
| Composite (Src 32) 32x32 | 7087/sec (7.26 Mpix/s) | 6382/sec (6.54 Mpix/s) | 6115/sec (6.26 Mpix/s) | 30312/sec (31.04 Mpix/s) |
| Composite (Src 32) 512x256 | 1845/sec (241.89 Mpix/s) | 3538/sec (463.78 Mpix/s) | 1731/sec (226.92 Mpix/s) | 286/sec (37.60 Mpix/s) |
and finally text:
| EXA | UXA | XAA | NoAccel | |
| Text 8px | 1726/sec (1.49 Mpix/s) | 2295/sec (1.98 Mpix/s) | 2490/sec (2.14 Mpix/s) | 6830/sec (5.87 Mpix/s) |
| Text 12px | 1658/sec (2.39 Mpix/s) | 2265/sec (3.26 Mpix/s) | 2274/sec (3.28 Mpix/s) | 4550/sec (6.54 Mpix/s) |
| Text 24px | 2099/sec (9.89 Mpix/s) | 2383/sec (11.24 Mpix/s) | 1978/sec (9.32 Mpix/s) | 1846/sec (8.69 Mpix/s) |
At the end the mx11mark gives also an total score. From these figures EXA and actually XAA gets the highets scores which is basically the same ~80 total score. XAA mainly getting these scores as it gets really high figures with pure rect rendering. UXA and NoAccel are also even with ~20 total score - NoAccel performing with small are renders and UXA with 512x256 rect's.
OK, my selection from these was the default one - EXA. Seems that at least with out_of_the_box UXA there is still quite much work to be done and I would definitely be interested to see how UXA for ex performs in tuned Intel 965 graphics device.
Thursday, 27 August 2009
The new Maemo 5 is finally out
This long speculated and waited "software behind our mobile computer" is finally released. See http://maemo.nokia.com/
Some interesting characteristics of this device
From the engine room's side this all will be running with TI's OMAP 3430 and PowerVR's SGX for HW accelerated graphics. Cool!
Some interesting characteristics of this device
- Including Clutter based compositing manager
- Using EGL and OpenGL ES 2.0 for rendering, and of course also providing the GLES API for developers
- Including Xorg X window system
- Support for Flash (not just the h.264 converted videos)
From the engine room's side this all will be running with TI's OMAP 3430 and PowerVR's SGX for HW accelerated graphics. Cool!
Sunday, 16 August 2009
Compositing Window Manager - how it works
Compiz and other compositors have been living in the x86 world already for an while but these compositors are also making their way to mobile devices. To understand a bit better how these compositing managers work, I decided to take an bit closer look at the de facto Linux composite window manager - Compiz. I know that there are several good articles about compositing in general (like composite_howto) and Compiz as it is, is not aimed for mobile devices but the idea with this was to understand in higher level how this compositing in general is working in Linux platform.
For this I set up an separate Xgl session and used the good old ltrace to see what happens under the hood. So, off we go...
When launching Compiz the first thing we do is query whether the needed extensions are in place. This by calling functions like XCompositeQueryExtension, XDamageQueryExtension, etc. This to ensure that the infra is there.
The actual magic starts to happen when we call the XCompositeRedirectSubwindows, which requests the XServer to redirect the entire window hierarchy to off-screen buffer. This including current and future windows. This redirection is done at XServer level and is completely transparent to the applications, and therefore does not require any changes to them.
In Compiz case the XCompositeRedirectSubwindows call requests for manual redirection (CompositeRedirectManual). This will mean that the server does not internally track damage for them but instead registers damage listeners for each window. Updating of the screen in this case is done by Compiz itself - allowing it to do the final presentation to the screen.
Next step is to notify the X Server that Compiz wishes to draw to an window of the XServer itself - window called the Composite Overlay. This is done by calling XCompositeGetOverlayWindow. This overlay window exists above all other windows and provides an surface for the compositing manager to draw to. The challenge with this overlay window comes when the user generates input events (key- / mouse presses, etc) and these would need to be passed trough this overlay window to the actual X Window for interaction. Compiz use XFixes shape to enable this to happen. This with an set of function calls - XFixesCreateRegion, XFixesSetWindowShapeRegion, XFixesSetWindowShapeRegion, XFixesDestroyRegion.
So now we would have the basics up and running and we are ready for the applications.
First thing with the applications executed through the compositing manager is to create an damage listener for the window. This is done by calling XDamageCreate. With this call we want to be notified through an event whenever the damage state changes.
For the applications we need to create the off-screen storage, as we have told the XServer that we would like to render to off-screen buffer. This is done by calling XCompositeNameWindowPixmap. With this we are creating an window sized pixmap and also getting as return value a handle to the pixmap as an reference to the off-screen storage. When rendering the screen to the display Compiz uses this handle to bind this pixmap as an texture with an GLX (not in EGL) extension texture_from_pixmap - glXBindTexImageEXT. Compiz uses these GLXPixmaps to do the final rendering to the screen.
For this I set up an separate Xgl session and used the good old ltrace to see what happens under the hood. So, off we go...
When launching Compiz the first thing we do is query whether the needed extensions are in place. This by calling functions like XCompositeQueryExtension, XDamageQueryExtension, etc. This to ensure that the infra is there.
The actual magic starts to happen when we call the XCompositeRedirectSubwindows, which requests the XServer to redirect the entire window hierarchy to off-screen buffer. This including current and future windows. This redirection is done at XServer level and is completely transparent to the applications, and therefore does not require any changes to them.
In Compiz case the XCompositeRedirectSubwindows call requests for manual redirection (CompositeRedirectManual). This will mean that the server does not internally track damage for them but instead registers damage listeners for each window. Updating of the screen in this case is done by Compiz itself - allowing it to do the final presentation to the screen.
Next step is to notify the X Server that Compiz wishes to draw to an window of the XServer itself - window called the Composite Overlay. This is done by calling XCompositeGetOverlayWindow. This overlay window exists above all other windows and provides an surface for the compositing manager to draw to. The challenge with this overlay window comes when the user generates input events (key- / mouse presses, etc) and these would need to be passed trough this overlay window to the actual X Window for interaction. Compiz use XFixes shape to enable this to happen. This with an set of function calls - XFixesCreateRegion, XFixesSetWindowShapeRegion, XFixesSetWindowShapeRegion, XFixesDestroyRegion.
So now we would have the basics up and running and we are ready for the applications.
First thing with the applications executed through the compositing manager is to create an damage listener for the window. This is done by calling XDamageCreate. With this call we want to be notified through an event whenever the damage state changes.
For the applications we need to create the off-screen storage, as we have told the XServer that we would like to render to off-screen buffer. This is done by calling XCompositeNameWindowPixmap. With this we are creating an window sized pixmap and also getting as return value a handle to the pixmap as an reference to the off-screen storage. When rendering the screen to the display Compiz uses this handle to bind this pixmap as an texture with an GLX (not in EGL) extension texture_from_pixmap - glXBindTexImageEXT. Compiz uses these GLXPixmaps to do the final rendering to the screen.
Subscribe to:
Posts (Atom)