When accleration came to browser

HW accelerating in website rendering has become an hot topic along the HTML5. Mozillas answer to this demand is their new Layers architecture.
Mozilla's Layers allows browser to take the suitable parts of the rendering to GPU and of course along that to get some eye candy to the end users. Idea is to divide the logically different parts of the rendered content to different layers allowing better division between the CPU and GPU.

The Layers will structure a layer tree which constructs the final webpage. To simplify, they have a container (representing the surface) and its leafs (including the actual graphical data). The final render is constructed from the textures generated from the leafs and composited to the screen.


The basic layer infra will include ThebesLayers, ContainerLayers and a LayerManager.

The Manager is responsible of constructing the layer tree and rendering that to the destination context. The rendering is always happening as an result of an transaction to the tree and only happening to the visible region parts of the texture.

The Container layers task is to group together the child layers beneath it. It represents the surface into which its children are composited and handles the Z order of the its children.

Thebes in general is nothing new. It has been part of the Mozilla gfx architecture already quite awhile as an C++ wrapper for Cairo API's and as an API for text handling.
The ThebesLayer abstracts the Thebes surface which is used to display items that will be rendered (surprise, surpise) using Thebes. So Cairo Image surface drawn to by Cairo. The question is of course - why not doing this directly with Cairo? Why the wrapper? - Some discussions about this from Bas's blog comments.

There is bit of an older bug in Mozilla bugzilla explaining this basic functionality in detailed. In principle what you get with the basic functionality is what you already have - Cairo content shown to the screen. Of course besides the basic functionality you will also need some additional layers for video, something for webgl plus you would also like to have optimized way of doing the rendering. So not all about just basics.


What from my point of view makes this layers especially in interesting are the plans to bring layers to also to Fennec 2.0 along the other goodies like Electrolysis. It will be most definitely interesting to see how it performs and behaves in a more limited environment.

BTW the first bits of layers are already available through the latest nightly builds of Firefox4. The usage of the layers is still limited to Windows/GL, full-screen video only but the sources are available for seeing and testing.

Difficulty of developing Open Source Linux drivers for GPU

During the last few days there as been an active discussion ongoing at dri-devel list about Open Source Graphics drivers. The discussion was initiated by Qualcomm's RFC message - [RFC] Qualcomm 2D/3D graphics driver.

Qualcomm's Linux team has been working on Open Sourced Kernel drivers for their chipset. What they have done so far is that they have build DRM driver with a limited set of supported functionality to enable DRI2. Their own userland parts of the drivers do still utilize the KGSL (Kernel Graphics Support Layer) interface to allocate memory making DRM just as an API to be more closer with the mainstream Linux Graphics stack. Qualcomm does still have some plans in the future to move to an standard design - possibly TTM.


The dri-devel list the opinions about drivers with an userland binary blobs was made quite clear - if you aren't going to open the whole drivers don't try to push them to mainline. Basically the door was slammed from the Qualcomm efforts for pushing the drivers in.

In between the lines I think there were some excellent point made in this discussion.
The bottom line is that the community cannot work with the drivers if there exists and dependency to closed sourced userland - this is definitely clear. With this kind of an setup it is just impossible to work with the drivers. On the otherhand if the community is going to bombard down all the drivers with the closed userland parts (even without making sure what are the attemts to open source them) how are we ever going to see open sourced mobile GPU drivers? From my point of view the community also has the means to guide this work to right direction - giving comments to the request. It really seems that the Qualcomm guys do have the enthusiasm to develop the open source drivers. How about making sure that these guys are taking the correct steps also with the userland and making sure that the Kernel bits are as they should?

MeeGo for handsets getting its UI

MeeGo for handsets has taken its next step and is now capable to provide MeeGo User Experience also for mobile users - see the blog entry.
There is some nice screenshots and video showing its functionality. Unfortunately the downloadable image is only available for Moorestown based Aava device. The N900 image still needs to be compiled by hand - instructions from here and repo from here.

GLES1.1/2.0 on Mesa DRI and a bit about Gallium

Mesa has had GLES 1.1/2.0 already since since last year through Gallium state trackers. What this means is that Gallium based drivers are capable of providing support GLES support for end users. Now there is also an activity ongoing to get the Mesa GLES1/2 support to work with DRI drivers and along that to expose multiple GL APIs without separate state trackers on Gallium side.

This work on GLES 1.1/2.0 on DRI made by Kristian received good feedback also Gallium driver developer and is now merged to Mesa.


The discussion about classic Mesa vs. Mesa Gallium State Tracker in general has not lately been this productive (see the Mesa thread). Intel guys are clearly wanting to stay with the classic drivers, and whom could actually blame after all the work that Intel has place to get their drivers to perform. Currently Nouveau camp is the only one that is really utilising Gallium3D and the RedHat guys are going to ship the latest free software NVIDIA bits with the upcoming Fedora 13.

Multitouch support for X

This is not directly grapihcs related but about interesting enablers for modern UI's anyway...

XInput2/MPX was introduced already in XServer1.7 and the Linux Kernel has had the multitouch support since 2.6.30. So we should have case closed and a working multitouch solution existing for Lixux environment. Right?
Well, what we currently have are couple of reference implementation done to show of the Linux multitouch capabilities, but the full pipeline coming directly from upstream is still missing. When we are discussing about X we should be capable to support all the different use cases from simple gestures with pinch, all the way to cases where we have multiple user interacting simultaneously with a big multitouch device. This makes things complicated.

From the current reference implementations people at ENAC have been working quite actively around the multitouch solutions - directly on top of Linux + they have also patched the existing evdev to propagate multiple pointers through MPX. Another reference implementation of multitouch X driver is done by Henrik Rydberg - found from here. Still either of these solutions have not made it to official XServer.

Now there has been an discussion ongoing in the xorg-devel mailinglist that there would be now an clear first step to include support multitouch features for X. This first step would be limited to have a single focus / input point with auxiliary information attach to the event. Most importantly X would propagate these events to the right clients.
The solution would be mainly for indirect touch devices (like touchpads), but would also cover single-application input cases. From mobile devices point of view, where we would have a simple window manager with top most application fullscreen, this solution would already be able to cover all your favourite two finger pinch gestures - making this basically a full fledged solution for these kinds of devices.

DRI2 driver for Mesa EGL

Couple of weeks ago there was an announcement that Wayland would be moving to Mesa EGL and drop its dependencies to its own Wayland specific Eagle EGL stack. The Eagle is a non-conformant EGL loader for the DRI drivers originally develped by Kristian Høgsberg as an "side product" of his DRI2 work. Eagle as an lightweight and with its way of integrating EGL directly to KMS/DRM, was the solution for Wayland - this at least until recently.

Major reason to allow Wayland to jump to Mesa EGL is Kristian's work for the work behind the DRI2 driver for EGL and to get the Eagle EGL parts as officially to Mesa. This work - EGL on DRI2 is firstly aiming to implement this support through X, but the longer term goal seems to be that the implementation would be standalone - working directly with KMS/DRM and making it suitable for Wayland purposes.

Current plan is that the DRI2 parts for the Mesa EGL would be integrated to the upcoming Mesa 7.8 - planned to be release during this month. This is good news especially for Wayland. Along the Mesa integration the Wayland enablers are now even more mainlined. This move will most definitely make Wayland it self more attractive to outside world.

Challenges in 2D acceleration - how about the future?

To achieve great 2D graphics performance from Linux Graphics stack has most definitely been an challenge for years. Trying to get the XRender extension more usable with XAA/EXA/UXA (see my previous blog post), utilising all-software image backends, utilise GL directly to accelerate the 2D and so on. The best solution really has been the CPU rasterization backend based on the fact that the CPU simply runs with higher clock speed and quite many of these 2D operations require raw processing power.
So are we stuck just to continue the everlasting optimisation of the existing solutions? Maybe not.

Now it seems that this there might be some light in the tunnel. Cairo has been getting some quite impressive performance figures with its cairo-drm backend. Instead of using XServer to do the rendering or build an abstraction of XRender with GL backend (what glitz was doing) it has the capabilities to use directly the GEM buffers and issues the rendering commands directly to HW. This will allow direct access to GPU and will give quite impressive performance figures.
The thing is still that cairo-drm is an experimental backend and that the backend is limited to Intel HW (i915 to be specific).

With the performance figures - what the cairo-drm can really do, the goal for the guys is now to get this same performance out from the GL interface directly. This would 1st of all remove the limitations mentioned above and 2nd, it would also allow all the rendering to happen under one driver - OpenGL. This of course would mean that there would be only one graphics stack to tune and maintain and would allow CPU to do other stuff. The cairo-gl project aims to do exactly this.
The cairo-gl backend would also make the solution usable with other than Intel specific HW. The questions of course is that how well the solution performs in non-Intel HW.


Time will show how the cairo-gl backend will perform and whether it is capable to beat CPU rasterization backends.
More info about the progress from:

1. http://anholt.livejournal.com/
2. http://ickle.wordpress.com/