The new Gnome-Shell has started to land and to take its final development steps. This landing has also raised some intensive discussions around Gnome - what is Gnome, whom are the competitors and what should get into Gnome.
From the major distros the Fedora camp have indicated themselves to utilize Gnome-Shell/Gnome3 with their upcoming Fedora 15 release, where as the Canonical/Ubuntu guys will be using their own homemade Unity Dekstop and Gnome3 applications. This division was initially fired already at October 2010 @Ubuntu Developer summit where Canonical stated that all the future releases would be based on Unity.
Currently ongoing discussions have been especially intensive around libappindicator - as it has been taken as _the_ item to discuss in various of different blogs [1]. Shortly the history of indicators have evolved from KNotificationItem to Canonicals StatusNotifier proposal and now to the rejection of libappindicator as part of Gnome 3.0 module set - "there is nothing in GNOME needing it". For Canonical this statement is quite clearly saying that Unity is not part of Gnome. Naturally this has raised some discussions.
This definitely becomes interesting when we are discussing about the future of Gnome. What we have here is freedesktop.org project aiming for "interoperability and shared technology for X Window System desktops" and two most popular Linux distros differentiating themselves. What we are also discussing about is free software. Surely in both efforts it is just about that but Canonical copyright assignments contribution agreements are something that Gnome does not want to depend on (this in a scheme where libappindicator would be part of GTK+).
The two projects do have quite different directions, making them competing against each other rather than cooperating. I personally do agree with the statement that healthy competition is always welcomed. The danger is that too much competition can also ruin the possibilities to compete the outside world - competing with the closed source solutions whom are running fast. Getting some new innovation emerging from these Linux based desktops is something that we all would want to see. This would require some big time cooperation instead of competition.
[1]
* collaboration's demise
* Application Indicators: A Case of Canonical Upstream Involvement and its Problems
* Has GNOME rejected Canonical help?
Saturday, 19 March 2011
Sunday, 6 February 2011
Way to Wayland - take 1
There is an interesting discussion ongoing regarding to Waylands current memory allocation model and how this could be scoped better to be more suitable for different graphics architectures.
The rendering model with Wayland is simple and therefor powerful. The key part is the possibility to share a video memory buffer between the client and compositor. This all is currently done through DRM which also makes DRM an essential part of the whole Wayland concept.
So what has now been under the discussion in the Wayland forum is how the current model could,
a) provide interoperability between client and server. Different Wayland clients could more easily discuss with different Wayland Server solutions
b) isolate the important memory handling part away from the core protocol. Allowing Wayland to be used more easily with other memory architectures.
The proposal is to split the DRM specific part to an separate file which would solve both of these problems and would once again put Wayland closer to take the place as the future Linux graphics solution.
The rendering model with Wayland is simple and therefor powerful. The key part is the possibility to share a video memory buffer between the client and compositor. This all is currently done through DRM which also makes DRM an essential part of the whole Wayland concept.
So what has now been under the discussion in the Wayland forum is how the current model could,
a) provide interoperability between client and server. Different Wayland clients could more easily discuss with different Wayland Server solutions
b) isolate the important memory handling part away from the core protocol. Allowing Wayland to be used more easily with other memory architectures.
The proposal is to split the DRM specific part to an separate file which would solve both of these problems and would once again put Wayland closer to take the place as the future Linux graphics solution.
Thursday, 6 January 2011
Some interesting statistics for the beginning of new year
Year 2010 has ended so its nice to check the couple of interesting OSS Graphics components have evolved during 2010. With the interesting ones meaning Mesa, XServer and the one that everyone is discussing about, Wayland. Surely along these components there has been a lot of activity on the Kernel side from graphics point of view by moving the logic to the Kernel side (DRI drivers, the kernel side GEM scheduler and kernel mode setting) plus there has also been a lot of activity around the toolkits but as said now sticking with these three.
So what has happened,
With Mesa we have:
157 people committing
and those people pushing in 12816 commits (of different sizes)
(where as in 2009 the figures were, 118 people pushing in 9719 commits)
With XServer we have:
99 people committing
and those people pushing in 1305 commits (of different sizes)
(where as in 2009 the figures were, 109 people pushing in 2057 commits)
With Wayland the figures are especially interesting as it has received a _lot_ of publicity lately (Unity, Fedora). Now during the year 2010 there were already close to 20 contributors for the project where as a year ago there were only two.
So a lot of things happening in the head of mainline. If we check these commits on daily basis - Mesa having average of 35 commits happening every single day and XServer having 3.5 commits on daily basis. And not to even mention the Kernel which has over 100 commits per day.
Starting to deviate from this with an custom build behind the closed doors will mean an big maintenance and development effort for the instances deciding to do so. On the other hand this is sometimes mandatory. When discussing about an commercial company which is still aiming at to make profit you are bound to keep something behind the closed doors.
So balancing between Open Sourcing code and potentially exposing the bits and pieces from the latest and greatest can get tricky and really needs to be thought carefully.
PS. with MeeGo the problem is tackled by doing the XServer development with the git head - see.
So what has happened,
With Mesa we have:
157 people committing
and those people pushing in 12816 commits (of different sizes)
(where as in 2009 the figures were, 118 people pushing in 9719 commits)
With XServer we have:
99 people committing
and those people pushing in 1305 commits (of different sizes)
(where as in 2009 the figures were, 109 people pushing in 2057 commits)
With Wayland the figures are especially interesting as it has received a _lot_ of publicity lately (Unity, Fedora). Now during the year 2010 there were already close to 20 contributors for the project where as a year ago there were only two.
So a lot of things happening in the head of mainline. If we check these commits on daily basis - Mesa having average of 35 commits happening every single day and XServer having 3.5 commits on daily basis. And not to even mention the Kernel which has over 100 commits per day.
Starting to deviate from this with an custom build behind the closed doors will mean an big maintenance and development effort for the instances deciding to do so. On the other hand this is sometimes mandatory. When discussing about an commercial company which is still aiming at to make profit you are bound to keep something behind the closed doors.
So balancing between Open Sourcing code and potentially exposing the bits and pieces from the latest and greatest can get tricky and really needs to be thought carefully.
PS. with MeeGo the problem is tackled by doing the XServer development with the git head - see.
Monday, 13 December 2010
SceneGraph in toolkits
SceneGraph architectures can be found from quite many of the new architectures introduced. Clutter with its latest rework on the SceneGraph to make it as retained as possible, Qt with its new SceneGraph architecture and also Mozilla's Layers architecture is getting its idea from SceneGraph. Earlier this has been the way the games render them selves but now this model is starting to define how toolkits render our 2D content.
First idea with the scene graph is that you will start to take the benefit from GPU rendering. Instead of having the applications doing the rendering in SceneGraph the compositing is done in retained mode, constructing an tree structure of the scene to be visualized. The SceneGraph paradigm and problem setting with the toolkit/widget and games is still quite different cmp'd to games. When we are discussing about 2D UI's we do not have complex geometry, fancy viewport transformations or dynamic lightnings. Instead we are mainly talking about getting our textures displayed on the screen and about blending them. This difference puts also requirements to the actual traverse algorithm and renderer to ensure that we can take everything out from the GPU from 2D toolkits point of view. Also the problem statement is rather aiming for optimizing of the performance than managing the complexity of the view.
With the GPU there is one particular thing that it cannot execute efficiently - changing states. What the GPU's are designed to do is to load in the shader program and do their stuff. Simple. With the imperative way of doing the rendering where the view is constructed by hierarchically rendering your scene by the application you are bound to do a lot of state changes. This as every single application is in charge of rendering itself - widget by widget and line-by-line, constantly loading content to the GPU. So what we want to do is to feed our GPU as much data as possible and letting it do its job. This by grouping our geometry, shaders and textures data together.
The most important data (or node) from toolkits point of view would be our appearance or geometry node defining our graphics primitives (vertices data and texture info. Data required for your widget rendering). This data is something that we want to store not only per-frame but per lifetime of an application to minimize the state changes.
Second important item is the drawing it self. Although GPU's are designed for drawing, the unnecessary drawing is naturally something that we don't want to do. An easy way to improve in this area is to draw less, or draw only what is visible on the screen. This is exactly what our SceneGraph architectures want to do. With its tree structure the SceneGraph model can smartly use the z buffer to avoid the overdraw. And by this not meaning the computation time clipping and depth buffering that the GPU's are able to do, but rather limiting the objects actually sent to the GPU. This is done by selecting the rendering order smartly (front-to-back when possible) or for ex in Mozilla's Layer case placing the transparent areas into their own layer. The challenge is of course that with the modern UI's we are discussing about a lot of transparency which will force us to do back-to-front rendering.
There are of course also challenges. The SceneGraphs are balancing between the memory consumption (caching too much) and performance. Other challenge is the different nature of different GPUs and drivers as they sometimes behave a little differently and do different things well.
With the toolkit SceneGraphs the challenge is also the target. Are they only aiming for optimized 2D rendering or would they also like to serve the games.
First idea with the scene graph is that you will start to take the benefit from GPU rendering. Instead of having the applications doing the rendering in SceneGraph the compositing is done in retained mode, constructing an tree structure of the scene to be visualized. The SceneGraph paradigm and problem setting with the toolkit/widget and games is still quite different cmp'd to games. When we are discussing about 2D UI's we do not have complex geometry, fancy viewport transformations or dynamic lightnings. Instead we are mainly talking about getting our textures displayed on the screen and about blending them. This difference puts also requirements to the actual traverse algorithm and renderer to ensure that we can take everything out from the GPU from 2D toolkits point of view. Also the problem statement is rather aiming for optimizing of the performance than managing the complexity of the view.
With the GPU there is one particular thing that it cannot execute efficiently - changing states. What the GPU's are designed to do is to load in the shader program and do their stuff. Simple. With the imperative way of doing the rendering where the view is constructed by hierarchically rendering your scene by the application you are bound to do a lot of state changes. This as every single application is in charge of rendering itself - widget by widget and line-by-line, constantly loading content to the GPU. So what we want to do is to feed our GPU as much data as possible and letting it do its job. This by grouping our geometry, shaders and textures data together.
The most important data (or node) from toolkits point of view would be our appearance or geometry node defining our graphics primitives (vertices data and texture info. Data required for your widget rendering). This data is something that we want to store not only per-frame but per lifetime of an application to minimize the state changes.
Second important item is the drawing it self. Although GPU's are designed for drawing, the unnecessary drawing is naturally something that we don't want to do. An easy way to improve in this area is to draw less, or draw only what is visible on the screen. This is exactly what our SceneGraph architectures want to do. With its tree structure the SceneGraph model can smartly use the z buffer to avoid the overdraw. And by this not meaning the computation time clipping and depth buffering that the GPU's are able to do, but rather limiting the objects actually sent to the GPU. This is done by selecting the rendering order smartly (front-to-back when possible) or for ex in Mozilla's Layer case placing the transparent areas into their own layer. The challenge is of course that with the modern UI's we are discussing about a lot of transparency which will force us to do back-to-front rendering.
There are of course also challenges. The SceneGraphs are balancing between the memory consumption (caching too much) and performance. Other challenge is the different nature of different GPUs and drivers as they sometimes behave a little differently and do different things well.
With the toolkit SceneGraphs the challenge is also the target. Are they only aiming for optimized 2D rendering or would they also like to serve the games.
Sunday, 17 October 2010
SW that matters
During the past few weeks/months we have been seeing a lot of news about the future HW of different vendors. Qualcomm has been pushing out news about their 1.5GHz dual-core Snapdragon (ARM cortex-A8 based, Qualcomm modified), Samsung about their Orion a 1GHz dual-core ARM Cortex A9 and Marvell its tri-core Armada 628 (once again ARM based, ARM v7 cores) which can have the main two cores running up to 1.5GHz and the third at 624 MHz. Quite impressive, right?
Lets first think about the figures that the different vendors are providing. What do we do with all of that power? at least form my point of a view this is an excellent questions.
Take for ex the first generation iPhone - ARM11 downclocked to run at 412Mhz, PowerVR MBX light (optimized for low power consumption instead of speed), 128MB of RAM. Still with these figures in the paper even the first iPhones can still beat many of the devices in the markets today. And bare in mind that iPhone 2G was announced already at at the beginning of 2007.
Why is that? Well the SW and especially the framework has been build in a respect of the underlaying HW. And because of the fact that Apple was ready to make compromises with the device - no multitasking which makes things a lot simpler (limited support existing nowadays), simple set of features which really work, extremely simple and limited set of widgets without theming or the 480x320 pixel resolution combined with a color depth of 18 bits (helping a lot with memory bandwidth usage [1]) - focusing on simplicity and usability.
And its not only about the framework from the SW side that matters - badly implemented drivers, SW which is not making the use of the architecture and features a chip offers (separate cores for video, image processing, blitting, etc.). This will effect to the overall performance.
The other thing with these nice MHz figures is the battery consumption. My self I would at least like to get my hands on one touch screen based device which would really last longer than my working day. Of course this is also how the power management of the device is done, which comes once again to well designed and implemented SW...
Instead of getting the MHz's up, how about as an ultimate goal to get the CPU to run as slow as possible and really shifting the operations to GPU. The GPU's can in many cases achieve far better efficiency than the CPU. This would of course still require something from the SW - framework build in such away that it can really utilize the GPU and not just making it to do some fancy effects when switching between applications.
[1] 320x480pixels, 18 bits for a pixel and rendering that 60 times per sec vs. that a frame would hold 800 x 480 pixels (as the Nexus has).
Lets first think about the figures that the different vendors are providing. What do we do with all of that power? at least form my point of a view this is an excellent questions.
Take for ex the first generation iPhone - ARM11 downclocked to run at 412Mhz, PowerVR MBX light (optimized for low power consumption instead of speed), 128MB of RAM. Still with these figures in the paper even the first iPhones can still beat many of the devices in the markets today. And bare in mind that iPhone 2G was announced already at at the beginning of 2007.
Why is that? Well the SW and especially the framework has been build in a respect of the underlaying HW. And because of the fact that Apple was ready to make compromises with the device - no multitasking which makes things a lot simpler (limited support existing nowadays), simple set of features which really work, extremely simple and limited set of widgets without theming or the 480x320 pixel resolution combined with a color depth of 18 bits (helping a lot with memory bandwidth usage [1]) - focusing on simplicity and usability.
And its not only about the framework from the SW side that matters - badly implemented drivers, SW which is not making the use of the architecture and features a chip offers (separate cores for video, image processing, blitting, etc.). This will effect to the overall performance.
The other thing with these nice MHz figures is the battery consumption. My self I would at least like to get my hands on one touch screen based device which would really last longer than my working day. Of course this is also how the power management of the device is done, which comes once again to well designed and implemented SW...
Instead of getting the MHz's up, how about as an ultimate goal to get the CPU to run as slow as possible and really shifting the operations to GPU. The GPU's can in many cases achieve far better efficiency than the CPU. This would of course still require something from the SW - framework build in such away that it can really utilize the GPU and not just making it to do some fancy effects when switching between applications.
[1] 320x480pixels, 18 bits for a pixel and rendering that 60 times per sec vs. that a frame would hold 800 x 480 pixels (as the Nexus has).
Sunday, 12 September 2010
Standards, agreenments and fragmentation
The graphics area can be seen through your OpenSource "standards" and industry _standards_. Even when discussion only about Linux and not even going to the Microsoft land the field is starting to be a quite messy.
The fragmentation in open source side is in many areas caused by the fact that the communities are competing with each other. They want to ensure that they can bring out as good as possible SW as soon as possibly - naturally. What this causes is that instead of making an effort and developing something together we fragment. Good examples of this is your TTM vs. GEM, classic Mesa vs. Mesa Gallium state tracker or EXA/XAA/UXA.
Then we have our industry standards by Khronos. What Khronos defines is an open standard mainly for Graphics and Media. We have the defacto OpenGL/ES and the counterparts GLX and EGL and then we have something else. We have things like OpenWF (display and Composition) which in OSS world is know as KMS (with limited composition support), OpenKODE with the OSS counterpart being SDL and OpenMAX which in OSS world is tackled with VAAPI or XvMC or VDPAU. Well you get the point.
So why should we care about this? What this all means is that the set of selection when developing an Linux based system - following the industry standards or selection one of the Open Source paths. Often the proprietary drivers do follow the industry standards but they have not adopted any of the OSS ways of providing the functionality and why would they as the OSS "standards" are often vendor specific. This will mean that reasonable way to build our platform would be the industry standards but then you would loose the possibility to try out the new nice things which are not executable in these platforms. Difficult choice and ideally a choice that we wouldn't need to do.
The fragmentation in open source side is in many areas caused by the fact that the communities are competing with each other. They want to ensure that they can bring out as good as possible SW as soon as possibly - naturally. What this causes is that instead of making an effort and developing something together we fragment. Good examples of this is your TTM vs. GEM, classic Mesa vs. Mesa Gallium state tracker or EXA/XAA/UXA.
Then we have our industry standards by Khronos. What Khronos defines is an open standard mainly for Graphics and Media. We have the defacto OpenGL/ES and the counterparts GLX and EGL and then we have something else. We have things like OpenWF (display and Composition) which in OSS world is know as KMS (with limited composition support), OpenKODE with the OSS counterpart being SDL and OpenMAX which in OSS world is tackled with VAAPI or XvMC or VDPAU. Well you get the point.
So why should we care about this? What this all means is that the set of selection when developing an Linux based system - following the industry standards or selection one of the Open Source paths. Often the proprietary drivers do follow the industry standards but they have not adopted any of the OSS ways of providing the functionality and why would they as the OSS "standards" are often vendor specific. This will mean that reasonable way to build our platform would be the industry standards but then you would loose the possibility to try out the new nice things which are not executable in these platforms. Difficult choice and ideally a choice that we wouldn't need to do.
Sunday, 5 September 2010
UI done right
I have always been a fan of what the guys at TAT are doing. These guys have really understood the whole thing of making great looking UI's in away that it is respecting the underlaying HW and taking every last bit out of it.
Their latest creation Velvet is just that - innovative, simple, great looking UI. Nice job!
PS. Would be nice to understand how these guys are working. You have your UI designer, framework developer and lower level graphics guy all working seamlessly together...
Their latest creation Velvet is just that - innovative, simple, great looking UI. Nice job!
PS. Would be nice to understand how these guys are working. You have your UI designer, framework developer and lower level graphics guy all working seamlessly together...
Subscribe to:
Posts (Atom)