I am one of our QWS developers. QWS, the Qt Window System, is the heart of Qt for Embedded Linux, formally Qtopia Core, formally Qt/Embedded. What’s great about working on embedded is that you have a view of the system as a whole - the complete stack. So it’s my job to have a pretty good idea how QPainter commands you write in a widget’s paint event end up as voltage levels rapidly alternating between +3.3v and 0v on wires going to your LCD. While QWS handles all the usual window system tasks such as keyboard focus and mouse events, the biggest component is probably graphics. The window system is inherently part of the graphics stack and I actually spend a lot of my time working with the graphics team.

Over the last year our clear focus has been on performance. This performance push has been in all areas on all platforms and architectures. When it comes to graphics, there’s a very broad range of hardware Qt can expect to see - from a simple MMU-less ARM with only a frame-buffer all the way to scary gamer PCs with thousands of graphics cards & neon lights installed. Our lives are made complicated because if we’re not careful, one can end up writing code which runs faster on that little arm than it does on the gamer PC. This post hopes to explain why this is so.

Let’s begin by categorising the range of hardware available:

First, we need to differentiate Unified Memory Architecture (UMA) devices from those with dedicated graphics memory. Generally, high-end hardware will have dedicated graphics memory whereas low-end devices will just use system memory (sometimes reserving a memory region, sometimes not). This is pretty strait forward on PCs - You can almost tell from a PC’s price tag if it has dedicated graphics memory. Sadly, in the world of embedded devices, this is not the case. High-end devices often have UMA and low-end devices (especially set-top-boxes) have dedicated graphics memory.

The next differentiation is the graphics operations supported by the hardware. Generally they are wide ranging but can be loosely categorised as:

1) No acceleration (framebuffer only)
2) Blitter & alpha-blending hardware
3) Path based 2D vector graphics
4) Fixed-function 3D
5) Programmable 3D

Hardware with no acceleration whatsoever or a simple video overlay is the most common we see in embedded devices. This will always be the case until someone figures out how to design and manufacture silicon for free. Blitter and alpha-blending hardware is almost non-existent on desktops these days, but it does seem to still be around in the current generation of embedded hardware. Path based 2D vector graphics is pretty new and looks ready to replace blitter-only style hardware. NOTE: This does not refer to hardware which can draw a 1-pixel wide, non-anti-aliased, non-dashed, solid-colour line without clipping. Fixed-function 3D tends to be the older generation of desktop graphics processors. Generally, fixed function has pretty much been replaced with programmable 3D. This is even the case on mobile hardware.

So, there’s five categories of operations and two types of memory architecture leading to ten different overall types of graphics hardware. I’ve collected an example of each, just so you know we don’t make this stuff up.

Type UMA Non-UMA None Marvel PXA270 Various* Blitter NXP PNX8935** Fujitsu Lime MB86276*** 2D vector Freescale i.MX35 Fixed-3D Freescale i.MX31 nVidia GeForce 2 Programmable-3D TI OMAP3530 AMD Radeon HD 4600

* Various: Some devices use dedicated framebuffer memory to reduce load on the system memory bus
** NXP PNX8935: http://www.nxp.com/applications/set_top_box/ip_stb/stb225/
*** Fujitsu Lime MB86276: http://www.fujitsu.com/downloads/MICRO/fma/pdf/MB86276.pdf

The next question then becomes: How can Qt off-load graphics operations to these different types of hardware? Well this is done through Qt’s QPaintEngine API. The idea is that Qt applications (& Qt itself of course) always uses QPainter, which in turn uses one of the paint engines. To take advantage of graphics acceleration, we write a new paint engine (like the OpenGL ES 2.0 engine we’ve added in 4.5.0). The advantage is that existing applications can benefit from new rendering back ends and new applications can still work on older or less advanced hardware (albeit with lower performance). There seems to be a misconception in the community that Qt is out-of-date because it has no OpenGL scene graph API. While that statement is technically correct, Qt does have QGraphicsView scene graph API which uses QPainter. Because it uses QPainter, if OpenGL (for example) is available, it can be used as the rendering back end.

So, now that’s cleared up, what QPaintEngines are there and do we have all the hardware acceleration types covered by them?

Well, for devices with no acceleration, Qt will use it’s raster paint engine. The raster engine has seen some very impressive optimizations in Qt 4.5, as Gunnar has previously blogged about. For higher end graphics hardware, there’s usually a nice high-level API which is powerful enough to express all of QPainter. I.e. OpenGL & OpenVG. The trouble we’ve recently hit is the hardware in-between, I.e. those with blitters but not much else. Such hardware is not powerful enough to express the whole of QPainter, so we must fall back to the raster paint engine for unsupported operations. The raster paint engine needs a pointer to the memory it renders to (and reads from). On UMA systems, this is not a problem as the buffer is obviously in system memory (that’s all there is!). It’s on systems with dedicated graphics memory where the fun begins…

First, on many systems, you simply can not map graphics memory into your process’ address space - The architecture simply has no way to do it. On such systems, the buffer must be copied to system memory, rendered to with the raster engine, then copied back. If this happens _every_ time you switch between a fall-back and the hardware, it’s going to be _slow_!!

On some systems (particularly PowerPC for some reason?), the graphics controller sits on the SoC’s external bus and can be addressed directly by an application. All that needs to happen is for the kernel to configure the process’s page table to point to the graphics controller’s memory range. It’s then up to the graphics controller to access data in it’s dedicated memory on behalf of the host CPU. Although this kind of set-up does allow the raster paint engine to get a pointer to graphics memory, all accesses go over this external bus - which is usually slow. On PC/x86 architecture, things get more even more complicated, the kernel has to fiddle with lots more hardware, cache controllers, PCI bus controllers, IOMMUs, etc. However, in all cases, if you’re lucky enough to get a pointer to graphics memory, all access must go over a slower external bus.

So now we know what’s going on, what conclusion can we draw? Well, reading & writing to external graphics memory is slow. If your on non-UMA, don’t have OpenGL or OpenVG available, but do want to use your blitter then you’d better make sure your mostly using QPainter::drawPixmap(). NOTE: Graphics view’s cache modes can help you out a lot there - see Andreas’ previous posts! Otherwise falling back to the raster engine is going to be slow. Fortunatly, this type of hardware is (finally) on it’s way out.

NOTE: I should also mention that there’s a similar issue with X11. There’s no API to get a pointer to an X pixmap and X11 does not provide enough API to implement the whole of QPainter. While the X11 paint engine does not inherit from the raster paint engine, it does make use of software fall-backs which involve copying the pixmap, executing the fall-back and then uploading the result. It’s for that reason that we’ve added the raster graphics system which uses system memory (via the MITSHM extension) in Qt 4.5. On desktop, this is a fairly temporary measure until our OpenGL 2.0 engine & graphics system is in a fit state to take over all of Qt’s rendering. No promises, but we hope that can happen for Qt 4.6. For X11 on low-end embedded devices (like the n810), MITSHM provides a pretty decent long term solution.

So, when we look to the future of Qt’s graphics architecture and the required paint engines, I think we’re well on the way to having all the bases covered:

Type UMA Non-UMA None Raster Raster* Blitter DirectFB DirectFB** 2D vector OpenVG*** OpenVG*** Fixed-3D OpenGL (ES) 1.x OpenGL (ES) 1.x Programmable-3D OpenGL (ES) 2.x OpenGL (ES) 2.x****

* When using raster on NUMA, rendering is actually done in system memory first, then flushed to VRAM
** This is the one which is going to be slow when doing anything other than QPainter::drawPixmap()
*** It shouldn’t be a big surprise we’re researching an OpenVG paint engine!
**** Qt 4.5 contains a new paint engine for OpenGL ES 2.x which we’re now making work on desktop OpenGL 2.0

I just want to finish by asking you to take another look at the above table. Do you notice anything interesting? All of the graphics systems (apart from DirectFB) are cross platform which means, when we make something faster in one engine, all platforms will benefit.

In the comments to my previous post, a few pointed out that the states & transitions API we’re researching seems generally useful (i.e. outside of Kinetic context). Hence, we’ve released a stand-alone state machine add-on that you can play with. It’s been tested with Qt 4.4 and 4.5. Enjoy, and don’t be shy about giving feedback.

Now into something a bit different. Today’s Graphics Dojo example is related to WebKit (again), namely using it as a basis for WYSIWYG (what-you-see-is-what-you-get) editor. Thanks to Apple engineers and other WebKit hackers, WebKit’s editing features have improved a lot so that it is perfectly possible to use it to let the user edit the web page.

These days, rich-text editing is supported quite well in many web browsers. Take a look at Mozilla support for editing, which gives an idea on how to start doing it. Head also to the short overview on HTML and JS-based editors to get more information.

The editor demo is available at the usual git repository. Too lazy to use git? Just grab the archive (65 KB) and unpack it. Check the editor under the htmleditor directory. You need Qt 4.5, which has just been released.

To get an idea what this little editor can do, have a look at this short screencast.

A couple of things I showed in the demo are character formatting (bold, italic, underline, font name, size, colors), paragraph settings (style, alignment, indentation, lists), image and hyperlink support. You can also insert any HTML code. For editing convenience, undo and redo are even possible. Switching to "HTML Source" tab will reveal the raw HTML code for the page (note the syntax highlight).

Since this is just a quick-and-dirty example, there are few things which are not done the right way. For example, due to the lack of our QtWebKit public API (which we plan to address for Qt 4.6), I need to resort to JavaScript code for some functionalities. Also, things like paragraph alignment, heading style, font size, etc are stateless, mainly because there is no easy path to get the state value. Of course, this can be solved using a few DOM and JS magic tricks commonly employed in popular online editors, however I am still looking for a much cleaner way. Feel free to point me to some leads and/or give some feedback!

In the spirit of doing releases and more specifically, Jasons doing releases (or at least blogging about them), I am happy to present to you the latest release of the Qt for S60 port! Go ahead, download it in the background while you keep reading:

Download Qt for S60 “Garden”

Following fresh on the heels of the Qt 4.5.0 release, our latest release, commonly referred to as “Garden”, is a real showcase for all the hard work that we have done over the past 4 months. Espen set the bar pretty high when he mentioned our ambitions for Garden back in December, but I am quite proud to say that we pulled it all off in the end. One of the most obvious differences you’ll notice from the previous (Temple) release is that this one is based on Qt 4.5. That means we got a bunch of performance improvements and other features practically for free!

One of the main goals of this release was to cleverly disguise Qt applications as native S60 applications and see if we could fool our grandmothers. This was no easy feat considering that S60 has interchangeable themes, input methods, and special menus and adding to that the fact that our previous release looked like a certain OS from 1995, we knew we had some work to do. Well, grandma put on her reading specs and fired up her XpressMusic 5800 to have a look and she was pleasantly surprised!

She immediately saw the application themed correctly and could input text like she did in other S60 applications (they call her “CrazyThumbs” at bingo) and after trying some other Qt applications she could use menus and even drag-n-drop worked! Well, needless to say she had no idea that she was using the same application framework that powered the desktop environment on her Linux box!

Ok, far fetched fantasies aside, there are tonnes of other improvements in the Garden release. We have rewritten large parts of our event loop and also QWidget so that QApplication and QWidgets can be easily embedded in native S60 applications. In fact, we even introduced a new static library called ‘qtmain’ to hijack the entry point and construct a full S60 application before calling an application’s main() function. With the full S60 environment at our disposal we could start integrating with things like the status pane and menu pane thus making functions like QWidget::setWindowTitle() work properly. Finally, we ported a few new classes such as QLocale, QClipboard and QDesktopServices. There are lots of other things I could mention, but I don’t want to steal all the joy from reading the changelog

To get you up and running with the build, here is Alessandro to explain the details:

Finally, here is a video from Espen where he takes you on a walk through the Garden (release):

The next steps for us will be to evaluate where we go next with the port. I won’t cover that here today since it warrants a blog post of its own so I’ll keep you in suspense. Enjoy!

Getting help and providing feedback

Pre-releases are not supported, but we still want your feedback of course. We have set up a special mailing list for feedback on the S60 port:

qts60-feedback@trolltech.com

This list is read by the developers working on the port, so please join if you want to provide technical feedback, bug reports or suggestions to us directly. In order to join the mailing list send a mail to:

qts60-feedback-request@trolltech.com

There is also a Nokia Forum available here.

Solid Dragon

It’s time to present the little hack that I’ve been working on for a while now: A Mac OS X backend (source code) for Solid, KDE’s hardware introspection library. To make a good screen shot, and to show how easy it is to use Solid, here’s also a small hardware browser:

The backend is still in progress, and not all Solid hardware interfaces are implemented yet. Still, the generic interface can be used to introspect any device and read out its properties. You can even introspect the famous Dont_Steal_Mac_OS_X device

One of the nice things about Solid is that it can be used from non-KDE applications as well - it just requires Qt (*)

64-bit Leopard

Next to playing with Solid, the release of Qt 4.5 was a good excuse to play with 64-bit KDE on 10.5 (Leopard).

First, we need the magic CMake switch to build 64-bit Mac OS X applications: -DCMAKE_OSX_ARCHITECTURES=”x86_64″ (**)

Unfortunately, some of kdelib’s files rely on deprecated API which doesn’t exist in the 64-bit versions of OS X’s libraries. So after some fixing and some more commenting out of non-compiling stuff, I finally got my first KDE 64-bit application:

minibrowser.app/Contents/MacOS/minibrowser: Mach-O 64-bit executable x86_64

Screen shot: see above

 

(*) Well, there’s also a D-Bus dependency

(**) You can produce universal binaries by separating the desired architectures with a semicolon, e.g. -DCMAKE_OSX_ARCHITECTURES=”x86_64;i386;ppc;ppc64″ to get all four currently supported architectures. Unfortunately, I have no clue how to create universal binaries for the third party libs that KDE depends on (e.g. D-Bus) - any automake expert, please leave a comment (or convince the projects to switch to a more humane build system).

Of course, I cheated. The trick is actually by using Google’s own iPhone web application, which is pure HTML and JavaScript. Despite the name Google Talk, unfortunately it has no support for voice chat, rather only text chat.

The secret is to find the actual URL and load it in QWebView. Since initially it presents a login page, this particular client also automatically fills the username and password for you, getting them from its own login page. Overall, the code is very simple and does not even reach 200 lines, including some rudimentary error handling (wrong password, failed login). The user interface was created completely using Qt Designer. Careful readers might notice that I deliberately added the flick support (kinetic scrolling) using Flick Charm. This quick-and-dirty chat client was completed in less than quarter an hour.

document.getElementById("webgtalk").style.maxWidth="800px";

Check the code at the usual Graphics Dojo git repository, find it under the webgtalk folder. This example requires Qt 4.5, which has just been released, in particular due to the use document.getElementByid. Exercise for the reader: tweak the little JavaScript snippets used in this program so that it works also with Qt 4.4. Another challenge: there is also
chat client for iGoogle which performs better but then it requires Flash plugin.

Since Qt 4.5 is released already, it is about time we - the QtWebKit team inside Qt Software - show few cool features that you can get from QtWebKit in this new release. This time let us focus on JavaScript. If you read the What’s New on our website, it mentions something about “lighting-fast JavaScript engine”. Fast is qualitative, right? As software engineers, we always want to know how fast is fast.

For Qt 4.5, the JavaScript engine is powered by a new bytecode interpreter, along with optional JIT (just-in-time) compile support. They are often referred as SquirrelFish and SquirrelFish Extreme, respectively. This interpreter has been developed by Apple engineers working on WebKit in order to speed-up JavaScriptCore, the backbone of WebKit’s JavaScript engine. The interpreter makes its first appearance in the recent Safari 4 Public Beta, where it is dubbed as Nitro JavaScript Engine.

Specific to QtWebKit in 4.5, we enable the JIT compiler for the following supported platforms: Linux with gcc 4.1 or newer and Microsoft Windows with MSVC. There are several reasons why other platform combinations are not supported (yet), but the main reason is our own priority and resource availability. However, even without the JIT, the new JavaScript engine should be much faster already compared to the one shipped with Qt 4.4.

To compare the performance, here is the result running SunSpider benchmark with our demo browser in Qt 4.4 and Qt 4.5, the latter with and without JIT. The test machine is a typical four-core desktop running OpenSUSE 10.3. The test duration is converted to the number of runs per minute. Longer is better.

document.getElementById("qtwebkit-4.5-sunspider").style.maxWidth="800px";

Let us try another one: V8 benchmark suite. The raw scores are shown here, again longer is better.

document.getElementById("qtwebkit-4.5-v8bench").style.maxWidth="800px";

So even without JIT, JavaScript-heavy application could already benefit from the speed-up. And if you plan to build hybrid web/C++ applications, do not be afraid to put more and more application logic in its JavaScript side.

PS: For Qt 4.6 we would evaluate the possibility of using JavaScriptCore as the back-end for QtScript. However, please treat it like a research project, hence do not hold your breath as right now we can not give any guarantee that it will make it for 4.6.

There’s a new example in the animations branch called “Animated Tiles”. It’s nothing very fancy, showing how to animate transitions between five defined states.

But it does qualify for youtube material :-).

If you want to access the code, visit the Kinetic labs page: http://labs.trolltech.com/page/Projects/Graphics/Kinetic

Discussions are on #qt-kinetic on irc.freenode.net.

Enjoy!

(update 13:41: updated the url)

Hidden behind the great news about Qt 4.5 and QtCreator 1.0 release, there is this:

Qt Software discontinues Qt Extended

But all is not lost, ye fans of open source phones! The Qt Extended source now lies solely in the hands of the community. You can find an unoffical git repo of Qtopia and Qtextended here

Not sure when this will be updated with 4.4.3..
git clone git://git.asheesh.org/qtopia_snapshot.git

Yes, it is a bit sad when all you have worked for the last 5 years or so gets axed, and we have had a few anxious weeks here in Brisbane finding new things to work on, but hey - when one door closes, another opens!

Qt Everywhere!

and if I may make a personal rant.. would you know it, just barely a week after I bought a mac mini… they announce a new version!

… and not only that. Qt 4.5 was also released today. Hence we decided to bundle up both Qt and QtCreator together into a “Qt SDK” that contains everything you need to begin developing with Qt.

I can say that the new Qt SDK packages containing precompiled Qt and Qt Creator for the different platforms kept some of us quite busy in Berlin the last weeks :-). All of you who already tried the Qt Creator package on Windows know that we did this packaging already for this one platform, to avoid that people need to download and install mingw, and download and compile(!) Qt with mingw. But we learned that doing it for one platform is no(t so much a) preparation for doing it for all platforms and license variants:
Windows (OpenSource, Commercial, Evaluation),
Linux 32 bit (OpenSource, Commercial/Evaluation),
Linux 64 bit (OpenSource, Commercial/Evaluation),
Mac OS X (OpenSource, Commercial, Evaluation).

And we had no binary packages for Linux at all in the past…

But the result was worth the effort, every minute of it. Just go to the download page and see how it feels to get going with Qt 4.5 right away.

On the QtCreator side I’m happy to tell you that we got yet another new application icon, better following the style of the other Qt icons. I think we made the best out of it :

And, of course, we made QtCreator available under the LGPL as well. Currently talks are going on with different packagers for Linux distributions, so hopefully you’ll be able to just press a button in your Linux distribution to get the newest Creator. All packagers who happen to read this: Please contact us, we are usually hanging around on irc.freenode.net, #qt-creator channel!

If you already used the QtCreator release candidate don’t expect too many differences - except for lots of bug fixes, crash fixes and so on. So please don’t be angry if your feature request XYZ didn’t make it into the final release. Especially since we are already hacking away for the next release, and this will not be far in the future. During the whole development time we had “release early, release often” prominent in our minds (in the very beginning doing internal releases), and this isn’t gonna to change. Though I won’t promise anything, we have something like 8 week cycles in our mind.

So just have a look at what is currently happening in QtCreator, either get the sources yourself from the open git repository (notice that you can get a .tar.gz source snapshot of the state at a certain point in time via the snapshot links), or download one of the binary snapshots we provide, which we will update to be based on the future-to-be-1.1-release.

As a last note, some of you who read Jason’s blog might ask what this strange looking building in the background of the Berlin group picture is: No, it’s not our offices. But it’s next to our offices. We are lucky to work in a historically important area, which was always a place of great (and not so great) invention and science the last hundred years. The funny spheres were thermodynamic labs with constant conditions inside of them.

I am just posting the link of Artur who work in INDT. They are working with us on animated layouts using Qt Kinetic.

Click for the full blog post

PS : Oops, i missed to say that it works on Windows too.

Today we is a day that will be remembered for a long time in Qt history (I expect that we’ll remember it all the way until next week at least — that’s at least a thousand commits). I made today two 280,000-line changes to Qt, touching over 6500 files in each. At the end of the day, three Qt branches (4.6, 4.5 and 4.5.0) now contain the LGPL license header in all Qt’s .cpp and .h files, plus an assorted set of scripts. Third-party code is obviously excluded from this change. That means the GPL era of Qt comes to its end — and LGPL starts.

Today, I stopped the cron job that creates and publishes the Qt snapshots. Mostly because the LGPL and other changes are very likely to break stuff. And that we don’t want the snapshots published under the LGPL until we actually release 4.5.0 next month. What’s more, I don’t know if snapshots will ever come back: maybe we will go directly to the open repository. That’s the end of the snapshot era.

Today, Alexis also made changes to the Qt repository in Git, removing the never-released files and adapting the the license files. He also refactored the license part of the configure script and the Windows configure.exe (it’s a good thing that I turned snapshots off, because he went to ski shortly afterwards in a classical example of “submit-and-run” ). That’s one of the last steps required for the open repository, though there are a few minor things to go.

This week, the temperature in Oslo reached 0°C again. That means the snow is starting to melt and the streets are very dirty now. The mountains of snow that we have collected over the past few weeks will gradually disappear. That’s the end of Winter, but I hear it will come back (I don’t put too much faith in those rumours).

This month, I’m also stepping down as Release Manager for Qt. I had pre-announced this at Developer Days last year, but it’s effective now: after 18 months and releasing Qt 4.3.3, 4.3.4, 4.3.5, 4.4.0-tp1, 4.4.0-beta1, 4.4.0-rc1, 4.4.0, 4.4.1, 4.4.2, 4.4.3, 4.5.0-tp1, 4.5.0-beta1, 4.5.0-rc1 (and two mac-cocoa alphas), it’s time I pass the torch to the next poor sob brave soul to take on the job. So this is also the end of my era as Release Manager. (No, I’m not leaving Qt Software, I’m just moving on to other responsibilites)

It’s interesting to note that the change in RM matches the change in environment. It’s tradition that each new RM gets to rewrite the release scripts from scratch. The RM before me had all of it working for our setup with Perforce. I rewrote it to use Git. After me, you can’t call them release scripts anymore, because I rewrote them in C++, using memory-mapped files and QtConcurrent. Now my successor will have the chance of rewriting it to match the open repository. It looks like packaging will be a lot simpler, since there will be no file editing or removal.

This is not a sad time. At least, I haven’t seen anyone crying their eyes out in the office. So I must conclude that this is a happy time: the end of an era marks the beginning of a new one. We can only hope that the new era will be even better than the current. We’re certainly going to make our best effort.

Disclaimer: this blog could be part of a conspiracy against Qt on Windows.

Qt already provides a set of standard icons. The nice thing about these icons is that they adapt to the current desktop environment. However the current selection of available icons is rather limited, at least compared to what KDE apps have at their disposal through the freedesktop icon specification. Since most of these icons only are available on the X11 platform, we still haven’t decided how to make them available to pure Qt applications such as Designer and Creator. While we are thinking about that, I thought I might as well wrap up the essential code and provide it as a simple convenience class, QtIconLoader, that you can use in your own project. You can find the source code here.

The class currently only provides one static function:

QIcon QtIconLoader::icon(const QString &iconName, const QIcon &fallback = QIcon()).

Usage should be fairly obvious:
new QAction(QtIconLoader::icon(”document-new”), tr(”&New”), this);

The fallback is of course for platform compatibility so you can provide your own masterpiece as a fallback for other platforms, which is probably the icon you are already using.

And since no blog post is complete without some eye candy, I modified our well know textedit example to QtIconLoader:

Here is how it looks running on KDE 4.2:

And since we also pick up GNOME settings, this is how it looks when running GNOME:

I’m sure there are bugs. For some reason freedesktop standardized on everything except a convenient way to detect which theme to use, so we need to do some trickery to get to the KDE and GNOME settings. Feel free to report issues and suggestions here.

Hi fellow hackers,

I have found a bug in my brain, it does not emit the readyRead signal properly which means all the public information that was meant to be published at appropriate intervals is not processed. Here follows a manual flush of what I’ve been doing since last time. Talking about everything is too much, so I’ll give you a brief overview.

Last year was pretty much spent on optimizing/re-factoring code related to the Qt Falcon project where the goal was to make Qt 4.5 fly. I mainly focused on painting performance for widgets, i.e. the backing store / window surface infrastructure. After spending a fair amount of time profiling different painting scenarios, I found out the backing store infrastructure was too poor to improve. It simply had to be re-factored in order to make it fly. I finally had an excuse to wear my latex laboratory gloves and white coat in public (I usually only wear this kind of equipment behind closed doors when lobotomizing colleagues). With all the equipment in place–including liters of coffee–the time had come. I was about to replace our platform dependent beast with something new and shiny–preferably cross-platform. After some tough work and patching throughout 2008 I ended up with something useful. Let me remind you that there are no problems, only solutions. I basically removed a lot of QRegion operations and made the update mechanism a whole lot smarter, in addition to many cut-offs for opaque widgets. I will leave out the details and let these numbers speak for themselves:

Qt 4.4 Qt 4.5 Boost (2.0x means twice as fast) Full update (transparent widgets) 4330 ms 2446 ms 1.8x Full update (opaque widgets) 3352 ms 1464 ms 2.3x Scroll (opaque widgets) 50231 ms 3746 ms 13.4x Partial update (opaque widgets) 4246 ms 1567 ms 2.7x Complex update (opaque widgets) 4966 ms 2265 ms 2.2x Full update (opaque children) 3376 ms 1464 ms 2.3x Move (opaque widgets) 50293 ms 4188 ms 12.0x Mass update (opaque widgets) 9560 ms 1679 ms 5.7x

Download the sources from here.

With a new and optimized backing store, we were one step closer the goal. While I was working on the new backing store, the other falcon guys were working on a brand new graphics system whose goal was to make it possible to switch the rendering back-end. In order to achieve that, all the painting had to go through our backing store. We had never used the backing store on the Mac before, so my next action point was to put the pieces together and make it happen. Voila, “-graphicssystem raster” on the Mac was no longer just a wet dream. And yes, it was a little bit harder than it sounds.

During all this hacking I also decided to go to DevDays in Munich and Redwood, which implied that I had to prepare two presentations; “Advanced Visualization, Widgets On Graphics View” and “Alien Widgets & Widget Rendering”. I spent quite some time making these presentations the best since sliced bread, and I really hope they were entertaining and interesting. Dev Days 2008 was such a great event and I really enjoyed my stay.

Few days later, back in business here in Oslo with turbo charged batteries, I felt a bit disappointed that I couldn’t show off a graphics view embedded inside graphics view embedded inside another graphics view etc. I knew it wasn’t going to work, so I didn’t even try. However, I made an example having exactly two redirections and that seemed to be more than enough for the hackers in the room, based on the applause it received. This limitation was made on purpose to ease and simplify the shared painter and QWidget::render. However, I was not satisfied with it and thought it would be cool to remove it without adding overhead, so I sat down and did some brain work. And as usual there was a solution to my problem. The shared painter, and hence QWidget::render, now supports an arbitrary number of redirections. Have a look at WolfenQt to see it in action, there’s no limitation of how many times you can “walk into” another graphics view. HTML-5 video in QWebView is another example. Pretty cool.

With that out of my way, I found time to look into a very annoying issue that has followed us since Qt 4.0: “Qt::WA_StaticContents does not work”. Well, it worked for Qt::WA_PaintOnScreen top-levels, but not in general which is really bad since the attribute can have an incredible impact on the performance on resize. It basically means “Dear Qt, do not repaint my entire rect on resize and only repaint newly exposed areas instead”. Just think of it next time you expose yourself for the sun; does it make sense to wear off all the clothes and put on sun lotion on the entire body just because parts of it is exposed? I hope not. I therefore decided to implement the Qt::WA_StaticContents attribute for all widgets in Qt 4.5.0 and here’s the result:

Download the sources from here (run with “QT_FLUSH_PAINT=10″ in debug mode to flush repainted areas with yellow).

I have done several other things too, like fixing lots of bugs, reviewed patches, fixed auto tests, helped out with recruitment, promoted Qt Software at various events and so on. All in all very exciting stuff, and I look forward to new challenges and exciting stuff in the months to come.

OK, that’s it for now. I promise to blog more often in the future, or even write a QQ article (mostly to give back something to Kavindra who has provided me with an unknown number of delicious muffins).

Have a nice weekend and happy hacking!

If you want to learn Qt, and French is your favourite language, we have good news for you: the main Qt tutorial for Qt 4.5 is now available with French explanation. Pierre and I have translated the tutorial with the help of Kavindra and David and we have successfully got everything to work (encoding problems are more easily solved in Qt than on the Web…)

If French is your mother tongue, you will be happy to learn that there is a lot of resources available in the French Qt community. One very active community is http://qt.developpez.com/, who have translated the complete Qt tutorial. They have a forum, many original articles and new translations coming soon. Also, there is another community, QtFr, with tons of articles and an active forum.

Si vous voulez apprendre Qt et que le français est votre langue préférée, nous avons une bonne nouvelle pour vous: le tutoriel de Qt 4.5 est maintenant disponible avec les explications en français. Pierre et moi avons traduit le tutorial avec l’aide de Kavindra et de David et nous avons réussi à tout faire fonctionner (les problèmes d’encodage sont plus facile à résoudre dans Qt que sur le web…).

Si le français est votre langue maternelle, vous serez heureux d’apprendre qu’il y a énormément d’aide disponible dans la communauté francophone. Une communauté très active est sur http://qt.developpez.com/, ils ont entièrement traduit le tutoriel Qt. Ils ont aussi un forum, tout plein d’articles originaux et de nouvelles traductions sont attendues prochainement. Sans oublier la communauté QtFr qui propose des tonnes d’articles et un forum.

It has some time gone since I implemented the plasma widget to show the in-time departure information for local public traffic in Dresden, so I thought it is the right time to do something similar again. My current location is Oslo and it is really cold here (-14 this night), so you really don’t want to wait 10 minutes at the bus station for the next bus. Fortunately the local transport services trafikanten.no provides a WAP service that allows you to retrieve the departure information of busses, trams and subways as WML pages. WML is a XML based markup language, so one would immediately start the favorite editor and hack together a bunch of code that uses QNetworkAccessManager to load the document from the internet and QDomDocument or QXmlStreamReader to parse the documents for the relevant information. However this approach needs a lot of glue code, and writing that wasts a lot of valuable time…

So today I want present a more efficient and smart way of doing that: QXmlQuery
This class is part of Qt since 4.4 and provides an implementation for XPath and XQuery (in 4.5 also XSLT), both are query languages to extract parts of XML documents with a rich set of filter rules. Extracting the time information from our WAP services needs only 5 - 10 lines of code + the code for representing everything on the screen:

I could now list the source code of this application here and explain how it works, however that would be duplicated work because that is already done here

So the application is only based on Qt, as it’s part of the example applications for Qt 4.5, but maybe there is some brave soul that converts it into a real plamsa widget? You can currently choose the station you want to departure information for and even filter for the bus/tram/subway number you are interested in. More features are not planned, to keep the example small and if you live outside Oslo, the departure times won’t help you anyway but it is an excellent example that shows, how to use QXmlQuery for accessing web services easily.

Sebastian Kügler blogged yesterday about why distributions shouldn’t jump on upgrading to Qt 4.5 while shipping KDE 4.2. This caused quite a stir in the community as well as inside Qt Software. Sebas then replied to Cyrille’s blog explaining a bit more (if the link doesn’t work, scroll down; blogspot comment links don’t do anything for me).

Sebas is right: Plasma developers did not test KDE 4.2.0 with Qt 4.5. So there may be issues that they have not seen or corrected. That’s entirely true and no one can blame the Plasma developers. After all, resources are limited and prioritising the work was necessary. Therefore, distributions should be careful when upgrading Qt, because they may introduce problems. But, then again, isn’t that the job of distributions? To make sure their package set works without issues? And in case patches are needed, they can backport fixes from upcoming releases or introduce a temporary hack.

However, Sebas is right only up to a point. We at Qt Software have done testing of Plasma with Qt 4.5. In fact, we know it works better with 4.5 than with 4.4.3, though some of the qt-copy patches help somewhat. Has anyone noticed that the system tray finally works in 4.5? But, more importantly, other parts of KDE (other than Plasma) benefit greatly from Qt 4.5.

Just to give you an example: I still build my KDE trunk with Qt 4.4, then run with 4.5 on my main development workstation, but with the standard 4.4 (pristine, unpatched) in my laptop. After logging in to KDE, I usually get Kontact crashing within 5 minutes of using: when I click on a folder, it crashes with a dangling pointer dereferencing. Then I remember I have to restart it with QT_NO_SHARED_PAINTER=1. But this crash never happens on my workstation: standard 4.5, pristine, unpatched.

What’s more:

• We did lots of testing and fixing of KDE 4.2 for Qt 4.5, but not with Qt 4.4
• We did a lot of fixing and stabilisation in 4.5 itself and most of these changes did not make into 4.4
• Performance improvements in 4.5 are noticeable, especially in the painting code and itemviews

I can hear the arguments saying that those fixes should have bene done in the 4.4 line and that we should have prioritised that over the feature release. I know those arguments because the discussion happens in Qt Software too: it’s only natural to have it. But the same way that Plasma developers were faced with a finite amount of resources, so were we: we had to prioritise work. Qt 4.5 is the future and we needed to stabilise it, so we did focus most of work there. (We did not forget 4.4, you can find several fixes in the Qt 4.4.4 snapshots)

And I don’t have to say that we can’t fix further issues unless they are reported to us. If people don’t use 4.5, those issues will not be found (read: our QA people can’t find every single issue). With the 4.4 release, we had a very good feedback from the KDE community: KDE 4.1 started using 4.4 very early, I think even before the beta (Feb 2008). That means we had a lot to work with in order to stabilise Qt 4.4. By the time of the 4.4 release candidate, we had had two months of continuous feedback and stabilisation work done. And we have one more month left, more or less, by which time KDE 4.2.1 (with its own set of fixes) should be out.

And my personal, subjective experience comparing now (4.5 RC) to 10 months ago (4.4RC), things are a lot better and more stable.

I am actually in Porto (Portugal) for the Tokamak version 2.0. Tokamak is a developer meeting for the Plasma KDE project. I arrived Friday and the first day was talks, people explaining on what they are working on for KDE 4.3. There will have some awesome features that make Plasma the best desktop ever. From my side i gave a talk about Qt Kinetic to introduce them the project. The first feedback i had was that this project is warmly welcome, and miss to plasma developers (and i guess to others). I introduced the Qt State machine framework, the Qt Animation API and talked about improvements in QGraphicsView. I had nice feedback from people and they were pretty happy about the API. I wrote all their remarks on a paper to work on them with other trolls. I have already implemented some of them inside my local copy of Qt Kinetic :D. The team is now waiting a Qt Solution in order to “kineticize” Plasma.

I have cooked a small demo showing Plasma running Qt Kinetic :

Ogg Video

You can see an animation on the opacity during switching pictures and an another to display one day meta data behind the picture.
Here is the code for the opacity animation :
QPropertyAnimation *opacityAnimation = new QPropertyAnimation(this,"opacity");
opacityAnimation->setEndValue(0.);
opacityAnimation->setDuration(1000);
opacityAnimation->start();

By the way Qt 4.5 RC is now in qt-copy and KDE trunk use it. After talking with all people here the switch is apparently nice. I have asked to all people to give a mark from 1 to 10 (ten is best) based to their user/developer experience. The average is 8 which is nice. According to Aaron Seigo the switch in Plasma was “pretty smooth” and Leo Franchi from Amarok said “it seems faster with raster graphic system”. Marco Martin added “QGraphicsLayouts start to work”. There are still some small problems with SVGs rendering in Plasma but i will sit down with Marco to fix them. It is encouraging for the final release and we will have now KDE folks reporting bugs they still have with 4.5 to make this release the best as we can.

I’m working on cleanups in Graphics View’s cache mode / offscreen rendering support. Did the first part together with Alexis (who’s on Tokamak these days btw!). Then I spent lots of time trying to get everything as simple and clean as it can be, in preparation for a few changes that will probably be useful for most QGV hackers. We started with a few clean-up goals:

• Removing as many string operations as possible from QPixmapCache handling
• Making a proper struct for storing all info associated with the off-screen buffer
• Ensuring that we use as little pixmap memory as necessary
• Removing rendering glitches / chopped off corners and so on
• Replacing today’s bounding-rect-only support for partial updates of the cache with proper QRegion support
• Clipping to the exposed area to speed up partial cache updates

Then we had some higher goals, what we’d like to achieve with the cache in the future. Some examples are:

• Optimal performance with software and hardware rendering, especially on embedded hardware
• Removing “out of memory” bugs when you zoom too far in on DeviceCoordinateCache items
• Adding sub-surface caching to allow items and their children to be cached into the same surface
• Using an intermediate buffer for QGraphicsItem::opacity to avoid source-over glitches even in NoCache mode
• Adding support for graphics effects that make use of the off-screen buffer

Well, the struct is in place, the stringops are gone, we added region support, and we made sure we don’t use more pixmapcache entries than strictly necessary. All the rendering glitches that have been reported (e.g., for flat items, chopped-off corners etc) are also gone. So today, I spent my Creative Friday experimenting with the “out of memory” bug in DeviceCoordinateCache and ended up with something pretty cool :-).

DeviceCoordinateCache stores the item in an offscreen buffer with the same as the item’s bounds in device coordinates (pixels on screen). The item is painted once only for each time you transform the item or the view, and when you move it around it’s just blitted from its buffer. KDE people can see this in action in plasmoids - moving them around is fast, and the plasmoids are not rerendered.

The problem is when the item is large, or you zoom in, this buffer typically gets pretty big. Zoom in more, and your app is likely to crash. Not so good. Try “print preview” in Assistant or the Text Edit demo and zoom in 8x to see what I mean (yes! it’s embarrassing!). The excessive memory used to store the whole item seems even more silly when the only part that’s drawn is quite small, and the rest is just wasted.

The solution I researched today is to clip this buffer to the viewport. In the image below, a large ellipse item is cached in DeviceCoordinateCache mode, but only the segment that’s visible in the viewport is actually cached.

By clipping, we effectively cap the amount of memory this buffer can use. So no matter how far you zoom in, the same max pixmap size is used. Now, when either the viewport scrolls, or if the item moves, it’s easy to just regenerate the cache by rerendering the whole item. But it’s even more tempting to do “smart stuff”, i.e., create a new pixmap, copy over the old content, and rerender the exposed areas tightly clipped, just like QWidget::scroll does for when you scroll a widget (blit+update instead of complete rerender). Something like this:

As the item moves by (dx, dy), the last (clipped) DeviceCoordinateCache buffer is in effect “shifted” by (dx, dy), and then copied into a replacement buffer which is (dx, dy) larger/smaller.

What’s cool about this approach is that it gives you almost the same effectiveness as QWidget::scroll, but it works with both scrolling the viewport, or moving an item, or even translating it, regardless of its current transformation. And as opposed to QWidget::scroll, this approach allows semitransparent surfaces to be “scrolled”. In fact the next step is to hook this up with QGraphicsItem::scroll, so that when you scroll an item that’s cached, you’re only scrolling its cache buffer (as opposed to today, where, ok I think this is pretty cool!, the scroll propagates all the way up to QGraphicsView::scroll, even for proxied widgets / try embedding a QWebPage and scroll it! try rotating 90 degrees and scroll again, it also works). I can’t wait to see the performance of embedded QWebPage scrolling :-).

With these changes, it’s easy to make a scrollarea in QGraphicsView that has several semitransparent layers (something Michael, a Brisbane troll, and I have discussed a while back), and when sliding the “content” layer, this is done by scrolling the offscreen buffer and reexposing. It’s also bringing us closer to integrating composition effects like dropshadows, bloom and colorize, pixelate or even mesh-grid-transformations. Next week will be fun :-).

For those interested, I made a patch that applies the the 4.5.0 RC which shows progress so far (wish I could just share my branch, in the future I hope I can!). Click to download patch.

PS: A sideeffect of this patch is that DeviceCoordinateCache items, which earlier were only rendered completely or not at all, are now rendered “partially” when new areas are exposed. The print preview mode in Assistant is actually slower than usual because of this, yet it gives you the same performance regardless of the scale. Obviously this isn’t good enough (it’s caused by slowness in QPicture, which performs the same no matter how much/little is rendered at a time) so it requires some more work.

PPS: If you really don’t want your items to be rendered “partially” like this, but rather keep the same behavior as today, well, I don’t have any answer to that. Make it an option? Or only apply this feature to “large” items? Ideas are welcome.

Available now for download, testing and feedback.