Developping for Qt 4.8 Embedded on Windows

"Yes we can!" [Obama '08]

I started developing with Qt for Embedded Linux 5 years ago (it was Qt 3.3.1 at the time) and I immediately fell in love with the "hands in the dirt" aspect of embedded programming! However, I have to confess that as much as I love embedded stuff, I'm still a Windows guy... (no judgements please). Actually, while in my case it's purely a matter of habits, many people are stuck with Windows as their mandatory development platform, e.g. due to company policies, Windows-only software/SCM...

So, one day while downloading Sourcery-G++, the popular gcc toolchain for ARM, I noticed that a Windows version was available as well. Out of curiosity I installed it, and just as I hoped what I got was a full-fledged ARM toolchain for Windows, with gcc, gdb, a sysroot, etc.

The first thing that then came to my mind was:

Will it blend^W be possible to cross-compile Qt 4.8?

Well, yes it certainly is possible, and it took surprisingly little effort to get there. This is mainly thanks to qmake which provides a lot of the support out-of-the-box.
And it gets even better: Using the upcoming Qt Creator 2.3.0 you can also cross-compile your applications and even debug them remotely, all within the IDE!

Want to give it a try too? Here's how:

1) What you'll need

• The target toolchain, such as Sourcery-G++ Lite for Windows
• The host toolchain, it can be either Mingw or Visual Studio 2008/2010 (Express versions are fine)
• Jom, if you're using Visual Studio for the host toolchain, or as a replacement for mingw32-make
• Git for Windows
• A Qt 4.8 source git checkout from gitorious
• An ARM-based development board for testing, such as the BeagleBoard

2) What you won't need

• Cygwin, MSYS, or any other linux shell/emulation layer. The standard command line can be used

3) Installing the tools

Follow the instructions to download and install both the host and target toolchains. The installers should make it only a couple of clicks. You may want to add the toolchains permanently in your PATH for convenience, but this is not necessary.

Optionally, install Jom and add it to your PATH.

4) Downloading the Qt source

In the directory where you installed Git for Windows, double-click on the file git-cmd.bat. This opens a command line environment with the git tools set up. Navigate to the place where you want the Qt source downloaded, e.g. C:\Dev\, then clone the Qt source repository from gitorious:
C:\Dev>git clone git://qt.gitorious.org/qt.git qt-4.8-src

Once the download is done, let's switch to the 4.8 branch:
C:\Dev>cd qt-4.8-src
C:\Dev\qt-4.8-src>git checkout -b 4.8 origin/4.8

5) Build preparations

Before going ahead with the build itself, we need to patch qmake.exe and configure.exe to add a few missing things and fix a couple of bugs. The patches are pending upstream and may be merged in soon, so hopefully this step won't be necessary by the time Qt 4.8.0 is released:

For qmake.exe: merge request or patch1 and patch2
For configure.exe: merge request or patch1 and patch2

Typically, Qt source checkouts come with a prebuilt configure.exe, and qmake gets built during the configuration step. However, we do need qmake to build the patched configure.exe, so we're in a chicken-and-egg situation. Not a big problem though, we could run configure.exe once first to build the patched qmake.exe, then rebuild configure.exe, then re-run it for the actual build. But a simpler alternative is to use another qmake.exe to rebuild configure.exe if you already have a Qt installed on your machine (any version >= 4.6.0 is fine):

Open a Qt Command Prompt from the Windows Start menu, and navigate to the Qt 4.8 checkout, then do:
C:\Dev\qt-4.8-src>cd tools\configure
C:\Dev\qt-4.8-src\tools\configure>qmake && nmake
(replace nmake with mingw32-make if Qt was built for MinGW)

This will rebuild configure.exe and automatically replace the one in the top-level directory. Note that you can even use Qt Creator to rebuild configure.exe: just open C:\Dev\qt-4.8-src\tools\configure\configure.pro, then click the "Build" button and that's it.

6) Setting up the target Makespec

We're now almost ready to run the configure.exe script, but we have one more thing to do before that.

An important aspect of configuring Qt is to select the proper Makespec that qmake will use. A Makespec is a file that specifies what toolchain to use, the target platform, the compiler and linker flags, the standard libraries to link, and so on. In the case of cross-compiling, we have to specify two Makespecs: one for the host platform (the platform we're building on, ie. Windows) and one for the target (the platform we're building for, ie. ARM/Linux).

Qt comes with many predefined Makespecs, found under the mkspecs/ directory. Now, Qt doesn't provide target Makespecs when cross-compiling on Windows, so we'll need to provide our own instead (note that we can't use the ones provided for cross-compiling on Linux, since those are made for compiling in a linux shell environment. They're still useful as a base to see what needs to be set though). So this is the Makespec I've written (also available here):


#
# qmake configuration for building with arm-none-linux-gnueabi-g++
#
include(../common/unix.conf)
include(../common/gcc-base-unix.conf)
include(../common/g++-unix.conf)
include(../common/qws.conf)

MAKEFILE_GENERATOR      = MINGW
CONFIG                 += no_import_libs no_generated_target_info
# modifications to g++.conf
QMAKE_CC                = arm-none-linux-gnueabi-gcc
QMAKE_CXX               = arm-none-linux-gnueabi-g++
QMAKE_LINK              = arm-none-linux-gnueabi-g++
QMAKE_LINK_SHLIB        = arm-none-linux-gnueabi-g++
QMAKE_LIB               = arm-none-linux-gnueabi-ar
QMAKE_AR                = arm-none-linux-gnueabi-ar cqs
QMAKE_OBJCOPY           = arm-none-linux-gnueabi-objcopy
QMAKE_STRIP             = arm-none-linux-gnueabi-strip
QMAKE_RUN_CC            = $(CC) -c $(CFLAGS) $(INCPATH) -o $obj $src
QMAKE_RUN_CC_IMP        = $(CC) -c $(CFLAGS) $(INCPATH) -o $@ $<
QMAKE_RUN_CXX           = $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $obj $src
QMAKE_RUN_CXX_IMP       = $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $<
QMAKE_INCDIR            =
QMAKE_INCDIR_QT         = $$[QT_INSTALL_HEADERS]
QMAKE_LIBDIR_QT         = $$[QT_INSTALL_LIBS]
QMAKE_MOC               = $$[QT_INSTALL_BINS]\\moc.exe
QMAKE_UIC               = $$[QT_INSTALL_BINS]\\uic.exe
QMAKE_IDC               = $$[QT_INSTALL_BINS]\\idc.exe
QMAKE_COPY              = copy /y
QMAKE_COPY_DIR          = xcopy /s /q /y /i
QMAKE_MOVE              = move
QMAKE_DEL_FILE          = del
QMAKE_MKDIR             = mkdir
QMAKE_DEL_DIR           = rmdir
QMAKE_CHK_DIR_EXISTS    = if not exist
QMAKE_IDL               = midl
QMAKE_ZIP               = zip -r -9
CODESOURCERY_ARM_CFLAGS = -march=armv7-a -mtune=cortex-a8 -mthumb -mfpu=neon -mfloat-abi=softfp -Wa,-mimplicit-it=thumb
TARGET_SYSROOT          = /CodeSourcery-sysroot
TARGET_QTDIR            = /QtEmbedded-4.8.0-arm
#modifications to gcc-base.conf
QMAKE_CFLAGS           += $$CODESOURCERY_ARM_CFLAGS
QMAKE_CXXFLAGS         += $$CODESOURCERY_ARM_CFLAGS
QMAKE_LIBS             += -lrt -lpthread -ldl
QMAKE_LFLAGS           += $${QMAKE_LFLAGS_RPATH}$$[QT_INSTALL_LIBS]
!isEmpty(TARGET_QTDIR) {
    QMAKE_LFLAGS       += $${QMAKE_LFLAGS_RPATH}$${TARGET_QTDIR}/lib
}
!isEmpty(TARGET_SYSROOT) {
    QMAKE_LFLAGS       += $${QMAKE_LFLAGS_RPATH}$${TARGET_SYSROOT}/lib:$${TARGET_SYSROOT}/usr/lib
    QMAKE_LFLAGS       += -Wl,--dynamic-linker=$${TARGET_SYSROOT}/lib/ld-linux.so.3
}
load(qt_config)


As you can see, we use the MINGW makefile generator since it supports gcc commands lines (which is what Sourcery-G++ is based on). Most of the work then consists in specifying the executables and compiler/linker options.

For the compiler options I'm assuming we're building for an ARM Cortex-A8 CPU, as found on the BeagleBoard. Depending on your CPU type and architecture, the compiler options might differ.

Also, the TARGET_SYSROOT and TARGET_QTDIR variables only need to be set if your target OS wasn't build with Sourcery-G++, or if it uses a different version, and if the Qt libraries aren't going to be in a standard path. More details about that further down.

Save this to a file called qmake.conf in a subdirectory of mkspecs. This directory will be the name of your Makespec:
C:\Dev\qt-4.8-src\mkspecs\linux-arm-gnueabi-g++\qmake.conf

You also need a file called qplatformdefs.h, which we can copy directly from the Linux Makespec in qws/linux-arm-gnueabi-g++:
C:\Dev\qt-4.8-src\mkspecs\linux-arm-gnueabi-g++>copy ..\qws\linux-arm-gnueabi-g++\qplatformdefs.h

7) Configuring Qt

We're now ready to run configure.exe, which will create qmake, set up the build-time variables and build configuration, then create the Makefiles for the Qt libraries and examples.

Open a command prompt with the host toolchain set up (ie. MinGW or Visual Studio), then go to the Qt source directory. This is the minimal configure.exe command line we need to enter:

C:\Dev\qt-4.8-src>configure.exe -debug -embedded -arch arm -neon -platform win32-msvc2010 -xplatform linux-arm-gnueabi-g++

The options are as following:

-debug: we want a non-optimized build with debug symbols. You could use -release instead for a optimized build without debug symbols.
-embedded: means we want to build the embedded version of Qt, not the desktop version. This activates the use of QWS and other features specific to Qt Embedded.
-arch arm: we're building for an ARM CPU. This is used internally to select the proper implementations for low-level stuff.
-neon: We want to make use of the NEON co-processor SIMD instructions, which considerably improves painting performance. However note that NEON is only available for Cortex-A8 and later ARM CPUs.
-platform win32-msvc2010: this is the host Makespec, ie. the one we'll use to build qmake, moc, uic, and other tools running on the host. In this case we're using Visual Studio 2010, but you could replace it with win32-g++ if you want to use MinGW instead.
-xplatform linux-arm-gnueabi-g++: this is the target Makespec, ie. the one we'll use to build libraries, tools, and examples. This will use the CodeSourcery compiler and linker.

You can of course add more options to it, for example to disable building some modules (like Qt3Support, Phonon, or Webkit) or to even skip building the additional tools like Assistant, Designer, etc., since those are useless on the target platform. I usually also skip building the examples and demos, since those increase the build time considerably and I never use them.

This is my typical command line:
C:\Dev\qt-4.8-src>configure.exe -debug -embedded -arch arm -neon -platform win32-msvc2010 -xplatform linux-arm-gnueabi-g++ -fast -no-phonon -no-webkit -no-qt3support -nomake tools -nomake translations -nomake examples -nomake demos

Normally the program should complete without errors. If you did get some, make sure your host toolchain is properly set up (tools in PATH, INCLUDE and LIB set if you're using Visual Studio), and that the proper makespec and architecture are selected (you can see it in the summary printed out by configure.exe before the makefiles are generated).

8) Cross-compiling Qt

The final step is to build Qt, which is as simple as running your make tool in the top-level build folder (there should be a Makefile in there now).

Note however that if you used Visual Studio as the host toolchain, do NOT use nmake to build Qt, as the target makefiles are generated for MinGW instead.

I recommend just using Jom, no matter which host toolchain you used. Jom handles both mingw and msvc makefiles, with -j support for parallel builds, and it's much faster than mingw32-make.

3) Profit

So now is a good time to get a cofee and take a break! Depending on your machine and what modules you're building, this may take up to four hours for the build to complete. Also, Sourcery-G++ on Windows is substantially slower than the MSVC compiler or even MinGW, so don't be surprised if it seems slower than normal.

Once the build is done, you'll find the Qt libraries as .so files in the lib/ directory, and the examples and demos in their respective directories if you built those.

The libraries and executables can be copied directly onto your target board and run from there. If the toolchain matches the toolchain used for building your board image (release and machine versions), it will work out of the box. Otherwise you'll need to copy the sysroot from Sourcery-G++ as well on the board and make sure the app can find it. Note that you can use the sysroot from the linux installers instead of copying it from windows, which is a little easier to deploy. More details about sysroot handling with Sourcery-G++ here.


...and that's it!

In the next post, I will show you how to cross-compile your own Qt applications as well, and how to use the new features in QtCreator 2.3 to cross-compile within the IDE, deploy the applications on the board, and even do remote debugging, all from your Windows desktop!

Oh boy, it's a boy!

The title says it all... I haven't made any "official" announcement on this blog yet, so here it is: we're having a baby! My sweet wife Sahar is 5 months pregnant, and during today's ultrasound we could now clearly see a little "knack" between the baby's legs :)

So it seems that the Pokrzywka line will keep going for a little longer :)
I love the position of the baby in that picture: the curve on the left is the baby's head, seen from the top, next to the "B" is the left arm, and on the top right is the butt and legs (which would be "coming out" of the picture).
We also got a couple of pics from showing a leg and an arm, but they're kinda hard to see:

We have another ultrasound on thursday, this time with more advanced machine and display, so check back for more picture of the little boy! :)

Short Story of a Long Struggle - Part 2: It's alive...

I'm freshly back from San Francisco, and among other things (like finding a place to live in Redwood Shores) I attended Camp KDE 2011 on Monday. It was great! Thanks again to Celeste, Claudia, Justin, Jeff, and all the people who made this possible - you guys rock ! :)

I was actually invited to give a talk about Kontact Touch as part of the KDE on Mobile track. For those who missed it and are interested, our awesome team of organizers have made it available online already :



You can skip the setup downtime until 1:45, but the really interesting stuff starts at 15:00, when I'm performing a live demo of Kontact Touch on the Nokia N900. Will you manage to find the hidden crash in the demo ?? ;)

At the end of my previous post we left off at the point where we solved all the memory and platform issues on Windows CE. All issues ? Well, not exactly, there was still a small area of resistance...

For an unknown reason, we would get random hangs during the initial account sync, or later on while trying to send mails, or browse folders. After some monitoring, we found out that those hangs were caused by socket notifications being lost on the way. We fixed a couple of bugs in Akonadi, solved a few dbus issues, but nope, the problem was still there. This was the beginning of a long hunt, involving multi-thread and multi-process debugging sessions, random reproducibility, and more fun.

Anyway, this post is called Short Story of a Long Struggle, so long story short: the problem was coming from the Qt event dispatching and socket notifiers implementation for WinCE. The implementation was roughly based on the same logic that for desktop Windows, however on WinCE the underlying priorities for threads and event dispatching are quite different, so we basically ended in situation where we would starve some threads and processes, or would never get the socket handlers to be called. By now this was nothing that could frighten Marcus Brinkmann anymore, so he simply went forward and implemented a whole new Qt event dispatcher for WinCE ! With this new event dispatcher, the problems finally disappeared and we finally started having a stable Kontact Touch on WinCE, also fixing a few obscure bugs on the N900 at the same time. Victory !!!

One we got that sorted, the last piece of the puzzle was to speed up a few parts and perform last-minute optimizations. As you can see on the video above, the result on N900 is really good. The apps are fast, responsive, and the animations are super smooth !

Unfortunately, we can't really say the same about the TouchPro2... Despite our best efforts, Kontact Touch is still really slow and sluggish there. But I believe this is a limitation of the hardware itself, as Windows Mobile itself is painfully slow at times on the TP2, at the limit of unuseable. It's too bad really, as the app would rock there too if it only had a little beefier hardware.

So this brings us to the end of this journey. This was a bumpy ride at time for sure, but we had an awesome team working together to make this happen against all odds. Thanks again to everybody involved at KDAB, Intevation and g10 code !

Now it's time to look to the future, and I'm already having "Kontact Touch on Android" as the number one item on my list to Santa for Christmas :)

Short Story of a Long Struggle - Part 1: the announcement that never came...

Back in September I was writing about some HUGE announcement to come related to KDE on Windows. I got everybody holding their breath, even a bit of speculation going on, but then nothing happened. The announcement never came...

So what happened ? What was that announcement about ? And why didn't it come ?

Well, this is the short story of a long struggle...

Our announcement was going to be about the porting of the KDE stack on Windows CE, including parts of kdelibs and kdepim, as well as all their dependencies: dbus, Akonadi, OpenSSL, SASL...

However, just as we were getting the announcement ready after seeing the first kde-based apps running (the first ever was kde4-config.exe) we started hitting wall after wall...

First, we got hit by the drastic process and memory limits of Windows CE5 (on top of which Windows Mobile is based), which were so tight that  even starting the simplest apps would be problematic. Just to name a few: maximum 32 processes can be running system-wide, and each process has a limit of 32MB of RAM, including loaded DLLs. So first we had to go through a complete overhaul of the build process to try and work around those limitations, for example by switching from DLLs to static libraries. This was so that the unused parts of the libs would be automatically discarded by the linker: bringing 20MB of DLL code for only a few used symbols is hardly a good idea when you're short on RAM. We also had to get Qt to go through a severe diet, by removing every single unused feature we could find, and of course having to go through the KDE code to #ifdef out parts that were relying on these features.

Once we were done, our next wall was the Visual Studio emulator, which is a great tool for testing when you don't have an actual Windows Mobile device, or even for faster debugging. But for some reason, the emulator couldn't handle more than 20 file handles being open or so. We looked for alternatives and it took us some time to work around it too, but as far as I know we never really found the cause for that one.

But then the memory issues hit us again: even with a slimmed Qt and static libraries, our executables were now quite large, and taking most of the allowed 32MB of process memory, which is heap+stack+program image+DLLs from all processes (yes, you read it right, but please don't even ask). This this left only very little heap space for the actual program, and that is not good news when you're running a QML app, with lots of images that have to be stored in heap memory. So we had to find a remedy. And it came from our fellow developer Marcus Brinkmann, from g10code, which was our partner on the project. The idea was very simple, and at the same time completely crazy : write a custom program loader which loads the program in high-memory space instead of the process space. This is based on the fact that Windows CE has a huge portion of its virtual memory space that's unreserved, so it can be allocated e.g. for shared memory segments. By managing to get the program loaded in this area (called hi-mem), all the per-process limits are lifted. This seemed insane but it was our only option at that point, so we gave it a try. And guess what : in less than 3 weeks, Marcus got a a fully functional and working hi-mem program loader: himemce.exe ! There are few things that really impress me in programming, but this was definitely one of them ! If you're interested, the code is available as part of gnupg's wincetools repository and should be usable for any program on WindowsCE, not only KDE.

But the dude didn't leave it here : since we could load the program in the hi-mem space, why not allocate our heap data there as well ? Well, no guts no glory ! So Marcus did it again, and faster than you can say "shallow grave" he brought us his infamous dlmalloc, which is a malloc() replacement that allocates memory blocks in the hi-mem space. The last step was just to integrate that into our program as the underlying malloc() used by the operator new. And even better, by reimplementing qMalloc() we could even get all of Qt's heap allocations to be done into high memory ! Success at last !

So, with our applications now running without hitting the platform limits, you may think this is where the story ends... Happy ever after ? Well, yes, if you're in a Disney movie. But this is the real world, and ponies don't stay pink for long...

It's that time of the year!

Finally ! After a long year of waiting, it's time again for Camp KDE, and this time we're going to rock San Francisco !

I can't wait to meet with our KDE developers and enthusiasts on the other side of the Atlantic. Like last year, I have the honor of giving a talk around KDE on Windows. However, this time I won't be talking about desktop Windows, but  mobile Windows...

In fact, I will be presenting the Kontact Touch project: KDE's professional Personal Information Management client targeting touch-enabled form factors, including smartphones and tablets. Topics will include porting the whole KDEPIM stack to mobile platforms including Maemo and Windows Mobile, as well as rewriting the whole UI using the bleeding edge Qt Quick framework, and it will of course include demos, which will be on multiple devices, including an HTC TouchPro2 running Windows Mobile, a Nokia N900 running Maemo, and an IBM tablet running Meego.

Update: a few teasers below...

So if you're around the Bay Area on Monday April 4th and you want to see how cool the KDE technologies can be on the mobile side, don't miss Camp KDE !