I am not talking about His Steveness or this kind of jobs (congratulations to both involved parties!), I am talking about these.

Traditionally we have been writing in a very start-to-end fashion, where execution starts at an entry point and ends when the task is done.
At first there wasn't any reason not to do this as there was no user interaction during a programs execution point or only at controlled points, e.g. a console program asking for a Y/N descision.

Then, with the switch to event based main execution, that changed to having multiple entry points but each usually still starting an execution chain that finished a complete task in one go.

One of the reasons for this "complete task in one go" style is that it is a lot easier to code. All necessary data only has to live during this uninterrupted execution, nothing else can interfere and maybe make some of the data invalid, etc.

This is of course no problem for anything that can be done fast, but long duration processing like the introduction of network I/O with its latency and overall time requirements made it necessary to come up with new styles.

One of them is staying with the start-to-end fashion but move the execution to a parallel processing unit. e.g. multi threading.

Another one is processing the task in smaller steps, using the a similar technique already used for separating the tasks, e.g. event based.

Given that Qt already has event driven processing, doing it for things like networking was a natural choice.
However, mainly because Qt's I/O classes are relatively low level and at that time weren't as unified as they are nowadays, KDE developers came up with an even nicer to use idea: jobs.

A job is basically a context for processing a single task (though such a task could of course be an aggregation of sub tasks). It is started and ends at some point, it might report progress inbetween, might even be able to be suspended and resumed.

Implementing a task as a job is usually more challenging than the traditional uninterrupted execution path, but it is really not much more difficult to use: you create the job object, connect to its result signal and start it.

Sure, compared to a normal method call you don't get the result at that part of your code where you start the thread, instead you need an addition method (the slot the result signal is connected to).
However, you also have advantages, like having all data necessary for the task stored in the job object and not having to keep it around elsewhere (but still having access to it from whereever you interact with the job).

Unfortunately they are not yet widely used outside of I/O bound tasks, quite some other long duration processing is implemented in keeping the event loop running but not actually returning from the current code context, e.g. "blocking" user input by showing some kind of process dialog, returning either when finished or when the user explicitly cancelled.

At first this looks just like another good option for doing long duration task processing, however this can lead to multiple execution contexts within applications that are not prepared for that (because they know they are not using threads so they don't expect things to be called twice).

This possibilty of unexpected re-entrancy is often overlooked because the magnitude of operations is caused by user input and thu,s by taking that out of the equation, makes the approach "safe" again.

I put "safe" in quotes because it isn't true. Events can not only caused by the application's user, it could be caused by a timer reaching its timeout, or a socket becoming available for reading.

One could work around some of these by means of flags or similar status variables but even for application code this becomes really messy at some point.
It is almost an invitation for disaster for any form of library code.

In KDE PIM we've got bitten by that a lot during the last couple of releases due to libraries and applications assuming things like instant access to data. We had to effectively split a lot of our code into smaller parts that could deal with having some data arrive in chunks or not at all.

That was a lot of work and mostly only possible because none of these libraries were public API, thus allowing us to change them without caring about even source compatibilty.

So my advice to anyone adding new libraries that involve long duration processing, don't assume that all applications using the library will be fine with the library blocking the user input as its sol method of avoiding re-entrancy.

Think about providing job based API and let application developers hook it up to a progress dialog if that's how they want it processed.
Heck, if it is really so difficult to connect the job's result signal to an appropriate dialog slot, provide a convenience function that does that and extracts and returns the result from the job.

api.kde.org is operational again.

I can help fix problems you might notice on the site (bad-links, etc), but please don't bother me about content -- contact the project authors, maintainers, module coordinators if you encounter missing/outdated/broken/wrong API documentation.

Bugs can reported to the KDE sysadmins filed under the api.kde.org component.

OpenChange is an important project, but it does require quite a lot of work to get it all to build. We're working on the process, but in the mean time, we've (ok, Julien Kerihuel with nothing from me except encouragement) has built a Virtual Box image that provides OpenChange all built, configured, set up and ready to try.

See http://tracker.openchange.org/projects/openchange/wiki/OpenChange_Appliance for the download (ftp or rsync) location and setup procedures.

Have fun, and let us know how it goes!

I haven't been doing a lot of KDE stuff recently (happy user, although I'd be happier if I could find some extra time for development...). Instead, I've been doing some "real" work, and also going some OpenChange work. [For those that tuned in late, OpenChange is an implementation of the Exchange RPC protocols on both the client (i.e. "Outlook") and server (i.e. "Microsoft Exchange") sides of the network protocol]

Some recent changes:
- we've arranged some new infrastructure servers, with the help of the Free Software Conservancy and FSF France (especially thanks to Loic Dachary for his work on this)
- we've migrated from Trac to Redmine, which is working out pretty well so far
- got the buildbot back up (on http://buildbot.openchange.org:8010 for those who'd like to check it out)
- significant progress on the server side (including getting outlook to show a message), although there is still a lot to do here
- better handling of some data types (especially unicode strings)
- a lot of little code cleanups

Thanks to the Novell (Evolution) team for their work in identifying issues and suggesting fixes, and to the Microsoft Open Specification team for following up on our obscure questions.

Future (near-term) plans mainly focus on server-side functionality, and fixing up some of the current bugs that are biting client users.

After all the blogging about our (as in KDE PIM developers) Akonadi porting, I thought I'll address a couple of things other application developer might consider for the next release cycle.

The most prevalent use of PIM API seems to be KABC::StddAddressBook.

Find By Uid

A common use case is to look for a contact object by its identifier, acquired by user selection somewhen in the past.
The code to do that usually looks like this:

KABC::AddressBook *addressBook = KABC::StdAddressBook::self();

KABC::Addressee contact = addressBook->findByUid( uidString );

This is quite easy but it always has a couple of issues, mainly the call to

KABC::StdAddressBook::self()

because its first invocation will load the whole address book and it will do so synchronously.
Potentially blocking the application for the whole duration or, sometimes even worse, introducing unexpected reentrancy in single threaded applications due to use of nested event loops.

Note: while the same sequence is possible with Akonadi, due to Akonadi jobs being based on KJob and it having the capability of "blocking" by use of a nested event loop, I really recommend that you use the porting effort as an opportunity to get rid of such emulated synchronousity whenever possible.

The equivalent using Akonadi API looks like this

Akonadi::ContactSearchJob *job = new Akonadi::ContactSearchJob( this );
job->setQuery( Akonadi::ContactSearchJob::ContactUid, uidString );

connect( job, SIGNAL( result( KJob* ) ), SLOT( contactSearchResult( KJob* ) ) );

and

void contactSearchResult( KJob *job )
{
  if ( job->error() != 0 ) {
    // error handling, see job->errorString()
    return;
  }

  Akonadi::ContactSearchJob *searchJob = qobject_cast( job );

  const KABC::Addressee::List contacts = searchJob->contacts();
}

Also see API docs for ContactSearchJob

Find by Email

Another use case I came across is searching by Email. The KABC code usually looks pretty identical:

KABC::AddressBook *addressBook = KABC::StdAddressBook::self();

KABC::Addressee contact = addressBook->findByEmail( emailString );

Unsurprisingly, the Akonadi code looks also quite similar to the previous use case:

Akonadi::ContactSearchJob *job = new Akonadi::ContactSearchJob( this );
job->setQuery( Akonadi::ContactSearchJob::Email, emailString );

connect( job, SIGNAL( result( KJob* ) ), SLOT( contactSearchResult( KJob* ) ) );

The code for the result slot is the same.

Who Am I

Another use case for StdAddressBook is not directly available in Akonadi:

KABC::AddressBook *addressBook = KABC::StdAddressBook::self();

KABC::Addressee contact = addressBook->whoAmI();

Even this had its shortcomings, because this depends on the users adding themselves to the address book and marking the respective entry as "this is me".

The concrete use cases I've seen this deployed for are getting the user's full name and/or their email address.

It might be better to use KPIMIdentities::IdentityManager instead.
It contains the identity users can configure in the Systemsettings module for personal information, so if there is data yet, you could embed the respective KCM in a dialog and let the user configure this centrally for all of KDE.

Modifying a contact

Using KABC::StdAddressBook modifying a contact (for example one retrieved by findByUid()) worked like this:

addressBook->insertAddressee( contact );

KABC::Ticket *ticket = addressBook->requestSaveTicket();
if ( !ticket ) {
  // error
} else if ( !addressBook->save( ticket ) ) {
    // error
    addressBook->releaseSaveTicket( ticket );
  }
}

With Akonadi, you will need the Akonadi::Item or at least the item's identifier.
Assuming you retrieved the contact using ContactSearchJob, consider keeping the item or Item::id() around, see

ContactSearchJob::items()

(inherited).

Lets say you have just the item's identifier:

Akonadi::Item item( itemId );
item.setPayload( contact );
item.setMimeType( KABC::Addressee::mimeType() );

Akonadi::ItemModifyJob *job = new Akonadi::ItemModifyJob( item );

connect( job, SIGNAL( result( KJob* ) ), SLOT( contactModifyResult( KJob* ) ) );

Slot code similar to the one for search.

Buildsystem adjustments
Since you are already linking against KDE PIM Libs classes, you already have the necessary

find_package( KdepimLibs 4.5 REQUIRED )

.

However, your target_link_libraries need to be extended to contain

${KDEPIMLIBS_AKONADI_LIBS}

for base things like Akonadi::Item and

${KDEPIMLIBS_AKONADI_CONTACT_LIBS}

for things like

Akonadi::ContactSearchJob

.

Includes for Akonadi core classes are like

#include 

for contact related specializations they are like

#include 

In both cases also available in lowercase variants with ".h" suffix.

Other useful classes

Our Akonadi porting efforts have resulted in a couple of useful classes for various aspects of dealing with PIM data.
For contacts such classes would be: ContactEditor (also available as a dialog), ContactViewer (again also available as a dialog), ContactsTreeModel and others.

As usual: in case of questions, our IRC channel is #akonadi, freenode network, or send an email to our mailinglist (kde-pim@kde.org).

I added some new options to Krazy in the past few days. Hope folks find them useful.

1. You can now control the types of files to process with the new --types and --exclude-types options.

   Use the --list-types option to see what file types are supported.

   Examples:
   

   # Run all checks against desktop and designer files in this directory:
% krazy2 --types=desktop,designer  *
   # Run all checks on all file types except C++ files:
% krazy2all --exclude-types=c++

2. Krazy now supports "check-sets" with the new --check-sets option. Check-sets are a collection of checkers with a similar high level purpose in mind.
   Check-sets currently available:
   * c++, pure C++ checks
   * qt4, Qt4 source (C++, QML, Qt Designer files)
   * kde4, everything (the default)
   * foss, checks specific to FOSS (eg. license and copyright)

   Use the --list-sets option to see what check-sets are supported.

   Examples:
   

   # Only check for pure C++ issues:
% krazy2 --check-set=c++ *
   # Only check for pure Qt4 issues:
% krazy2all --check-set=qt4

Wait! There's more... you can control all these new features by using TYPES, EXCLUDETYPES, and CHECKSETS directives in your .krazy file. Please see the krazy2, krazy2all and krazyrc man pages for more info.

Pick up the new Krazy in $KDESVN/trunk/quality/krazy2 and run ./install.sh from there.
Please tell me about bugs and feature requests by submitting a report to the Krazy component in bugs.kde.org

You most probably have already run into this at least once. You use the computer, try to do something and you get an error message saying "sorry, application foo is not installed", "the required plugin bar is not installed" or similar. And that's it, there it stops. You have to find out what package the required functionality is in, install it manually and try again. Like if the computer couldn't ask "but maybe I can install that, do you want me to try?" and handle it itself.

And that's what my goodie for this openSUSE release is about. I've been examining a bit about what various parts of the desktop could do this and there indeed are some cases. For example, clicking in Dolphin on a file that has no associated application installed usually results in the "Open with?" dialog. And that dialog has nowhere in it the option "the application that can open it, silly". Especially given that if the installation medium is accessible (i.e. usually if the network connection is up), it's rather easy to find out the right application for the file and install it:

This specific case is actually a bit tricky. The file, example.kvtml, is a file with pairs of words or expressions that e.g. KWordQuiz can use for teaching (words for language lesson, for example). The problem is that technically the data is stored as XML and the mimetype (file types) specification has a concept of subclasses that is and in not useful depending on how you look at it. A C++ source file is a subclass of a plain text file, so you can edit it just like a plain text. Good. An XML file is also a subclass of a plain text file, so you can view XML as plain text. Good? I'm not quite sure on this one, since while XML is human-readable, any decent up-to-date XML is certainly not human-understandable anyway, so I fail to see the point. But, since .kvtml files are XML files, they are a subclass of them, and that means you can view them just like a plain text.

Good??? Probably not. They are supposed to be opened in an application that can show the lesson nicely, who'd be crazy enough to decipher it from the XML? Well, but that's the reason for the dialog looking this way, the above is what the dialog is trying to tell you. Sorry . This should get eventually sorted out somehow in the mimetype specification, but for now I had to go with this.

The short version is: Just say you want an application that can handle exactly the file type, that's usually the right choice here. And if there are more applications that can handle it, you'll get a choice:

There is another, rather obvious case, where this can be useful. Amarok on its first start usually likes to complain about lack of support for certain well-known and widely used multimedia format and 11.3 will be no different. However, Amarok has also some support for solving this problem and has this dialog:

And that is where the new feature comes into play.

This time, however, there is the usual problem: The openSUSE distribution is not allowed to include the necessary support, because <a lot of ugly legal babble that causes headache>. Some distributions may try to include it and hope that residing in a country without such laws solves the problem. Or, even better, not having a load of money in bank avoids a lot of trouble too (what's the point of sueing somebody who can't pay afterwards, these merry patent folks don't do it just for the sports). That doesn't quite work for openSUSE, being supported by Novell, and I bet every big company has been already sued for much more stupid things than this, just in case it'd work out. So openSUSE simply can't include the support and can't even really tell you where to get it. As long as the world is the way it is, there can't even be any "install all I need" button in openSUSE. Sorry. That's the way it is .

So what happens in this case it that the required packages will not be found. However, there are many repositories for openSUSE not provided by openSUSE, and you can add the right one and try again (BTW, the URL in the link doesn't work yet, that will be fixed in time for 11.3).

Choosing to enable additional repositories will simply launch the YaST module for configuring repositories. And, as I said, it cannot point you "here" and tell you which repository to add (because, if nothing else, it doesn't know anyway). But adding a repository is not really that hard.

After adding the right repository it will proceed with installation. Again, the usual allmighty YaST. Nothing hard about it, and this part should be mostly automatic anyway.

There it is. As simple as possible (sigh) and now it's ready (but the dialog is actually right, restart is required for technical reasons).

There of course can be more places where this could be useful. The crash handler has already support too, so generating proper bugreports with full backtraces should be now much simpler as well. This is so far just experimenting with the feature and seeing how it works out, if it works well, even more can be added after 11.3.

As you may have followed, I compiled my KDE in a virtual machine, described an example how it goes and even compared compile times in a mini-benchmark. I stopped using a virtual machine later because VirtualBox only gives you one virtual CPU, QEmu as well, and VMware Server only two. The good thing is I found KVM virtualization and learned how to set it up. It is a bit tedious and you will have to create a network bridge after every reboot, but the result is astonishing - I could compile Qt 4.6 in 20 minutes instead of 80.

I read an interesting blog this morning Freedom vs. The Cloud Log where Glyn Moody interviewed Eben Moglen. Eben Moglen was General Council of the FSF for 13 years and helped draft various versions of the GPL. He talks about the implications for software freedom caused by the rise of services in the 'cloud' where your data is owned by the service provider, and the fact that they don't usually release the code of their applications that run on the servers.

With the recent ownCloud initiative, the KDE community is doing something about the problem, by allowing you to have your own data stored in a variety of types of places of your choice, so that it is independent of a particular machine. It gives you a option of putting it on a host providers machine, or on you own network - somewhere where you personally are in control of it.

I had been thinking of getting some sort of home server, that was always connected to the internet, to use as a music server. The Apple Mac Mini running either Mac OS X or Kubuntu (like I run on my small laptop) seemed to be the best bet. Most servers are still really big and ugly with noisy fans, and not really suitable for running in your living room. The Mac Mini is one of the few attractive and quiet options, but it is quite expensive.

When Eben Moglen talked about how we could construct a distributed infrastructure peer to peer style, instead of client/server as used by Facebook and the like, he mentioned a small ARM based server called the 'ShivaPlug'. That got me thinking, and I went off googling for everything I could find about this tiny server the size of a wall wort power supply. It turned out that there is a new model called a GuruPlug which has additional features like WiFi, eSATA interface for cheap fast hard disks, and an SD slot. The more I thought about it, the more fun the idea sounded. There are so many things you can do with one of these little servers.

So I just went ahead and ordered one. The basic server was 91.47 euros, a JTag board for debugging was another 26.82 euros, and shipping by FedEx was a bit pricey at 49.76 euros. The total cost was 168.05 euros, which is actually quite a lot relative to how much a cheap netbook costs these days. I'm sure in a years time you will be able to walk into a computer shop and get them for more like 50 euros or so. I think these things will be subject to Metcalfe's Law where their usefulness will rise according to the square of the numbers of users. If Eben Moglen is right we should be able to undermine the efforts of authoritarian governments, such as China, the UK, Australia, France and so on, by going completely peer to peer with strong encryption and cut off the government snoopers from invading our privacy and stealing our civil liberties. So for reasons like that, I think my 170 euros, plus a bit of my time will be a good investment.

Do you know this one?

Phoronix tested md5sums of ISO images of distributions. The winner was openSUSE, scoring e29311f6f1bf1af907f9ef9f44b8328b, which gave it a noticeable lead before second Slackware (b026324c6904b2a9cb4b88d6d61c81d1), which is quite closely followed by Fedora (9ffbf43126e33be52cd2bf7e01d627f9) and Debian (9ae0ea9e3c9c6e1b9b6252c8395efdc1). The difference between these two distributions, as you can see, is only very small. Ubuntu completely flopped in this test, achieving only 1dcca23355272056f04fe8bf20edfce0, which is surprising, especially considering that its previous release scored a very nice c30f7472766d25af1dc80b3ffc9a58c7. (source).

Ok, that's just a joke, but the sad part is, as somebody pointed out, that it wouldn't be really that surprising if Phoronix actually did something like that. And, probably even more sad, there would be people who'd really take it as if it meant something and started adding comments about how last openSUSE is pretty good, last Ubuntu is so disappointing, and Fedora and Debian are not really that different.

So take this from somebody who has already done a lot of performance work: Benchmarks, on their own, mean almost nothing if you don't understand them. Especially if they are seriously flawed (I mean, testing filesystem performance by doing CPU-intensive tasks? Hallo? Probably even FAT16 could provide the same results in those tests on an SSD.), but even if the results are useful numbers, it is still necessary to understand what the numbers actually say. I think I wouldn't even have a big problem forging a "benchmark" where KDE would get better (and correct) numbers than LXDE by finding a scenario that'd be twisted enough.

And even then, it is still necessary to keep in mind what is compared. Comparing the default setup of Fedora and openSUSE means also comparing GNOME and KDE, which you may or may not want, but if you compare the distributions this way, it is affected by differences between the desktops, and if you compare the desktops, it is affected by the differences between the distributions. And in either case, it may or may not apply to another distribution or another desktop.

Yet one more thing to understand is what is measured and how it affects performance as a whole. Ages ago there was a Dot article that also mentioned performance improvements to be brought by Qt4 in some specific areas, yet even now there are numbers of people seriously disappointed by KDE4's performance only because they thought that since Qt4 improves in some areas, KDE4 will get exactly the same improvement, regardless of how much these improvements matter for the whole of KDE4 or how much of KDE4 was rewritten when porting from KDE3. When I fixed Exmap to work again and did a little benchmark as a part of it, there wasn't really much more to it than to show that Exmap works again and that it is very easy to lose a lot of advantage by a simple mistake, yet people had no problems drawing all kinds of strange conclusions from that. Since making right conclusions is unexpectedly difficult for most people when it gets to benchmarks, I really need to remember not to just post numbers again without also saying what it means.

And, finally, there is still the question of other costs and whether it is worth it. Various KDE components often have resource demands, but then they are also often worth it. I mean, we all could still use TWM, or, heck, Windows 95, but seriously, how many of us really would? This, ultimately, is always about what works the best for you.

With only a few hours a week to work on Konsole, I often view the logs and other reports for simple issues to investigate. The compile warnings were fixed sometime ago. Most of the simple Krazy and Apidocs issues were also fixed. The remaining Krazy issues would require some heavy duty changes.

Recently I've been trudging through the -Weff++ logs. I coded a simple Ruby script to parse them as Qt generates a ton of warnings. Currently there are around 300 issues. The vast majority of these fall into 2 categories:

1. ... should be initialized in the member initialization list
2. ?class ...? has pointer data members
   but does not override ?Class(const Class&)?
   or ?operator=(const Class&)?

These 2 can be fix easily enough.
1. Add them to the constructor watching the order of the initialization list
2. Add these to the private section of the .h file:
   // Disallow copy and assign
   Class(const Class &);
   Class & operator=(const Class &);
   If anyone tries to use the Copy or Assign, it will fail at build time.

Thoughts on these 2 issues?

I did manage to get the Apidocs to generate the class hierarchy in graphic form which is very helpful. Although I'm still struggling with using valgrind for memory issues (and unfortunately, valgrind doesn't work on Mac Snow Leopard).

Today I'm leaving Tokamak 4, so I thought I'd write a post about what I've been up to while I've been here.

The first day or so I spent getting my machine sorted out as trunk was causing some issues with my graphics driver leading to a hang in the DRI layer of the kernel. I also had a chance to triage some ksnapshot bug reports and look over some patches.

The next day was largely taken up with talks about the current status of the various parts of plasma, kwin and related technologies. We got a little hacking done but not much as it was more important to make sure we're all coordinating our development.

The remaining days became a bit of a blur, so in no particular order:

• I refactored the plasma javascript scriptengine to make it easier to use it with QScriptEngines that are created externally. This is required before we can integrate things like QML.
• Worked out what is required for QML integration to be implemented in a way that is compatible with our existing javascript bindings etc. This will require some small changes to the API offered by Qt for QML, which may be a problem. If we don't get those changes then QML is likely to be trapped outside the main engine which will reduce what it can do unless we write
  another set of bindings.
• Made some fixes to the webslice applet to improve error handling.
• Started work on an idea I've had in the back of my head for a while, I won't say too much about it here, but I'll include a pretty picture to whet your appetite:
  
  
  

  In addition to this I've taken part in lots of discussions on topics ranging from activities/contexts, scripting of animations, silk and more. I'd like to thank Will and the rest of the openSuSE team for making this such a pleasant and productive sprint.

No, Locus OS concept design, has apparently no association with MS.

"This interface was designed before iPhone 3.0, Palm Pre, Android etc, making the ideas original at the time :)"

Is the last sentence true at all?

A topic that I've not mentioned in any of my blog posts is threading, not because I have anything against it, simply because a simple use-case hadn't come up. Today I was coding something easy to describe, where using threads was a good solution, so let's take a look at it.

The problem I needed to solve was to calculate cryptographic hashes of files that could be very large - for example 4 gigabyte DVD isos. As with any graphical application it is very important to ensure that the GUI remains responsive while this work is done. Since the problem involves very little communication between the part of the application calculating the hash, and the rest of the code a worker thread is an ideal solution.

The code that actually does the work is very simple, it uses the handy QCryptographicHash class to do the work (stored in the variable hasher):

HasherThread::HasherThread(QObject *parent) :
    QThread(parent)
{
    hasher = new QCryptographicHash( QCryptographicHash::Sha1 );
}

void HasherThread::run()
{
    QFile f( filename );
    if ( !f.open(QIODevice::ReadOnly) ) {
        emit error( QString("Unable to open file %1").arg(filename) );
        return;
    }

    hasher->reset();

    char buffer[16*1024];
    qint64 count;
    do {
        count = f.read( buffer, sizeof(buffer) );
        if ( count == -1 ) {
            emit error( QString("Read error") );
            break;
        }
        hasher->addData( buffer, count );
    } while( !f.atEnd() );

    emit completed( hasher->result().toHex() );
}

The code opens the file to be hashed, then reads through it in 16K chunks which it passes to the hasher. If this code were executed in the application's gui thread then this would cause the interface to freeze until the hash was ready, but the code lives in a class that inherits QThread.

One important feature to note about the code above is that the communication with the rest of the application is via signals (the error and completed signals to be precise). This is very important as it means that when we use the class, we can simply use it like this:

    hasherThread = new HasherThread( this );
    connect( hasherThread, SIGNAL(error(const QString &)),
             ui->resultEdit, SLOT(setText(const QString &)) );
    connect( hasherThread, SIGNAL(completed(const QString &)),
             ui->resultEdit, SLOT(setText(const QString &)) );

The special thing in the code above, is that we didn't have to do anything special - Qt just solved our inter-thread communication issues for us. What I mean, it that if our code had simply tried to output the result by calling ui->resultEdit->setText() directly then we would be attempting to access the GUI from outside the GUI thread which would most likely cause our application to crash. Instead, since we're using the default type of connections 'AutoConnection', what this means is that Qt will spot if the thread in which a signal is emitted is different from the one in which it is received and handle the necessary synchronisation for us - nice!

Some of Akonadi's resource agents (usually just called resources) work on local files, some on files containing more than one data object, some on directories containing one data object per file.

For example the "VCard Resource" has one vcf file to work with which in turn contains any number of vcards, i.e. contacts.

Those single file storage containers have a couple of things in common so of course we want to share as much of code between their respective resources as possible.

Unfortunately the data inside the different files is formatted quite differently, so the parsing and creation of C++ objects is rather type specific.
Another difference which can make common code difficult is the expected size of the files and each of its entries, i.e. a VCard file will most likely just contain a couple of dozen contacts, maybe in the lower hundret range while an MBox file can easily reach several thousands of entries and each of its entries (emails) is usually larger than one of the contacts.

So code that works well for contacts or calendars does not necessarily work well for messages and vice versa.

While thinking about possible ways to improve our situation I had the idea of using the same level of abstraction we are already using in Akonadi, i.e. generic "Items" which hold the type specific data as their "Payload".

In other words, if we had an "Akonadi Item File" we would get Akonadi items out if it and not have to care about whether those items transport contacts or emails.

This would still leave us with the problem of different file and item sizes. Again I decided to use concepts already proven useful in Akonadi: payload parts and gettings things on demand.

Payload parts refer to a concept where allow for a payload (remember that could be a contact or an email, etc) be split into parts that make sense for the respective data type, e.g. splitting an email into "Headers", "Body" and so on.

Getting things on demand refers to get whatever parts of an item you are interested in at any time, e.g. only getting headers when listing a mail folder and getting the rest when displaying a selected one.

In Akonadi we do this through jobs and telling those jobs what we expect them to return to us.

The respective Akonadi code would look similar to this:

const Collection collection = someModel->selectedCollection();

ItemFetchJob *job = new ItemFetchJob();
job->fetchScope().fetchPayloadPart( MessagePart::Header );

connect( job, SIGNAL( result( KJob* ) ), this, SLOT( collectionListed( KJob* ) );

with collectionListed() doing something like that

ItemFetchJob *job = dynamic_cast( job );

const Item::List items = job->items();
// fetch the first item as a whole

job = new ItemFetchJob( items[ 0 ] );
job->fetchScope().fetchFullPayload();

connect( job, SIGNAL( result( KJob* ) ), this, SLOT( itemFetched( KJob* ) );

In order to do something similar with files I came up with a concept I called the Akonadi Filestore.
The main interface looks like this (omtting some of the methods not important for our example above)

class StoreInterface
{
  public:
    virtual Collection topLevelCollection() const = 0;

    virtual ItemFetchJob *fetchItems( const Collection &collection, const ItemFetchScope *fetchScope = 0 ) const = 0;

    virtual ItemFetchJob *fetchItem( const Item &item, const ItemFetchScope *fetchScope = 0 ) const = 0;
};

Assuming we have an implementation that operates on a file which contains messages, e.g. an MBox store, we can implement the example above quite similar to the respective Akonadi code.
(note: all jobs are named like the ones from Akonadi but live in their own namespace. Collections, items, fetchscope are directly the classes from Akonadi)

// lets assume mStore is of type StoreInterface* and has been
// initialized properly with an MBox store implementation

// lets list the mails in the stores top level collection
ItemFetchJob *job = mStore->fetchItems( mStore->topLevelCollection() );
job->fetchScope()->fetchPayloadPart( Message::Header );

connect( job, SIGNAL( result( KJob* ) ), this, SLOT( collectionListed( KJob* ) );

with collectionListed() doing again something like that

ItemFetchJob *job = dynamic_cast( job );

const Item::List items = job->items();
// fetch the first item as a whole

job = mStore->fetchItem( items[ 0 ] );
job->fetchScope().fetchFullPayload();

connect( job, SIGNAL( result( KJob* ) ), this, SLOT( itemFetched( KJob* ) );

As you can see we get on demand, payload part fetching with the only type specific thing being the name of the header payload part.
In Akonadi resource this difference would removed because Akonadi forwards the payload part from the client requesting it, so the resource does not have to know any of those identifiers itself.

This is already quite nice but it gets better
- We no longer require the items to be of the same type, say a ZIP file containing contacts, calendars and emails (or like an Outlook PST file)

- We can include metadata that is not part of the common payload formats, e.g. flags of email messages if the store's format supports that (or if it is actually working with more than one file, e.g. KMail is saving this things into index files it keeps alongside the actual mail files).

Neither design nor implementation are fully production ready, I consider it a testbed for the concepts mentioned above.
You can find it in PIM Playground (to get correct paths either checkout playground/pim/akonadi or create a directory "akonadi" into which you checkout "filestore")