GSoC Timeline in KOrganizer

25 04 2011

Google just announced the list of accepted summer of code projects for this year. Together with Torgny Nyblom, I’ll be mentoring Sudhendu Roy in his quest to bring support for HTML replies and forwards to KMail. Looking forward to this project!

If you are a student or a mentor, you need to know the timeline of this year’s summer of code. Adding it as a calendar to KOrganizer (here for KDEPIM 4.4) is very easy:

  1. Right-click in the calendar area to add a new calendar
  2. Choose “Calendar in Remote File
  3. Enter the correct URL and give it a nice name
  4. Done, the timeline is displayed in your calendar




Akonadi Meeting and the KDE SC 4.5 release

14 05 2010

The meeting

We are at the Akonadi meeting at the KDAB offices in Berlin right now, which was quite nice so far. We had the first round of API review of new methods in KDEPIMLIBS for 4.5, and already cleaned up quite a bit. Having multiple eyes look at the API is a nice way to improve the overall quality of the API. We met with Andrey Moiseenko and Alvaro Manera of Nokia, who work on calendaring for the next Meego phone from Nokia. They use our KCal library, which they have forked/extended for some special requirements they have. We’re now making plans with them to integrate their changes back to our version of KCal, so that both sides will profit from changes and have a single point of maintenance.

The release

Many of you are interested in the state of KMail 2, which is the port of KMail to the Akonadi PIM framework. We haven’t communicated much about that lately, as most of us aren’t keen bloggersยน. Finally, here is an update for you now. Yesterday, we had an intensive discussion about the state of KMail 2 and our release plans for it. While we made some good progress on it in the last few weeks, we came to the conclusion that KMail is not yet in a state that is suitable for a beta release. With the KDE SC 4.5 beta 1 release approaching fast, we had to go for some plan B. KDEPIM will not be included in KDE SC 4.5 beta 1. Instead, we will concentrate on making KMail more stable and usable, and release the first beta version of KDEPIM one month later, at the time when KDE SC 4.5 RC1 will get released. That gives us some time to finish important bits and bring KMail 2 to a state in which it can be used.
This means that there will be no fresh KDEPIM packages in KDE SC 4.5.0, and distributions should ship the KDE 4.4.x version of KDEPIM instead. The final KDEPIM release will then follow a bit after the SC 4.5.0 release, at the same time as KDE SC 4.5.1.
KDEPIMLIBS is not affected, it will be released together with KDE SC 4.5.0 as planned. The rest of KDEPIM progressed very nicely: Tobias fixed a lot of problems in KAddressbook and brought back some missed features. The Akonadi port of KOrganizer is also working quite well. However, releasing KDEPIM without KMail is not an option, as it is a quite important part of the PIM suite. We briefly pondered replacing KMail with an older version for the release, but that would involve ripping apart and moving around the dependencies, which is a lot of work and has a high risk of regressions. I think delaying the release is the best option here, as it gives us some more time for stabilizing without too many downsides.
If you’re a KDE developer, please try out the beta version of KDEPIM when it gets released and help us making it work for your usecases. I hope the revised release schedule will work out, keep your fingers crossed ๐Ÿ™‚
So basically the release of KDEPIM will happen a bit after the KDE SC 4.5.0 release. We’re not sure about the release name yet, maybe we should call it “Preview” or something like that, just to prevent the endless discussion and flames about naming that happened with the KDE SC 4.0 release.

Features

KMail 2 is a major rewrite of KMail: We throw out the ancient brittle storage layer and replaced it with Akonadi. On top of that, we ported it from the mimelib library to KMime. Finally, we factored out much of KMail into separate libraries, such as Akonadi itself, the messageviewer component, the messagecomposer library and more. That allows us to re-use those components in other applications, such as the mobile version of Kontact, and makes it possible to unit-test the components, which was not possible before, as everything was tightly glued together in KMail.
Such a major rewrite is not without consequences: On the one hand, it will finally modernize the complete architecture and make it maintainable. On the other hand, such rewrites always introduce regressions and remove features. This is unavoidable, and it is always a compromise between releasing and between stabilizing. Software would never get released if the requirement would be to make it regression-free and feature-complete.
What does that mean for KMail 2? With the first release, some features will be missing:

  • There will be no on-server filtering for POP3. I guess most people weren’t using this anyway, as it was actually quite hard to configure this correctly in KMail 1. Normal client-side filtering will still work, which is what everyone uses.
  • There will be no attachment loading on demand for IMAP. In KMail 1, with online IMAP, it was possible to activate loading attachments on demand, which means attachment will not be downloaded when viewing a message, but only when the attachment is actually saved to disk. This had some problems in KMail 1, like attachments having a size of 0 bytes, but it also sometimes worked. While the Akonadi architecture supports retrieving only parts of items, we don’t use that yet in the message viewer.
  • The most hated KDE bug will not get fixed. The bug is that KMail’s UI will freeze when the incoming mails are filtered, which is especially annoying for slow filters, like the filters that pipe the mail through spamassassin. That is not a regression from KMail 1, but we had hoped that the bug would be fixed with Akonadi. The idea is to use a filtering system which runs in a separate process in the background, instead of doing the filtering inside of KMail. Such a filtering system was created in a summer of code project last year, but due to lack of time, it is not yet integrated in KDEPIM. That means filtering will still be done on the client-side in KMail, with the consequence of freezing UI. Once we get a release out of the door and have some time, we will integrate the new filtering framework that will finally solve this bug.
  • We’re not sure if we manage to be able to migrate the IMAP cache if you’re using disconnected IMAP. That would mean re-downloading the mail again.
  • Probably IMAP server-side searching will not make it either

As said earlier, KMail is not yet ready for being released as a beta version. One major blockers right now is the lack of data migration, which includes reading the old KMail 1 index files and storing those flags in Akonadi, and also migrating over the account configuration. This is being worked on at the moment by Kevin Krammer, so we’ll have data migration before the first KDEPIM beta. The other blocker is the lack of client-side filtering, which Volker Krause is working on. As said above, that will at first be almost the same filtering system as we had in KMail 1, which allows us to have a working client-side filtering system for the first KMail 2 release. After those two blockers are fixed, which is before the first KDEPIM beta release, KMail 2 should be usable for other developers and testers, and we can concentrate on making KMail 2 usable for daily usage.

So the major changes in KMail 2 happen in the background, mainly not visible to the user, as the user interface largely stays the same. We have talked about the advantages of the Akonadi framework in other places, so I won’t repeat that here in detail. Overall, the porting to Akonadi is an important step in long-term stability and maintainability, and also fixes the old architectural issues we had with our 10-year-old system. I’m looking forward to the release of KMail 2 and of the ported version of KOrganizer and Kontact, which finally gives as a new architecture that meets modern requirements.

ยน: No wonder I’m not blogging much, writing this blog took a long time ๐Ÿ™‚





Nepomuk in KMail 2

18 01 2010

As you might now, we are currently porting KMail to Akonadi. The Akonadi-based KMail will be called KMail 2 and released together with KDE SC 4.5 if everything goes well.

Just a quick summary of what Akonadi is, for those who don’t know: Akonadi is an abstraction layer/proxy and a cache for PIM data.
PIM data can by anything like mails, contacts or calendar entries, and they can come from different sources, like an IMAP server, a
local vCard file or an Exchange server. Akonadi provides an easy API for the client application developer to access that PIM data
in a transparent way.

This post is about Nepomuk, not Akonadi. Akonadi uses Nepomuk to index the mails. Basically this means every mail that was seen by the Akonadi cache is also indexed by Nepomuk. This makes some great features possible, which I’ll describe below. With screenshots!

Powerful searches

The main benefit of Nepomuk is that we can do very powerful searches. KMail 1 had a search function as well, and even virtual folders. Those however were so slow that they were basically unusable.
With Nepomuk-powered searches, this is different: The searches are quite fast, and because they run in another process, the KMail GUI is not slowed down by searches anymore. Now it is for example possible to have a virtual folder with all messages marked as to-do or important.
Speed is not the only thing that is improved: The search results are now much, much better. We can now search in attachments and also optionally in encrypted mails. We can even do powerful semantic searches, for example linking mails with contacts. One example is “search all mails sent by persons who are tagged with ‘boss'”, since you can now tag contacts in KAddressbook.
Because Nepomuk indexes the mails, they will now show up in KRunner when you type something there. The amount of mails it finds for a given search term is so powerful that it is downright scary: when typing your name, it finds mails sent by you, sent to you, mentioning you or even mails in which you are mentioned in some attachment.
To conclude, searches in KMail 2 will rock!

Tagging

In KMail 2, we now use Nepomuk for all our tagging needs, instead of relying on some home-brewn system. The big advantage of course is integration with the rest of KDE, for example an application showing you every thing that is tagged with a certain tag can now also show the tagged mails. And KRunner will also display mails when you search for a tag.
We also have a Nepomuk tag resource, which shows one folder for each tag, with the tagged mails inside. When you drag a mail into such a folder, it will even get tagged automatically. Again this is powered by Nepomuk searches behind the scenes.

Below two screenshots showing how to tag a mail, and how the tag folders look. Please ignore the horrible colors, I was just testing the function that tags can change the text and background color.

Annotations

Having annotations for mails is a much requested feature. Often, people want to attach a little note to a mail where they can put some text in. Now, which technology could we use for annotations? Right, Nepomuk! I implemented annotation support today, and I have to say it was pleasantly easy to do so.
In KMail 2, you can now add a note to a mail. The mail list shows a little icon when a mail has a note attached, and you can edit the note by clicking it, or by using the appropriate action from the main menu.

Below are some screenshots which shows how to use annotations.

Did you notice the tooltip in the last screenshot? The note you have added to the mail also shows up in the tooltip. Even better: The tooltip shows a short preview of the mail text now, for mails that are indexed by Nepomuk. It was again really simply to get that information out of Nepomuk.

Closing

I hope I could give you a glimpse at the nice improvements related to Nepomuk which will be available in KMail 2.

So when can you use that cool stuff?
KMail 2 is currently under heavy development, right now it should not be used at all ๐Ÿ™‚ In the past, we have been working in a separate branch, the akonadi-ports branch. That branch will now be merged back to trunk during the next days. If you are a user who follows trunk, you should switch to the 4.4 branch. Otherwise you’ll see a big fat warning when starting KMail 2.

Right now there are quite many bugs and regressions in KMail, the Akonadi port is probably comparable with the KDE3 to KDE4 port, if not worse. We hope to stabilize KMail 2 so that it becomes usable soon, and if everything goes well, it can be released together with KDE SC 4.5.
Helping hands for the work on KMail 2 are very welcome, there is a huge amount of stuff to fix where you could help us. Find us in #kontact or #akonadi on IRC, or on the kde-pim mailing list.





Junior Job Achievements

24 08 2009

I my last post, I talked about Junior Jobs in KMail a bit. Now, I want to write about the progress of those Junior Jobs that was made during the last few weeks.

I think the Junior Jobs program was a success, the wiki page where I listed ideas was almost empty at one point. Some of the developers who started with Junior Jobs later picked something else up by themselves, which is the way it should go. Especially since it is not easy to actually come up with ideas ๐Ÿ™‚

Now, without too much further talking, let me present you the progress, including pretty screenshots. Note that the order here is totally random, and also I probably forgot many things here, so don’t feel left out when you committed something that isn’t listed here.

James Bendig improved the usability of the options of the new message list. There is now an unified way to configure tooltips for the folder list and the message list. Remember all the buttons next to the quick search field that appeared in KDE 4.2? Those buttons were confusing to new users, who often didn’t discover how to change the theme or the aggregation. Also, those buttons cluttered the UI a bit. This is how it looked like:

Quick search line before the changes

This is how it looks now:

Quick search after the changes

As you can see, the buttons have been removed. Vincent Dupont helped with converting the filter to a combo box again. But where are the options to change the theme, aggregation and sort order now? They are now in two places: The View menu, and in the context menu of the header. The global theme and aggregation now can also be changed in the config dialog. Setting a per-folder theme or aggregation is now also easier, it can be done in the folder properties dialog now:

Folder Properties Dialog

Overall the options for the new message list are now much more user-friendly.

Bruno Bigras ported over some long forgotten features from the old kdepim 3.5.5+ branch, for example an improved recipients picker that shows the alternative mail addresses as children of the contact and has more grouping capabilities like grouping per address book category.

New recipient picker

Bruno also added a new filter action that can add people to the addressbook automatically. Also, you can now filter messages in KMail before they are sent.

Torgny Nyblom again converted one hardcoded dialog into an UI file. I remember a year ago or so, there were no UI files in KMail, but now those keep increasing.

Jonathan Armond brought back searching by status and added searching by tag. Tags can now also be added by filters. Switching the identity in the composer now switches the template as well, if you have not modified the message already. For some people, adding the signature at the beginning or the end is not enough, so now the %SIGNATURE command is supported in templates.

Jaime Torres, whom you probably know as a member of the bugsquad, also contributed a couple of bugfixes.

Apart from the Junior Jobs listed above, there were of course more commits in KMail, but I don’t want to talk about those now. One person deserves special mention though: Martin Koller. He is by no way a “junior”, since he was listed in the KMail about dialog long before I was added there.ย  But recently he started coding for KMail again and fixed a lot of bugs, over 30 I think. He also went through bugzilla and closed a lot of bugs there as well.

A big thank you to all the people who contributed to KMail and help making it better!

Last but not least, there has been much progress in the akonadi-ports branch. Kevin blogged about progress with the ports of the message list and the reader widget, you should read that if you haven’t already. Constantin made good progress with the port of the composer, which does not sound exciting, but it is a very important step. His work will eventually make it easier to implement HTML replies, share the composer library with other applications, make it easier to support native Exchange sending methods instead of SMTP and much more. But there is so much stuff in Akonadi-land that I really should do a separate post about this.





Hello Planet & KMail Junior Jobs

6 06 2009

Hello Planet! If you don’t know me yet, let me introduce myself: I’m Thomas McGuire, and I’m a KDE developer. My main work area is KMail, as I’m the KMail maintainer, but I do touch other bits occasionally.

Why did I start a blog? Not many members of the KDEPIM team blog often, and I think it is important to tell the outside world what is going on, I plan to give updates when something interesting related to KMail or KDEPIM happens. Also, when googeling for my name, the first entry still is a certain (dead) air force major, that has to change ๐Ÿ˜‰

Ok, now that I got mandatory introduction finished, let me get to today’s topic: Junior Jobs in KMail.

As you maybe know, for example through Allen’s blog, we have relatively few developers, but a huge codebase that needs to be maintained, and thus are always in the need of new blood.

Getting into KDE development is not always easy, especially when there is lot of code like in KMail. To make it easier for potential developers, I’ve created a wiki page on techbase with Junior Jobs for KMail. Those are little coding tasks that don’t require much knowledge about the internals of KMail, and are self-contained, so working on those issues should be easy and fun.

So if you know C++ and a bit of Qt and want to help out with KMail development, look at the wiki page: http://techbase.kde.org/Projects/PIM/KMail_Junior_Jobs!

I’m glad that two developers, Jonathan and Frank, already joined in the fun.

Frank fixed a rather annoying regression which badly affected the speed of disconnected IMAP syncing in 4.2.3 and fixed a bug that renaming a sending account had no immediate effect.

Jonathan added support for tags that change the background color (will be in KDE 4.4, not 4.3 due to the feature freeze) and fixed two bugs, one relating to grouping of messages if the start of the week is not on Monday, and one with the filter dialog.

So thanks to Frank and Jonathan!

Hopefully we’ll see more new developers picking up Junior Jobs soon!

The next posts will probably be about Akonadi or the Summer of Code project by my student Constantin, so stay tuned! Tell me if there is any topic you are particularry interested in.