Nepomuk in KMail 2

18 01 2010

As you might now, we are currently porting KMail to Akonadi. The Akonadi-based KMail will be called KMail 2 and released together with KDE SC 4.5 if everything goes well.

Just a quick summary of what Akonadi is, for those who don’t know: Akonadi is an abstraction layer/proxy and a cache for PIM data.
PIM data can by anything like mails, contacts or calendar entries, and they can come from different sources, like an IMAP server, a
local vCard file or an Exchange server. Akonadi provides an easy API for the client application developer to access that PIM data
in a transparent way.

This post is about Nepomuk, not Akonadi. Akonadi uses Nepomuk to index the mails. Basically this means every mail that was seen by the Akonadi cache is also indexed by Nepomuk. This makes some great features possible, which I’ll describe below. With screenshots!

Powerful searches

The main benefit of Nepomuk is that we can do very powerful searches. KMail 1 had a search function as well, and even virtual folders. Those however were so slow that they were basically unusable.
With Nepomuk-powered searches, this is different: The searches are quite fast, and because they run in another process, the KMail GUI is not slowed down by searches anymore. Now it is for example possible to have a virtual folder with all messages marked as to-do or important.
Speed is not the only thing that is improved: The search results are now much, much better. We can now search in attachments and also optionally in encrypted mails. We can even do powerful semantic searches, for example linking mails with contacts. One example is “search all mails sent by persons who are tagged with ‘boss'”, since you can now tag contacts in KAddressbook.
Because Nepomuk indexes the mails, they will now show up in KRunner when you type something there. The amount of mails it finds for a given search term is so powerful that it is downright scary: when typing your name, it finds mails sent by you, sent to you, mentioning you or even mails in which you are mentioned in some attachment.
To conclude, searches in KMail 2 will rock!

Tagging

In KMail 2, we now use Nepomuk for all our tagging needs, instead of relying on some home-brewn system. The big advantage of course is integration with the rest of KDE, for example an application showing you every thing that is tagged with a certain tag can now also show the tagged mails. And KRunner will also display mails when you search for a tag.
We also have a Nepomuk tag resource, which shows one folder for each tag, with the tagged mails inside. When you drag a mail into such a folder, it will even get tagged automatically. Again this is powered by Nepomuk searches behind the scenes.

Below two screenshots showing how to tag a mail, and how the tag folders look. Please ignore the horrible colors, I was just testing the function that tags can change the text and background color.

Annotations

Having annotations for mails is a much requested feature. Often, people want to attach a little note to a mail where they can put some text in. Now, which technology could we use for annotations? Right, Nepomuk! I implemented annotation support today, and I have to say it was pleasantly easy to do so.
In KMail 2, you can now add a note to a mail. The mail list shows a little icon when a mail has a note attached, and you can edit the note by clicking it, or by using the appropriate action from the main menu.

Below are some screenshots which shows how to use annotations.

Did you notice the tooltip in the last screenshot? The note you have added to the mail also shows up in the tooltip. Even better: The tooltip shows a short preview of the mail text now, for mails that are indexed by Nepomuk. It was again really simply to get that information out of Nepomuk.

Closing

I hope I could give you a glimpse at the nice improvements related to Nepomuk which will be available in KMail 2.

So when can you use that cool stuff?
KMail 2 is currently under heavy development, right now it should not be used at all🙂 In the past, we have been working in a separate branch, the akonadi-ports branch. That branch will now be merged back to trunk during the next days. If you are a user who follows trunk, you should switch to the 4.4 branch. Otherwise you’ll see a big fat warning when starting KMail 2.

Right now there are quite many bugs and regressions in KMail, the Akonadi port is probably comparable with the KDE3 to KDE4 port, if not worse. We hope to stabilize KMail 2 so that it becomes usable soon, and if everything goes well, it can be released together with KDE SC 4.5.
Helping hands for the work on KMail 2 are very welcome, there is a huge amount of stuff to fix where you could help us. Find us in #kontact or #akonadi on IRC, or on the kde-pim mailing list.


Actions

Information

57 responses

18 01 2010
Björn Ruberg

Great stuff! I can instantly think of then bug reports I can close when this becomes reality🙂

18 01 2010
Thomas McGuire

And that is only the tip of the iceberg🙂

Many other bugs will be gone, like index corruptions, all kind of crashes involving KMFolder* or KMMessage, and much more.

OTOH, KMail 2 will of course have a new set of bugs, but overall, the bug count will go down, thanks to the new architecture.

I closed the feature request about annotations already, fwiw.

18 01 2010
Anders

Can I be put to sleep until KDE SC 4.5?🙂 Many things will be just sooo nice with akonadi and nepomuk. Well, i guess we will get a little of the goodness with KDE SC 4.4 already…

18 01 2010
Christoph

Hm, reading that you are going to switch to Akonadi in the next days, I wonder why starting KMail from last weeks trunk already starts Akonadi. Is it possible to run KMail/Akregator from 4.4 branch without Akonadi?

18 01 2010
sebas

@Christoph

Akonadi will already be used for contact information in KDE PIM 4.4. KAddressbook is the first application in KDE SC to make use of Akonadi. As Thomas writes, kmail follows in 4.5.

18 01 2010
Stefan

Wow

I can not wait!

“Now it is for example possible to have a virtual folder with all messages marked as to-do or important.”

Will it be possible to create virtual folder with other criteria? For example I would really like to have virtual folder with messages that has “Kmail” inside message…

18 01 2010
Thomas McGuire

Sure, virtual folders with all kinds of search criteria will be possible, including the example you mentioned.

18 01 2010
Gabor

This sounds fantastic! I hope this will get into the kubuntu packages soon!

18 01 2010
Romain

Great stuff !
Let’s make it happen🙂

18 01 2010
ben

You’re really doing a great job. Thank you!

As you describe, Nepomuk is used to a) index existing data, but also to b) save data generated by the user (e.g. tags). With that great power comes great responsibility to you: To make sure this data is always safe. Not just when moving a mail between folders, but also when kde/nepomuk/akonadi/kmail and friends are updated, .kde folders are moved etc.

This might sound simple, but I think its not: I’m often disappointed by (the otherwise great) amarok. When I take the time to rate/tag/comment songs, then the app should *never* forget/lose this information. Amarok did, I hope kmail and its pim friends never will.

Again, thanks!
ben

18 01 2010
Thomas McGuire

When moving mail between folders, the Nepomuk data is preserved, as each mail in Akonadi has an unique ID, which stays the same after moving.

The other point is importing tags from KMail 1: We have to read the old index files for migration anyway, for mail flags, and so we can migrate the tags in the same go, that should be possible.

The case of moving the .kde folder is indeed problematic, that will probably loose the Nepomuk data altogether. This not only affects KMail, but all applications which use Nepomuk. We or the Nepomuk guys need to come up with a good solution for this, it should be possible to backup and import Nepomuk data in an easy way.

18 01 2010
Fri13

I am not sure do I like that the KMail is getting number 2 after it so it would be KMail2 or KMail 2. It just looks so tech name:/

So this does then help removing the most hated bug in BKO (from 2002 on KDE 3.0.0 !!!!) that KMail interface is freezed when querying mails, like changing tab etc?
https://bugs.kde.org/show_bug.cgi?id=41514

And while porting KMail to Akonadi, please tip someone to add the Ctrl+M support. It is asked many times on BKO but devs have not choosed to include that great feature. I have forced to use QtCurve style to hide the menu with the experimental Ctrl+Alt+M function. I would love to see it with Oxygen style as polished clean UI.

I can not wait to start tagging files to include them queried on nepomuk searches! And the virtual folders…. Mmmm..😀

18 01 2010
DanaKil

“And while porting KMail to Akonadi, please tip someone to add the Ctrl+M support. It is asked many times on BKO but devs have not choosed to include that great feature.”

It’s not very related to the post, but indeed, a global shortcut to hide de menubar would be great, especially for some applications (ie akregator… once the toolbar well configured, I never use the menu)

18 01 2010
Thomas McGuire

I am still not sure if the application will actually be named “KMail 2”, but at least the version number will jump from 1.13.x to 2.0.0.

Regarding the most hated bug: There was a summer of code project last year the implemented filtering support for Akonadi. Once that is hooked up with KMail, that bug is gone, since filtering then happens in a different process and can therefore not freeze the KMail GUI. This is not done currently, but the plan is to do this, of course.

As for the Ctrl+M shortcut: Nice feature, someone should indeed implement that. Volunteers welcome, this should be an easy task🙂

18 01 2010
Fri13

@Thomas

Well, I have big hope they will keep it as “KMail” and move the version numbering. Because I just love the name “KMail”, it is like “GMail”, short and very unique.

And if someone would implent the Ctrl+M feature, it will make KMail look very easy to use, at least by my opinion. I have added only buttons to toolbar what most my clients needs. I myself dont use some actions like Copy/Cut/Paste or Next/Previous but many feels they are great.

But when used, there is no need for menu on daily basics because KMail has so rich features already. And this Nepomuk thing is just making it even better.

http://imagebin.ca/view/BCZLCmw.html

18 01 2010
mutlu

Wow! Wonderful developments, Thomas!🙂

Ben, AFAIK tags and annotations automatically point to the right file when they are moved, at least whe using Dolphin.

But I agree with you in regard to backup and transfer of the database. Nepomuk really needs a backup and transfer system to move annotations, tags, etc. from one computer to another and to enable easy backups.

Also, it would be great if Nepomuk would copy the metadata when a file is copied. Right now, the metadata only points to the source file. But that is probably not relevant to KMail.

Looking forward to what the future will bring.

18 01 2010
arwa

Adding a note to a mail is very interesting. But IMHO it would be better to display the note in the opened mail e.g. between the headers and the body, so it cannot be overseen. Also I have switched off all tooltips as they are in the way too often.

18 01 2010
Thomas McGuire

Yes, having the note (or just the note icon, not sure) displayed in the opened mail would be even nicer.
This would be a great junior job, it should be easy to implement. If anybody volunteers, please contact me.

I agree with the tooltips, I have switched them off myself.

18 01 2010
Dotan Cohen

How about having the note beside the message? Most non-HTML messages don’t need the full width, but vertical space is valuable.

18 01 2010
Luis

Hi. This is great work. My only regret is that it will only come out with KDE SC 4.5. I have a couple of doubts, though. What will happen when a user has a few tens of tags available. I still remember the old digikam where you could tag pictures with users from the address-book and the pop-up menu would just fill the whole screen… One other thing. Does this akonadi port mean that I don’t have to have kmail running in order to receive e-mails? For my personal usage I much prefer to have a simple notification (a persistent one) that tels me that a new mail arrived and from whom without having to have one more icon in the tray.

18 01 2010
Dotan Cohen

> since you can now tag contacts
> in KAddressbook

I have tested the new Kaddressbook, I do not see how contacts can be tagged. Where is this?

18 01 2010
Thomas McGuire

I think they are called “categories” there, but internally Nepomuk tags are used, as far as I know.

18 01 2010
Dotan Cohen

No, there are no Categories in the new Kaddressbook:
https://bugs.kde.org/show_bug.cgi?id=222678

This is actually one of two showstopper Kaddressbook bugs preventing me from upgrading to KDE SC 4.4.

18 01 2010
Thomas McGuire

I think categories in KAddressbook are not available for 4.4, only in trunk/4.5. I testify that categories are in trunk, I have seen them with my own eyes🙂

18 01 2010
Dotan Cohen

Thanks. I need to find an easy way to test Trunk. I’m no software dev, but I have a vested interest in keeping KDE useful🙂

Thanks, Thomas!

18 01 2010
AdeBe

Really nice to hear about these changes.
Personally I’m still looking forward to possibility of an all-out integration with GMail (Kaddressbook using GMail contacts, ‘Archive’ button in KMail etc.)

Keep up with this great job!

18 01 2010
Thomas McGuire

You’ll be pleased to hear that there is a Google contacts and a Google calendar resource for Akonadi: Check out http://www.my-guides.net/en/content/view/178/26/ on how to set those up. With KDE SC 4.4, setting up KAddressbook for Google should be easier, as it now uses Akonadi natively.

As for better GMail integration in KMail: Yes, that would certainly be nice, but there is currently no one planning to do that.

18 01 2010
AdeBe

Wow, thanks for the info, very interesting.

18 01 2010
redm

Cool, this sounds so awesome! I’m eagerly waiting for it since Akonadi has been promised to us in the beginning of KDE4!🙂

Btw. Is Nepomuk integrated into Akonadi (so every app using Akonadi will get it for free) or specifically in KMail?

18 01 2010
Thomas McGuire

Nepomuk is generally integrated in Akonadi, every app will get it for free basically.
We are currently indexing mails, contacts and calendar events with Nepomuk.

18 01 2010
shamaz

I think the next most wanted feature will be :
add a file import/export mechanism for kmail (including metadatas from nepomuk!)
IMHO, you should already start thinking about this while refactoring everything😉

18 01 2010
arekm

Nice but I hope that team works also on bugzilla reported stuff (like “kmail is damn slow” bug #135717) and not only new features…

18 01 2010
Onno Molenkamp

How will this new Nepomuk-based tagging work with IMAP? Will tags be stored on the server as well?

18 01 2010
Thomas McGuire

The tags are not yet synced with the IMAP server. The IMAP resource is technically easy to extend to support this, the problem is more on the conceptual side of things: I don’t know which tags on the IMAP server and in Nepomuk are changed, and therefore I can’t decide which tags to add/delete on which side. I am not quite sure how to solve this.

19 01 2010
Karellen

Is this related to just IMAP? Are there other online/remote resources that might have native tagging, to which the user may or may not have write access, where if they have read-only access they might want to create local tags, or if they have read-write access they may still want to keep local tags separate from on-line tags, or they might want to synch local/remote tags?

Is it worth thinking about making a distinction between local tags and remote tags for online content? Might it be possible to have a local “negative tag” that tells nepomuk to ignore a certain tag for a particular resource locally, without removing it from the remote resource itself?

One example that springs to mind for read-only posts is other people’s blog posts. Some people mark their blog posts with tags, which could get picked up by an RSS resource and integrated with nepomuk. Alice may tag a blog posts as “important”, but Bob think it isn’t, and doesn’t want it to show up when he searches for “important” blog posts in his feed reader. He then applies the “!important” local tag which overrides Alice’s “important” tag, to do what he wants. (Although you still might need some way to see “negative” tags in order to remove them and let the original through)

With IMAP email, maybe making a difference between local tags and online tags could help? Local tags are never sent to the server and always override remote tags, no problem. Remote tags cannot be set unless they can be immediately written to the server.

Maybe?

20 01 2010
Karellen

Extending from this, could local/remote tagging be an option on the IMAP account? “Local” means no syncing needed, no problem. “Remote” means that a connection to the server is required to push tag updates immediately, else the tagging operation simply fails.

18 01 2010
vvaz

Great things are coming, thank you!

18 01 2010
Timo

Thank you very very much for your work!

Please let me ask a question if there is a plan to make an option to rename a subject of mails in inbox? Because a lot of people are sending mails with strange subject and I’d like to rename them.

I’m very glad to see, that you are working with option to change colors of background, fonts etc.
I’m waiting for this feature since I use KDE. I believe this will be possible in every app in KDE SC.

Annotations are great too.

Hooray and thanks again!

19 01 2010
Thomas McGuire

Changing the colors and fonts of KDE applications is already possible, just have a look into System Settings. The tags in KMail can change the color and font of tagged messages on top of those system colors/fonts.

About changing the subject: Yes, that would be a very nice feature. And again it needs a volunteer who has time who implements this.
Somehow funny how many of those blog comments end up mentioning the commentator’s favorite bug or wish🙂 It is good to know those, though.

20 01 2010
Kishore

Fair enough that kmail will take until 4.5. However in the mean time is mailody usable in 4.4? I would like to use akonadi for mail starting with 4.4 if that is possible at all. I tried mailody 1.5.0-alfa1 and it allowed me to create a new mail account but crashed the moment i selected a mail to read it!

7 02 2010
Laurent

Hi ! Good news for tagging feature.
Tagging email will be nice, for sure. But i think tagging mail with rigth mouse menu is not very usefull for current usage with lot tags. Today i tag my email – and only use tag to navigate it – and i have more than 50 tags … Not easy to select one with right mouse menu. A combobox/textfield on the menu bar will be the must😉
Virtual folder are good way to retrieve tagged mails but you need to know filters before doing the search. I ‘m dreaming a tag cloud/list and a delicious like tagging navigation, more powerfull with dynamic tag combination.

9 04 2010
Roland

I hope that somebody is going to reorganise KAdressbooks way of tagging multiple contacts. Version 4.3.2 still does not allow to select a bunch of contacts and add a tag. Or assign it to a group, like in GMail or in Apple’s addressbook. This approach sounds to me much more useful, if I can have a contact in multiple groups. KAdressbook seems still to handle each resource as a separate entity, so when I add an adressbook for my mobile phone, I will never be able to sync it with my standard resource contact, but I have two entries. But maybe I am just to silly to understand the concept of KAdressbook’s categories, adressbooks and tags. To me, it should be reduced to the concept of groups, where I can assign contacts to ;-/

24 07 2010
rolland

I’d like to be informed when Kmai 2.0 will be avalaible for Ubuntu 10.04

13 10 2010
Reply SMS Hindi

Tagging is the future. Overcoming typos in tags, synonyms is a conflicting feature, but does not limit the wide applicability of being able to tie information together. Now, Nepomuk is a bit strict on the type of metadata allowed, which is why Strigi has a super ontology (as in super set), to which we can add chemistry bits.

Thanks in Advance

4 01 2011
மோகன்

Hi,

I am using kmail to read my IMAP mails. Usually I configure the account to be “Disconnected IMAP” so that I can access the mails when I am not connected to internet. It works perfectly. But I have some issues.

When I change my distro (like Fedora Kubuntu) or upgrade distro, all this information is lost. So I have to restart downloading entire IMAP mails once again. Is there any solution available to solve this problem?

5 01 2011
Thomas McGuire

Hi,

a random blog from over a year ago is not the best place to ask user questions, better to ask on the kdepim-users mailinglist or in the #kde IRC channel next time.
The best way to solve this problem is to copy all of your KMail configuration and data files to your new installation. See http://userbase.kde.org/KMail/FAQs_Hints_and_Tips#Transfer_mail_and_settings_to_another_computer_.28or_another_user_account_on_the_same_machine.29 for details.

26 01 2011
Leonardo

Hello Thomas,

Are there any news regarding this? KDE is in 4.6 as of today, and I was wondering (I’m still in 4.5.5), will this work in the latest release or there is work to be done yet?

Regards,
Leonardo

28 01 2011
Thomas McGuire

Hi Leonardo,

KDEPIM was not released with 4.6, it is not ready yet. You can use KDE 4.6 togehter with KDEPIM 4.4 though, that works.

Regards,
Thomas

1 02 2011
Leonardo

Thomas,

I meant if the nepomuk integration was ready in KDEPIM 4.6 (if its not out yet, when 4.6.X is). As I’m really looking forward this feature. Searching for an email right now is quite a pain!!

Regards,
Leonardo

1 02 2011
Thomas McGuire

The Nepomuk integration in KDEPIM is finished now, it’s just the rest that needs more work.

5 04 2011
kdenutter

“The Nepomuk integration in KDEPIM is finished now, it’s just the rest that needs more work.”

Mr McGuire – I am currently using KMail Version 1.13.6 on KDE 4.6.2, am I correct in thinking this is still KMail One? (not certain how the KMail version numbering works)

Thank you.

5 04 2011
Thomas McGuire

Version 1.13.6 is KMail 1, yep.
KMail 2 is version 2.x.x, and not released yet.

6 04 2011
kdenutter

Version 1.13.6 is KMail 1, yep.
KMail 2 is version 2.x.x, and not released yet.

Thank you Mr McGuire

Let me see if I can become the million and one person to ask you🙂 will KMail2 be released on it’s own or as part of the KDE 4.x upgrade cycle? and if so, which update will it be a part of?

Let me add that I don’t think KDE has ever looked and performed better, it has become a brilliant DE.

6 04 2011
Thomas McGuire

KMail2 will be released with the rest of KDEPIM, not on its own.
When the new KDEPIM will be released is not yet clear, the first release of it will probably be separate from the KDE SC release. Might happen during KDE 4.6.x, might happen during 4.7.x, not clear yet.

7 04 2011
kdenutter

Thank you for your update.

14 01 2012
Phillb

Is it possible to disable nepomuk completely from kmail2? I have renamed the binary nepomukserver so it can’t run for now. I use a 64G usb key and nepomuk slows things down so badly I don’t want to use it.

I’m also having lots of trouble with the interface simply ignoring my commands. Often the pane that shows the email is left in a state of “retrieving folder” and just never comes back.

5 06 2013
Emre Erenoglu

I know this is a very old thread, but I’m using Kmail 4.10.3 but can’t make an advanced search on the search bar. When I do it after clicking search icon on the right, I can put some criteria but that’s too long to do for my work. How can I make a quick search on the search bar for

find messages from aaaa with subject bbbb

In thunderbird, I use GmailUI/ExpressionSearch add-on, and then I can just do:

from:aaaa subject:bbbb

and it returns the results. Kmail doesn’t work this way.




%d bloggers like this: