• Pimp my Nepomuk

Being both a really old KDE user and a semantic desktop partisan at the same time, I am, of course, keeping my eye on the progress in Nepomuk project.  It was apparently close to my old dream of a tagging framework supported natively and consistently across the whole desktop environment, so I highly appreciated this effort and it was nice to hear that Nepomuk would be officially included into KDE – my desktop of choice for many years.

Unfortunately, the experience was rather disappointing. It’s turned out to be painfully slow, not only slow by itself, but being a brake for overall desktop navigation. Even hovering the cursor over files and folders in Dolphin made Nepomuk process to eat above 50% of CPU time and caused annoying delays. The simple operations like assigning a tag to a file took seconds, the responsiveness which is obviously inappropriate for a real-world desktop system. It thus was turned off in a hope that the things would be improved in future versions (I was confused a bit by how it might appear in the production release, but, frankly, early KDE 4 was full of much more disastrous things). Since then, I checked it after every KDE version upgrade, but there was no visible progress in performance, alas.

It was really good news – the author of the post above argues that the performance issues are in fact, caused by a storage backend which Soprano, an RDF framework underlying to Nepomuk, uses to keep RDF data. By default, it’s shipped with Redland (aka librdf), an RDF database library written in C. Luckily, the backend is easily replaceable and it’s worth to try to install a faster alternative seeking for a better performance. The author recommends Sesame2 – a 100% pure Java RDF framework which works (surprisingly for many, I think – but not for me!) much faster than it’s native code counterpart.

I tested Nepomuk with Sesame and convinced that now it works really faster – as it should, in fact. There is of course, a room for improvements in Nepomuk to be a real end-user tool – e.g. a tag navigation interface without which the tags are rather useless, but its another story. At least, the performance is not a blocker anymore, so Nepomuk now is enabled in my KDE all the time.

For Kubuntu users: How-to

I tested the Sesame2 backend for Nepomuk on Kubuntu 9.04 Jaunty Jackalope, KDE 4.3.0 and Sun JRE 6 (I have no idea if it works with GNU Java, but you can give it a try).

1. Install soprano-backend-sesame package (sudo apt-get install soprano-backend-sesame)
2. Make a symlink from $JAVA_HOME/jre/lib/i386/client/libjvm.so in the /usr/lib directory
3. Restart Nepomuk server

Check if Sesame2 backend is used now:

qdbus org.kde.NepomukStorage /nepomukstorage usedSopranoBackend

It should answer “sesame2“. If it still answers “redland“, something was wrong. You may need also to replace the value “redland” to “sesame2” in ~/.kde/share/config/nepomukserverrc file manually and restart Nepomuk again.

http://rdfbeans.sourceforge.net

What’s this

It’s a prototype of a social network service where people are connected automatically via shared topics of interests extracted from their texts. You can find more on the PCN theory in this post.

If you’re interested to play with it, we are glad to invite you to join the testing.

How to join

1. Request a personal invitation code by filling this form. The code will be sent to the provided email address.

2. Register on PCN with your invitation code. Your username must be the same you used in the request above.

3. Download, install and configure the PCN client (Conex) as proposed by the system.

To run the client, an installed Java Runtime Environment is required. You also may need the Java plugin installed into your browser for visualisation of the network in your server profile.

The PCN Client

The PCN client software (Conex, from “CONcept EXtraction”) is a desktop content aggregator collecting pieces of content from different locations (local folders, webs, email, RSS feeds, Del.icio.us, SharePoint servers …). The content is analyzed to extract valuable terms from the texts and assign them as the document tags (manual tagging is possible also). A user can browse her content collections with the tag cloud or  metadata facets, edit document metadata, annotate documents, search in the content collections and do a lot of other things. Those who are interested into details of client functionality and in a underlying technology, can check this site.

To configure PCN connection, select  “PCN Client → Configure PCN Client” menu. At the first time, it will ask your username/password on the server and offer to create one or more contexts for your data. The contexts serves as navigation facets to browse the content collection and are useful to organize the content resources depending on your activities. After the contexts are defined, you can assign existing Conex content locations to them.

Autotagging is applied to individual documents by selecting them and choosing “Autotagging…” in the context menu. This operation can also be automated for all documents in a specific location, if “Apply autotagging for new documents” option is set in the location properties (“Settings” tab of the location dialog box). You can edit the results of autotagging or add new tags for selected documents using “Edit tags” option of the context menu.

As the content is tagged, Conex sends the tagging data and resource descriptions to the server, where a profile of user’s interests is created. All updates in the monitored locations are also sent to the server (in the specified time interval) to keep the user profile up-to-date.

By default, Conex works with the files in your local folders. To enable other types of content locations, you need to install the plugins. A plugin is installed with a single click in the Plugin Management console (“Tools → Manage plugins…” menu) and will be activated after restart.

PDF, MS Office, OpenOffice, HTML, XML and plain text documents are supported out of the box.

For SCAN users

To clarify the things, Conex is neither a proprietary fork, nor a new version of SCAN. It’s basically a rebranded distribution bundled with the common document plugins and integrated into the PCN solution. There is a plugin providing connectivity with the PCN Server and it’s possible to install this plugin into “native” SCAN to use it as a full-featured PCN client. This likely won’t work with the released version, but should be ok with a SVN snapshot. Anyway, new SCAN version enabling the PCN plugin will be released soon, so stay tuned.

The PCN Server

The server receives data from the clients and builds the socio-semantic network of users connected via the tags they have in common. After you’ve submitted some data from Conex, you can check your profile on the server to see your tags and  people appeared in your personal network:

The people listed in your network are the users who have the similar interests. On their profile pages, you can see a detailed information on compatibility between you and a profile owner:

The Network view displays a visualization of your complete network as a map of the overlapping clusters of tags you share with the members of your network:

The map is interactive — double-clicking an element will navigate to a user profile or to a tag page.

The people search is also implemented. You can find users by their names, other profile data or by the tags:

And finally, you can explore the content collections of the network members using faceted navigation with tags and contexts (supplemental metadata facets, such as document authors and  creation dates are also available).

Your privacy

⚠ The full text of the documents is not submitted to the server, however, Conex can use excerpts from the documents to fill the title and description metadata (if it is not defined explicitly). So, if your documents contain some sensitive information, parts of it may appear on the public. Please check and make sure that there are no private documents in the Conex locations you are about to assign to the PCN contexts. Well, you’ve been warned.

Report issues

It’s a beta, so  the bugs are likely to be there. Feel free to report the bugs, either in the Server or the Client,  to the bug tracker (registration is needed). Any other feedback, of course, is also more than welcome.

Disclaimer: Like it says on the author page, I work for UoN and Laboranova project, but I cannot speak for it officially in this blog. These are my own views as a project insider and the other’s may differ.

Back story

As far back as in early 2008, I’ve been introduced into PCN proposal and invited for collaboration on a R&D for its prototype implementation. It was very interesting for me and we developed a vision and basic design principles of the PCN solution to start the prototype development at the beginning of 2009.

This work is a part of Laboranova — a large EU project aimed at new ways of collaboration between knowledge workers and sharing ideas and competencies. Apart from PCN, there are lots of other interesting thngs the project partners do there.

PCN basics

The idea is a social networking where the members were connected via shared areas of expertise. These areas are identified by the topics of interests, discovered at user’s content.

Like other social network services connect people via shared professional activities, schools, hobbies or music tastes, PCN approach employs the shared semantics of texts the people read or write. There is an optimistic assumption that such semantics, expressed as sets of the weighted keywords (or, tag clouds), may reflect an area of user’s expertise and constitute a profile of his professional interests.

Multiple user profiles are aggregated to create expertise information and form a socio-semantic network of people and concepts. Comparing profiles of different users, we can evaluate a similarity of their expertises and thus, estimate their social distance in the network. We also can take a specific area of expertise to find who are the best experts relevant to it, and explore the related overlapping areas. With this network analysis, it is possible to generate a variety of individual recommendations to help people to discover new collaboration opportunities and areas of knowledge.

The Platform

There is a client-server architecture including a central PCN server and a number of clients connected to it. The PCN client software is installed locally and analyses content from different locations specified by its owner: file folders, webs, email, RSS, delicious accounts, SharePoint servers and so on.

A user can let the software extract the keywords from the content automatically, edit the results or choose to tag the content manually. Then he chooses what locations should be submitted to the server and assigns them to one or more named contexts which help to organize the concepts within the user profile. Document metadata and tagging information is uploaded to the server where an individual profile of tags is created and published. As a location is submitted to the server, it is monitored for changes to synchronize the profile with the actual state of the content (by sending incremental updates at a specified time interval).

The Client

The  client part is based on SCAN, so there is a zero barrier to start working with it for SCAN users. Actually, from a SCAN user perspective, there is no difference from usual everyday work — you can enjoy a full set of features one can find in native “offline” SCAN, but also use it for populating your public profile of interests at the PCN server.

Also, variety of location types and document formats are supported by the client thanks to the plugins from the SCAN repository.

The Ontology

The server receives metadata about content resources as SIOC RDF, so in theory, it may work with any SIOC provider, apart from the default PCN client. The server augments resource metadata with relationships to the user profile, concepts and contexts, thus forming the quadripartite PCN ontology model:

PCN quadripartite model

To describe the PCN model, we adopted SCOT (Semantic Clouds Of Tags) ontology aimed at conceptualization of the structure and  semantics of tagging data with strong focus on social interoperability. It is an extension and further development of the Tag Ontology project that describes the relationship between an agent, an arbitrary resource, and one or more tags.

The SCOT (Tag) ontology is based on a tripartite (User—Tag—Resource) model. These three core concepts are connected together via a central concept of Tagging representing the tagging activity. Every Tagging instance can be considered as a result of a single tagging action defining a user who performed it, the tagged resource and what tags have been used. It also can carry auxiliary information about the action, such as the time of tagging.

For users and resources, SCOT relies upon concepts from SIOC — specifically, sioc:User and sioc:Item classes. For our PCN ontology, we extended the SCOT model with the notion of  context by adding another class of entities and a property to relate them with the Tagging instances.

Extended SCOT Tagging as a skeleton of the PCN model

Using SCOT Tagging model, it is possible to avoid excessive verbosity in the PCN ontology, as the relationships between core PCN classes (the edges of the tetrahedron) are inferable from their relationships with the central Tagging class (the skeleton). In the diagram below, the implicit relations are shown as dashed.

PCN Ontology

An interesting possibilities comes from the fact that the scot:Tag is actually a subclass of the SKOS Concept, so all kinds of SKOS reasonings about the concepts are possible in the future. Moreover, MOAT features incorporated in SCOT open a way to integrate the tag descriptions with the Linked Data web.

The API

The client talks to the server via a simple RESTful API for adding and modifying the users metadata. As said above, the metadata is described with SIOC, so it is theoretically possible to use that API and integrate the PCN server with any system which can provide SIOC metadata about the content.

An endpoint for SPARQL queries is planned also.

For more information:

• An early description of the work package (Laboranova Profile System) on Laboranova web-site.
• Marc Pallot et al, “Future Workplaces, towards the ‘Collaborative’ Web“
• Peter Mika, “Ontologies are us : A unified model of social networks and semantics“
• Hak-Lae Kim et al, “Review and Alignment of Tag Ontologies for Semantically-Linked Data in Collaborative Tagging Spaces“ , “The State of the Art in Tag Ontologies: A Semantic Model for Tagging and Folksonomies“
• SCOT
• SIOC

Working on my current project, I had to deal with a rich object model which is persisted into a RDF triple-store, so quickly I stuck in writing the object-triples translation code. I thought it would be a good idea to automate this by providing a simple data binding framework for transparent mapping of the Java objects to RDF (like Hibernate and other tools does so for SQL or XML) with the following requirements in mind:

• It should be easy to make existing classes compatible with the framework with minimum modifications. This should not affect the business object model (no special interfaces and superclasses) and should not interfere with common JavaBeans-oriented frameworks (like Spring, etc.). Any JavaBean-like POJO class can be RDF-serializable just with few Java annotations added.
• No predefined ontologies and RDF-schemas are required.
• Class information is stored in the RDF model for transparent instantiation of the objects.
• Cascade binding for related objects should be supported.
• Basic Java Collection types should be supported.

This framework is implemented using Java annotations and Reflection APIs and works with major RDF triple-stores (via RDF2GO abstraction layer).

RDF Beans

In order to be mapped to an RDF resource, a Java class must obey certain conventions:

• The class must have a public default (no-argument) constructor.
• The class declaration must have a @RDFBean annotation to associate it with appropriate RDFS class:

  @RDFBean("http://xmlns.com/foaf/0.1/Person")
  public class Person { ...
  

See also: A RDFBean class example.

Properties

The class properties must be accessible using standard getter and setter methods. The properties are mapped to their RDF counterparts with @RDF annotations:

@RDF("http://xmlns.com/foaf/0.1/name")
String name;

@RDF("http://xmlns.com/foaf/0.1/mbox")
String email;

public String getName() { return name; }
public void setName(String name) { this.name = name; }
public String getEmail() { return email; }
public void setEmail(String email) { this.email = email; }

The argument of @RDF is a valid URI of a RDF property (predicate) in a domain of the specified RDFS class. The following types of properties are permitted:

• A Literal (String, Boolean, Date, Integer, Float, Double, Long, Short, Byte, java.net.URI)
• Another RDFBean class
• A Java Collection (List or Set) of Literals or RDFBeans

Literal values are represented in RDF with corresponding XML-Schema datatypes and Collections with RDF Containers (rdf:Bag or rdf:Seq depending on a Collection type). References to other RDFBean objects will be kept as RDF object properties and the values will be represented as separate RDF resources.

RDF subject

There must be a special String property to hold an unique URI value to identify the RDF resource (the subject URI). It must be accessible with getter and setter methods and be marked with @RDFSubject annotation:

@RDFSubject
String id;

The property value must be a valid absolute URI, otherwise you can use the prefix argument for a common prefix to construct URIs from arbitrary String identifiers:

@RDFSubject(prefix="http://example.com/persons#")
String id;

Data Binding

RDFBeans data binding framework is based on RDF2Go API which creates an abstraction layer above physical RDF storages. It makes an RDF model independent from specific RDF frameworks (such as Sesame and Jena), thus it can work with any, if a RDF2Go adapter is implemented. The data binding is performed using single RDFBinding class, which is initialized with existing RDF2Go model:

import org.cyberborean.rdfbeans.RDFBinding;
import org.ontoware.rdf2go.RDF2Go;
import org.ontoware.rdf2go.model.Model;
...
Model model = RDF2Go.getModelFactory().createModel();
model.open();
RDFBinding binding = new RDFBinding(model);

The marshal method converts a Java object into a RDF resource in the model:

Person person = new Person();
person.setId("http://example.com/staff#johndoe");
person.setName("John Doe");
person.setEmail("johndoe@example.com");
...
binding.marshal(person);

This code will create a set of RDF statements representing the properties of the given object in the model:

<http://example.com/staff#johndoe> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .
<http://example.com/staff#johndoe> <http://xmlns.com/foaf/0.1/name> "John Doe"^^<http://www.w3.org/2001/XMLSchema#string> .
<http://example.com/staff#johndoe> <http://xmlns.com/foaf/0.1/mbox> "johndoe@example.com"^^<http://www.w3.org/2001/XMLSchema#string> .

Additionally, it adds a special statement to associate the RDFS class with Java:

<http://xmlns.com/foaf/0.1/Person> <http://cyberborean.org/rdfbeans/1.0/bindingClass> "com.example.Person" .

The marshal method inspects the properties which link the object with other RDFBeans and initiates their cascade binding to the RDF model, thus simplifying the development and ensuring referential integrity of the RDFBeans.

To reconstruct a Java object from RDF model, there is unmarshal method:

import org.ontoware.rdf2go.model.node.URI;
...
URI subject = model.createURI("http://example.com/staff#johndoe");
Person person = (Person) binding.unmarshal(subject);

The method creates new instance of the Java class (defined by http://cyberborean.org/rdfbeans/1.0/bindingClass property) and fill the properties with data from the RDF model. If a property holds a reference to another RDFBean, it results into cascade unmarshalling of related objects, thus restoring the original object model.

Code

• RDFBeans JAR and source code
• Javadocs

[Update: 2009-08-04]

RDFBeans framework is released on SourceForge: http://rdfbeans.sourceforge.net. The API has been changed, please check the documentation and Javadocs for details.

RSS of Cyberborean Chronicles now is http://blog.cyberborean.org/feed.

Changes

Main taxonomy of the content (tags) is preserved, but the structure of categories is completely redesigned. In 1.0 version, categories were used to identify topics and thus duplicated the tags in many cases. In fact, there were two independent systems of topics classification.

In new version, topics are identified only by tags (java, linux, coding etc) while “categories” are used to classify  posts by their type, format or purpose (Announcements, Essays, Howtos etc). Thus, there are two  clearly distincted taxonomies: topics and types.

Reading a post, you may notice “See also” menu in the sidebar. It contains the posts similar to the current one – those you might want to read if you are interested in that topic. This magic is a courtesy of Similar Posts Wordpress plugin.

Photos

I also launched a standalone photo gallery and moved all pics from my Flickr account to there. Now the gallery is another part of Cyberborean Chronicles: http://photos.cyberborean.org. The pics are organized with their own system of tags.

I’m not going to post gallery updates in the blog (former Photos category) anymore, except for the shots I have to tell something special about. You can see gallery updates (the latest pics) at the widget on the right sidebar. You also can subscribe to the updates feed: http://photos.cyberborean.org/main.php?g2_view=rss.SimpleRender.

Gears

Wordpress just rocks as usual and gears everything on this site with a help of the following plugins (alphabetically):

• ActiveLink lets quickly insert links to Wikipedia definitions.
• Add to Any: Share/Save/Bookmark Button: you can see this button on every post page to add the post to your favorite bookmarking services.
• Akismet keeps spammers away from my blog.
• Contact Form 7 powers the form to contact me on the Feedback page.
• Similar Posts does all magic to generate “See also” menus.
• Simple Tags helps me to tag the content and generates the tag cloud (see on the left)
• SyntaxHighlighter Plus converts code snippets into eye candies.
• WPG2 integrates the photogallery with the blog

The photogallery is powered by Gallery2 software. To photos from Flickr are imported with Gallery2Flickr plugin (hacked to fix the dates of the imported pics).

Please note that everything is still a beta, so any feedback is welcome. Thanks!

If you haven’t noticed already, there was no simple way to just enter a web URL in SCAN and get a document in the repository. One had to add web documents via del.icio.us or fetch them from RSS. Now it is fixed – with Web plugin you can add web-pages with their URL’s and more – you can tell the plugin to follow hyperlinks and crawl web-pages recursively, thus adding whole web-sites to SCAN document repository.

More on Web-plugin »

To install new plugin, click on “Check for updates” button in Plugins manager window (“Tools?Manage plugins”). Web plugin will appear in “Location type plugins” section.

• Download SCAN 1.3.1

What’s fixed actually:

• [1976004] Wrong document date breaks the application
• [1976266] No validation of date values in Document properties
• [1976036] Indexing fails with String index out of range error
• [1976045],[1925137] Function keys starts editing the table
• [1976066] Application window doesn’t open immediately on start
• [1903350] Exclusion Filter Regular Expression

As I skipped 7.10 “Gutsy” release, I preferred to do a fresh install from Hardy Heron LiveCD instead of two-step “Feisty » Gutsy » Hardy” network upgrade. I installed Hardy right into the existing Feisty partition replacing the old installation. Separate home partition was left untouched to keep all my personal preferences and data in new system.

It took ~25 minutes from booting the installation CD to logging into a working system.

Unsurprisingly good

No doubts, Kubuntu developers and packagers spent this year not for nothing. In general, Hardy is a better system than previous Kubuntu versions. It loads essentially faster and many little annoyances was fixed. Good job!

Hardware compatibility

Ubuntu systems are known to have great hardware support. As far back as 7.04 version, it recognized my hardware configuration including such exotic devices like DVB-card and CDMA modem without any problem. The only trouble was my HP LaserJet 1020 printer for which I had to install and configure foo2zjs package manually. So, it was nice to see it’s fixed in Hardy; now my printer is supported via native HP open source drivers and working out the box.

A bad surprise was that Hardy could not detect my monitor automatically and set failsafe 640×480 resolution as a result. It was strange as it was no problem with it in Feisty. Even when I’ve clicked “Detect monitor” button manually, it detected it as default “plug-n-play” monitor. I had to remember a model of my monitor and select it from the list. Not a big deal, but a pity though.

KDE 4

Looking great and nice but apparently, a lot of work is still needed there. Crashes are not unusual everywhere and the single Plasma panel looks more like a prototype than a part of a desktop for the real world. It’s half-baked, almost not configurable and I cannot setup a desktop I need for my tasks (IMHO, an essential fault for any Linux software).

So, I’m use the “solid rock” KDE 3.5.9 as a primary desktop environment for work and sometimes switch to KDE 4 just to get a feeling of  bleeding edge desktop technologies.

Grumbles

• Dolphin – I already grumbled at it and I still have no idea what are the reasons to have yet another desktop file manager. Konqueror was a default KDE file manager for years and a whole generation of Linux users grew up with an idea of this software inspired by Konq. Dolphin’s user experience is different in many ways and totally unusual for me (and I think, I’m not alone). No matter if Dolphin is good or bad, I need Konqueror, thanks!
• Samba didn’t see my home network after installation. And I was unsuccesfull to fix it using configuration GUI in KDE Control Center (though I’m not a sysadmin guru and might miss some options). Finally I managed to configure the network by manual editing ’smb.conf’ taken file from a backup of previous installation. Maybe it sounds trivial for experienced network administrators, however I remember that I didn’t edit any configs to get Samba working in Kubuntu 7.04.