There is however, a lot of important stuff I’m backing up regularly: projects, documents, Basket notes, emails and so on. A long ago, I have developed a custom automated backup procedure that has been greatly improved with the help of a dedicated server in my home network. The solution is simple, based on standard Linux tools and works perfectly for me.

On the desktop

I have a special directory in my home space, ~/.toBackup, that contains symbolic links to the places I want to include into the backup. A tiny script is called by cron every hour to loop over this list and send incremental updates to the server using rsync:

#! /bin/sh
 
TO_BACKUP=$HOME/.toBackup
REMOTE=192.168.0.100::backup
 
for D in $TO_BACKUP/* do
  rsync -avzL $D $REMOTE
done

The target directory (“backup“) of the server’s rsync daemon is shared via NFS, so I always have a mirror of my things one hour back maximum.

On the server

Every night, the server runs a cron job to copy the contents of the backup directory to another hard drive, thus making a daily snapshot of my work. If something horrible would happen with both the original data and with the mirror, there is a yesterday’s copy to restore.

#! /bin/sh
 
BACKUP=/mnt/disk1/backup
DAILY_BACKUP=/mnt/disk2/backup-daily
 
rm -rf $DAILY_BACKUP
mkdir $DAILY_BACKUP
cp -R $BACKUP/* $DAILY_BACKUP

And, finally, once a week (every Friday), yet another cron job packages the latest daily backup into tar.gz archives and puts them into new directory, named after the current date:

#! /bin/sh
 
DAILY_BACKUP=/mnt/disk2/backup-daily
WEEKLY_BACKUP=/mnt/disk2/backup-weekly/`date +%Y-%m-%d
 
mkdir $WEEKLY_BACKUP
cd $DAILY_BACKUP
for D in ./* do
   tar czhf $WEEKLY_BACKUP/$D.tgz $D
done

All what I have to do manually is to burn the DVD’s with latest archives once in a couple of months and clean up the weekly backups directory (though it is possible to automate the latter too).

The last year, an opportunity to bring myself to do it right is appeared, thanks to two things: First, I got new Core-i7 box as my  developer workstation, so my old good Pentium-4 came out of work. Second, a large roll door closet has been built in the hallway where I reserved a room for the server stuff.

The server room is 70×50×45 cm space in the top section of the closet, accessible with a ladder: good against kids, pets and unscheduled accidents of all sorts. The room contains the server machine, LAN/WiFi router and the power supply, as well as some empty space for accessories, backup disks, instruments and other stuff.

The Server

The server machine is Pentium-IV/3.00GHz CPU with 2GB of RAM, built upon ASUS P4S motherboard. The machine is rebuilt from a common 2005-year desktop PC by repackaging it into a smaller case and removing the videocard and DVD-ROM. Two brand new 1.5TB SATA drives were installed to provide enough disk space.

The box runs Ubuntu 10.04 LTS Server with 2.6.32-21 kernel.

Power and ventilation

The server closet is powered by 600VA Ippon UPS unit monitored via NUT.

The hardware in the closet produces a considerable amount of heat, so ventilation system was necessary. Hot air is pulled out of the closet space through a vent grille with a fan fitted into the drop ceiling and connected to the house ventilation grid. Inward fresh air is delivered through the clearances above and below the roller door and through the perforation at the back of the shelf. Initially, it was planned to mount another fan for forced air supply from outside of the closet but the idea had been postponed for the sake of noiselessness. For better air circulation, the internal shelves were made shorter to provide an empty space along the back wall.

The air vent carries a common 12V computer case cooler fan that is powered  directly from the motherboard and is controlled by BIOS, therefore. The more warm is registered by the motherboard thermistors, the faster is  the speed of the ceiling fan, that provides a good balance between cooling and noise.

The sensors display constant temperature of 35°C for the server motherboard and 50°C for the CPU (with the roll door closed), that’s ok, I think.

Networking

There is D-Link DIR-300 router that serves both as an Ethernet cable switch for stationary devices and as a WiFi hotspot for mobile stuff across the house, like the laptops and the phones. A special hole was drilled in the closet wall to put the WiFi antenna outside for better coverage.

Twisted pair Ethernet cables were laid inside the walls during house repairs last summer. The cables are ended with the wall outlets in convenient places (with some spare endpoints).

The server connects to Internet via an EV-DO broadband modem with external antenna on the house roof. The client devices use the server machine as an Internet gateway. To provide the gateway functions, the iptables firewall with NAT, caching DNS proxy (pdnsd) and transparent Squid web-proxy are configured on the server.

File server / NAS

The most part of the server disk space is shared over the network.  As there are Linux and Windows clients, the server provides both Samba protocol and NFS to access the shares. Also, there is rsyncd daemon running to provide incremental remote backup service for the client systems.

Media server

The server runs MediaTomb UPnP MediaServer software to host our movie and music collections. It primarily serves a home theatre setup in the living room, centered around  a UPnP-compliant Blue-ray player.

Developer’s stuff

I am a full-time software developer, working from home most of the time. Almost all my projects are hosted on remote repositories, but there are some internal and experimental things living on local Subversion server. Also I found that running a local Trac does a great help for scheduling and task management.

And, there is Tomcat I need for  testing of the web-apps.

Control and monitoring

To access the server  from the local network, plain old Telnet works great. To monitor the server status, there is Munin software that displays nice real-time diagrams of server load, temperature, memory and disk usage, network traffic and other useful information via the web-interface.

Problems

• It’s a real pain if the headless machine goes unbootable due to misconfiguration or by another reason. I got this recently when attempted connection of an external HDD brought to major maintenance: I had to get the box from the closet, open the case and mount a videocard and a CDROM just to comment a single line in fstab.
• Free disk space is vanishing with menacing speed (mostly because of Full-HD movie rips) and the old motherboard has only two SATA slots. Perhaps, I would need to think about a SATA hub and RAID at the nearest future.
• I’m worrying how my cooling solution would work at summer season.

Pimp my Nepomuk

Being both a really old KDE user and a semantic desktop partisan at the same time, I am, of course, keeping my eye on the progress in Nepomuk project.  It was apparently close to my old dream of a tagging framework supported natively and consistently across the whole desktop environment, so I highly appreciated this effort and it was nice to hear that Nepomuk would be officially included into KDE – my desktop of choice for many years.

Unfortunately, the experience was rather disappointing. It’s turned out to be painfully slow, not only slow by itself, but being a brake for overall desktop navigation. Even hovering the cursor over files and folders in Dolphin made Nepomuk process to eat above 50% of CPU time and caused annoying delays. The simple operations like assigning a tag to a file took seconds, the responsiveness which is obviously inappropriate for a real-world desktop system. It thus was turned off in a hope that the things would be improved in future versions (I was confused a bit by how it might appear in the production release, but, frankly, early KDE 4 was full of much more disastrous things). Since then, I checked it after every KDE version upgrade, but there was no visible progress in performance, alas.

It was really good news – the author of the post above argues that the performance issues are in fact, caused by a storage backend which Soprano, an RDF framework underlying to Nepomuk, uses to keep RDF data. By default, it’s shipped with Redland (aka librdf), an RDF database library written in C. Luckily, the backend is easily replaceable and it’s worth to try to install a faster alternative seeking for a better performance. The author recommends Sesame2 – a 100% pure Java RDF framework which works (surprisingly for many, I think – but not for me!) much faster than it’s native code counterpart.

I tested Nepomuk with Sesame and convinced that now it works really faster – as it should, in fact. There is of course, a room for improvements in Nepomuk to be a real end-user tool – e.g. a tag navigation interface without which the tags are rather useless, but its another story. At least, the performance is not a blocker anymore, so Nepomuk now is enabled in my KDE all the time.

For Kubuntu users: How-to

I tested the Sesame2 backend for Nepomuk on Kubuntu 9.04 Jaunty Jackalope, KDE 4.3.0 and Sun JRE 6 (I have no idea if it works with GNU Java, but you can give it a try).

1. Install soprano-backend-sesame package (sudo apt-get install soprano-backend-sesame)
2. Make a symlink from $JAVA_HOME/jre/lib/i386/client/libjvm.so in the /usr/lib directory
3. Restart Nepomuk server

Check if Sesame2 backend is used now:

qdbus org.kde.NepomukStorage /nepomukstorage usedSopranoBackend

It should answer “sesame2“. If it still answers “redland“, something was wrong. You may need also to replace the value “redland” to “sesame2” in ~/.kde/share/config/nepomukserverrc file manually and restart Nepomuk again.

As I skipped 7.10 “Gutsy” release, I preferred to do a fresh install from Hardy Heron LiveCD instead of two-step “Feisty » Gutsy » Hardy” network upgrade. I installed Hardy right into the existing Feisty partition replacing the old installation. Separate home partition was left untouched to keep all my personal preferences and data in new system.

It took ~25 minutes from booting the installation CD to logging into a working system.

Unsurprisingly good

No doubts, Kubuntu developers and packagers spent this year not for nothing. In general, Hardy is a better system than previous Kubuntu versions. It loads essentially faster and many little annoyances was fixed. Good job!

Hardware compatibility

Ubuntu systems are known to have great hardware support. As far back as 7.04 version, it recognized my hardware configuration including such exotic devices like DVB-card and CDMA modem without any problem. The only trouble was my HP LaserJet 1020 printer for which I had to install and configure foo2zjs package manually. So, it was nice to see it’s fixed in Hardy; now my printer is supported via native HP open source drivers and working out the box.

A bad surprise was that Hardy could not detect my monitor automatically and set failsafe 640×480 resolution as a result. It was strange as it was no problem with it in Feisty. Even when I’ve clicked “Detect monitor” button manually, it detected it as default “plug-n-play” monitor. I had to remember a model of my monitor and select it from the list. Not a big deal, but a pity though.

KDE 4

Looking great and nice but apparently, a lot of work is still needed there. Crashes are not unusual everywhere and the single Plasma panel looks more like a prototype than a part of a desktop for the real world. It’s half-baked, almost not configurable and I cannot setup a desktop I need for my tasks (IMHO, an essential fault for any Linux software).

So, I’m use the “solid rock” KDE 3.5.9 as a primary desktop environment for work and sometimes switch to KDE 4 just to get a feeling of  bleeding edge desktop technologies.

Grumbles

• Dolphin – I already grumbled at it and I still have no idea what are the reasons to have yet another desktop file manager. Konqueror was a default KDE file manager for years and a whole generation of Linux users grew up with an idea of this software inspired by Konq. Dolphin’s user experience is different in many ways and totally unusual for me (and I think, I’m not alone). No matter if Dolphin is good or bad, I need Konqueror, thanks!
• Samba didn’t see my home network after installation. And I was unsuccesfull to fix it using configuration GUI in KDE Control Center (though I’m not a sysadmin guru and might miss some options). Finally I managed to configure the network by manual editing ‘smb.conf’ taken file from a backup of previous installation. Maybe it sounds trivial for experienced network administrators, however I remember that I didn’t edit any configs to get Samba working in Kubuntu 7.04.

How it works

It’s simple and easy. You press and release a special key called “Compose” (or, “Multi_key”) and then, two or three character keys – a mnemonic ASCII sequence identifies a Unicode character you want to see. For example, the sequence of letter keys “o” and “c” produces the copyright symbol (©) and it is much easier to remember than Unicode 00A9.

Where is my Compose key?

There is no Compose key defined in X.org by default. Before using it, you need to do some configuration to tell X which key you want to use as a Compose. Usually, right “Win” (“Super”) key is a good choice.

In KDE, open “Keyboard layout” configuration panel (Control Center → Regional & Accessibility → Keyboard Layout) and select “Enable xkb options” checkbox on “Xkb options” tab. Then scroll the options list down to “Compose key position” section and select a checkbox for a key you want to use as a Compose.

Alternatively, you can edit /etc/X11/Xorg.conf (as a root) to add the line to the keyboard section:

Option "XkbOptions"  "compose:rwin"

After configuration is done, restart X (simply hit Ctrl+Alt+Backspace).

Note that you’ll need Unicode fonts installed (to see all exotic characters) and a default locale with UTF-8 support (e.g. en_US.UTF-8 for US English). This seems to be the default in modern Linux distributions.

Predefined characters

Diacritics, ligatures and currency symbols

Entering extended latin letters with diacritic marks follows the pattern:

Compose diacritic_character letter

The diacritic characters are:

• ' (apostrophe) ⇒ A letter with acute (Áá)
• " (double quote) ⇒ A letter with diaeresis (Ää)
• ` (grave) ⇒ A letter with grave (Àà)
• _ (underscore) ⇒ A letter with macron (Āā)
• ^ (circumflex) ⇒ A letter with circumflex (Ââ)
• ~ (tilde) ⇒ A letter with tilde (Ãã)
• , (comma) ⇒ A letter with cedilla (Çç)
• . (period) ⇒ A letter with dot above (Ȧȧ)
• ! (exclamation) ⇒ A letter with dot below (Ạạ)
• ? (question) ⇒ A letter with hook (Ảả)
• / (slash), - (minus) ⇒ A letter with stroke (Øø, Đđ)
• ; (semicolon) ⇒ A letter with ogonek (Ąą)
• + (plus) ⇒ A letter with horn (Ơơ)
• = (equals) ⇒ A letter with double acute (Őő)
• o ⇒ A letter with ring (Åå)
• c ⇒ A letter with caron (Ǎǎ)
• b ⇒ A letter with breve (Ăă)

Some characters may be combined to create mixed diacritical marks, e.g. Compose _ " a produces ǟ (‘a’ with diaeresis and macron), and so on.

Other letters and ligatures:

• Compose A/a E/e ⇒ Æ/æ
• Compose O/o E/e ⇒ Œ/œ
• Compose T/t H/h ⇒ Þ/þ
• Compose N/n G/g ⇒ Ŋ/ŋ
• Compose s s ⇒ ß
• Compose e e ⇒ ə

Currency symbols:

• Compose c / ⇒ ¢
• Compose C = ⇒ €
• Compose L = ⇒ ₤
• Compose F r ⇒ ₣
• Compose Y = ⇒ ¥
• Compose o x ⇒ ¤

Punctuation marks and other symbols

Quotation marks:

• Compose , ' ⇒ ‚
• Compose , " ⇒ „
• Compose < ' ⇒ ‘
• Compose > ' ⇒ ’
• Compose < " ⇒ “
• Compose > " ⇒ ”
• Compose < < ⇒ «
• Compose > > ⇒ »
• Compose . < ⇒ ‹
• Compose . > ⇒ ›

Others:

• Compose . . ⇒ ·
• Compose - - - ⇒ — (em-dash)
• Compose - - . ⇒ – (en-dash)
• Compose ? ? ⇒ ¿
• Compose ! ! ⇒ ¡
• Compose + - ⇒ ±
• Compose : - ⇒ ÷
• Compose 1 2 ⇒ ½
• Compose 1 4 ⇒ ¼
• Compose 3 4 ⇒ ¾
• Compose _ 0-9 ⇒ ₀ – ₉ (subscript digit)
• Compose ^ 0-9 ⇒ ⁰ – ⁹ (superscript digit)
• Compose ( 0-9 ) ⇒ ⓪ – ⑨ (circled digit)
• Compose o c ⇒ ©
• Compose o r ⇒ ®
• Compose o o ⇒ °
• Compose o s ⇒ §
• Compose x x ⇒ ×
• Compose P P ⇒ ¶
• Compose T M ⇒ ™
• Compose m u ⇒ µ
• Compose % o ⇒ ‰

Defining custom Compose sequences

Default Compose sequences are defined in ‘Compose‘ text file in the current locale directory (/usr/share/X11/locale/xxx). This file contains the rules to define Compose keyboard sequences and corresponding Unicode characters:

# UTF-8 (Unicode) compose sequence
# David.Monniaux (at) ens.fr
#
# $XFree86: xc/nls/Compose/en_US.UTF-8,v 1.11 2004/01/06 13:14:04 pascal Exp $
...
<Multi_key> <less> <less>        	: "«"   guillemotleft # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
<Multi_key> <greater> <greater>  	: "»"   guillemotright # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
<Multi_key> <less> <apostrophe>  	: "‘"   U2018 # LEFT SINGLE QUOTATION MARK
...

To define new Compose sequence, we should add another rule using the syntax:

<Multi_key> <key1> [<key2> ...]   	: "character"

For instance, for a smiley character (☺, Unicode 263A), the rule could be

<Multi_key> <colon> <parenright> : "☺"  # Compose : )

There is a number of utilities (like KCharSelect in KDE) for to select the Unicode characters and insert them into the file via clipboard.

It is possible to add new rules just into this system file (if you are root), but it’s better to create ‘.XCompose‘ file in the user home directory:

# ~/.XCompose
# This file defines custom Compose sequence for Unicode characters 

# Import default rules from the system Compose file:
include "/usr/share/X11/locale/en_US.UTF-8/Compose"

<Multi_key> <colon> <parenright> : "☺" U263A   # Compose : )
<Multi_key> <minus> <less>   : "←"  U2190 # Compose - <
<Multi_key> <minus> <greater> : "→" U2192 # Compose ->
...

You also can assign whole strings to the Compose sequences — for instance, syntax constructions of your favorite programming language or HTML tags:

<Multi_key> <less> <p> : "<p></p>" # Compose < p
<Multi_key> <less> <a> : "<a href=""></a>" # Compose < a
...

I should say I hate disks. They are taking a lot of space, cluttering all around, getting scratched and getting lost sometimes. I am too lazy to stand up and find a CD/DVD on a shelf just to get a movie or a song. I already got all my music collection in the computer as MP3 files and used to grab every new audio CD immediately. I’d like to do the same with DVD movies – rip them from the disks, convert them to something like MPEG4 and provide shared access to the media collection in our home network.

Our home LAN is small and simple – there are only two workstations connected by an Ethernet cable. There are my main work machine running Kubuntu Feisty Fawn and the children room computer owned by my daughters. The last box runs XP Home Edition, it is weaker and got a smaller harddrive than the Linux box – so server/client roles in the media network were obvious.

As the Linux box already runs Samba, the solution for sharing the media files over the network was ready. I only created a Windows network drive on the client machine and linked it to the Samba share on a server filesystem. I’d like to have more sophisticated solution for media organizing – something like a specialized media server with advanced metadata/annotation/categorization features but found no one so far. All in all, simply movie titles and preview thumbnails are good enough, and my children have no problems to navigate over the movie collection.

Well, my only task was to rip the movies from DVD’s. My first try of ripping DVD using the familiar tools was unsuccessful. Strangely, K3b – a swissknife for all CD/DVD tasks in KDE – could not rip video DVD’s in Feisty. As it was turned out, it is Kubuntu-only limitation and I have no idea is it a result of that copyrights paranoia (though grabbing the audio CD’s works fine), or just a miss of packagers in Canonical. The people said it is still unworkable in 7.10 Gutsy. Of course, I might recompile K3b from sources but I decided to look for another ripping tool first.

There is a nifty command line utility: transcode. It can convert video from one format to another using lots of import and export codecs – and directly from a DVD too. In fact, you need to install it if you want to rip DVD’s with K3b, though it had no help in my case. Using transcode has only problem – you should be a video engineering guru to understand its manpage with lots of all transcoding options. I didn’t want to learn all that stuff – I only needed to convert videos from DVD into AVI/MPEG4 files with the same frame size and without visible loss of quality.

I finally chose dvd::rip – a GUI frontend to transcode written in Perl and GTK+. In Kubuntu, you can found it in multiverse/graphics repository section. If you want to get compressed MPEG-4 files, you will also need XVid codec (multiverse/libs/libxvidcore4) or another DivX/XVid library for transcode.

Ripping DVD with dvd:rip is an easy two-step process: at first stage it copies the selected titles from DVD to the harddisk and lets you clip and scale the resulting movie. The program allows to choose from few presets of movie format and quality with previewing the results. Video gurus can also get into all codec’s fine-tuning options, while others can just go to the next stage by clicking on “Transcode” button letting the program to do its job with defaults. The second stage is quite long and finishes with a MPEG4 file in a container of a selected type (AVI, OGG or MPEG).

http://shipit.kubuntu.org

Thanks, Ubuntu!

I told I was not going to upgrade to Gutsy so far. Now I’m thinking that perhaps, I will need to do so because an important thing is going to happen soon – KDE 4 has been announced to be finally released in January, 2008. I am not sure it will be available through the Feisty repo’s.

Regarding the Gutsy received: It’s nice to see that you’ve got what you’ve ordered. Before, I’ve received Ubuntu (without “K”) disks instead of Kubuntu (with “K”) I needed. Also, it was delivered really quickly – about two weeks (yeah, quickly for my geographical location, of course). I was waiting for Feisty a month or so.

But hmm… they forgot the stickers! My Kubuntu (with “K”) box is still decorated with Ubuntu (without “K”) sticker :-)

After spending some time on googling and reading the comments and opinions of those who have answered “yes”, I finally decided not to upgrade. Well, not now.

Don’t get me wrong – I am not devoid of neophilia attacks and not afraid of experimenting with my environment and tools. I even love it. But I want to be sure that the results would improve my life’s quality or, at least, the things are exciting new and interesting to try.

The sum of innovations in Kubuntu 7.10 doesn’t worth a time, a bandwith, and (as it turns out) a degree of a risk to break working Feisty installation. I am pretty happy with it – perhaps it is the best Linux I ever had – and I am definitely unwilling to change the things without essential reasons.

The reasons they tell why do I need an upgrade look strange. Dolphin as a default file manager? Sorry, but I don’t need a file manager – I need also a local documentation browser and a client for SSH and for FTP and for Samba and WebDAV and SVN and for other stuff and it should have the tabs and a console at the bottom and… In short, I need Konqueror. Replacing it with a file manager (even if it is a really good one) – it’s nothing but downgrade.

Strigi search? Everyone says it is still an alpha so putting it into production release looks weird. KDE old file search worked fine and it is stable and well tested. And (excuse me for this product-placement) I use SCAN which does a lot more except basic full-text search.

OpenOffice 2.3? But to install it I don’t need to upgrade a whole system.

No, thanks. What might drive me to upgrade is KDE 4 Final, for instance. Or Beryl/Compiz, gettin’ fuckin’ stable, integrated seamlessly and not conflicting with basic KDE stuff. Or maybe, NEPOMUK or similar bleeding edge technologies released into production.

I am far from blaming Kubuntu maintainers and pretty well understand them. This is what happens sometimes when you have to provide a fixed release cycle but fully depend on other’s work. The fact is that the Linux world had no visible technology breakthrough in the last six months. Alas.

So let’s wait for the April, 2008.

We assuming Memoranda is already installed on your Linux system (that is, ZIP archive is unpacked) into, say, “/opt/memoranda” directory. To be sure that the executables have proper permissions, run:

chmod 755 /opt/memoranda/memoranda.sh
chmod 755 /opt/memoranda/lib/kde/systray4jd

Creating a desktop shortcut

1. Right-click anywhere on your desktop, select “Create New->Link to application…” and enter “Memoranda”.
2. Go to “Application” tab and enter into the fields
   Command: /opt/memoranda/memoranda.sh
   Work path: /opt/memoranda
3. Go back to “General” tab and click the icon. In the icon dialog box select “Other icons”, “Browse” and point it to ‘/opt/memoranda/lib/icons/memoranda48x48.png’
4. Press “Ok”

To put new shortcut into KDE panel also, simply drag-n-drop it from the desktop.

Creating a menu item

1. Right click the desktop shortcut created before and select “Copy”
2. Open Konqueror and navigate to “/home/you/.kde/share/applnk” directory. Tip: if Konqueror does not display “.kde” directory, select “Show hidden files” in its “View” menu.
3. Create new subdirectory (say, “Utilites”, “Office” or “PIM”) or go into an existing one. These subdirectories are the sections of your K menu. Right click anywhere in this directory and select “Paste file”.

Done.

http://shipit.ubuntu.com/

Thanks, Ubuntu!

[Upd]
Actually, I’ve requested Kubuntu disks :-) but, as I downloaded and installed it already, it doesn’t matter what to have as a souvenir… Thanks anyway!