new desktop: phase 2 - versioned backup

29 Mar 2007 I've been looking at several ways of doing versioned backup these last couple days. Remember, I want to put entire directories under automatic version-control so that all changes to files in those directories are recorded. Then I want to have a copy of this data on my server, which should sync automatically also.

There is WayBack, which describes itself as "User-level Versioning File System for Linux". The nice thing about this, is that it's based on FUSE (userspace filesystem). The sad thing is that it's no longer maintained :(

I couldn't find anything to fit my purpose, so I guess I'll have to build my own, using prebuilt components.

I basically want to have a local versioning system (fully inside my encrypted homedirectory) and sync that to my server from time to time.

There are a couple of versioning systems around. The most famous is no doubt Subversion. Then there is CVS, which is a predecessor of Subversion. CVS works on files instead of directories. There is also SVK, which is built on top of Subversion. It's advantage over Subversion (SVN) is that it doesn't have .svn directories all over the place which pollute filesystems. But it's documentation leaves alot to be desired...
RCS should also be mentioned, because it's great to do versioning on files without having to do complicated networked setups. RCS just works locally and does a very good job.

On top of that, there are also: BZR (bazaar-ng) which claims to be able to track renamed files and directories, Cogito, CVSNT, MCVS, Mercurial, Monotone and PRCS.

Besides the first 4, none of those ring a bell.

At this particular instant, I'm looking at Bazaar-ng and I'm impressed by passages like this:

We believe that the revision control system should not impose 
arbitrary restrictions on your project. In other words, your 
community should be able to treat your tree as entirely malleable, 
reshaping it to improve clarity and structure of the code layout as 
much as the contents of the files that make up the tree.

That's how I like things :) I don't want a system to impose limits on me. If the system can't cope with my odd and insane behaviour, then I need something else.

Also

Bazaar is a free software project with a large community of 
contributors, sponsored by Canonical Limited, the founders of Ubuntu 
and Launchpad.

If Canonical supports it, then I'm sure it will integrate well with Ubuntu.

[A quick test later]
I'm very impressed with bzr. It's usage is so simple, and it allows me to automatically track all new files in a directory so it will be easy to do auto-checkin.

Apparently, I can just rsync the entire working copy to a server, and things will just work. The bzr people advise to use bzr push though, which is fine by me :)

I'll be working on a script to automatically commit my changes. More on this later.

[Some tests later]

My current setup looks like this: on the local machine I have a file called .bzr-dirlist, which contains a list of directories under revision control (like Documents and Projects).

The script then looks like this:

#!/bin/bash

dirlist=$HOME/.bzr-dirlist

cd $HOME

for dir in `cat $dirlist`;
do
        bzr add $dir
done

bzr commit --message "Auto-commit `whoami`@`hostname` on `date`"

This script will examine the new files in the directories and commit them to bzr.

Next, I need to mirror these directories to a remote server.
Just using push isn't enough because that won't "checkout" the files.
There is a plugin that does a push and an update, called "push-and-update"

I can install it by doing:

guest@deepstar-desktop:~/.bazaar$ mkdir plugins
guest@deepstar-desktop:~/.bazaar$ cd plugins/
guest@deepstar-desktop:~/.bazaar/plugins$ 
guest@deepstar-desktop:~/.bazaar/plugins$ bzr branch https://launchpad.net/bzr-push-and-update push_and_update

The plugin even provides hooks for bash_completion, which is a plus for me :)

This command will push and update the files:

guest@deepstar-desktop:~$ bzr push-and-update sftp://guestbackup@localhost/home/guestbackup/something

But before doing that, the directory must be created and initialised on the remote server (with bzr init)

I'll have to use SSH-keys to do all this without entering a password...

[later]
The bazaar-ng developers also suggest putting this line in /etc/apt/sources.list to get the latest bzr packages:

deb http://bazaar-vcs.org/releases/debs/edgy ./