I have inboxes across several hosts stored in djb's Maildir format.

One of the things I would like to be able to do is merge/distribute all of those Maildirs to other hosts.

Discounting how much simpler this would be to with rsync, I decided to try monotone for this.

This perl script issues the appropriate monotone add/drop/rename commands, for new/deleted and modified messages.

It does this by keeping track of the inodes and filenames in the Maildirs, and calculating the difference between the last current and last state of the Maildir.

This script is run from cron. I'm currently using it to sync a few of my inboxes of about 40k messages (~300MiB), and about 1000 add/drop/rename operations a day. The script typically takes about a minute to run, and a netsync takes about 2 minutes.

There are a few annoyances:

• The inital sync of the maildir branch takes a long time
• Each update requires a merge, which can take a few minutes

I'm happy with it. It means that I don't have to worry as much about my mail going away when ice is on the wrong side of a network schism.

#!/usr/bin/env perl
use warnings;
use strict;

use DB_File;
use File::Find;
use File::Basename;
use Data::Dumper;

my $MAIL_BASE="$ENV{'HOME'}/Mail";
my $INODE_CACHE="${MAIL_BASE}/inode_cache.db";
my @FOLDERS=('inbox', 'sent');

my (@add,@drop,@rename);
my (%new_inodes,%done);

tie my %old_inodes, "DB_File", $INODE_CACHE, O_CREAT|O_RDWR, 0644, $DB_HASH
        or die "Cannot open file $INODE_CACHE: $!\n";

chdir $MAIL_BASE;

print "scanning inodes...\n";

foreach my $folder (@FOLDERS) {
        print "${MAIL_BASE}/${folder}\n";
        File::Find::find({no_chdir => 1, wanted => \&wanted}, "${folder}");
}

my @keys  = keys %old_inodes;
push @keys, keys %new_inodes;

print "finding differences...\n";

foreach my $k (sort(@keys)) {
        next if defined $done{$k} ;

        # In old, but not in new -> drop
        if (not defined $new_inodes{$k}) {
                push @drop, $old_inodes{$k};
                $done{$k}=1;
                next;
        }

        # In new, but not in old -> add
        if (not defined $old_inodes{$k}) {
                push @add, $new_inodes{$k};
                $done{$k}=1;
                next;
        }

        my $nfile=$new_inodes{$k};
        my $ofile=$old_inodes{$k};

        # inodes same, and files are the same -> do nothing
        if ($nfile eq $ofile) {
                $done{$k}=1;
                next;
        }

        $nfile =~ s/:.*//;
        $ofile =~ s/:.*//;
                # strip off the trailing garbage. 

        # inodes same, and the msg basename is the same -> move
        if ( basename($nfile) eq basename($ofile) ) {
                push @rename, [ $old_inodes{$k}, $new_inodes{$k} ] ;
                $done{$k}=1;
        } else {
        # inodes same, and the msg basename different. Reused inode. Drop old,
        # add new
                push @drop, $old_inodes{$k};
                push @add, $new_inodes{$k};
                $done{$k}=1;
        }
}

#XXX: Bleech. There's gotta be a better way to chunk the arrays through to monotone. 

my $commit=0;
my $work=0;

print "marking differences...\n";

while ( $#add > $work ) {
        my @cmd=('monotone', 'add');
        my $chunk=(($#add - $work) > 500) ? 500 : $#add-$work;
        push @cmd, @add[$work..$work+$chunk];

        system(@cmd) == 0 or die "system @cmd failed: $?";
        $work+=$chunk;
        $commit=1;
}
$work=0;
while ( $#drop > $work ) {
        my @cmd=('monotone', 'drop');
        my $chunk=(($#drop - $work) > 500) ? 500 : $#drop-$work;
        push @cmd, @drop[$work..$work+$chunk];

        system(@cmd) == 0 or die "system @cmd failed: $?";
        $work+=$chunk;
        $commit=1;
}

# have to iterate through the renames. 
foreach my $e (@rename) {
        my @cmd=('monotone', 'rename');
        push @cmd, @{$e};
        system(@cmd) == 0 or die "system @cmd failed: $?";
        $commit=1;
}

if ( $commit != 0 ) {
        my @cmd=('monotone', 'commit', '-m', 'automatic commit');
        system(@cmd) == 0 or die "system @cmd failed: $?";
}

# XXX: Bleech, There's probably a better way of doing this. 

print "updatin inode cache...\n";

foreach my $k (keys %old_inodes) {
        delete $old_inodes{$k};
}

foreach my $k (keys %new_inodes) {
        $old_inodes{$k} = $new_inodes{$k};
}

untie %old_inodes;

##############################

sub wanted {
        my $f=$_;
        ! -f $f && return ;
        my ($dev,$ino) = lstat($f);


        $new_inodes{$ino}=$File::Find::name;
}