maildir-mirror

Introduction
Running it
How it works
Files it uses to keep information
License
Downloading it
Requirements
Installing it
Authors

Introduction

maildir-mirror is a program useful to keep multiple copies of a given maildir directory synchronized, allowing you to make changes to each copy and commit them to a central location. The synchronization is bi-directional, with a simple set of rules for solving conflicts (which occur very rarely). maildir-mirror runs on top of ssh using its encryption, authentication and compression functionality.

I wrote it when I noticed that I would be taking trips with spare internet access. It allows me to get online for a few seconds to sync my maildirs –downloading new mail and uploading local changes (mostly deletions and changes in flags such as New, Replied-to, Important)– and then work offline on my email for hours, repeating the process everyday. It also allows me to work locally on my mail regardless of whether I'm home, in my office or in another country.

Note: This is very experimental yet. Although I've been using it for around one month, you might want to wait for a few more before starting using it or you could lose information and even, like Kenji Urada, get crushed to dead.

Edit Link Top

Running it

There are two ways of running maildir-mirror. The first, useful for marking a specific directory as a mirror of a remote maildir, is:

maildir-mirror --create [USER@]HOST REMOTEPATH LOCALPATH

This command sets the directory LOCALPATH as a mirror of a remote maildir. It does not make any network connections, all it does is store a few files in LOCALPATH with the parameters specified.

USER

and HOST are the parameters that should be

passed to the ssh command when connecting to the remote host to update this maildir. Note that maildir-mirror must be installed in HOST. REMOTEPATH is the path in HOST of the maildir of which LOCALPATH should be made a local copy.

To update one local mirror use:

maildir-mirror [PATH ...]

You can specify as many paths as you want (if you specify none, the current working directory will be used). Each of them must have been passed as LOCALPATH to one invocation with the –create parameter. For each path specified, maildir-mirror will run ssh to connect to the HOST you specified when you created the copy. maildir-mirror is run in HOST and both processes (the one running in HOST, called the server, and the one running in your local machine, called the client) communicate and apply changes to their local maildirs, leaving them in an identical state.

Edit Link Top

How it works

Right after the connection has been established, both the server and the client send the version of the protocol they speak. Currently there is only one version but in the future we might make new versions.

Then the client reads the list of files in the maildir and compares it with the state the last time the mirror was synchronized. Any differences (files that have changed) are sent to the server, along with a version number produced the last time the client copy was updated. The version numbers are required to allow multiple clients to keep copies.

The server receives the changes and also produces a list of all local changes since the last time any client was synchronized. The listings are mixed into one, solving any conflicts that arrise (files that have been changed in both the server and the client). If there are any changes, a new version is produced and the changes registered.

Finally, the server loads all the changes since the client was last updated and sends them along with the new version number. There are two optimizations. First, if two different changes affect the same file, the first can be ignored and does not need to be sent. Second, the server doesn't send back to the client changes that it has already applied. The client receives all the changes and applies them, storing the version number received.

There is an important optimization. Normally, updating multiple maildirs would require establishing one separate ssh connection for each. However, if two or more have the same USER and HOST, only one connection is actually used to update them. As a consequence, the user only needs to authenticate once for each (USER, HOST) pair. In this case the client sends the changes for one maildir, reads and applies the results from the server and repeats the process for all remaining maildirs with the same USER and HOST. After sending all the changes for a given maildir, the server waits for the remote changes for the next.

Edit Link Top

Files it uses to keep information

Chances are you won't need to know this, but in case you are curious, in each remote copy of a maildir we keep the following files:

maildir-mirror-source: Contains two lines, the first being the HOST and the second the REMOTEPATH specified when maildir-mirror was run with the –server option.
maildir-mirror-version: Contains the version number received the last time the client was updated.
maildir-mirror-state: Contains the list of files in the maildir when the client was last updated.

In each copy that acts as a server we keep the following files:

maildir-mirror-version-X, where X is a number: Contains the list of changes that produced version X.
maildir-mirror-state: Contains the list of files when the last version was produced.
maildir-mirror-version: Contains the current version number of the maildir.

Edit Link Top