The Woodshed

Behind here, no one can hear you scream

If You Wish Upon A Cloud...

Keeping Those Config Files Synced

A compadre of mine recently asked how I go about keeping my config files synced. If you are someone who uses customizable programs (IDE/text editors, OS shells, version control software, and the like), and those programs allow storing your settings in some sort of standalone file (whether plain text, XML, binary, or what have you), than this is an issue worth looking at. Reason being, if a program offers customization, and you customize it, you expect it to act the same no matter which computer you're sitting at (or remoted into). And even if the program has a convenient interface for said customizations, you don't really want to spend the time redoing what you've already done.

In fact, I know some (very bright) people who are daunted enough by the task of customization and its maintenance that they prefer to work with programs in their vanilla state, knowing full well that it is suboptimal to do so. (Admittedly these people don't use something like Emacs, which is basically useless without customization.)

What I Use

I will outline my personal scheme for synchronization in this section; alternative schemes will follow.

My method involves a simple directory structure for organizing your config files, the programs Git and Dropbox for grabbing your personalized versions of said files, and some quick setup. The aforementioned tools are free, and clients/front-ends for both are available for all the major OSes. But see the Alternatives section for scenarios when using them might not be straightforward or practical. On with the show...

On your main machine (one-time setup):

  1. Create a folder somewhere, and name it something straightforward (I use "config_files").
  2. Inside that, I like to make a folder for each OS I use, as well as a folder that will contain files common to all platforms. This is purely for organization; you can throw everything in one folder if you want.
  3. Copy all the configuration/customization files/directories that you want to synchronize into this new location.
  4. Assuming you've already installed Dropbox, go to your local Dropbox folder, and within it create another directory. You might want to end its name with ".git", as in config_files.git.
  5. Enter said directory, and use your preferred Git client to init an empty repository there.
  6. Now head back to your config_files directory (where you copied all your stuff). Ensuring that you're in the "top-level" folder of the directories you created, use your Git client again to init another fresh repository.
  7. Still in Git, add and commit all your copied configs. (If you're using the command line, git add . && git commit -m Initial works fine for this. If you're on Windows and have never used Git, a simple guide like this one will get you up and running.)
  8. Finally, use Git to link this repo with the one in your Dropbox. This is called "adding a remote." At the command line: git remote add origin /path/to/your/Dropbox/config_files.git && git push

On your other machines (one-time setup per machine):

  1. Make sure Git and Dropbox are installed (or see Alternatives).
  2. Use Git to clone the repository from your Dropbox into a folder on this machine, e.g.: git clone /my/Dropbox/folder/config_files.git

...And your config files are now on this machine. This seems like a good time for an aside.

Aside: Why Bother With a Repository?

"But Blog Author Guy," I hear you saying, "That Git stuff seemed redundant. Why not just copy the config files into a folder in my Dropbox?"

First of all, the name's Jordan! But yes, I did this myself for about two years. It's fine for maintaining a basic setup on, say, your desktop and your laptop, as long as you're diligent about recopying to and from your Dropbox folder every time you make a change to a file on either machine. But there are some limitations to this approach. For example, you might not be able to install [even the command-line version of] Dropbox on some remote machines. Specifically, the transitory nature of shared hosting screws up Dropbox's authentication scheme. Or you may plain just not want your Dropbox on a computer you don't own.

Using Git opens up the alternative of using a different service to host your config file repository. My personal preference is Bitbucket, which offers free private repositories.

Oh, and of course it never hurts to have the benefits that come with version control anyway—easy to revert a change, and you never have to worry about backups. This is especially true if you tend to change configs on different machines from time to time, and maybe you don't always remember to immediately sync the changes. This can lead to so-called "version conflicts," which programs like Git were designed to handle.

Perhaps most importantly, I've simply found it easier to keep things synced when all I have to do is type git pull. Manually copying files doesn't seem labor-intensive at first glance, but you'd be surprised what a drag it can be.

What I Use, Continued

In fact, I like to eliminate copying files from the procedure altogether. If possible, I like to be free to make modifications to a config file on any machine, and then be able to sync those changes with one line of Git (which I've even made an alias for, heh). Symlinks to the rescue!

Important note for Windows users: Your OS does actually support symlinks! I'll say some more about this in a bit, but tl;dr Don't skip this section.

In any version control software I've used, it's expected that you keep tracked files in the directory structure of the repository. But as you know, programs expect to find their configuration files in various specific places. For example, a common place for one's Emacs init file is ~/.emacs.d/init.el (or in %APPDATA% instead, on Windows). Instead of copying init.el to that location, though, I place a symlink there that points to the file in my repo.

This works great, because the programs don't know the difference, and yet the files are still version controlled and easy to sync.

If you're on Linux or OS X, you probably already know how to symlink a file (just remember the arguments go in the same order as for cp, and be careful with relative paths).

I was surprised to learn that Windows supports symbolic and hard links, since for some stupid reason nobody ever uses them (probably since you can't create them directly in the Explorer GUI; but note you can view and use the links fine from the GUI). The filesystem has had this support since the Win2K days, and then in Vista they added a command-line utility that makes link creation as easy as on *nix. So open up cmd and type:

mklink "C:\path\where\program\looks.whatever" "C:\my\repo\path\file.whatever"

(The only two differences to note from unix ln -s are the reversed argument order, and that if you're linking a directory you should provide the /D switch.)

Alternatives

  • If you don't want to use Dropbox at all, I'd definitely recommend using a (likely private) hosted repository from a service like Bitbucket or GitHub. The process outlined above will work much the same, except the "remote" that you add to your repo (and from which you'll clone on other computers) will be the URL your service provides, instead of a local file path. Note that in this method, every time you sync it will ask you for password confirmation, though (unless you go through the effort to set up public-private key pairs).
  • If you don't want to use any sort of cloud-y service, you can get away with using git archive on your repo, and sending the result to other machines via something like (S)FTP or even email. This will make it a pain to sync any future changes, though. (You can also use this method plus a thumbdrive if you're going to some godforsaken planet that lacks Internet access.)
  • Similarly, if you use Dropbox most of the time, but you can't get it working on a particular machine, you can just use the Dropbox website to get a download link to your repo's folder. (PROTIP: If you want to use wget to fetch a link from Dropbox, append ?dl=1 to said link [which will conveniently skip the dumb confirmation redirect page])
  • You don't have to use Git, of course. In fact, there used to be some advantages to using Mercurial instead (it had better Windows support for a long time, for example). Nowadays they're pretty equivalent from a usability standpoint (same would be true of Darcs or pretty much any other DVCS—you don't need to worry about their differences for a simple usage like this). You could alternatively use Subversion or another centralized server (which obviates Dropbox, of course). If you do, make sure your server will be accessible from the various machines you can expect to be working on (keeping in mind possible blocked ports and firewalls, for example).