The Fundamentals of OSTree

I’ve had some time at work to get my head around the OSTree tool. I summarised the tool in a previous post. This post is an example of its basic functionality (as of January 2014).

The Repository

OSTree repos can be in one of multiple modes. It seems ‘bare’ is for use on deployed systems (as the actual deployment uses hardlinks so disk space usage is kept to a minimum), while ‘archive-z2’ is for server use. The default is ‘bare’.

Every command requires a –repo=… argument, so it makes sense to create an alias to save typing.

$ cd ostree-demo-1
$ mkdir repo
$ alias ost="ostree --repo=`pwd`/repo"

$ ost init
$ ls repo
config  objects  refs  remote-cache  tmp
$ cat repo/config
[core]
repo_version=1
mode=bare

Committing A Filesystem Tree

Create some test data and commit the tree to ‘repo’. Note that none of the commit SHA256s here will match the ones you see when running these commands, unless you are in some kind of time vortex.

$ mkdir tree
$ cd tree
$ echo "x" > 1
$ echo "y" > 2
$ mkdir dir
$ cp /usr/share/dict/words words

$ ost commit --branch=my-branch \
    --subject="Initial commit" \
    --body="This is the first commit."
ce19c41036cc45e49b0cecf6b157523c2105c4de1ce30101def1f759daafcc3e

$ ost ls my-branch
d00755 1002 1002      0 /
-00644 1002 1002      2 /1
-00644 1002 1002      2 /2
-00644 1002 1002 4953680 /words
d00755 1002 1002      0 /dir

Let’s make a second commit.

$ tac words > words.tmp && mv words.tmp words

$ ost commit --branch=my-branch \
    --subject="Reverse 'words'"
    --body="This is the second commit."
67e382b11d213a402a5313e61cbc69dfd5ab93cb07fbb8b71c2e84f79fa5d7dc

$ ost log my-branch
commit 67e382b11d213a402a5313e61cbc69dfd5ab93cb07fbb8b71c2e84f79fa5d7dc
Date:  2014-01-14 12:27:05 +0000

    Reverse 'words'

    This is the second commit.

commit ce19c41036cc45e49b0cecf6b157523c2105c4de1ce30101def1f759daafcc3e
Date:  2014-01-14 12:24:19 +0000

    Initial commit

    This is the first commit.

Now you can see two versions of ‘words’:

$ ost cat my-branch words | head -n 3
ZZZ
zZt
Zz
error: Error writing to file descriptor: Broken pipe

$ ost cat my-branch^ words | head -n 3
1080
10-point
10th
error: Error writing to file descriptor: Broken pipe

OSTree lacks most of the ref convenience tools that you might be used to from Git. For example, where providing a SHA256 you cannot abbreviate it, you must give the full 40 character string. There is an ostree rev-parse command which takes a branch name and returns the commit that branch ref currently points too, and you may use ^ to refer to the parent of a commit as seen above.

Checking Out Different Versions of the Tree

The last command we’ll look at is ostree checkout. This command expects you to be checking out the tree into an empty directory, and if you give it only a branch or commit name it will create a new directory with the same name as that branch or commit:

$ ost checkout my-branch
$ ls my-branch/
1  2  dir  words
$ head -n 3 my-branch/words
ZZZ
zZt
Zz

By default, OSTree will refuse to check out into a directory that already exists:

$ ost checkout my-branch^ my-branch
error: File exists

However, you can also use ‘union mode’, which will overwrite the contents of the directory (keeping any files which already exist that the tree being checked out doesn’t touch, but overwriting any existing files that are also present in the tree):

$ ost checkout --union my-branch^ my-branch
$ ls my-branch/
1  2  dir  words
$ head -n 3 my-branch/words 
1080
10-point
10th

One situation where this is useful is if you are constructing a buildroot that is a combination of several other trees. This is how the Gnome Continuous build system works, as I understand it.

Disk Space

In a bare repository OSTree stores each file that changes in a tree as a new copy of that file, so our two copies of ‘words’ make the repo 9.6MB in total:

$ du -sh my-branch/
4.8M    my-branch/
$ du -sh repo/
9.6M    repo/

It’s clear that repositories will get pretty big when there is a full operating system tree in there and there is no functionality right now that allows you to expire commits that are older than a certain threshold. So if you are looking for somewhere to start hacking …

In Use

OSTree contains a higher layer of commands which operate on an entire sysroot rather than just a repository. These commands are documented in the manual, and a good way to try them out is to download the latest Gnome Continuous virtual machine images.