End of April I spent a week in Marrakech to work on MirageOS at our annual retreat. This was the 12th time when it actually took place, and it was amazing. We were a total of 17 people attending, of which 5 people used ground transportation. I have respect for them, to take the 3 or more day trip from Berlin or Glasgow or Paris or whereever to Marrakech - with a ferry in between (which availability depends on the wind etc.). This time, I didn't take ground transportation since I had appointments just before and just after the retreat nearby Berlin where I had to be in person. The food an weather was really nice and motivating.

Of the 17 people there were lots of new faces. It is really nice to see an active community which is welcoming to new people, and also that there's change: a lot of fresh ideas are brought by these people and we rethink what we are doing and why.

Our daily routine included breakfast, circle, lunch, dinner, (sometimes) presentations. There wasn't more formal structure, and there was no need for it since people kept themselves busy working on various projects. We had a bigger discussion about "deploying MirageOS", which is a reoccurring theme. Even running a MirageOS unikernel on a laptop is for people who are not very familiar with networking pretty hard. In addition, our website only briefly described network setup, and included DHCP very early (which is overwhelming to setup and use on your laptop).

I went to the retreat with some ideas in my head what I want to achieve (for example, replace all the bigarray allocations by bytes & string), but the discussion about deployment made me prioritise this more.

Operator handbook

I sat down with some people to start off a MirageOS operator handbook - we talked about the scope and the early chapters (and what is not in scope). This is still very much work in progress, issues and pull requests are welcome. The target audience is someone who wants to execute a MirageOS unikernel with hvt on a laptop or server. It will not be necessary to install OCaml, but the focus is really on the operational aspects with binaries that we provide from our reproducible build infrastructure.

In contrast to earlier attempts, this time we started with the focus and the pain points that we encountered at the retreat. We didn't plan all the chapters upfront or tried to start by developing and compiling a MirageOS "hello world". I have confidence that our approach is a great start for a handbook that has been missing in the last decade.

Mollymawk

Also related to the deployment discussion, I re-started work from last year's retreat on a Web UI for Albatross. Welcome mollymawk. The idea is to have both a UI but also a HTTP API for deploying MirageOS unikernels via e.g. GitHub actions by a simple HTTP POST request. Observation and management of unikernels is also part of mollymawk. Pixie worked a lot on the JavaScript and web side of it, while I put some effort into the MirageOS and Albatross interaction. Within days we got an initial prototype up and running and could show it to others. This already is able to gather information of running unikernels, the console output, and create and destroy unikernels. We will continue on mollymawk in the future.

From the design point, mollymawk communicates via TLS to albatross. It contains the TLS certificate and private key to access albatross (and generates temporary certificates for all the actions required). This also means that mollymawk needs to protect its secrets, otherwise anyone can mess around with albatross. Mollymawk itself is a unikernel that runs on top of albatross. What could possibly go wrong? We plan to add authentication soon to mollymawk, and can then use it for our own purposes. This will greatly improve the state of our running unikernels, since updating our unikernels from our reproducible build infrastructure will then be a click on a button, with a revert button if there are issues.

Defunctorising MirageOS

Other discussions were about porting existing OCaml code to MirageOS, and what are the hurdles. The excessive use of functors are usually cumbersome for existing projects that need to revise their internal code structure. I have some issues with the use of functors as well, especially at places where they don't seem to be necessary. A rule of thumb I learned is that a functor is great if you want to have something in the same application instantiated with different modules -- i.e. a set of string and a set of integers. Now, in MirageOS we use even for time (providing the function sleep), clocks (monotonic and posix time), random (random data), network interface functors. There's some history about it: mirage before 4.0 used opam packages for "cross-compilation". Since mirage 4, a monorepo is used to compile the unikernel. This allows to reconsider the options we have.

Talking about time - I don't think anyone wants to have two different sleep functions in their application, similar for monotonic clock and posix time. Also the existing open source unikernels only use a single time and clock. The random interface has as well some history connected, since our cryptographic library used to bundle everything in a single opam package, and required arbitrary precision integers (via zarith using gmp), and we didn't want to impose a dependency on gmp for every unikernel. Nowadays, the CSPRNG code is separated from gmp, and we can just use it. The network interface: indeed if you're using solo5 (hvt or virtio or whatnot), you will never need a unix (tap-based) network implementation, neither xen netfront/netback. Thus, the functorisation is not necessary.

At the retreat we discussed and thought about replacing the functors in MirageOS with other mechanisms - conditional compilation (depending on the target) or the linking trick (dune variants) or a ppx are candidates, apart from other things (maybe first-class modules fit the requirement as well).

Another benefit of conditional compilation and dune variants is editor support -- jumping to the definition of time (clock, etc.) now simply works because the compiler knows which code is used at runtime (in contrast to when a functor is used).

I got into coding a bit, and first looked into conditional compilation with the select clauses from dune. They turned out to be not very flexible (you've to decide in the package what are all the options, and which one to take). I switched to dune variants, where the idea is that you provide a virtual interface, and only when compiling the executable you specify the implementation. This is great, since the mirage tool (at mirage configure time) can emit the dependencies. I went ahead and defunctorised the whole network stack - it was nice to see it is possible, but this is not easily upstreamable. There's as well at least one issue in the dune variant code, I'm sure it will get debugged and fixed once we decide that dune variants are the solution we want to try out.

The current result is a mail to the development list, and a defunctorising time PR, as well as a bootvar PR (relying on other PRs).

Conclusion

I found the retreat very inspiring, there were lots of interest in various topics, and a lot of projects have been worked on. I'm also very happy that I managed to contribute and start some projects that will hopefully ease MirageOS adaption -- both on the deployment side, but as well on the developer experience side.

I'm looking forward to other writeups and future retreats, or other events where I can say hi to attendees.

We at Robur are working as a collective since 2018 on public funding, commercial contracts, and donations. Our mission is to get sustainable, robust, and secure MirageOS unikernels developed and deployed. Running your own digital communication infrastructure should be easy, including trustworthy binaries and smooth upgrades. You can help us continue our work by donating (select Robur from the drop-down or put "donation Robur" in the purpose of the bank transfer).

If you have any questions, reach us best via eMail to team AT robur DOT coop.