Back in 2015 Docker was awesome on Linux, but the experience on Mac and Windows was a little rough. Rather than shipping a coherent app, Docker shipped a bundle of software components loosely glued together: Virtual Box, docker machine, boot2docker and Kitematic (as "Docker Toolbox") There was a lot to fix. Installations often failed (for various reasons). The networking often failed (interactions with firewall rules and VPNs). Worst of all nothing worked unless you remembered to set a bunch of arcane environment variables in your shell (DOCKER_TLS_VERIFY, DOCKER_HOST etc).
We used our systems background (and unikernel libraries) to create a seamless "Docker for Mac" and "Docker for Windows" experience. Later the same code evolved into a platform for running containers and AI agents in microVMs and is used across the Docker product suite. Here's a list of some of the things I've worked on, newest first. Almost all of it was done with other people, so where I can I've linked to the code, the blog posts and the papers.
Anil Madhavapeddy, Justin Cormack and I wrote up how Docker got from Linux to Mac and Windows, the systems research underneath it, and where things are heading with coding agents. The ACM made a nice video and cover art!
Agents need to run somewhere where access to the outside is carefully controlled i.e. a "sandbox". The unikernel derived VMM tech used in Docker Desktop is now also the isolation layer in Docker Sandboxes, now enhanced with ultra-fast networking and support for running GUIs safely inside the VMs.
why microVMs? from Srini and Craigan example safe GUI from Cody
Our ICFP networking paper (see 2025) won the Cambridge Computer Lab Ring's Publication of the Year award.
An experience report on ten years of running an OCaml, unikernel-inspired network stack on millions of developer machines: what worked, what didn't and what we'd do again. Written with Anil Madhavapeddy, Patrick Ferris, Ryan Gibb and Thomas Gazagnaire; Anil gave the talk.
Our 2013 Unikernels paper won one of the ASPLOS Influential Paper Awards. The Mirage OS libraries from that work are the same ones that ended up inside Docker Desktop.
Although Apple's virtualization.framework is a great general-purpose VMM, there are various ways to optimize specifically for Linux containers. Therefore we created Docker VMM using Apple's lower-level hypervisor.framework, built using a fork of the excellent libkrun project. Filesharing perf improved to the point where a directory search over virtiofs would be "faster than native macOS" if caches are warm. We shipped Docker VMM as a beta in Desktop 4.35.
In the cloud people often talk about "scaling to zero". I've always been a fan of the principle that your laptop should only be executing code if you actually tell it to, i.e. definitely no battery reducing background polling loops. I built a mechanism for shutting down the Docker Desktop helper VM, keeping it offline as much as possible (e.g. by not booting it up for read API requests) and starting it on demand to actually run containers.
I originally switched Docker from kernel networking to a custom userspace
stack
vpnkit (based on the Mirage unikernel libraries)
to work around OS security issues and to allow us to interoperate more easily
with corporate firewalls and VPNs.
Now it was time to switch Mac again to a
google/gvisor-based solution.
In our published
iperf3 benchmark on a first-generation M1 Mac Mini, throughput went
from about 3.5 to 19 Gbit/s — over five times faster.
Kubernetes was built on top of Docker, but in 2022 the method of talking to Docker was changing. Various misleading headlines saying alarming things like "Kubernetes is deprecating Docker" started appearing so we needed to act quickly. To ensure zero hassle for our users, I migrated Desktop's Kubernetes integration to cri-dockerd. This migration shipped in Desktop 4.7.0. To explain what this all means properly I blogged about it in an April article ahead of Kubernetes 1.24.
Enterprises often have sophisticated HTTP proxy setups, especially if they operate worldwide. Originally Docker Desktop's handling of HTTP proxies was fragmented, each component doing its own thing. A customer told us the obvious truth, that proxy configuration should be "one and done", not something that needs per-component consideration. I consoliated all proxy handling across the product so every subcomponent behaved identically, irrespective of programming language, and all supported the same feature set. Previously for example one component would support PAC files proxy auth, while another relied on Unix-style environment variables.
A common topic when speaking with customers is how exactly Docker Desktop's networking works and how well will it interop with other network products. I wrote a detailed blog post all about the unikernel-inspired userspace networking stack that connects containers to the outside world in Docker Desktop. Later this work was refined and used by Valentin and Julius from CARIAD to produce a white paper.
Businesses want to control which registries their developers can pull from. Inspired by Cilium I defined a network policy and then built an enforcement mechanism using a combination of DNS + eBPF + cgroup hooks.
We shipped the GA version of Apple Silicon support! In the meantime Apple had published a nice higher-level VMM API called virtualization.framework which we intended to become our default. So I added support for this too and it shipped as an experimental choice, while QEMU remained the default.
The first M1 Macs arrived and Docker didn't run on them yet. I put together the early preview builds: integrating the engine and CLI, fixing startup problems and iterating until the tech preview shipped in December, immediately after the hardware itself shipped. My screenshot of an early build ended up in an AppleInsider post by Malcolm Owen.
A talk about the new file sharing on Windows and Mutagen on the Mac, and how they speed up the edit-compile-test loop.
After Windows, I brought the gRPC-FUSE file-sharing implementation to the Mac as the default, reducing file-event CPU overhead and allowing shared paths to change without restarting the VM. It shipped as the default in Docker Desktop 2.4.0.0 on 30 September.
I also implemented virtio-blk DISCARD in hyperkit so deleted data could be given back to the host.
A deep-dive into how the logging system works, joint work with Magnus Skjegstad.
We replaced Samba with FUSE over gRPC over Hyper-V sockets. It's much faster, and file change notifications finally work, so tools that watch for edits (like hot reload) behave properly. I implemented the inotify event injection and spent a lot of time on filesystem semantics and caching. The deep dive explains how it all fits together.
Rewrote host port forwarding in Go with a simple HTTP control protocol, replacing several slightly different copies with one.
Why the VM's clock drifted away from the Mac's (laptops go to sleep!), why that breaks builds and file sharing, and how we fixed it.
I changed Docker Desktop's Windows DNS handling to use the native APIs. Using Windows' own resolver instead of reimplementing its behaviour meant Docker followed the same DNS and VPN rules as other Windows applications.
Docker for Windows gained Kubernetes. I worked on networking for it, and looked at how to change the VM's IP address without forcing everyone to reset their cluster.
Added pluggable logging to LinuxKit, building on Magnus Skjegstad's earlier work, and taught vpnkit to forward Unix sockets and Windows named pipes, including connections to services such as the Docker socket.
I added HTTP proxy modes for networks where only the proxy could resolve DNS, with CONNECT support for TLS traffic and tests for the different request paths. I also built more realistic HTTP proxy tests using host firewalls.
Two hyperkit instances writing to the same disk file is a recipe for corruption. I added locking on the backing file, replacing a fragile separate lock file, and added TRIM for sparse raw disks.
I implemented online qcow2 compaction so Docker could return disk space without a restart, enabled it by default, and later used sparse raw disks where APFS made the simpler approach possible. Building on TRIM support from late 2016, the OCaml algorithm recycles clusters in the background while the VM keeps running.
I introduced docker.for.mac.localhost and
docker.for.win.localhost so containers could reach services on the
host without hard-coded IP addresses. These were precursors to the later
host.docker.internal name.
With Magnus Skjegstad and Anil Madhavapeddy I found that apps using VMnet.framework could end up with a DNS proxy listening on all network interfaces. Apple fixed it in macOS Sierra and credited us.
Lots of early users had VPNs, and VPNs didn't get along with the VM's networking. So I plugged the Mirage OCaml TCP/IP stack into Docker for Mac: Ethernet frames from the VM are turned into ordinary socket calls on the Mac, so as far as the VPN is concerned it's just another app. On 2016-04-18 it became the default. It was open-sourced as vpnkit in May, and I merged the Mac and Windows frontends (using Hyper-V sockets on Windows) in July.
moby/vpnkitopen-source announcementbeta announcementmoby/vpnkit#93
I integrated the Mirage qcow2 block layer into Docker for Mac's VM, giving containers a disk that grew as needed instead of reserving its full size on the laptop. This provided sparse storage even on HFS+, which lacked sparse-file support, and let us offer a thin-provisioned 64 GiB container disk.
I integrated Irmin into Docker for Mac to store the VM's configuration, using transactions, snapshots and watches to react to changes. Configuration became versioned data: committing a new memory setting made the VM exit and pick up the change on its next start.
Alongside this, I debugged alpha builds and worked out how diagnostics reports should work. Early changes fixed symlinks in shared folders and kept the VM's clock in sync with the host. That code now lives in the LinuxKit repo.