articles

There’s a popular myth in the Docker community. The myth is that it’s possible to mount /var/run/docker.sock with read only options. Even large container projects like Traefik get this wrong. Bind mounting the Docker socket with read only options doesn’t work the way one thinks it would. This post will explain how using the ":ro" option when mounting the Docker socket is little more than security theater and to discuss alternatives. Please read this post if you’re adding /var/run/docker.sock:/var/run/docker.sock:ro to Docker compose files. ...

Who Should Not Read This Post This post does not discuss: data support for multiple writers having zero downtime cloud-scale best practices Instead I discuss embracing single points of failure (SPOF) in exchange for simplicity within environments that can tolerate downtime. Permission to Not Over-Engineer In environments that tolerate downtime, having a SPOF is great because they are simple to deploy and it’s obvious what failed. Lets examine the typical home router. In addition to routing packets between the LAN and the Internet, the home router also provides DHCP and DNS services. Then someone learns of PiHole and runs it on a separate device like a Raspberry PI. Now the router queries PiHole for adblock entries during DNS resolution. Ad-free browsing works works great; until HTTP requests stop working. With this scenario, it’s not obvious if it’s an issue on the router or an issue with PiHole. ...

This post is the README for my home network and something I can refer to later. It’s the story of how I replaced a underperforming consumer-grade router with a fully virtualized RouterOS instance using QEMU and PCI passthrough for the network. Like any software project, this project is source controlled and rebuildable from scratch. If the router dies, I simply deploy a new router instance. This is the reproducibility you’d expect in a datacenter, not a home office. ...

Adding Subcommands to Go CLIs Command Line Interfaces (CLIs) use subcommands and flags to enable different program features. A subcommand is a grouping of related features, and flags are options for controlling those features. The openssl command provides a great example of subcommands and flags. openssl rand -base64 8 will generate 8 random bytes of data with hexadecimal output. The subcommand is “rand” and “-base64” is the flag. Other openssl subcommands like “s_client” or “x509”, provide different features and each has their own options. ...

Until recently, I haven’t had many opportunities to use Go’s generics. I ran into a case where generics make sense. Best of all, this isn’t a contrived example. I’m working on a project and using openAPI to generate API contracts. One of the generated structs contains optional fields implemented as pointers. The only required field is Name. const ( Gzip PostPailsCompression = "gzip" None PostPailsCompression = "none" ) type PostPails struct { Compression *PostPailsCompression `json:"compression,omitempty"` // MaxArchiveSize Max size (bytes) before rotating to a new archive. MaxArchiveSize *int `json:"max_archive_size,omitempty"` // Name Name of the new pail Name string `json:"name"` } I need to populate the struct with values when writing unit tests. But dealing with pointers in Go test code usually results in using temporary variables. It’s not bad, but there’s some visual noise. ...

OpenAPI is a specification for documenting HTTP APIs for both humans and machines to consume. As OpenAPI is a specification, it is language agnostic. OpenAPI relies on generators for translating the specification. There’s more than just documentation that’s generated. Generators also create language-specific interfaces, tooling, and contracts. In some ways the OpenAPI pattern reminds me of either protobuf with gRPC or ORM schema-first design. As a result, a declarative API is created by the tooling. ...

With Docker, it’s not always obvious what storage options exist beyond the built-in local volume driver or a traditional bind mount. Exploring Docker volume drivers often turns up archived GitHub repositories or commercially backed plugins tied to specific cloud storage products. The volume ecosystem is especially limited for on-premise storage, and many plugins require more privileges than you’d expect. In this post, I’ll cover how Docker handles volume storage under the hood. I’ll also walkthrough how to create a volume plugin that interacts with remote storage without needing CAP_SYS_ADMIN privileges. ...

I’m looking at how object storage systems manage data on disk. Especially the idea of using append only archives with an index for fast retrieveal. While reading Facebook’s Haystack design, I noticed similarities to the tar file format and the potential to implement something similar at the local scale. Haystack Overview There are several components mentioned in the original Haystack paper, but at the core is the Haystack Store, where end user image files are physically kept. Instead of writing files directly to the filesystem, images are appended to a large file called a volume, which acts as an append-only archive. Each volume is typically capped at around 100 GB and is aligned to 8-byte offsets. Image files within this volume are referred to as needles. ...

UTM and Multipass are great apps for virtualization on macOS. But I wanted a lighter-weight approach by invoking QEMU directly. Which meant I needed to understand how QEMU’s networking options interact with the vmnet virtualization API on macOS. This becomes especially important when dealing with VM-to-VM connections, network isolation, and bridging on macOS. In this post, I’ll walk through creating a simple QEMU-based networking lab. Set up RouterOS and Alpine Linux VMs using QEMU on macOS Connect VMs with Apple’s Hypervisor vmnet networking APIs Use unified logging to troubleshoot QEMU network issues on macOS Lab Setup Overview The network diagram shows the network topology used in this lab. Both VMs run on on the same macOS host and connected to virtual network interfaces using QEMU’s support for Apple’s vmnet virtualization API. ...

QEMU uses files to emulate storage devices, and the features available depend on how those files are created. While QEMU can emulate disks from Parallels and VirtualBox, I’m going to focus on the formats most commonly used in automation and scripting, raw and qcow2. The default format is raw and raw offers the fewest features. It’s just plain storage. The other format qcow2 supports compression, snapshots, and copy-on-write in addition to storage. ...