Reviving the blog
I'm back!
The last time I blogged was in 2017: a lot has changed since then, and after a decade of ignoring blogging, I will attempt to put some regularity into it again.
The times call for it: having a personal space that is really "our own" is extremely important, and it is as important as having something to read that is written by other humans, and not slop.
I mainly stopped blogging for two reasons:
- Wordpress and similar tools are terrible, for rarely changing content. I'd rather not blog, than host a dynamic website just for serving static webpages
- My static site generation pipeline heavily relied on my workstation's software dependencies, which shifted continuously, breaking the website build at all times.
Stabilizing the build
Note: This section describes the Nixification of the blog, done in this pull request. You can skip this, if you prefer reading the PR instead.
The first thing to do is to get everything under control again.
Since a few years back, I started heavily relying on Nix, a lazy functional language that is perfect to achieve reproducible builds and environments.
At the time of this writing, this website is built via Sculpin, a static website generator whose dependency upgrades I've neglected for far too long.
In order to "freeze" the build in time, I used a Nix Flake to pin all the dependencies down, preventing any further shifts in dependency versions:
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils, composer2nix, ... }@inputs:
flake-utils.lib.eachDefaultSystem (
system: {
packages = {
# things that will stay extremely stable will go here
};
}
);
}
The above will "pin" dependencies such as composer or php,
preventing them from drifting apart, unless a commit moves them.
This is also thanks to the built-in flake.lock mechanism of Nix Flakes.
Because Composer does not compute content hashes of PHP dependencies, NixOS
cannot directly use composer.json and composer.lock to
download dependencies: that is an obstruction to reproducible builds, and requires
a little detour.
Luckily, Sander van der Burg built a very
useful composer2nix tool,
which can be used to scan composer.lock entries, and compute their content
hashes upfront:
{
inputs = {
# ...
composer2nix = {
url = "github:svanderburg/composer2nix";
flake = false;
};
};
As you can see, composer2nix is not a flake: we still manage to use it ourselves,
to process composer.lock locally:
update-php-packages = pkgs.writeShellScriptBin "generate-composer-to-nix.sh" ''
set -euxo pipefail
TMPDIR="$(${pkgs.coreutils}/bin/mktemp -d)"
trap 'rm -rf -- "$TMPDIR"' EXIT
mkdir "$TMPDIR/src"
mkdir "$TMPDIR/composer2nix"
${pkgs.coreutils}/bin/cp -r "${./app}" "$TMPDIR/src/app"
${pkgs.coreutils}/bin/cp -r "${composer2nix}/." "$TMPDIR/composer2nix"
${pkgs.coreutils}/bin/chmod -R +w "$TMPDIR/composer2nix"
${pkgs.php84Packages.composer}/bin/composer install --working-dir="$TMPDIR/composer2nix" --no-scripts --no-plugins
${pkgs.php}/bin/php $TMPDIR/composer2nix/bin/composer2nix --name=${website-name}
${pkgs.coreutils}/bin/rm -f default.nix
'';
We can now run nix run .#update-php-packages to generate a very
useful php-packages.nix, which will be used to produce our vendor/
directory later on.
The generated php-packages.nix looks a lot like this:
let
packages = {
"components/bootstrap" = {
targetDir = "";
src = composerEnv.buildZipPackage {
name = "components-bootstrap-fca56bda4c5c40cb2a163a143e8e4271a6721492";
src = fetchurl {
url = "https://api.github.com/repos/components/bootstrap/zipball/fca56bda4c5c40cb2a163a143e8e4271a6721492";
sha256 = "138fz0xp2z9ysgxfsnl7qqgh8qfnhv2bhvacmngnjqpkssz7jagx";
};
};
};
# ... and more
With that, we can then prepare a stable installation of the website generator:
# ...
built-blog-assets = derivation {
name = "built-blog-assets";
src = with-autoloader; # an intermediate step I omitted in this blogpost: check the original PR for details
builder = pkgs.writeShellScript "generate-blog-assets.sh" ''
set -euxo pipefail
${pkgs.coreutils}/bin/cp -r $src/. $TMPDIR
cd $TMPDIR
${pkgs.php}/bin/php vendor/bin/sculpin generate --env=prod
${pkgs.coreutils}/bin/cp -r $TMPDIR/output_prod $out
'';
inherit system;
};
Running nix build .#built-blog-assets now generates a ./result directory
with the full website contents, and we know it won't break unless we update flake.lock, yay!
Let's publish these contents to Github Pages:
publish-to-github-pages = pkgs.writeShellScriptBin "publish-blog.sh" ''
set -euxo pipefail
TMPDIR="$(${pkgs.coreutils}/bin/mktemp -d)"
trap 'rm -rf -- "$TMPDIR"' EXIT
cd "$TMPDIR"
${pkgs.git}/bin/git clone git@github.com:Ocramius/ocramius.github.com.git .
git checkout master
${pkgs.rsync}/bin/rsync --quiet --archive --filter="P .git*" --exclude=".*.sw*" --exclude=".*.un~" --delete "${built-blog-assets}/" ./
git add -A :/
git commit -a -m "Deploying sculpin-generated pages to \`master\` branch"
git push origin HEAD
'';
We can now run nix run .#publish-to-github-pages to deploy the website!
Self-hosting: a minimal container
Since you are one of my smart readers, you probably already noticed how GitHub has been progressively enshittified by its umbilical cord with Microslop.
I plan to move the blog somewhere else soon-ish, so I already prepared an OCI container for it.
Since I will deploy it myself, I want a container with no shell, no root user, no filesystem access.
I stumbled upon mholt/caddy-embed, which embeds an entire static website into a single Go binary: perfect for my use-case.
The Caddy docs suggest using XCaddy for installing
modules, but that is yet another build system that I don't want to have anything to do with.
Instead, I cloned caddy-embed, and used
NixPkgs' Go build system to embed my website into it:
caddy-module-with-assets = derivation {
name = "caddy-module-with-assets";
builder = pkgs.writeShellScript "generate-blog-assets.sh" ''
set -euxo pipefail
${pkgs.coreutils}/bin/cp -r ${./caddy-embed/.} $out
${pkgs.coreutils}/bin/chmod +w $out/files/
${pkgs.coreutils}/bin/cp -rf ${built-blog-assets}/. $out/files/
'';
inherit system;
};
embedded-server = pkgs.buildGo126Module {
name = "embedded-server";
src = caddy-module-with-assets;
# annoyingly, this will need to be manually updated at every `go.mod` change :-(
vendorHash = "sha256-v0YXbAaftLLc+e8/w1xghW5OHRjT7Xi87KyLv1siGSc=";
};
Same as with PHP, I'm pretty confident that Nix won't break unless flake.lock changes.
We can now bundle the built server into a docker container with a single Caddyfile attached.
The following is effectively a Dockerfile, but reproducible and minimal:
runnable-container = pkgs.dockerTools.buildLayeredImage {
name = website-name;
tag = "latest";
contents = [
(pkgs.writeTextDir "Caddyfile" (builtins.readFile ./caddy-embed/Caddyfile))
];
config = {
Cmd = [
"${embedded-server}/bin/caddy-embed"
"run"
];
};
};
We can now:
- build the container via
nix build .#runnable-container - load the container via
cat ./result | docker load - run it via
docker run --rm -ti -p8080:8080 ocramius.github.io:latest
The running container uses ~40Mb of RAM to exist (consider that it has all of the website in memory), and a quick test with wrk showed that it can handle over 60000 requests/second on my local machine.
❯ wrk -t 10 -d 30 -c 10 http://localhost:8080/
Running 30s test @ http://localhost:8080/
10 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 165.35us 75.09us 3.91ms 90.49%
Req/Sec 6.12k 365.53 7.05k 92.46%
1833101 requests in 30.10s, 23.32GB read
Requests/sec: 60901.74
Transfer/sec: 793.44MB
Cleaning up the website
While cleaning up the builds, I found some really horrible stuff that should've gone away much earlier:
Google Analytics: kill it with fire! I'm not here to "convert visits": I'm here to help out my readers and make new connections. I am not a marketing department, and the privacy of my website visitors is more important than a website ticker that sends data to a US-based company.
Leftover JS/CSS: the website had various external CDNs in use, with CSS and JS files that were not really in use anymore. Cleaning these up felt good, and also reduced the number of external sites to zero.
Navigation menu simplified: this is a static website. An animated "burger menu" was certainly interesting a decade ago, but nowadays, it is just an annoying distraction, and extra navigation steps for visitors.
Disqus: this used to be a useful way to embed a threaded comment section inside a static website, but it is no longer relevant to me, as it becomes an extra inbox to manage. Disqus was also cluttered with trackers, which should not be there.
Next steps?
This first post is about "being able to blog again", but there's more to do.
I certainly want to self-host things, having my blog under my own domain, rather than under
*.github.io.
I also want comments again, but they need to come from the Fediverse, rather than being land-locked in a commenting platform. Other people have attempted this, and I shall do it too.
Perhaps I may remove that 3D avatar at the top of the page? It took a lot of time to write, with Blender, THREEJS, and it uses your video card to run: perhaps not the best energy-efficient choice for a static website, but I'm still emotionally attached to it.
Also, this website is filled with reference information that no longer holds true: a decade has passed, and my OSS positions have vastly changed, and so will pages that describe what I do.
Finally, I want it to be clear that this is a website by a human, for other humans: I will therefore start cryptographically signing my work, allowing others to decide whether they trust what I wrote myself, without a machine generating any of it.
And you?
If you are still here and reading: thank you for passing by, dear fellow human.
Hoping that this has inspired you a bit, I'm looking forward to seeing your own efforts to self-host your own website!