Release Engineering (releng) Design
Status: Draft
Version: 4
Multiple recipe monorepos as upstreams for the collection releng git repo
The main recipe monorepo will be the foundation of Serpent OS. The base subset of main (implemented as a named group/set under the main recipe monorepo) will be what the core of Serpent OS requires to work and be self-hosting.
Essentially, a collection of moss binary .stone packages built from this base set (and considered a part of the virtual base moss .stone collection) will be bootable in a systemd-nspawn container and enable the user to write/update recipes, commit them and build corresponding .stone packages.
All other Serpent OS official recipe monorepos will be used to compose and build the rest of the moss .stone collection. This collection will be built from a combination main (where most things will go) and kernel, hardware, gnome, plasma, etc. recipe monorepos, which each live under its own snekpit org namespace.
The collection releng (shorthand for release engineering) monorepo will use git submodules to combine these recipe monorepo "source feeds" into something that we present in each moss .stone collection release for consumption by users.
A release should be thought of as a successful build (w/necessary rebuilds) of the collection releng monorepo that has the property of transitive closure and is known to be dependency complete.
Moss source and binary .stone collections
Our official moss binary .stone collections (w/associated moss-readable index files) will be as follows:
volatile - Super unstable, raw dump of all built and cached .stones. Must be stabilised for each "release"testing - Points to a newly cut release that hasn't been approved as workingcurrent - Points to a not-so-newly cut release that we believe to work
Individual stones will have a mechanism indicating which set(s) they belong to, including the base set etc.
Thus, within the same single moss-readable index file describing the moss .stone collection contents, it will be possible to configure a local profile that only enables a virtual moss .stone collection containing the base set of .stones.
In addition, we'll have a build-from-source-only community recipe monorepo on GitHub which will allow anyone to contribute recipes which aren't (yet?) suitable or ready for inclusion in main etc. This is essentially our version of an AUR. This recipe monorepo will be indexed and published independently as a moss from-source-only collection index (no .stones).
Moss itself will always install/remove only binary .stones to/from the local filesystem and will by default prefer to download existing .stones if any are available that match the configured system profile (x86-64-v2 by default until we build out our portfolio of profiles).
If a cached .stone exists that matches the desired build profile, it will be installed directly. Otherwise, the source recipe in question (and its reverse dependencies!) will be automagically built on the local system and cached in a local moss .stone collection, from where moss will install the cached .stones.
If an updated recipe becomes available, moss will automagically build and install any from-source recipes and necessary rev-dep rebuilds. This aspect is inspired by how the FreeBSD synth tool works.
Recipe Migration Path
We aim to build a history-preserving recipe migration path from the build-from-source-only community recipe monorepo to the official Serpent OS maintained recipe monorepos.
This implies that new stacks that aim for inclusion in the official moss .stone collections can be landed and fleshed out in the community recipe monorepo first before being migrated to a suitable official recipe monorepo.
Similarly, recipes initially landed in a personal github from-source-only repo can be migrated to the community recipe repo using the exact same process.
When recipes become obsolete or unmaintained, we aim to support migrating the recipes in the opposite direction, first back to the community recipe repo and then to the unmaintained recipe repo.
If a recipe isn't picked up by a maintainer (i.e. brought up to date and tested within a set period), it will be moved to the graveyard recipe repo. Here, the recipe will be known to exist (and its last known providers will be captured for moss lookup purposes), but it won't be installable. If someone resurrects it, it can be brought back into community recipe repo.
Cutting a Release
Each week, and likely multiple times a week, we'll cut a release from the volatile collection and version it. There will be a direct correlation between our source recipe repos and moss .stone collections - with each recipe monorepo "domain" (main, etc.) distinctly versioned via tags.
We'll cut a release by updating submodules in the collection releng monorepo to the latest "domain" monorepo tags (i.e. main, kernel, hardware, gnome, plasma etc.) and releasing a new tag for the collection releng monorepo itself.
When we're happy that the release on the testing channel created via the above tagging process holds up, we'll publish it on the current channel to push it to users.
(The author would like to thank @Ikey for writing an initial and perfectly cromulent draft version of this post, which summarised recent development discussions quite succintly)
(If you don't know how to read between the lines, this means I stole what he wrote and pretended I wrote it!)
