It's that time (ish) again here in Serpent OS land, and as usual, a lot has happened since the last Tuesday Update.

Development Developments

The focus since the last update has been squarely on getting build infra sorted, including getting existing packages brought up to snuff.

@Ikey has been hard at work on prototyping and fleshing out our build controller dashboard "summit" and our "boulder as a service" builder middleware called "avalanche". This has resulted in the creation of a shared "moss-service" module, which encapsulates the necessary security and authentication/authorisation code for easy use in both avalanche, summit and our binary collection manager "vessel". It has also resulted in some necessary additions to our boulder-generated build manifest files, which are used for providing metadata for source-only packages.

At a high level, summit sits in the middle of everything, reaching out to our GitHub release-engineering "collection" git repository, which will be composed (via git submodules) from several logically grouped, separate git recipe-repositories for easier delegation of responsibilities.

Summit Collection view

Summit can parse and display package metadata too:

Summit Package Metadata view

From there, summit does a git clone of a specific commit in the release engineering "collection" git recipe repo (the one composed of individual git recipe monorepos), analyses said clone for build-dependencies and subsequently creates a build queue of pending jobs.

These pending jobs are then farmed out to authenticated and authorized avalanche builder instances (each having been successfully paired with summit beforehand), which are then responsible for invoking their local installs of our boulder build tool to build one package at a time.

Avalanche stats page

Upon completion (or failure) of each build, the avalanche builder in question will send back a report to summit, which will record the result and -- if the build was successful -- notify vessel to fetch the binary build artefacts from the relevant avalanche builder. After a successful fetch, vessel will then update its source package (srcpkg) and/or binary package (binpkg) repository index with the newly minted package, thus making the package available in our official binary repository for download and installation via our moss package manager.

Ikey is currently putting the finishing touches on the authentication and authorisation code, which means that (fingers crossed, touch wood, etc. etc.) he hopes to begin building packages with summit, avalanche and vessel at the end of this week/beginning of next week.

@ermo has set up our first physical builders and has been on hand for Ikey to offer encouragement and act as a live rubber duck/teddy bear programming buddy whenever Ikey's been stuck on a particularly vexing issue.

He's also contributed a bit of code to moss.core.cpuid, which now allows e.g. moss and avalanche to query and report which ISA levels are supported on the system on which they're running, which in the future will allow for (opt-in) automatic installation of optimised x86-64-v3x binaries on systems with CPUs that support them.

The "x" in -v3x stands for "sunnyflunk's carefully selected extra instructions sets not part of the standard x86-64-v3 psABI group of instruction sets". These additions have been shown to make the generated code faster than stock x86-64-v3 code, while maintaining compatibility with modern Intel Core branded CPUs since Haswell and modern AMD Ryzen branded CPUs. Our x86-64-v3x binaries might also be supported on the "Excavator" versions of previous generation AMD CPUs, but we don't have any on hand to test against at this point in time.

@sunnyflunk has been chipping in with design suggestions and updates to critical toolchain packages. Among other things, he's rebuilt and updated our gcc and clang packages manually on one of our builders via boulder and tinkered with various optimisations and benchmarks.

He's also been keeping a keen eye out for toolchain posts on the various toolchain-related mailing list, such that he's also ready to hit the ground running when the build infra is powered on. He's a bit hung up IRL and doesn't have 24/7 access to his machine currently.

@ulaas is still snowed under with IRL related stuff but stays in touch with the team as time allows.

Sponsorship via GH Sponsors

If you like what you're seeing, don't forget to chip in with what you can spare. Every penny counts for Ikey and his family. Thank you in advance for your generosity.

FAQ

... is available here.