moss-rs progress and the road ahead

ermo

Hi folks,

Yes, it's been a bit quiet in here lately. However, that does not mean that things have not been happening.

After a lot of deliberation, Ikey decided that it was in the best interests of Serpent OS and the people contributing to it to shift to writing the tooling in languages with better ecosystems and employer mindshare. You can read more about that decision here.

Ikey and new contributor tarkah have been hitting the moss-rs rewrite out of the park, and in the process, they have cleaned up a lot of the inevitable PoC niggles from the previous incarnation of the tooling.

Ikey estimates that the moss-rs rewrite is currently 85-90% done, and has just now begun putting in place the ground work for simulating VFS transactions before they are blitted to disk as a set of hardlinks pointing to files in the Content Addressable Store.

This will be a new addition to the moss tooling and will in turn help power the upcoming upgrade support.

A few days ago, Ikey created this video showcasing the current status of moss-rs in action:

Coincidentally, Ikey had a flash of inspiration just now about how to leverage the existing build infra and tooling to begin the conversion process of Solus 4 into the Serpent OS-based Solus 5. Read more here.

That's all for now. We're really excited about how things are shaping up!

P.S. We are looking for contributors, particularly around a port of our website to Next.js. If this sounds right up your alley, get in touch!

Ikey

Thanks for posting this ermo ! Yeah I'm really pleased with progress of late and moss-rs is making great progress.

Currently we're working on the vfs::Tree type in order to handle file path conflict detection, as well as optimising the blit of the final filesystem.

Apart from that the logic of installs, removals, states, databases, etc, is all in place, so we're almost back to feature parity already. Worth noticing the new implementation is taking more optimal paths now, benefit of hindsight

Ikey

Progress update: https://fosstodon.org/@ikey/111320662405326285

The Rust port is now officially "more better", more robust, and quicker than the original.
It lacks search and upgrade but we'll whack them in quickly, then roll up trigger support, the works.

ermo

Just a quick update that Ikey is now working on thinking through and implementing the moss upgrade support.

https://fosstodon.org/deck/@serpentos/111360546956696273

This is a major milestone for us, and is made possible in part by the work that was done on the new Virtual File System code in moss-rs, which enables the code to resolve and apply transactions up-front in memory.

The point of this is that it ensures that any transactions blitted to disk will be known to be fully resolved ahead of time, thus eliminating file conflicts by design.

Once Ikey lands this, the moss-rs code will have officially superseded the original proof-of-concept moss in terms of functionality and capabilities.

ermo

Following on from the last post on blitting, last week tarkah added the ability to blit very quickly to a new root from the moss store shared with the host system, meaning that standing up a build container is virtually instantaneous, as long as the container root is composed of packages which have already been staged to the local system.

Tarkah and Ikey also landed the ability for moss-rs to distinguish between explicitly installed and implicitly installed packages. This is important for being able to traverse the install digraph for packages that are no longer used after a moss sync.

"Wait, what is moss sync?" you might ask? Well, as moss-rs uses atomic transactions, we started thinking about how useful it would be to be able to sync against the current state of the configured and active repos (a bit like you would git pull from an origin), as a way to reset the installed system state to a predefined system-model for e.g. a GNOME spin etc.

So instead of the usual update and upgrade operations, the primary operation for getting the newest version of the packages in any given repo is now moss sync. This has a bunch of interesting properties, but more on that in a future update.

As an aside, Ikey plumbed in the ability to update repositories, which is useful in case of typos or wanting to switch the install order of local and remote repos for testing. While he was at it, he also fixed a few small niggles regarding symlinks, which enabled him to successfully chroot into moss-rs managed root filesystems.

This week, tarkah started adding the ability for moss-rs to write moss-format v1 .stones, which means moss-rs will soon support the moss index command.

This is important, because it implies that, for all intents and purposes, moss-rs will become drop-in compatible with the existing moss-d code base, while even surpassing it on a couple of fronts.

In turn, all of the above opens up the door for work to begin in earnest on bringing the Serpent base system up to scratch again, after a lot of work and effort has gone into stabilising Solus for the past six months. In particular, it allows the existing boulder-d code base to begin using moss-rs to install roots for use when building packages.

In short, the future prospects for Serpent just got a whole lot brighter.

Bring it on!

Ikey

I'm really quite pleased with the moss-rs progress lately. I think its about time we start refreshing the ISOs!

ermo

After some back and forth discussions concerning build container requirements, tarkah landed a boulder-rs MVP PR that can start a bash shell from within the new rust-based tooling code base using user name spaces. This is significant, because it demonstrates how the new boulder-rs container build tool will be able to run both as a normal user and with elevated privileges out of the box.

Related to this milestone, we want to highlight the work that Fabio (livingsilver94) did on this in the DLang-based moss-container code base, so a big shout out to him for his prior work in this area.

Ikey did some work on wrapping up the Linux-specific renameat2 syscall for use inside moss-rs, modeled on how the Rust nix crate does things. This means that moss-rs is now built statically against the musl libc, which is a pretty big deal, because Ikey managed to demonstrate that moss-rs is now largely drop-in capable on existing systems, thus enabling trivial rescue scenarios of moss-based installations.

Additionally, at Ikey's request, our devops guy (Philipp) built a pipeline that spits out optimised moss-rs nightly builds, enabling much simpler onboarding for people who want to play with the tooling compared to our previous onboarding setup.

Philipp has also done a lot of work behind the scenes to automate dependency updates in our crates and ensuring that we have good PR workflows etc., which is something we have really come to appreciate over time.

Ermo (hi!) is currently in the process of cleaning up and updating the existing onboarding repo, which -- when work is completed -- will once again enable people to experiment with building and installing their own .stone packages and moss-based roots.

ermo

Just a quick note that the onboarding repo should now be in a working state again.

It still needs a bit of refinement re. the Rust-based moss version, but at least it now it updates and compiles the new moss-rs plus the existing legacy DLang tooling automagically.