Package Dependency Graph in Moss/Boulder/etc. (And Even for ABI Symbols)

GZGavinZhao

I wish moss (or boulder, not sure which one this should apply to) can have the ability or provide APIs to spit out the dependency graph of a package's run-time (also build-time, if possible) dependencies. This can save a lot of trouble for package maintainers, like now for Solus I need to write my own script that converts the entire eopkg-index.xml to a huge dependency graph, and then do some graph theory magic (usually by doing topological sorting) to tell me the safe order to rebuild packages.

I also hope we can know, say after a package update and some ABI symbols were changed/deleted, what packages (i.e. its reverse dependencies) will be affected by that. In this way, package maintainers don't have to rebuild every package down the dependency tree and be confident that things won't break. This is probably more suited to include in summit/vessel/avalanche since the amount of ABI symbols can quickly get very large and likely requires a database. If doing this for every package is not possible, the ability to do a one-shot search for a single package is also enough.

Serpent OS will be "smart" in the users' eyes, therefore I hope it can also be "smart" for package maintainers.

Ikey

I'm totally up for this. If you want we can have an APK style option for emitting the dotgraph for feeding into graphviz? As long as life is easier then thats what matters for the maintainers.

We could emit a resolution from the now resolution given a specific repository config to see how the reverse graph would look and make it easier to make decisions based on that? Graph theory is swell n all but I'd sooner leave it to the deep dark internals of moss instead of doing it myself all the time! =)

GZGavinZhao

Ikey Yep! Just to clarify, I also leave the graph theory stuff to my scripts. That's too much work for my brain 😛

ReillyBrogan

Just to kind of flesh out something I suggested in Matrix for moss:

User Story:
I, an intrepid and easily distractable package maintainer, often get "genius" ideas for how to improve/overhaul certain packages. Often these ideas don't pan out, require more work than I'm willing to commit at the moment, or I just got distracted by some other new shiny.

Let's say for example that I have an idea to improve mesalib. I will make my change and relbump mesalib, then relbump dependent packages (and perhaps make some other changes along the way). Then I will install mesalib and the modified packages to test, but then discover that while my system is functional that I would still need to do some additional work to get my changes ready for submitting. Let's say that sometime after this point someone else does a breaking update to the mesalib package, pushes it along with dependent package rebuilds, but because the relnum of my installed packages is the same as the updated packages those updates do not get installed to my system. This could lead to breakages locally, or more importantly (from the perspective of a system maintainer) it could mean that my system is functional and that I don't see issues that are present in the new version of the package (issues that I could otherwise help contribute fixes for).

So as a fix for this situation it would be great to get a moss diff or similar that would compare the state of the local package database against the configured repos. Installed packages that different in some aspect of the metadata would be displayed and additionally packages that were installed locally that were not in any repo would be shown (for instance if my changes to a package required that I package new deps).

Something like:

moss diff

Packages different than repo packages:
mesalib
dependentPackage1
dependentPackage2

Packages not in repo:
mesalibFancyDependency

Perhaps also with a --detailed option to show exactly what's different:

moss diff --detailed

Packages different than repo packages:
mesalib (different version, checksum mismatch)
dependentPackage1 (checksum mismatch)
dependentPackage2 (checksum mismatch)

Packages not in repo:
mesalibFancyDependency