I've thought about this too.

The problem is where to put this code and how to make sure any changes to it are picked up. Currently the only possibility is the standard library hardcoded inside lucicfg. It means whenever we make a change to how we want e.g. recipe roller to be, we need to release a new binary, make sure all people who manager configs get it (so.. release at least to depot_tools and to fuchsia's buildtools), and then somehow force them to run it :)

Presumably it is a lesser problem for the core code, since it is presumably will be more stable, being generic and low level...

Yeah I'm not sure; it's definitely not a fully baked idea, but it would be a nice end-state :)

We could potentially also have the lucicfg binary always pull some luci-stdlib stuff from some cipd ref (and allow this to be overridden on the command line, if you wanted to have reproducibility and accept the responsibility for keeping it up to date yourself).

Since the produced proto files are the actual source of truth, it would look something like:
 * Change stdlib .star files
 * Canary them (by regenerating infra/staging configs).
 * Promote them to production (and promote old stable to "previous_release" ref) and send PSA "please regenerate configs, pass -luci-stdlib-vers previous_release if you don't want to right now"

Seems reasonable.

I also had the following deps/lockfile idea which complimentes your proposal: when generating configs, all external starlark packages are by default fetched from ToT (e.g. 'stable' cipd ref). But at the same time we generate a sort-of "lockfile" which has a concrete version of a cipd package used during this generation run.

This lockfile is just a byproduct of the config generation and it is committed to the repository along with other configs. So the committed state in the repository is sufficient to reproduce what happened EXACTLY (e.g. for debugging).

If for whatever reason 'stable' is not good enough, any other version can be specified in the "deps" file. 

This "deps" file can either be a real file in some format, or we can perhaps have:

core.external_package(
    name = "lucitools",
    cipd_package = "infra/lucicfg/...",
    cipd_version = "latest")

load("@lucitools//recipe_roller.star", "recipe_roller")

recipe_roller(...)

+1 for lock file.

also sgtm :)

Though I would like the entrypoint to be just "luci_stuff" (but with a better name) not necessarily 'recipe_roller', etc. So that way we can add other maintenance tasks (or remove them!) as necessary.

In my example the "entry point" is '@lucitools' package. From owners points of view, I think this is mostly equivalent:

load("@lucitools//stuff.star", "stuff")
stuff.recipe_roller()

vs

load("@lucitools//recipe_roller.star", "recipe_roller")
recipe_roller()

In both cases to add something, we need to extend some existing namespace (either namespace of files, or the 'stuff' struct), and then ask users to go an pick up this new thing.

This comes down to what feels like more natural. Core Starlark advertises importing end-symbols via "load". But it is also a common pattern (as far as I've seen) to import structs :-/

In any case, the base implementation is identical and each individual external package can choose whatever pattern it prefers. Details like this are not important for 90% of the implementation.

Yes, I know, I was just meaning that client .star files should not contain the string `recipe_roller`, but instead `stuff.stuff()` (whose implementation is in lucitools//stuff and does mention recipe_roller).

I agree that 0% of this is relevant, considering how much left there is to do :)