Currently the third-party packages recipe (3pp) requires recipe modification when adding new packages, and this modification process is less straightforward than one would hope (in particular, what would be 3-4 lines of shell requires modification of a couple different files, copying and modifying a boilerplate recipe and regenerating expectations).
Additionally, the current recipe doesn't support cross-compilation, and it's not obvious how to add it without making things more complicated.
Thirdly, the recipes don't treat the fetch, build, package, upload cycle in a uniform way; some packages we build are meant to be consumed as dependencies and so are rebuilt every time, some are not, each package roughly manages its own cipd interactions, etc.
The goal for the rework is:
* Adding/changing packages, their dependencies or build procedure should not require any recipe modification. The recipe should publish a fixed API and the package definitions should live elsewhere, e.g. in the infra repo in a mostly-declarative format.
* Cross compilation/toolchain management should be trivial (at least for cross-arch on linux, but ideally built so that we can add cross-platform cross-compile as well, e.g. with clang for windows and mac from linux).
* Heavily document the API of the recipe so interacting with it is very predictable.
* Allow local execution of the recipe to build any subset of packages known to it.
* Allow other teams to define packages in their own repo and re-use this recipe verbatim (preferably from the infra recipe bundle).
Because this rework is pretty extensive architecturally, I'll be writing it as a parallel recipe/module with parallel builders instead of rebuilding it in-place.
