Need a better testing mechanism for uploaded toolchain packages |
||||
Issue descriptionThe toolchain package that gets uploaded by the chromiumos-sdk in the SDKPackage stage (and which then gets downloaded & used by the SimpleChrome workflow, is NOT the same toolchain package that gets tested by the rest of the chromiumos-sdk builder. This has bitten us several times. We need a better way of testing this.
,
Sep 28 2017
the chromiumos-sdk bot doesn't test the Simple Chrome workflow. i guess it should considering it produces the standalone toolchains that Simple Chrome relies upon ? but i think your discussion earlier was also about trying to run new toolchain CLs (i.e. not landed) against remote trybots with --latest-toolchain and having the simple chrome flow there use the new toolchain too ?
,
Sep 28 2017
the issues I have found are when using --latest-toolchain on bots that test simple chrome.
,
Sep 29 2017
I don't remember all the ins-and-outs now, but the basic issue I remember seeing is that we had a compiler/toolchain CL and we tested it on trybots for each arch (arm32, aarch64, amd64) AND we tested it on the chromiumos-sdk, and all 4 of the trybot jobs, including the chromiumos-sdk, passed the CL with flying colors, so we would then commit the CL only to find 2 or 3 days later that it broke the Simple Chrome workflow. I seem to recall at the time that we figured out that this was because the toolchain being tested by the chromiumos-sdk (with our patch) was NOT the same toolchain as the one that was packaged and uploaded for Simple Chrome.
,
Sep 29 2017
what you describe sounds correct: when the sdk bot runs as a remote tryjob, it shouldn't be uploading any files at random for other bots to be pulling down or for users to come across. i think we can distill this to two requests: (1) if a tryjob runs with --latest-toolchain, the Simple Chrome stage should create local standalone toolchain packages to use. at a glance, we do this already for the sysroot (we point Simple Chrome at the board in the local chroot). (2) the sdk bot should include a Simple Chrome phase. for remote tryjob requests for toolchain peeps, i guess the first request meets most of your needs ? having the sdk bot also test the simple chrome stage wouldn't really help ? but it would be good for the waterfall bot as we'd be able to include that in the normal updates rather than wait for the Chrome PFQ. we might be able to leverage GOMA in those stages too to help speed things up. would we do Simple Chrome for each board the sdk bot tests ? i guess that makes sense.
,
Jan 4 2018
,
Feb 5 2018
|
||||
►
Sign in to add a comment |
||||
Comment 1 by cmt...@chromium.org
, Jun 5 2017