Fix V8 infra end-to-end test |
|
Issue descriptionThe bot is red since a while and should be greenified: https://build.chromium.org/p/chromium.tools.build/builders/v8-linux-end-to-end The bot tests the complete v8 auto-bisect process on a wrapped tester bot. Main issue: The test fixture is based on uploaded isolates that run out of date and don't exist on the servers after a while. Solutions: Suggestion 1) Also wrap the three relevant builder tasks and compile and upload each time. Suggestion 2) Test a builder_tester bot instead. It would upload isolates itself. This is actually a partially orthogonal test and won't go through some interesting build/tester paths, like uploading isolate hashes to GS. Both solutions require compilation. Not sure if we can use goma in a wrapped recipe? Maybe we can switch it off. Either way we'd need a separate slave for this. Suggestion 3) Upload the fixture (e.g. all three builds) to GS once, let the recipe download them and then push again to isolate server. This'd require a new home-grown recipe.
,
Nov 29 2016
Suggestion 4) - Add a new v8-side build target that produces a file containing: bool(commit_position % 3 == 0). - Make a v8-side script that fails dependent on this file's flag - Isolate this new script - Add a dedicated builder bot that builds every revision and only builds and uploads this target - The isolate hashes won't ever change, there are just two different ones (one for the failing and one for the passing version of the script) - Let the special builder mark that last uploaded hash + CP in google storage - Build-side, add a dummy tester recipe that runs tests for only this executable - In the end-to-end test: o Check the last uploaded hash + CP o Calculate next x < CP for which x % 3 == 0 o Use [x-2, x-1, x] as fake blame list o Wrap the dummy tester recipe and simulate bisect with it o There are only two different swarming tasks resulting in this based on the two isolates - so we get mostly dedupes Caveat: We might need to go through the V8 test driver to get full coverage of all features. I.e. the simple script might need to be a stand-alone v8 test suite called through the test driver. This means we get new hashes for each test driver change (rather rare though). If we bypass the v8 test driver, we might need special infrastructure for this on the build-side. |
|
►
Sign in to add a comment |
|
Comment 1 by machenb...@chromium.org
, Nov 29 2016