vm89-m1 needs serious help / reimaging |
||||
Issue descriptionI was investigating update_scripts failures as part of bug 674759 and I noticed that this vm was high on the list. I took a look and saw that it was failing with Aborting could not detach HEAD Failed to update depot_tools. Looking for .gclient starting from E:\b\build Traceback (most recent call last): File "E:\b\depot_tools\gclient.py", line 2146, in <module> sys.exit(main(sys.argv[1:])) File "E:\b\depot_tools\gclient.py", line 2132, in main return dispatcher.execute(OptionParser(), argv) File "E:\b\depot_tools\subcommand.py", line 252, in execute return command(parser, args[1:]) File "E:\b\depot_tools\gclient.py", line 1883, in CMDsync client = GClient.LoadCurrentConfig(options) File "E:\b\depot_tools\gclient.py", line 1250, in LoadCurrentConfig path = gclient_utils.FindGclientRoot(os.getcwd(), options.config_filename) File "E:\b\depot_tools\gclient_utils.py", line 608, in FindGclientRoot exec(FileRead(entries_filename), scope) TypeError: expected string without null bytes Hopping on the machine, I found that the .gclient_entries file was completely full of null bytes. I deleted that file and sync'd again and it complains now that the .git/index file is corrupted (it's also completely full on null bytes). I suspect a crash on the VM may have committed journaled metadata but not actually the data for these and possibly other files. I would suspect the integrity of the entire VM.
,
Jan 12 2017
chromium gpu / GPU Win Builder (dbg)
,
Jan 12 2017
Wow. Thanks Robbie for finding that. We definitely need alerts for this sort of thing. chromium.gpu is supposed to be a tree closer so upgrading this to P1.
,
Jan 12 2017
,
Jan 12 2017
update_scripts is intentionally not a tree closer because <reasons>. I'm currently adding better monitoring for it across the fleet. The lack of update_scripts means that that bot's base depot_tools & friends won't be updated, but I believe that it's using remote_run for recipes anyway, which means that it ignores those repos, so it's not a fatal error (but I do highly suspect the integrity of the machine otherwise)
,
Jan 12 2017
vm89-m1 has been re-deployed and is back on the waterfall. Thanks.
,
Jan 12 2017
thanks :)
,
Jan 12 2017
n/p. bot update looks good on the current run. |
||||
►
Sign in to add a comment |
||||
Comment 1 by kbr@chromium.org
, Jan 12 2017Labels: -Pri-3 Pri-2