Many SuiteTimeOut failures on release builders |
||||
Issue descriptionWe just got another round of SuiteTimedOut failures. Assigning to on-call. nyan_big-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941125367367690832 nyan_kitty-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941125366640528512 peach_pit-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941125363897998000 wolf-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941125349278427936 From yesterday: nyan_kitty-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941215432188509408 peach_pit_paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941215429166806224 peppy-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941215428750727440 winky-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941215416122367808 link-paladin: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941215436996772000 From the 2 days ago: daisy-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941335069720270672 daisy-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941395393730622816 daisy-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941425838850157024 daisy-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941546963549535856 daisy-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941665577766221792 veyron_mickey-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941334976006637728 eve-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941335061735746480 kip-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941335036853874528 monroe-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941335022502945600
,
Jul 13
This is a lot of stuff to look at, and I doubt it is all the same root cause. ----- Examining wolf-paladin https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941125349278427936 The majority of the suite ran just fine, but one test failed 07/13 04:03:44.654 INFO | server_job:0216| START ---- wolf-paladin/R69-10872.0.0-rc2/suite_attr_wrapper/login_OwnershipApi timestamp=1531479824 localtime=Jul 13 04:03:44 07/13 04:03:44.655 INFO | server_job:0216| ABORT ---- wolf-paladin/R69-10872.0.0-rc2/suite_attr_wrapper/login_OwnershipApi timestamp=1531479824 localtime=Jul 13 04:03:44 07/13 04:03:44.655 INFO | server_job:0216| END ABORT ---- wolf-paladin/R69-10872.0.0-rc2/suite_attr_wrapper/login_OwnershipApi timestamp=1531479824 localtime=Jul 13 04:03:44 What is really strange to me is the path to this change. suite_attr_wrapper/login_OnwershipApi? That's a mystery I'm filing separately at Reading its job_reporter logs, I see a django.db.utils.DatabaseError: (2006, 'MySQL server has gone away') error which suggests to me this is another occurence of tko flake being investigated at Issue 852083 -----
,
Jul 13
peach_pit paladin https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941125363897998000 A few tests failed. One was due to another job_reporter issue https://stainless.corp.google.com/browse/chromeos-autotest-results/216811208-chromeos-test/chromeos6-row2-rack10-host10/debug/ _mysql_exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)") though this time in lucifer, not in tko/parse as far as I can see. I've filed for follow up at Issue 863504
,
Jul 13
,
Jul 13
Some more instances: clapper-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941093113145353856 platform_DMVerityCorruption [ FAILED ] platform_DMVerityCorruption ABORT: Timed out, did not run. kip-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941093080588412688 autoupdate_Rollback [ FAILED ] autoupdate_Rollback ABORT: Timed out, did not run. kefka-release: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8941093082419423152 provision_AutoUpdate.double [ FAILED ] provision_AutoUpdate.double ABORT: Timed out, did not run. cheets_StartAndroid.stress.0 [ FAILED ] cheets_StartAndroid.stress.0 ABORT: Timed out, did not run.
,
Jul 16
Copying current sheriffs
,
Jul 16
There are a range of different issues related to this bug. This bug itself is about too many different things to be actionable. Closing. See Issue 862685 Issue 863504 Issue 852083 |
||||
►
Sign in to add a comment |
||||
Comment 1 by gmeinke@chromium.org
, Jul 13