New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 619615 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Last visit 15 days ago
Closed: Jun 2016
Cc:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocking:
issue 594336
issue 595988
issue 618528
issue 618545
issue 618583
issue 618612



Sign in to add a comment

network_DefaultProfileCreation failing consistently on tricky

Project Member Reported by steve...@chromium.org, Jun 13 2016

Issue description

network_DefaultProfileCreation is failing consistently in the bvt-cq stage with the following message:

network_DefaultProfileCreation,Unhandled DBusException: org.freedesktop.DBus.Error.NoReply: Message did not receive a reply 

This is blocking the PFQ

 
Blocking: 595988 594336
Cc: kirtika@chromium.org steve...@chromium.org
Owner: ihf@chromium.org
Status: Assigned (was: Untriaged)
Cc: ejcaruso@chromium.org pstew@chromium.org
This appears to be due to a shill crash, e.g. from the failure in issue 618612:

http://cautotest.corp.google.com/afe/#tab_id=view_job&object_id=66214744

-> 'raw results logs' link -> network_DefaultProfileCreation/sysinfo/iteration.1/var/spool/crash/

https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/66214744-chromeos-test/chromeos4-row2-rack3-host12/network_DefaultProfileCreation/sysinfo/iteration.1/var/spool/crash/

Comment 3 by kirtika@google.com, Jun 13 2016

Lots of possible duplicates - there's one being tracked at the partner bug here: https://code.google.com/p/chrome-os-partner/issues/detail?id=52804


Re comment #3 - that issue was filed on Apr 27. These failures on tricky just started happening consistently since Jun 11

Also, it looks like we had some similar but slightly different failures starting June 8:

FAIL: Unhandled DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name :1.116 was not provided by any .service files

Owner: jdufault@chromium.org
jdufault@ - Have you had a chance to look at this one? Go ahead and re-assign this to ihf@ once he gets in.


Cc: keta...@chromium.org
I have a revert open at [1], but I cannot verify the revert fixes the build just yet because the of the compile error, which should be fixed tomorrow.

1: https://codereview.chromium.org/2061093002/
Another tricky failure instance in the latest build. 

FAIL: Unhandled DBusException: org.freedesktop.DBus.Error.NoReply: Message did not receive a reply (timeout by message bus)

jdufault: should this one be fixed too with your revert? I need a green PFQ by latest tomorrow morning. Any chance we can get this in quick?
That looks like a similar symptom. Not sure if the revert will fix it quite yet, but there's a good chance.

Comment 11 by kirtika@google.com, Jun 14 2016

Re comment #5 - Hi stevenjb@, we are pretty certain we have seen this failure through April and May. See the thread I forwarded to you. It probably began manifesting consistently again since Jun 8/11. 

Comment 12 by kirtika@google.com, Jun 14 2016

Hi jdufault@, stevenjb@,
What is the rationale for reverting ejcaruso@'s commit? I might be missing something here, but the code looks unrelated to the bug. 
This test has been failing on tricky with in different ways since early March. Please see https://bugs.chromium.org/p/chromium/issues/detail?id=594336 which is assigned to me. ejcaruso@'s commit is dated April 19. 


We have seen crashes in Shill, which show up as failures in this test, on
and off.

This particular failure has been happening consistently only on tricky
since that CL landed.

We are not certain it is the cause, which is why we are testing the revert
before landing it, but that CL does cause Shill to exercise new code, so
it's not entirely a shot in the dark.
The previous tryjob failed due to a compile failure in ToT, I started a new one prior to the suspected compile failure:
https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/chrome-pfq-informational/builds/23

Owner: ejcaruso@chromium.org
The revert passed, so that CL does indeed appear to be triggering the shill crashes. I'm going to commit the revert so that we can get a green pfq build.

https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/chrome-pfq-informational/builds/24 includes the change and also passed. What should we do about this?
I don't know. network_DefaultProfileCreation passed in the last tricky run
with the revert.

That said, BuildPackages has been failing since yesterday morning, so it's
possible that some other fix went in?

Let's:
1. Let tricky-tot-chrome-pfq-informational cycle green
2. Do another tot trybot run with your CL
3. Commit your CL and watch tricky-tot-chrome-pfq-informational

Let's do #3 first thing tomorrow if that is OK?
Labels: -Pri-0 Pri-1
OK, tricky-tot-chrome-pfq-informational is now green.

I started a tryjob with:
cbuildbot --remote tricky-tot-chrome-pfq-informational -G 2067913002

Tryjob is here:
https://uberchromegw.corp.google.com/i/chromiumos.tryserver/builders/chrome-pfq-informational/builds/25

If that cycles green, we can put 2067913002 into the CQ then keep an eye on tricky-tot-chrome-pfq-informational.

Downgrading to P1 since the PFQ informational builders went green.

Sounds good. Thanks.
New failure for network_DefaultProfileCreation immediately after the reland.

https://uberchromegw.corp.google.com/i/chromeos.chrome/builders/tricky-tot-
chrome-pfq-informational/builds/1152
Project Member

Comment 22 by bugdroid1@chromium.org, Jun 17 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/874b2bd05deb6506a09463f970e5271e17579c9e

commit 874b2bd05deb6506a09463f970e5271e17579c9e
Author: Eric Caruso <ejcaruso@chromium.org>
Date: Wed Jun 15 19:54:07 2016

wifi: do not send random MAC request until supplicant appears

If supplicant hasn't appeared yet, wait to send the request to
enable random MAC until we have connected to it.

BUG= chromium:619615 , chromium:620395 
TEST=cbuildbot --remote tricky-tot-chrome-pfq-informational

Change-Id: I5d9dd915774d9eeaf6e25deaa35c3db1ee90351c

[modify] https://crrev.com/874b2bd05deb6506a09463f970e5271e17579c9e/wifi/wifi.cc

Status: Fixed (was: Assigned)
Status: Verified (was: Fixed)
Bulk verified
Project Member

Comment 25 by bugdroid1@chromium.org, Sep 6 2016

Labels: merge-merged-release-R52-8350.B
The following revision refers to this bug:
  https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/2a2d915bbe877fc9b9a999a2885afb5bade2c202

commit 2a2d915bbe877fc9b9a999a2885afb5bade2c202
Author: Eric Caruso <ejcaruso@chromium.org>
Date: Wed Jun 15 19:54:07 2016

wifi: do not send random MAC request until supplicant appears

If supplicant hasn't appeared yet, wait to send the request to
enable random MAC until we have connected to it.

BUG= chromium:619615 , chromium:620395 
TEST=cbuildbot --remote tricky-tot-chrome-pfq-informational

Change-Id: I5d9dd915774d9eeaf6e25deaa35c3db1ee90351c
(cherry picked from commit 874b2bd05deb6506a09463f970e5271e17579c9e)

[modify] https://crrev.com/2a2d915bbe877fc9b9a999a2885afb5bade2c202/wifi/wifi.cc

Project Member

Comment 26 by bugdroid1@chromium.org, Sep 6 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/aosp/platform/system/connectivity/shill/+/2a2d915bbe877fc9b9a999a2885afb5bade2c202

commit 2a2d915bbe877fc9b9a999a2885afb5bade2c202
Author: Eric Caruso <ejcaruso@chromium.org>
Date: Wed Jun 15 19:54:07 2016

wifi: do not send random MAC request until supplicant appears

If supplicant hasn't appeared yet, wait to send the request to
enable random MAC until we have connected to it.

BUG= chromium:619615 , chromium:620395 
TEST=cbuildbot --remote tricky-tot-chrome-pfq-informational

Change-Id: I5d9dd915774d9eeaf6e25deaa35c3db1ee90351c
(cherry picked from commit 874b2bd05deb6506a09463f970e5271e17579c9e)

[modify] https://crrev.com/2a2d915bbe877fc9b9a999a2885afb5bade2c202/wifi/wifi.cc

Sign in to add a comment