New issue
Advanced search Search tips

Issue 717500 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android
Pri: 1
Type: Bug


Previous locations:
webrtc:7546


Sign in to add a comment

WebRTC Android perf bots failing in "gclient runhooks" due to problem with FireFox download

Reported by lliuu@webrtc.org, May 1 2017

Issue description

Temp workaround submitted in https://codereview.chromium.org/2858473002/
I have triggered new builds for the failing bots. 

It seems something has changed regarding the download harness provided by Mozilla. Possibly we have to update it. It thinks it shall find a build named 2017-04-28-10-04-38 at https://archive.mozilla.org/pub/firefox/nightly/2017/04/ but there isn't one such.

Next thing to look at would be to find the last successful download, which should be ~3 days before the first failing one.
Project: chromium
Moved issue webrtc:7546 to now be  issue chromium:717500 .
Components: Infra>Client>WebRTC
Owner: kjellander@chromium.org

Comment 4 by nisse@chromium.org, May 2 2017

Cc: nisse@chromium.org
Labels: -Pri-2 OS-Linux Pri-1
Owner: ehmaldonado@chromium.org
Adding Linux since this also affects:
https://build.chromium.org/p/chromium.webrtc/builders/Linux%20Tester
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Linux%20Tester
(those are the only actual users of the downloaded Firefox)

Edward, are you up for some Python debugging?

The firefox download still fails today, and we'll have to fix it today or extend the number of days we allow an old build even further to avoid breaking runhooks again.

Recent failure:
https://build.chromium.org/p/chromium.webrtc/builders/Linux%20Tester/builds/29047/steps/gclient%20runhooks/logs/stdio

________ running '/usr/bin/python webrtc.DEPS/download_firefox_nightly.py --clean-old-firefox-archives --target-dir firefox-nightly' in '/b/c/b/Linux_Tester'
 INFO | Retrieving the build status file from https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/
/b/build/third_party/requests_2_10_0/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/b/build/third_party/requests_2_10_0/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
 INFO | Retrieving list of builds from https://archive.mozilla.org/pub/firefox/nightly/2017/05/
Failed to download firefox: Folder for builds on 2017-05-02-10-04-21 has not been found: https://archive.mozilla.org/pub/firefox/nightly/2017/05/.
Using firefox-nightly/2017-04-24-10-03-13-mozilla-central-firefox-55.0a1.en-US.linux-x86_64.tar.bz2 instead; it is 7 days old.
Extracted firefox-nightly/2017-04-24-10-03-13-mozilla-central-firefox-55.0a1.en-US.linux-x86_64.tar.bz2
Hook '/usr/bin/python webrtc.DEPS/download_firefox_nightly.py --clean-old-firefox-archives --target-dir firefox-nightly' took 19.03 secs

It's not an error since we allow 7 (used to be 3) days old archives to be re-used:
https://chromium.googlesource.com/chromium/deps/webrtc/webrtc.DEPS/+/master/download_firefox_nightly.py#97

I suspect this might get resolved if we just update the mozdownload package that we use (DEPS pinned here: https://chromium.googlesource.com/chromium/deps/webrtc/webrtc.DEPS/+/master/DEPS#12).
We've been manually dumping it there. We might be able to just update it with a newer version from https://github.com/mozilla/mozdownload.
If that works, after fixing this bug, the obvious next step would be to file a Infra>Git>Admin ticket to get that repo (and dependent repos) mirrored by Chrome infra and update our DEPS file to point at that mirror instead of the code copy in https://chromium.googlesource.com/chromium/deps/mozdownload

Finally, I also created  issue 717870  to address improving how we handle these special resources long term.
I can take a look at this and  issue 717870  :)
Cc: jansson@chromium.org
Even using a version installed by https://github.com/mozilla/mozdownload fails.
i.e. running
$ mozdownload --type=daily 
fails.
Can you look for if a bug exists at Mozilla's bugzilla database? or google search for the error message?
It's a long bug, but I hope you can find something in there to patch our copy of mozdownload?
It's not an issue with mozdownload.
It's an issue with the Mozilla build system somewhere. It reports the wrong build IDs.
Status: Fixed (was: Assigned)

Sign in to add a comment