sdk bot: uploading perf data is blocked |
|||
Issue descriptioni guess we need to have some IP whitelisted or something ? https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8806 SDKPackage stage: 22:21:16: INFO: Generating manifest for new sdk [1;31m22:21:16: ERROR: Error when uploading perf data to the perf dashboard for test sdk. Traceback (most recent call last): File "/b/c/cbuild/repository/chromite/lib/perf_uploader.py", line 489, in UploadPerfValues formatted_data, dashboard=dashboard) File "/b/c/cbuild/repository/chromite/lib/retry_util.py", line 244, in GenericRetry return _run() File "/b/c/cbuild/repository/chromite/lib/retry_util.py", line 177, in _Wrapper ret = func(*args, **kwargs) File "/b/c/cbuild/repository/chromite/lib/retry_util.py", line 243, in _run return functor(*args, **kwargs) File "/b/c/cbuild/repository/chromite/lib/perf_uploader.py", line 342, in _SendToDashboard (e.code, e.msg, data_obj['data']), e) PerfUploadingError: HTTPError: 403 Forbidden for JSON [{"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/armv7a-cros-linux-gnueabihf", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 191258024.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3089341316.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_x86_64-cros-linux-gnu", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3281780180.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/arm-none-eabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 173737376.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_armv6j-cros-linux-gnueabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3277336472.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/armv6j-cros-linux-gnueabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 187995156.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/x86_64-cros-linux-gnu", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 192438864.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_arm-none-eabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3263078692.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/i686-pc-linux-gnu", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 189763300.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_aarch64-cros-linux-gnu", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3281803588.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/armv7m-cros-eabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 173404684.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_i686-pc-linux-gnu", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3279104616.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/armv7a-cros-linux-gnueabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 191362468.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_armv7a-cros-linux-gnueabihf", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3280599340.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/aarch64-cros-linux-gnu", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 192462272.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_armv7m-cros-eabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3262746000.0, "revision": 20180517144000}, {"units": "bytes", "master": "ChromeOSPerf", "supplemental_columns": {"a_default_rev": "r_cros_version"}, "error": 0, "test": "cbuildbot.sdk/cros-sdk-size/base_plus_armv7a-cros-linux-gnueabi", "higher_is_better": false, "bot": "cros-chromiumos-sdk", "value": 3280703784.0, "revision": 20180517144000}] caused by: HTTP Error 403: Forbidden[0m
,
May 18 2018
Reaching out to Chrome Perf team to check about whitelisting. Didn't find much info on it therefore we're not sure what is whitelisted or not. Odd that this just popped up now as that builder was moved to GCE in March: https://chrome-internal.git.corp.google.com/infra/infra_internal/+/b21b8b7f803aa5e813c0354c34d254cd5df279ec/ccompute/scripts/expectations/chromeos_central.json Will update, and document, what is whitelisted today and apply and missing updates. -- Mike
,
May 19 2018
i don't think it's new, just that someone noticed. Apr 13 failure: https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8700 Mar 14 failure: https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8600 Feb 27 failure: https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8570 Jan 17 failure: https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8450 Dec 24 failure: https://uberchromegw.corp.google.com/i/chromiumos/builders/chromiumos-sdk/builds/8350
,
May 21 2018
Good point. Thanks for pointing that out, Mike. That does help to identify the timing of the issue. -- Mike
,
May 21 2018
The whitelists have all been updated to allow for the following ranges (what we know today): Golo 74.125.248.64/27 GCE 104.196.64.0/24 104.154.124.0/24 104.154.125.0/24 104.154.112.0/24 (coming soon) Will keep an eye on upcoming builders to determine whether this has solved the upload issues. -- Mike
,
May 22 2018
I believe this is now resolved. Logs from a recent execution: 18:52:54: INFO: Successfully uploaded perf data to the perf dashboard for test sdk. Please let me know if you run into any further issues. -- Mike
,
May 22 2018
|
|||
►
Sign in to add a comment |
|||
Comment 1 by dgarr...@chromium.org
, May 18 2018