New issue
Advanced search Search tips

Issue 747974 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Oct 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: ----

Blocking:
issue 690921



Sign in to add a comment

Dashboard Upload Failing on Multiple Bots for Multiple Tests

Project Member Reported by ashleymarie@chromium.org, Jul 24 2017

Issue description

Dashboard Upload Failing

Builders failed on: 
- Android Nexus5 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5%20Perf
- Android Nexus5X Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5X%20Perf
- Android Nexus6 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf
- Android Nexus7v2 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf
- Android One Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Android%20One%20Perf
- Win 10 High-DPI Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Win%2010%20High-DPI%20Perf
- Mac Pro 10.11 Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Mac%20Pro%2010.11%20Perf
- Mac Retina Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Mac%20Retina%20Perf
- Win 7 ATI GPU Perf: 
  https://build.chromium.org/p/chromium.perf/builders/Win%207%20ATI%20GPU%20Perf


https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus5_Perf%2F264%2F%2B%2Frecipes%2Fsteps%2Fv8.runtimestats.browsing_mobile.reference_Dashboard_Upload%2F0%2Fstdout

Sending result 1 of 2 to dashboard.
HTTPError: 500. Reponse: 
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>500 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered an error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>
JSON: {"is_ref": false, "test_suite_name": "v8.runtimestats.browsing_mobile", "master": "ChromiumPerf", "versions": {"webk
... rest of json here ...
Error uploading to dashboard.
!@@@STEP_EXCEPTION@@@
step returned non-zero exit code: 1
 
Cc: nedngu...@google.com simonhatch@chromium.org
Labels: Pri-0
searching go/chromeperf-logs for:

path:/add_point status:500

yeilds a lot of deadline exceeded errors, almost always here:

File "/base/data/home/apps/s~chromeperf/clean-sullivan-7a8b1fc5.402644400137199241/dashboard/add_point.py", line 550, in _ConstructTestPathMap
    last_added_revision_entities = ndb.get_multi(last_added_revision_keys)

Ned, do you know of any changes to the v8.runtimestats tests recently?

Simon, any ideas?
Yes. Recently Mythri enabled more metrics for v8.runtimestats ( issue 690921 ). The idea is to deprecate the v8.browsing* benchmarks & always use runtimestats for v8 benchmarks*.
Blocking: 690921
Owner: mythria@chromium.org
Mythri: let's work together to figure out if we can reduce the number of metrics. Right now we went from ~3000 -> ~8000 metrics, which looks like about ~130 per page to ~350. Is it possible to prune some?

Simon also has some thoughts on making a dashboard change to allow us to upload more data.
Thanks for reverting. I will take a look at the metrics and see if we can remove some of them. 
It was my mistake. When I added metrics tracked by v8.browsing to v8.runtimestats.browsing benchmark I forgot to ignore some of the memory metrics and v8 metrics. So for all memory metrics v8.browse benchmark ignores std, count, min, sum and pct. There are several memory metrics (~30) so if we ignore these in v8.runtimestats benchmark as well the total number of metrics should come down significantly. I will upload a cl after double checking we don't increase the number of metrics by large amount.

Ok I've swept through the bots, looks like mostly android ones were still carrying the bad data so that's been deleted and I'm seeing uploads working now. Will check back in the morning to confirm.
Labels: -Pri-0 Pri-2
Lowering priority, since this isn't turning bots red.
This is now fixed. I enabled memory metric after ignoring data from memory:unknown_browser and also ignoring memory from different processes. Cl here: https://chromium-review.googlesource.com/738096.
Status: Fixed (was: Available)

Sign in to add a comment