New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 773372 link

Starred by 1 user

Issue metadata

Status: Fixed
Merged: issue 772741
Owner:
Last visit > 30 days ago
Closed: Mar 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 3
Type: Bug

Blocking:
issue 772741



Sign in to add a comment

fetch_telemetry_binary_dependencies & fetch_benchmark_deps are super duper noisy

Project Member Reported by thakis@chromium.org, Oct 10 2017

Issue description

https://chromium-review.googlesource.com/708049 made us download telemetry binary deps by default. That's cool, but the script that downloads them are super spammy.

50% of the output of `gclient runhooks` is from this script.

WARNING:root:Dependency perfhost_trusty not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency gdb not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency battor_agent_binary not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency perfhost_precise not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency host_forwarder not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency md5sum_bin not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency perf not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency ipfw not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency tcpdump not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency purge_ashmem not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency push_apps_to_background_apk not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency avrdude_binary not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency memtrack_helper not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency file_poller not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency ipfw_mod not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency hprof-conv not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency device_forwarder not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency avconv not configured for platform mac_x86_64. Skipping prefetch.
WARNING:root:Dependency perfhost not configured for platform mac_x86_64. Skipping prefetch.


Please make it so that this script doesn't print anything when it's not doing anything. (See also other CLs on  issue 772741 .)
 

Comment 1 by thakis@chromium.org, Oct 10 2017

(I'd do this myself, but the script doing this is in something called "catapult" which I don't know how to commit to, and the code printing this was > 2 imports from the script that's run by runhooks)
Components: Build
I reverted the change for now. Let's try to fix this before landing it again.
(the revert wasn't specifically for this, it was for bug 773357).
Status: WontFix (was: Unconfirmed)
Closing as change was reverted.

Comment 5 by thakis@chromium.org, Oct 17 2017

Cc: erikc...@chromium.org
And the plan isn't to reland it?
Status: Assigned (was: WontFix)
Owner: nedngu...@google.com
The plan is to reland it only on certain types of bots that need it (which already executing the script today). Though I can certainly reduce the verbosity of this script's log.
Project Member

Comment 8 by bugdroid1@chromium.org, Oct 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/catapult/+/75257303afa9fc66dffde2e1397930628f5af9bd

commit 75257303afa9fc66dffde2e1397930628f5af9bd
Author: Nghia Nguyen <nednguyen@google.com>
Date: Tue Oct 17 11:09:10 2017

Make the log of dependency_manager less spammy

This remove the log which notify whether depdency is skipped due to no matching
platforms found for it. This almost always happens, and is usually benign, hence
this log can be removed.

Bug:  chromium:773372 
Change-Id: I5a6ce78a03152f473b41f3ed0962c978f01c32a7
Reviewed-on: https://chromium-review.googlesource.com/722042
Reviewed-by: Juan Antonio Navarro Pérez <perezju@chromium.org>
Commit-Queue: Ned Nguyen <nednguyen@google.com>

[modify] https://crrev.com/75257303afa9fc66dffde2e1397930628f5af9bd/dependency_manager/dependency_manager/manager.py

Mergedinto: 772741
Status: Duplicate (was: Assigned)
Status: Started (was: Duplicate)
This is better, but fetch_telemetry_binary_dependencies is still very noisy. Here's the runhooks output on a bot: https://logs.chromium.org/v/?s=chromium%2Fbb%2Ftryserver.chromium.win%2Fwin10_chromium_x64_rel_ng%2F111265%2F%2B%2Frecipes%2Fsteps%2Fgclient_runhooks__with_patch_%2F0%2Fstdout

Over 75% of that is due to the fetch_telemetry_binary_dependencies hook. Can we please make that step print less?
Hmhh, strange. I thought we remove all that logs. Lemme look into it.
Owner: nednguyen@chromium.org
Actually this time it's the log spam caused by "src/tools/perf/fetch_benchmark_deps.py -f"
Oooh sorry, I fail at reading log output :-( Want me to file a new bug and close this one again?
(`WARNING:root:Failed to import devil.android.forwarder: No module named fcntl` is still printed by fetch_telemetry_binary_dependencies which looks a bit worrying, so you may or may not want to fix that one line)
Summary: fetch_telemetry_binary_dependencies * fetch_benchmark_deps are super duper noisy (was: fetch_telemetry_binary_dependencies is super duper noisy)
No problem, I am working on fetch_benchmark_deps already, so lemme retitle this.
Summary: fetch_telemetry_binary_dependencies & fetch_benchmark_deps are super duper noisy (was: fetch_telemetry_binary_dependencies * fetch_benchmark_deps are super duper noisy)
Project Member

Comment 18 by bugdroid1@chromium.org, Mar 30 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8e2a2162848a0791eaf874748e291870dfdf1e17

commit 8e2a2162848a0791eaf874748e291870dfdf1e17
Author: Ned Nguyen <nednguyen@google.com>
Date: Fri Mar 30 09:13:51 2018

Move all debug print of fetch_benchmark_deps to log print & supress them by default

R=laszio@chromium.org
TBR=perezju@chromium.org

Bug:  773372 
Change-Id: I9b1e8b0df73ecf05cd323c305ddcc5a8011a035c
Reviewed-on: https://chromium-review.googlesource.com/984381
Commit-Queue: Ned Nguyen <nednguyen@google.com>
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Cr-Commit-Position: refs/heads/master@{#547133}
[modify] https://crrev.com/8e2a2162848a0791eaf874748e291870dfdf1e17/tools/perf/fetch_benchmark_deps.py
[modify] https://crrev.com/8e2a2162848a0791eaf874748e291870dfdf1e17/tools/perf/fetch_benchmark_deps_unittest.py

Status: Fixed (was: Started)
Components: Test>Telemetry
Components: -Speed>Telemetry

Sign in to add a comment