New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 856189 link

Starred by 4 users
Status: Fixed
Owner:
█ wnwen@chromium.org
OOO until Feb 4th
Closed: Nov 19
Cc:
hua...@chromium.org
agrieve@chromium.org
estevenson@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

android_lint is too slow

Project Member Reported by agrieve@chromium.org, Jun 25 2018 
A one-line change to base_java results in a 14 second android_lint step on my machine. base_java has few dependencies and no resources, so why is it so slow!? Worth investigating...

Compiling is closer to 2-3 seconds, so this will be a ~10 second improvement if it can be made a lot faster.

Comment 1 by agrieve@chromium.org, Jul 4

Owner: hua...@chromium.org
Status: Assigned (was: Available)
Our wrapper script is here:
https://cs.chromium.org/chromium/src/build/android/gyp/lint.py

I've been using ninja -v to get the lint command-line for a target, then running the command-line directly to take ninja out of the picture.

ninja base:base_java -v

Comment 2 by wnwen@chromium.org, Aug 27

Cc: -wnwen@chromium.org agrieve@chromium.org hua...@chromium.org
Labels: -Pri-3 Pri-2
Owner: wnwen@chromium.org
Taking this one back as I have a project proposal to work on it.

Comment 3 by estevenson@chromium.org, Sep 12 

Stumbled across b/71868725 tracking a lint perf regression

Comment 4 by wnwen@chromium.org, Oct 30 

ninja -C out/Debug base:base_java -j 1120 -l 56
ninja: Entering directory `out/Debug'
[1/1] Regenerating ninja files
[466/466] STAMP obj/base/base_java.stamp
    Longest build steps:
           0.9 weighted s to build gen/third_party/auto/auto_common_java.javac.jar, gen/third_party/auto/auto_common_java.javac.jar.md5.stamp, gen/third_party/auto/auto_common_java.javac.jar.info (1.6 s CPU time)
           0.9 weighted s to build gen/third_party/auto/auto_service_processor.javac.jar, gen/third_party/auto/auto_service_processor.javac.jar.md5.stamp, gen/third_party/auto/auto_service_processor.javac.jar.info (0.9 s CPU time)
           1.0 weighted s to build gen/build/android/prepare_android_lint_cache/result.xml, gen/build/android/prepare_android_lint_cache/config.xml (3.6 s CPU time)
           1.0 weighted s to build gen/base/base_java.dex.jar (2.0 s CPU time)
           1.1 weighted s to build gen/third_party/android_deps/com_android_support_support_core_ui_java__classes.dex.jar (2.2 s CPU time)
           1.2 weighted s to build gen/tools/android/errorprone_plugin/errorprone_plugin_java.javac.jar, gen/tools/android/errorprone_plugin/errorprone_plugin_java.javac.jar.md5.stamp, gen/tools/android/errorprone_plugin/errorprone_plugin_java.javac.jar.info (1.2 s CPU time)
           3.0 weighted s to build gen/third_party/android_deps/com_android_support_support_fragment_java__classes.dex.jar (3.0 s CPU time)
           3.7 weighted s to build gen/third_party/android_deps/com_android_support_support_compat_java__classes.dex.jar (3.7 s CPU time)
           5.2 weighted s to build gen/base/base_java__lint/result.xml, gen/base/base_java__lint/config.xml (6.9 s CPU time)
           6.0 weighted s to build gen/base/base_java.javac.jar, gen/base/base_java.javac.jar.md5.stamp, gen/base/base_java.javac.jar.info (6.0 s CPU time)
    Time by build-step type:
           1.5 s weighted time to generate 12 .srcjar files (2.5 s CPU time)
           1.8 s weighted time to generate 51 .o files (32.1 s CPU time)
           7.7 s weighted time to generate 4 .xml files (15.3 s CPU time)
          11.9 s weighted time to generate 9 .info files (17.8 s CPU time)
          21.3 s weighted time to generate 105 .jar files (37.3 s CPU time)
    44.9 s weighted time (109.0 s CPU time, 2.4x parallelism)
    466 build steps completed, average of 10.39/s

Comment 5 by wnwen@chromium.org, Oct 30 

That was a clean build. The following is after a one byte change to ContextUtils.java:
ninja -C out/Debug base:base_java -j 1120 -l 56
ninja: Entering directory `out/Debug'
[18/18] STAMP obj/base/base_java__analysis.stamp
    Longest build steps:
           0.0 weighted s to build obj/base/base_java__lint.stamp (0.0 s CPU time)
           0.0 weighted s to build lib.java/base/base_java.jar (0.0 s CPU time)
           0.0 weighted s to build lib.java/base/base_java.interface.jar (0.1 s CPU time)
           0.0 weighted s to build obj/base/base_java__process_prebuilt-filtered.jar (0.1 s CPU time)
           0.1 weighted s to build gen/base/base_java_aidl.srcjar (0.1 s CPU time)
           0.2 weighted s to build obj/base/base_java__process_prebuilt-bytecode-rewritten.jar (0.5 s CPU time)
           0.4 weighted s to build obj/base/base_java__process_prebuilt-desugar.jar (0.8 s CPU time)
           0.9 weighted s to build gen/base/base_java.dex.jar (1.9 s CPU time)
           5.5 weighted s to build gen/base/base_java__lint/result.xml, gen/base/base_java__lint/config.xml (7.2 s CPU time)
           5.9 weighted s to build gen/base/base_java.javac.jar, gen/base/base_java.javac.jar.md5.stamp, gen/base/base_java.javac.jar.info (5.9 s CPU time)
    Time by build-step type:
           0.0 s weighted time to generate 9 .stamp files (0.0 s CPU time)
           0.1 s weighted time to generate 1 .srcjar files (0.1 s CPU time)
           1.7 s weighted time to generate 6 .jar files (3.3 s CPU time)
           5.5 s weighted time to generate 1 .xml files (7.2 s CPU time)
           5.9 s weighted time to generate 1 .info files (5.9 s CPU time)
    13.2 s weighted time (16.6 s CPU time, 1.3x parallelism)
    18 build steps completed, average of 1.36/s

Comment 6 by wnwen@chromium.org, Oct 30 

Example full build again but not depending on compile_java or jar_path/classpath.

ninja -C out/Debug base:base_java -j 1120 -l 56
ninja: Entering directory `out/Debug'
[1/1] Regenerating ninja files
[466/466] STAMP obj/base/base_java.stamp
    Longest build steps:
           0.8 weighted s to build gen/third_party/android_deps/android_arch_lifecycle_runtime_java__classes.dex.jar (0.8 s CPU time)
           0.9 weighted s to build gen/build/android/buildhooks/build_hooks_java__lint/result.xml, gen/build/android/buildhooks/build_hooks_java__lint/config.xml (2.4 s CPU time)
           1.1 weighted s to build gen/tools/android/errorprone_plugin/errorprone_plugin_java.javac.jar, gen/tools/android/errorprone_plugin/errorprone_plugin_java.javac.jar.md5.stamp, gen/tools/android/errorprone_plugin/errorprone_plugin_java.javac.jar.info (1.2 s CPU time)
           1.1 weighted s to build gen/build/android/prepare_android_lint_cache/result.xml, gen/build/android/prepare_android_lint_cache/config.xml (3.5 s CPU time)
           1.7 weighted s to build gen/third_party/android_deps/com_android_support_support_core_ui_java__classes.dex.jar (3.0 s CPU time)
           1.8 weighted s to build gen/base/base_java.dex.jar (1.8 s CPU time)
           3.0 weighted s to build gen/third_party/android_deps/com_android_support_support_fragment_java__classes.dex.jar (3.0 s CPU time)
           3.0 weighted s to build gen/base/base_java.javac.jar, gen/base/base_java.javac.jar.md5.stamp, gen/base/base_java.javac.jar.info (6.1 s CPU time)
           3.3 weighted s to build gen/base/base_java__lint/result.xml, gen/base/base_java__lint/config.xml (6.7 s CPU time)
           3.8 weighted s to build gen/third_party/android_deps/com_android_support_support_compat_java__classes.dex.jar (3.8 s CPU time)
    Time by build-step type:
           1.2 s weighted time to generate 51 .o files (16.5 s CPU time)
           1.6 s weighted time to generate 12 .srcjar files (2.6 s CPU time)
           6.2 s weighted time to generate 4 .xml files (15.0 s CPU time)
           7.5 s weighted time to generate 9 .info files (17.5 s CPU time)
          23.3 s weighted time to generate 105 .jar files (38.7 s CPU time)
    40.6 s weighted time (94.2 s CPU time, 2.3x parallelism)
    466 build steps completed, average of 11.47/s

After one byte change:
ninja -C out/Debug base:base_java -j 1120 -l 56
ninja: Entering directory `out/Debug'
[18/19] ACTION //base:base_java(//build/toolchain/android:android_clang_arm)
    Longest build steps:
           0.0 weighted s to build obj/base/base_java__dex.stamp (0.0 s CPU time)
           0.0 weighted s to build gen/base/base_java_aidl.srcjar (0.1 s CPU time)
           0.0 weighted s to build lib.java/base/base_java.jar (0.0 s CPU time)
           0.1 weighted s to build lib.java/base/base_java.interface.jar (0.1 s CPU time)
           0.1 weighted s to build obj/base/base_java__process_prebuilt-filtered.jar (0.1 s CPU time)
           0.2 weighted s to build obj/base/base_java__process_prebuilt-bytecode-rewritten.jar (0.4 s CPU time)
           0.7 weighted s to build obj/base/base_java__process_prebuilt-desugar.jar (0.7 s CPU time)
           1.8 weighted s to build gen/base/base_java.dex.jar (1.8 s CPU time)
           3.0 weighted s to build gen/base/base_java.javac.jar, gen/base/base_java.javac.jar.md5.stamp, gen/base/base_java.javac.jar.info (6.0 s CPU time)
           3.3 weighted s to build gen/base/base_java__lint/result.xml, gen/base/base_java__lint/config.xml (6.6 s CPU time)
    Time by build-step type:
           0.0 s weighted time to generate 9 .stamp files (0.0 s CPU time)
           0.0 s weighted time to generate 1 .srcjar files (0.1 s CPU time)
           2.9 s weighted time to generate 6 .jar files (3.2 s CPU time)
           3.0 s weighted time to generate 1 .info files (6.0 s CPU time)
           3.3 s weighted time to generate 1 .xml files (6.6 s CPU time)
    9.2 s weighted time (15.8 s CPU time, 1.7x parallelism)
    18 build steps completed, average of 1.95/s
Project Member

Comment 7 by bugdroid1@chromium.org, Nov 15 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3943e93ebe81d301d080e479f16a8b9d9e347930

commit 3943e93ebe81d301d080e479f16a8b9d9e347930
Author: Peter Wen <wnwen@chromium.org>
Date: Thu Nov 15 16:07:30 2018

Android: Remove javac dep for lint

Remove android_lint targets' dependency on the corresponding
compile_java target. This allows javac and lint to run concurrently.

Add srcjar_deps for android_lint target since R.java is necessary to
determine which resources are used. These are no longer transitively
depended upon through compile_java.

Bug:  856189 
Change-Id: If8b8abb1796bd50eb602443a36e3a198c3d35160
Reviewed-on: https://chromium-review.googlesource.com/c/1308100
Commit-Queue: Peter Wen <wnwen@chromium.org>
Reviewed-by: agrieve <agrieve@chromium.org>
Cr-Commit-Position: refs/heads/master@{#608384}
[modify] https://crrev.com/3943e93ebe81d301d080e479f16a8b9d9e347930/build/android/lint/suppressions.xml
[modify] https://crrev.com/3943e93ebe81d301d080e479f16a8b9d9e347930/build/config/android/internal_rules.gni
Project Member

Comment 8 by bugdroid1@chromium.org, Nov 15 

The following revision refers to this bug:
  https://webrtc.googlesource.com/src.git/+/25a3a97f591a6b6184b8a17826228830ff246980

commit 25a3a97f591a6b6184b8a17826228830ff246980
Author: Artem Titarenko <artit@webrtc.org>
Date: Thu Nov 15 19:48:53 2018

Android: ignore LintError for absent class files

This is required to support changes in build/config repo
see https://chromium-review.googlesource.com/c/chromium/src/+/1308100

Bug:  chromium:856189 
Change-Id: I9571c52d46ee57e8d7aa45d17301529bd16d56d5
Reviewed-on: https://webrtc-review.googlesource.com/c/111102
Reviewed-by: Oleh Prypin <oprypin@webrtc.org>
Commit-Queue: Artem Titarenko <artit@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#25664}
[modify] https://crrev.com/25a3a97f591a6b6184b8a17826228830ff246980/tools_webrtc/android/suppressions.xml
Project Member

Comment 9 by bugdroid1@chromium.org, Nov 16 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/812df6877cdf582a8ae242bfd4efc0db2c344a48

commit 812df6877cdf582a8ae242bfd4efc0db2c344a48
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Fri Nov 16 14:49:19 2018

Roll src/third_party/webrtc 302134240b5d..7dbb7c311f00 (6 commits)

https://webrtc.googlesource.com/src.git/+log/302134240b5d..7dbb7c311f00


git log 302134240b5d..7dbb7c311f00 --date=short --no-merges --format='%ad %ae %s'
2018-11-16 mbonadei@webrtc.org Adding missing build target for audio_device_default.
2018-11-16 srte@webrtc.org Removes templating from CompositeMediaEngine.
2018-11-16 srte@webrtc.org Adds interfaces for audio and video engines.
2018-11-16 nisse@webrtc.org Tweak ChannelSend interface, to make it closer to ChannelSendProxy
2018-11-16 nisse@webrtc.org Delete class ChannelReceiveProxy.
2018-11-15 artit@webrtc.org Android: ignore LintError for absent class files


Created with:
  gclient setdep -r src/third_party/webrtc@7dbb7c311f00

The AutoRoll server is located here: https://autoroll.skia.org/r/webrtc-chromium-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

CQ_INCLUDE_TRYBOTS=luci.chromium.try:linux_chromium_archive_rel_ng;luci.chromium.try:mac_chromium_archive_rel_ng

BUG= chromium:856189 
TBR=webrtc-chromium-sheriffs-robots@google.com

Change-Id: Idb96613892a627f2dea77c1e306d193bf29cc53c
Reviewed-on: https://chromium-review.googlesource.com/c/1340145
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#608775}
[modify] https://crrev.com/812df6877cdf582a8ae242bfd4efc0db2c344a48/DEPS

Comment 10 by wnwen@chromium.org, Nov 19

Status: Fixed (was: Assigned)
Currently lint is fully parallel for one-line java file changes (at least in base:base_java). Closing this out until we find a way to speed up javac in general.

Lint disabled:
/usr/local/google/home/wnwen/lib/depot_tools/ninja -C out/Debug base:base_java -j 1120
ninja: Entering directory `out/Debug'
[15/16] ACTION //base:base_java(//build/toolchain/android:android_clang_arm)
    Longest build steps:
           0.0 weighted s to build obj/base/base_java__process_prebuilt__filter.stamp (0.0 s CPU time)
           0.0 weighted s to build obj/base/base_java__compile_java.stamp (0.0 s CPU time)
           0.0 weighted s to build lib.java/base/base_java.jar (0.0 s CPU time)
           0.1 weighted s to build lib.java/base/base_java.interface.jar (0.1 s CPU time)
           0.1 weighted s to build obj/base/base_java__process_prebuilt-filtered.jar (0.1 s CPU time)
           0.1 weighted s to build gen/base/base_java_aidl.srcjar (0.1 s CPU time)
           0.7 weighted s to build obj/base/base_java__process_prebuilt-bytecode-rewritten.jar (0.7 s CPU time)
           0.7 weighted s to build obj/base/base_java__process_prebuilt-desugar.jar (0.7 s CPU time)
           1.8 weighted s to build gen/base/base_java.dex.jar (1.8 s CPU time)
           6.0 weighted s to build gen/base/base_java.javac.jar, gen/base/base_java.javac.jar.md5.stamp, gen/base/base_java.javac.jar.info (6.0 s CPU time)
    Time by build-step type:
           0.0 s weighted time to generate 7 .stamp files (0.0 s CPU time)
           0.1 s weighted time to generate 1 .srcjar files (0.1 s CPU time)
           3.4 s weighted time to generate 6 .jar files (3.4 s CPU time)
           6.0 s weighted time to generate 1 .info files (6.0 s CPU time)
    9.4 s weighted time (9.4 s CPU time, 1.0x parallelism)
    15 build steps completed, average of 1.59/s

Lint enabled:
/usr/local/google/home/wnwen/lib/depot_tools/ninja -C out/Debug base:base_java -j 1120
ninja: Entering directory `out/Debug'
[1/1] Regenerating ninja files
[18/19] ACTION //base:base_java(//build/toolchain/android:android_clang_arm)
    Longest build steps:
           0.0 weighted s to build obj/base/base_java__dex.stamp (0.0 s CPU time)
           0.0 weighted s to build lib.java/base/base_java.jar (0.0 s CPU time)
           0.0 weighted s to build obj/base/base_java__process_prebuilt-filtered.jar (0.1 s CPU time)
           0.1 weighted s to build lib.java/base/base_java.interface.jar (0.1 s CPU time)
           0.1 weighted s to build gen/base/base_java_aidl.srcjar (0.1 s CPU time)
           0.3 weighted s to build obj/base/base_java__process_prebuilt-bytecode-rewritten.jar (0.7 s CPU time)
           0.3 weighted s to build obj/base/base_java__process_prebuilt-desugar.jar (0.7 s CPU time)
           1.5 weighted s to build gen/base/base_java.dex.jar (1.8 s CPU time)
           2.9 weighted s to build gen/base/base_java.javac.jar, gen/base/base_java.javac.jar.md5.stamp, gen/base/base_java.javac.jar.info (5.8 s CPU time)
           3.9 weighted s to build gen/base/base_java__lint/result.xml, gen/base/base_java__lint/config.xml (7.8 s CPU time)
    Time by build-step type:
           0.0 s weighted time to generate 9 .stamp files (0.0 s CPU time)
           0.1 s weighted time to generate 1 .srcjar files (0.1 s CPU time)
           2.3 s weighted time to generate 6 .jar files (3.3 s CPU time)
           2.9 s weighted time to generate 1 .info files (5.8 s CPU time)
           3.9 s weighted time to generate 1 .xml files (7.8 s CPU time)
    9.3 s weighted time (17.1 s CPU time, 1.8x parallelism)
    18 build steps completed, average of 1.94/s
►

Sign in to add a comment