New issue
Advanced search Search tips
Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Dec 14
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment
link

Issue 915336: Fail to collect coverage from web tests: --batch-size not recognized by run_web_tests

Reported by robertma@chromium.org, Dec 14 Project Member

Issue description

$ python tools/code_coverage/coverage.py blink_tests -wt -b out/coverage -o out/report -f third_party/blink

[2018-12-14 15:15:06,132 INFO] Running command: "python testing/xvfb.py python third_party/blink/tools/run_web_tests.py --additional-driver-flag=--no-sandbox --additional-env-var=LLVM_PROFILE_FILE=<llvm_profile_file_path> --batch-size=1 --child-processes=28 --disable-breakpad --no-show-results --skip-failing-tests --target=coverage --time-out-ms=30000", the output is redirected to "/usr/local/google/home/robertma/chromium/src/out/report/linux/logs/blink_tests_output.log".
[2018-12-14 15:15:06,444 WARNING] Command: "python testing/xvfb.py python third_party/blink/tools/run_web_tests.py --additional-driver-flag=--no-sandbox --additional-env-var=LLVM_PROFILE_FILE=/usr/local/google/home/robertma/chromium/src/out/report/linux/blink_tests.%4m.profraw --batch-size=1 --child-processes=28 --disable-breakpad --no-show-results --skip-failing-tests --target=coverage --time-out-ms=30000" exited with non-zero return code.
Traceback (most recent call last):
  File "tools/code_coverage/coverage.py", line 1036, in <module>
    sys.exit(Main())
  File "tools/code_coverage/coverage.py", line 989, in Main
    args.targets, commands, args.jobs)
  File "tools/code_coverage/coverage.py", line 273, in _CreateCoverageProfileDataForTargets
    targets, commands)
  File "tools/code_coverage/coverage.py", line 372, in _GetTargetProfDataPathsByExecutingCommands
    'does not crash. %s' % (target, FILE_BUG_MESSAGE))
AssertionError: Running target "blink_tests" failed to generate any profraw data file, please make sure the binary exists, is properly instrumented and does not crash. If it persists, please file a bug with the command you used, git revision and args.gn config here: https://bugs.chromium.org/p/chromium/issues/entry?components=Tools%3ECodeCoverage


====================
out/report/linux/logs/blink_tests_output.log:

Additional test environment:
    CHROME_DEVEL_SANDBOX=/opt/chromium/chrome_sandbox
    CHROME_HEADLESS=1
    LANG=en_US.UTF-8
Command: /usr/bin/python third_party/blink/tools/run_web_tests.py --additional-driver-flag=--no-sandbox --additional-env-var=LLVM_PROFILE_FILE=/usr/local/google/home/robertma/chromium/src/out/report/linux/blink_tests.%4m.profraw --batch-size=1 --child-processes=28 --disable-breakpad --no-show-results --skip-failing-tests --target=coverage --time-out-ms=30000

Usage: run_web_tests.py [options] [tests]

run_web_tests.py: error: no such option: --batch-size
 

Comment 1 by robertma@chromium.org, Dec 14

I think the flag to run_web_tests.py should be `-j 1` instead of `--batch-size=1`?

Comment 2 by robertma@chromium.org, Dec 14

Oops. `-j1` is a different thing. I think `--batch-size=1` should now be `--restart-shell-between-tests=always` (changed in https://crrev.com/c/1300236).

Comment 4 by mmoroz@google.com, Dec 14

Ooh, right, I think we've fixed it on the bots but not in the script.

Comment 5 by robertma@chromium.org, Dec 14

Owner: robertma@chromium.org
Status: Started (was: Untriaged)
Re #4: I'm doing some local experiments using the script which is how I found out.

Comment 6 by bugdroid1@chromium.org, Dec 14

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/339fc47c5cf415833d367b927935a522097fc771

commit 339fc47c5cf415833d367b927935a522097fc771
Author: Robert Ma <robertma@chromium.org>
Date: Fri Dec 14 21:22:42 2018

[Code Coverage] Fix invocation of run_web_tests

The flag --batch-size has been removed in https://crrev.com/c/1300236.
This CL removes the flag from coverage.py.

Bug:  915336 
Change-Id: I22e5ccd5f10bf55795442f87deda767a4954fd32
Reviewed-on: https://chromium-review.googlesource.com/c/1378812
Commit-Queue: Robert Ma <robertma@chromium.org>
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Reviewed-by: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#616827}
[modify] https://crrev.com/339fc47c5cf415833d367b927935a522097fc771/tools/code_coverage/coverage.py

Comment 7 by robertma@chromium.org, Dec 14

Status: Fixed (was: Started)

Sign in to add a comment