New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 827220 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug

Blocked on:
issue 908638



Sign in to add a comment

Coverage taint tracking: provide information on which targets have touched a particular source file

Project Member Reported by mmoroz@chromium.org, Mar 29 2018

Issue description

Ideally, we'd like to see that information for every line or even every region, but that sounds like a huge computational task. It's still worth to explore though.

At least, we need to see which targets affect each source file. An example of use:

1) I've noticed some suspicious code laying deeply inside blink or some other component

2) I've checked code coverage and realized that we have fuzzers that exercise that code

3) I want to know which fuzzer is doing that in order to improve it or debug it


Not a priority right now, but would be a very good improvement so we'll do it in Q2.
 

Comment 1 by mmoroz@chromium.org, Apr 14 2018

Labels: -Pri-2 Pri-1
Owner: mmoroz@chromium.org
Status: Assigned (was: Untriaged)

Comment 2 by mmoroz@chromium.org, May 11 2018

Summary: Coverage taint tracking: provide information on which targets have touched a particular source file (was: Provide information on which targets have touched a particular source)

Comment 3 by mmoroz@chromium.org, May 16 2018

Labels: Coverage-v2-Blocker
Cc: csharrison@chromium.org
I'm imagining a UI that looks like this:
- On the dashboard for foo.cc, we have a selectable dropdown with all the targets 
  that touch this file.

- Each target has a separate indexed profile associated with it.

- When a target it selected, the UI updates line/region hits based on adding
  that target

- By default, all targets are selected.

Of course, this means we would need to keep N indexed profiles handy for N = # total targets. Still, it would be extremely useful.

For instance, the inverse of your example in the original comment is more useful to me. I have some code that is fully exercised by fuzzers, but not exercised at all in tests. That is something that we currently can't express in the UI but is very important to know!

Comment 5 by mmoroz@chromium.org, May 17 2018

csharrison@, your proposal sounds good to me! We will start with a non clickable list of targets. Later on, when we implement another solution for reports representation it will be possible to switch from the aggregated report to another one produced by a single target.

As of now, there is a secret trick, you can replace revision number in any URL (e.g. https://chromium-coverage.appspot.com/reports/559420/linux/chromium/src/report.html) with "%REVISION%_fuzzers_only, and see the report produced by fuzz targets (without the tests), e.g. https://chromium-coverage.appspot.com/reports/559420_fuzzers_only/linux/chromium/src/report.html
Owner: ----
Status: Available (was: Assigned)
I think this is pretty doable now and should not require any LLVM changes. I'm drafting the plan below and unassigning myself for now, just in case someone else is interested in implementing this.

0) We generate and upload summary only JSON files for fuzz targets: https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/scripts/run_single_fuzz_target.bash#74

1) We should add the same thing to the tests runner and provide summary only JSON files for individual test targets: https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/scripts/run_test_targets.bash#42

2) In a cron task which processes new reports (https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/src/tasks/tasks.go#42), we would need to loop through those individual JSON files

3) From each file we would extract the list of file paths, each of which needs to be converted to a list of paths including all parent directories (e.g. "net/http/some_code.cc" -> ["net/http/some_code.cc", "net/http", "net"])

4) For every path in the list we would need to add a record of the following format: (INTEGER revision_number, STRING target_name, STRING path, BOOLEAN is_coverage_non_zero [or INTEGER / DOUBLE number to store the actual coverage number, why not]) to the database*

* as for the database I suspect that Cloud SQL (https://cloud.google.com/sql/docs/) would be a better choice compared to Datastore or BigQuery, but this would be a good exploration task and important design decision which needs more thought

5) Having the data described above in the database, we would need to add an API endpoint that would return a list of targets affecting a particular file / directory for a given revision (https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/src/api/)

6) Add a front-end button or something else that would call that API endpoint and surface the list of targets to the user when they're looking at a particular file / directory


Blockedon: 908638
Blockedon: -759794
Blocking: 759794
Blocking: -759794

Sign in to add a comment