Coverage taint tracking: provide information on which targets have touched a particular source file |
|||||||
Issue descriptionIdeally, we'd like to see that information for every line or even every region, but that sounds like a huge computational task. It's still worth to explore though. At least, we need to see which targets affect each source file. An example of use: 1) I've noticed some suspicious code laying deeply inside blink or some other component 2) I've checked code coverage and realized that we have fuzzers that exercise that code 3) I want to know which fuzzer is doing that in order to improve it or debug it Not a priority right now, but would be a very good improvement so we'll do it in Q2.
,
May 11 2018
,
May 16 2018
,
May 17 2018
I'm imagining a UI that looks like this: - On the dashboard for foo.cc, we have a selectable dropdown with all the targets that touch this file. - Each target has a separate indexed profile associated with it. - When a target it selected, the UI updates line/region hits based on adding that target - By default, all targets are selected. Of course, this means we would need to keep N indexed profiles handy for N = # total targets. Still, it would be extremely useful. For instance, the inverse of your example in the original comment is more useful to me. I have some code that is fully exercised by fuzzers, but not exercised at all in tests. That is something that we currently can't express in the UI but is very important to know!
,
May 17 2018
csharrison@, your proposal sounds good to me! We will start with a non clickable list of targets. Later on, when we implement another solution for reports representation it will be possible to switch from the aggregated report to another one produced by a single target. As of now, there is a secret trick, you can replace revision number in any URL (e.g. https://chromium-coverage.appspot.com/reports/559420/linux/chromium/src/report.html) with "%REVISION%_fuzzers_only, and see the report produced by fuzz targets (without the tests), e.g. https://chromium-coverage.appspot.com/reports/559420_fuzzers_only/linux/chromium/src/report.html
,
Nov 9
I think this is pretty doable now and should not require any LLVM changes. I'm drafting the plan below and unassigning myself for now, just in case someone else is interested in implementing this. 0) We generate and upload summary only JSON files for fuzz targets: https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/scripts/run_single_fuzz_target.bash#74 1) We should add the same thing to the tests runner and provide summary only JSON files for individual test targets: https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/scripts/run_test_targets.bash#42 2) In a cron task which processes new reports (https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/src/tasks/tasks.go#42), we would need to loop through those individual JSON files 3) From each file we would extract the list of file paths, each of which needs to be converted to a list of paths including all parent directories (e.g. "net/http/some_code.cc" -> ["net/http/some_code.cc", "net/http", "net"]) 4) For every path in the list we would need to add a record of the following format: (INTEGER revision_number, STRING target_name, STRING path, BOOLEAN is_coverage_non_zero [or INTEGER / DOUBLE number to store the actual coverage number, why not]) to the database* * as for the database I suspect that Cloud SQL (https://cloud.google.com/sql/docs/) would be a better choice compared to Datastore or BigQuery, but this would be a good exploration task and important design decision which needs more thought 5) Having the data described above in the database, we would need to add an API endpoint that would return a list of targets affecting a particular file / directory for a given revision (https://chrome-internal.googlesource.com/chrome/tools/code-coverage/+/eae16cb3b0fcbbde29410be202c12917998df5e9/src/api/) 6) Add a front-end button or something else that would call that API endpoint and surface the list of targets to the user when they're looking at a particular file / directory
,
Nov 26
,
Jan 3
|
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by mmoroz@chromium.org
, Apr 14 2018Owner: mmoroz@chromium.org
Status: Assigned (was: Untriaged)