Fracas - Findit Pass JSON objects for explaining 'confidence score'.
Reported by
mimee@chromium.org,
May 5 2016
|
|||||||||
Issue descriptionFindit currently passes a paragraph of reasons, separated by \n. I recently learnt that \n is used to separate different reasons. Each reason is a string with a score associated. In the end, there is a summary sentence. It would be much better for display if these can be formatted in a dictionary, then Fracas frontend would have the flexibility to explain the scoring better, such as using a donut chart or some other customized view.
,
May 5 2016
+1. This would also be better when we migrate ClusterFuzz.
,
May 5 2016
For requests for change on Findit side, please add "Tools>Test>FindIt" so they will show up on our radar and we will prioritize. We stop using label "Findit" now.
,
May 20 2016
I have a proposal. Compared to the old message format, there are 2 changes: (1) Add "suspected_regression" and "suspected_regression_url". (2) Right now findit only return a string of reason, I think we can change this to be a metadata, so we can better display result. Replace "reason" with a dict, which has a field "changed_files". Since now the findit results seems a little confusing, after some discussion with TEs, I think we can display the findit results like below: Suspected regression range: "52.0.2740.0: 52.0.2741.0"(with changelog link) Suspected component: "Blink>DOM" Suspected project: "chromium" (circle) 88% COMMIT, CODE REVIEW, author@chromium.org Changed files: file1(with blame link): - #0, crashed in line 3, distance (LOC) 0 - #2, crashed in line 6, distance (LOC) 20 file 2(with blame link): - #5, crashed in line 99, distance (LOC) 30 A sample message would look like this: { "crash_identifiers": { "platform": "win", "chrome_version": "51.0.2695.1", "process_type": "renderer", "channel": "canary", "signature": "blink::FramePainter::paintContents" }, "client_id": "fracas", "result": { "suspected_regression_url": "https://...", "suspected_cls": [ { "reason": { "changed_files": [ [ "file_path1", { "frames": [ { "distance": 0, "crashed_lines": [ 2, 3, 4 ], "frame_index": 0 }, { "distance": 6, "crashed_lines": [ 4 ], "frame_index": 3 } ], "blame_url": "https://chromium.googlesource.org/..." } ], [ "file_path2", { "frames": [ { "distance": 0, "crashed_lines": [ 4 ], "frame_index": 0 } ], "blame_url": "https://chromium.googlesource.org/..." } ] ] }, "time": "2015-08-17 03:38:16", "author": "who@chromium.org", "url": "https://chromium.googlesource.com/chromium/src/+/commit-hash", "project_path": "third_party/pdfium", "review_url": "https://codereview.chromium.org/issue-number", "confidence": 0.6, "revision": "commit-hash" } ], "suspected_project": "chromium", "found": true, "suspected_regression": "52.0.2740.0: 52.0.2741.0", "suspected_components": [ "Blink>Paint" ] } }
,
May 20 2016
Any suggestion?
,
May 25 2016
What are "crashed_lines": [
2,
3,
4
?
,
May 26 2016
In a frame: #2 0x7ff9 in function(int) src/file.cc:87 'file.cc' crashed in line [87] in this frame. In clusterfuzz crash, a frame can be like this: #2 0x7ff9 in function(int) src/file.cc:87:2 This means 'file.cc' crashed in line range [87, 88, 89].
,
May 26 2016
Can you tell me what exactly substitutes in for the placeholder numbers? Would they just be numbers, or would they be more verbose?
,
May 26 2016
They would be just numbers
,
May 26 2016
,
May 27 2016
,
Jul 6 2016
mimee@, how do you think this proposal?
,
Jul 6 2016
I think it could work, potentially, though I am uncertain how all of this information can possibly be displayed in a clear fashion. Can we finalize on a design of the view, see what the sheriffs actually see and use, and go from there?
,
Jul 6 2016
Also line numbers don't matter unless you are constructing the code search URL. Are we doing that?
,
Jul 11 2016
As discussed offline, the updated the message format is listed in https://docs.google.com/document/d/1fDhJ2suyTsCLXukJwNDqtTAx_hK0hjHrOuUFFR_jDtE/edit#heading=h.o9pruw3u1jti
,
Jul 11 2016
,
Jul 21 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/bf641fc0f205214cc03c192f1e1473465630507f commit bf641fc0f205214cc03c192f1e1473465630507f Author: katesonia <katesonia@chromium.org> Date: Thu Jul 21 23:14:21 2016 [Findit] Pass changed files info to Fracas, 2 face design. Message format from Findit to Fracas (updated in red at the bottom of the document https://docs.google.com/document/d/1fDhJ2suyTsCLXukJwNDqtTAx_hK0hjHrOuUFFR_jDtE/edit#heading=h.o9pruw3u1jti) (1)Added changed files info to results (2)changed reason from a str to a list of tuple explaining the scores. Split previous aggregator.py into aggregators.py and aggregated_scorer.py. Add ChangedFilesAggregator. Fix some old tests due to the changes. BUG= 624928 , 609621 Review-Url: https://codereview.chromium.org/2157433002 [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/callstack.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/findit_for_crash.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/parse_util.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/results.py [add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/aggregated_scorer.py [delete] https://crrev.com/ed2d821a5d2acc248419b48cbf6a032a3a2a65f0/appengine/findit/crash/scorers/aggregator.py [add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/aggregators.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/min_distance.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/scorer.py [add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/aggregated_scorer_test.py [delete] https://crrev.com/ed2d821a5d2acc248419b48cbf6a032a3a2a65f0/appengine/findit/crash/scorers/test/aggregator_test.py [add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/aggregators_test.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/min_distance_test.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/top_frame_index_test.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/top_frame_index.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/callstack_test.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/crash_test_suite.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/findit_for_crash_test.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/parse_util_test.py [modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/results_test.py
,
Jul 29 2016
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra.git/+/e9c78ef32eb9f14eda83c20b8e47721b31ec3265 commit e9c78ef32eb9f14eda83c20b8e47721b31ec3265 Author: katesonia <katesonia@chromium.org> Date: Fri Jul 29 18:36:52 2016 [Findit]Post-process findit-to-fracas results. (1) Add feedback_url to crash analysis. (2) Delete 'reason' in each suspected cl. BUG= 624928 , 609621 Review-Url: https://codereview.chromium.org/2168173003 [modify] https://crrev.com/e9c78ef32eb9f14eda83c20b8e47721b31ec3265/appengine/findit/crash/fracas_crash_pipeline.py [modify] https://crrev.com/e9c78ef32eb9f14eda83c20b8e47721b31ec3265/appengine/findit/crash/test/fracas_crash_pipeline_test.py
,
Aug 8 2016
Findit can deploy the change once fracas side is ready.
,
Nov 30 2016
|
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 by mimee@chromium.org
, May 5 2016