New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 609621 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 609626
issue 624928



Sign in to add a comment

Fracas - Findit Pass JSON objects for explaining 'confidence score'.

Reported by mimee@chromium.org, May 5 2016

Issue description

Findit currently passes a paragraph of reasons, separated by \n. I recently learnt that \n is used to separate different reasons. Each reason is a string with a score associated. In the end, there is a summary sentence.

It would be much better for display if these can be formatted in a dictionary, then Fracas frontend would have the flexibility to explain the scoring better, such as using a donut chart or some other customized view.

 

Comment 1 by mimee@chromium.org, May 5 2016

Blocking: 609626
Cc: -aarya@google.com -kateso...@chromium.org infe...@chromium.org
Owner: kateso...@chromium.org
Status: Assigned (was: Unassigned)
+1. This would also be better when we migrate ClusterFuzz.

Comment 3 by st...@chromium.org, May 5 2016

Components: Tools>Test>FindIt
Labels: -Findit
For requests for change on Findit side, please add "Tools>Test>FindIt" so they will show up on our radar and we will prioritize. We stop using label "Findit" now.
Status: Started (was: Assigned)
I have a proposal.

Compared to the old message format, there are 2 changes:
(1) Add "suspected_regression" and "suspected_regression_url".
(2) Right now findit only return a string of reason, I think we can change this to be a metadata, so we can better display result. Replace "reason" with a dict, which has a field "changed_files".

Since now the findit results seems a little confusing, after some discussion with TEs, I think we can display the findit results like below:


Suspected regression range: "52.0.2740.0: 52.0.2741.0"(with changelog link)
Suspected component: "Blink>DOM"
Suspected project: "chromium"

(circle)  88%
COMMIT,  CODE REVIEW,  author@chromium.org
Changed files:
  file1(with blame link):
  - #0, crashed in line 3, distance (LOC) 0
  - #2, crashed in line 6, distance (LOC) 20
  file 2(with blame link):
  - #5, crashed in line 99, distance (LOC) 30



A sample message would look like this:

{
  "crash_identifiers": {
    "platform": "win",
    "chrome_version": "51.0.2695.1",
    "process_type": "renderer",
    "channel": "canary",
    "signature": "blink::FramePainter::paintContents"
  },
  "client_id": "fracas",
  "result": {
    "suspected_regression_url": "https://...",
    "suspected_cls": [
      {
        "reason": {
          "changed_files": [
            [
              "file_path1",
              {
                "frames": [
                  {
                    "distance": 0,
                    "crashed_lines": [
                      2,
                      3,
                      4
                    ],
                    "frame_index": 0
                  },
                  {
                    "distance": 6,
                    "crashed_lines": [
                      4
                    ],
                    "frame_index": 3
                  }
                ],
                "blame_url": "https://chromium.googlesource.org/..."
              }
            ],
            [
              "file_path2",
              {
                "frames": [
                  {
                    "distance": 0,
                    "crashed_lines": [
                      4
                    ],
                    "frame_index": 0
                  }
                ],
                "blame_url": "https://chromium.googlesource.org/..."
              }
            ]
          ]
        },
        "time": "2015-08-17 03:38:16",
        "author": "who@chromium.org",
        "url": "https://chromium.googlesource.com/chromium/src/+/commit-hash",
        "project_path": "third_party/pdfium",
        "review_url": "https://codereview.chromium.org/issue-number",
        "confidence": 0.6,
        "revision": "commit-hash"
      }
    ],
    "suspected_project": "chromium",
    "found": true,
    "suspected_regression": "52.0.2740.0: 52.0.2741.0",
    "suspected_components": [
      "Blink>Paint"
    ]
  }
}

Any suggestion?

Comment 6 by mimee@chromium.org, May 25 2016

What are "crashed_lines": [
                      2,
                      3,
                      4
?
In a frame: #2 0x7ff9 in function(int) src/file.cc:87

'file.cc' crashed in line [87] in this frame.

In clusterfuzz crash, a frame can be like this: #2 0x7ff9 in function(int) src/file.cc:87:2

This means 'file.cc' crashed in line range [87, 88, 89].


Comment 8 by mimee@chromium.org, May 26 2016

Can you tell me what exactly substitutes in for the placeholder numbers? Would they just be numbers, or would they be more verbose?
They would be just numbers

Comment 10 by mimee@chromium.org, May 26 2016

Cc: mmand...@chromium.org

Comment 11 by st...@chromium.org, May 27 2016

Blocking: 615262
mimee@, how do you think this proposal?
I think it could work, potentially, though I am uncertain how all of this information can possibly be displayed in a clear fashion. Can we finalize on a design of the view, see what the sheriffs actually see and use, and go from there?
Also line numbers don't matter unless you are constructing the code search URL. Are we doing that?
As discussed offline, the updated the message format is listed in https://docs.google.com/document/d/1fDhJ2suyTsCLXukJwNDqtTAx_hK0hjHrOuUFFR_jDtE/edit#heading=h.o9pruw3u1jti
Blocking: 624928
Project Member

Comment 17 by bugdroid1@chromium.org, Jul 21 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/bf641fc0f205214cc03c192f1e1473465630507f

commit bf641fc0f205214cc03c192f1e1473465630507f
Author: katesonia <katesonia@chromium.org>
Date: Thu Jul 21 23:14:21 2016

[Findit] Pass changed files info to Fracas, 2 face design.

Message format from Findit to Fracas (updated in red at the bottom of the document https://docs.google.com/document/d/1fDhJ2suyTsCLXukJwNDqtTAx_hK0hjHrOuUFFR_jDtE/edit#heading=h.o9pruw3u1jti)
(1)Added changed files info to results
(2)changed reason from a str to a list of tuple explaining the scores.

Split previous aggregator.py into aggregators.py and aggregated_scorer.py. Add ChangedFilesAggregator.

Fix some old tests due to the changes.

BUG= 624928 ,  609621 

Review-Url: https://codereview.chromium.org/2157433002

[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/callstack.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/findit_for_crash.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/parse_util.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/results.py
[add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/aggregated_scorer.py
[delete] https://crrev.com/ed2d821a5d2acc248419b48cbf6a032a3a2a65f0/appengine/findit/crash/scorers/aggregator.py
[add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/aggregators.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/min_distance.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/scorer.py
[add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/aggregated_scorer_test.py
[delete] https://crrev.com/ed2d821a5d2acc248419b48cbf6a032a3a2a65f0/appengine/findit/crash/scorers/test/aggregator_test.py
[add] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/aggregators_test.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/min_distance_test.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/test/top_frame_index_test.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/scorers/top_frame_index.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/callstack_test.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/crash_test_suite.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/findit_for_crash_test.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/parse_util_test.py
[modify] https://crrev.com/bf641fc0f205214cc03c192f1e1473465630507f/appengine/findit/crash/test/results_test.py

Project Member

Comment 18 by bugdroid1@chromium.org, Jul 29 2016

Status: Fixed (was: Started)
Findit can deploy the change once fracas side is ready.
Blocking: -615262

Sign in to add a comment