New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 682580 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 707443
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: ----



Sign in to add a comment

Duplicate data points in MasterFlakeAnalysis.data_points

Project Member Reported by st...@chromium.org, Jan 19 2017

Issue description

Not sure why, but for quite a few cases, there are duplicate data points for the suspected build number. This affects the confidence score.

One example is : https://stgao-dot-findit-for-me.appspot.com/waterfall/check-flake?key=ag9zfmZpbmRpdC1mb3ItbWVytAELEhdNYXN0ZXJGbGFrZUFuYWx5c2lzUm9vdCJ-Y2hyb21pdW0ubWFjL01hYzEwLjkgVGVzdHMgKGRiZykvMzAxNDQvdW5pdF90ZXN0cyBvbiBNYWMtMTAuOS9SR1Z6YTNSdmNFVnVaMkZuWlcxbGJuUlRaWEoyYVdObFZHVnpkQzVVWlhOMFZHbHRaVzkxZEVScGMyTnZkVzUwDAsSE01hc3RlckZsYWtlQW5hbHlzaXMYAQw

findit.regressedBuildNumber = 30124;
findit.passRates = [{"build_number": 30117, "commit_position": 418349, "git_hash": "7532c960faf5f59ef7031e4488fce36b6e1c1cf0", "pass_rate": -1.0, "previous_build_commit_position": 418347, "previous_build_git_hash": "873f27d84aaf55a3ee7ccfb0583639464efaf7bf", "task_id": null, "try_job_url": null}, {"build_number": 30118, "commit_position": 418356, "git_hash": "54279516e75ad5f5d6d1e94d26d65930cfa9dbdc", "pass_rate": -1.0, "previous_build_commit_position": 418349, "previous_build_git_hash": "7532c960faf5f59ef7031e4488fce36b6e1c1cf0", "task_id": null, "try_job_url": null}, {"build_number": 30119, "commit_position": 418368, "git_hash": "f714d2bd4450811bae2c849b075344eb31814cd6", "pass_rate": -1.0, "previous_build_commit_position": 418356, "previous_build_git_hash": "54279516e75ad5f5d6d1e94d26d65930cfa9dbdc", "task_id": null, "try_job_url": null}, {"build_number": 30120, "commit_position": 418380, "git_hash": "6e0ec0d0f1f709a1e2de02d439bcedbf6c3cb66a", "pass_rate": -1.0, "previous_build_commit_position": 418368, "previous_build_git_hash": "f714d2bd4450811bae2c849b075344eb31814cd6", "task_id": null, "try_job_url": null}, {"build_number": 30121, "commit_position": 418386, "git_hash": "f769f364f71068930d971dafb86dc1c5f298a8a7", "pass_rate": -1.0, "previous_build_commit_position": 418380, "previous_build_git_hash": "6e0ec0d0f1f709a1e2de02d439bcedbf6c3cb66a", "task_id": null, "try_job_url": null}, {"build_number": 30122, "commit_position": 418393, "git_hash": "198198900ec8d059d4d92be422cd65765c5a9ecf", "pass_rate": -1.0, "previous_build_commit_position": 418386, "previous_build_git_hash": "f769f364f71068930d971dafb86dc1c5f298a8a7", "task_id": null, "try_job_url": null}, {"build_number": 30123, "commit_position": 418395, "git_hash": "0df48d0d705860efe7904dd943669691eebff1d8", "pass_rate": -1.0, "previous_build_commit_position": 418393, "previous_build_git_hash": "198198900ec8d059d4d92be422cd65765c5a9ecf", "task_id": null, "try_job_url": null}, {"build_number": 30124, "commit_position": 418399, "git_hash": "0bb91bc3ffc6b11ba7d5c024e1fd89ddca3ffadb", "pass_rate": 0.040000000000000001, "previous_build_commit_position": 418395, "previous_build_git_hash": "0df48d0d705860efe7904dd943669691eebff1d8", "task_id": null, "try_job_url": null}, {"build_number": 30124, "commit_position": 418399, "git_hash": "0bb91bc3ffc6b11ba7d5c024e1fd89ddca3ffadb", "pass_rate": 0.040000000000000001, "previous_build_commit_position": 418395, "previous_build_git_hash": "0df48d0d705860efe7904dd943669691eebff1d8", "task_id": null, "try_job_url": null}, {"build_number": 30130, "commit_position": 418446, "git_hash": "846a59b93f9911517c7aeb05b47af9b4c0cad175", "pass_rate": 0.12, "previous_build_commit_position": 418441, "previous_build_git_hash": "24430ece16aa10aa57644daeddafe25032fd359c", "task_id": null, "try_job_url": null}, {"build_number": 30135, "commit_position": 418475, "git_hash": "87b8f7c3fc3e5bbcc9601023df0fe79cb79234e8", "pass_rate": 0.12, "previous_build_commit_position": 418474, "previous_build_git_hash": "d4f486c125772068950e8e43e3805dcf74eb23f5", "task_id": null, "try_job_url": null}, {"build_number": 30139, "commit_position": 418489, "git_hash": "d21e91636020a277b52bdbdde73d1e07ee0c57c0", "pass_rate": 0.089999999999999997, "previous_build_commit_position": 418484, "previous_build_git_hash": "ab3b40011e106bca7a83b880f332347bc7e7aae4", "task_id": null, "try_job_url": null}, {"build_number": 30142, "commit_position": 418494, "git_hash": "66c7a4cb89abd6a94f36c1ef976e08f7f9878255", "pass_rate": 0.10000000000000001, "previous_build_commit_position": 418491, "previous_build_git_hash": "14f15f11744445a7adb34030719afeb41a9432d0", "task_id": null, "try_job_url": null}, {"build_number": 30144, "commit_position": 418501, "git_hash": "4279cdda650058bbf7b82ddf6448927a532b1c76", "pass_rate": 0.070000000000000007, "previous_build_commit_position": 418499, "previous_build_git_hash": "517b1298eb6d07a30f07eade189a792c880c00df", "task_id": null, "try_job_url": null}];

For the time-being, I will write a script to clean that up, but it might be a bug in the code.
 

Comment 1 by st...@chromium.org, Jan 19 2017

Cc: -st...@chromium.org robert...@chromium.org
Project Member

Comment 2 by bugdroid1@chromium.org, Jan 19 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/ecbb8a15c18212cc562f7221be1937b7720ea68f

commit ecbb8a15c18212cc562f7221be1937b7720ea68f
Author: Shuotao Gao <stgao@chromium.org>
Date: Thu Jan 19 07:29:18 2017

[Findit] Workaround duplicate data points of the suspected build.

TBR=chanli
BUG= 682580 

Change-Id: I7bc158590055eb43e71729da6cd89dcecfd96b00
Reviewed-on: https://chromium-review.googlesource.com/430219
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Commit-Queue: Shuotao Gao <stgao@chromium.org>

[modify] https://crrev.com/ecbb8a15c18212cc562f7221be1937b7720ea68f/appengine/findit/waterfall/flake/confidence.py

Comment 3 by st...@chromium.org, Apr 25 2017

Cc: -lijeffrey@chromium.org chanli@chromium.org
Labels: -Pri-1 Pri-2
Owner: lijeffrey@chromium.org
assigned to Jeff instead for further fix. Chan, please comment if you know the root cause. 

Comment 4 by st...@chromium.org, Apr 25 2017

Labels: analysis

Comment 5 by chanli@chromium.org, Apr 25 2017

This should be before reruns on the stable build. So I'm not sure my fix on 707443 will fix this or not. We should check if there are any recent case for this issue.
Should we repair legacy data?

Comment 7 by st...@chromium.org, Apr 29 2017

If no value in the legacy data, no need to fix them.

But is the root cause identified and fixed? I just did a quick workaround.
Mergedinto: 707443
Status: Duplicate (was: Assigned)
Looking at several analyses since chan's fix in 707443 this doesn't appear to be happening any more. resolving as a duplicate to 707443

Sign in to add a comment