New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 792590 link

Starred by 1 user

Issue metadata

Status: Verified
Owner:
Closed: Jan 2018
Cc:
Components:
EstimatedDays: ----
NextAction: 2018-01-03
OS: ----
Pri: 2
Type: ----



Sign in to add a comment

Test failure analysis is buggy

Project Member Reported by st...@chromium.org, Dec 6 2017

Issue description

AssertionError: services.parameters.BaseSubFailure._dynamic_definitions not defined in TestFailedStep
https://pantheon.corp.google.com/errors/4357050944668676218?time=P30D&filter&project=findit-for-me

TypeError: issubclass() arg 1 must be a class
https://pantheon.corp.google.com/errors/CIOLjor0iqai8AE?time=P1D&filter&project=findit-for-me


 
Project Member

Comment 1 by bugdroid1@chromium.org, Dec 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/f9f33cd2b56823dc9c43ace03c043ab7344e00d2

commit f9f33cd2b56823dc9c43ace03c043ab7344e00d2
Author: Shuotao Gao <stgao@chromium.org>
Date: Wed Dec 06 19:59:34 2017

[Findit] Fix a bug in naming attribute definition metadata.

The name of the attribute metadata should be prefixed with "_". Otherwise, it just becomes a normal defined attribute and causes trouble in encoding/decoding.

Bug:  792590 
Change-Id: I6de12122c1d9bbf3361b6df286949df28fd98d70
Reviewed-on: https://chromium-review.googlesource.com/811832
Commit-Queue: Shuotao Gao <stgao@chromium.org>
Reviewed-by: Jeffrey Li <lijeffrey@chromium.org>
Reviewed-by: Roberto Carrillo <robertocn@chromium.org>

[modify] https://crrev.com/f9f33cd2b56823dc9c43ace03c043ab7344e00d2/appengine/findit/libs/structured_object.py
[modify] https://crrev.com/f9f33cd2b56823dc9c43ace03c043ab7344e00d2/appengine/findit/libs/test/structured_object_test.py

Comment 2 by st...@chromium.org, Dec 6 2017

Status: Fixed (was: Started)
Although the fix landed, I'm still wondering why the bug didn't affect compile failure analysis pipeline...

Comment 3 by st...@chromium.org, Dec 6 2017

Labels: -Pri-1 Pri-2
Owner: chanli@chromium.org
Status: Assigned (was: Fixed)
https://chromium-review.googlesource.com/c/infra/infra/+/804998 made test failure analysis pipeline buggy, I'm gonna to revert it.
Project Member

Comment 4 by bugdroid1@chromium.org, Dec 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/d8a5bcc1971eb86978c1b800f325abdaf3424cce

commit d8a5bcc1971eb86978c1b800f325abdaf3424cce
Author: Shuotao Gao <stgao@chromium.org>
Date: Wed Dec 06 22:01:20 2017

Revert "[Fndit] Make HeuristicAnalysisForTestPipeline inherit from new BasePipeline."

This reverts commit 34007b88a1e9ebfef758d6dcf3a2c496c75e616d.

Bug:  792590 
Change-Id: Iba67331f3ccafd92b8a07207c9e0b4dd482bb15e
Reviewed-on: https://chromium-review.googlesource.com/812007
Reviewed-by: Brandon Wylie <wylieb@chromium.org>
Commit-Queue: Shuotao Gao <stgao@chromium.org>

[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/libs/structured_object.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/test_failure/test/test_failure_analysis_test.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/compile_failure/compile_failure_analysis.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/pipelines/test_failure/analyze_test_failure_pipeline.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/build_failure_analysis.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/pipelines/test_failure/test/heuristic_analysis_for_test_pipeline_test.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/parameters.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/test_failure/test_failure_analysis.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/test_failure/extract_test_signal.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/test/build_failure_analysis_test.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/test_failure/test/extract_test_signal_test.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/pipelines/compile_failure/heuristic_analysis_for_compile_pipeline.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/pipelines/test_failure/test/analyze_test_failure_pipeline_test.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/services/test/deps_test.py
[modify] https://crrev.com/d8a5bcc1971eb86978c1b800f325abdaf3424cce/appengine/findit/pipelines/test_failure/heuristic_analysis_for_test_pipeline.py

Project Member

Comment 6 by bugdroid1@chromium.org, Dec 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/0bfe44fde69bd16520c66399fc2f765f7549c27a

commit 0bfe44fde69bd16520c66399fc2f765f7549c27a
Author: Shuotao Gao <stgao@chromium.org>
Date: Thu Dec 07 07:01:02 2017

Revert "[Findit] Merge Heuristic pipelines for test failures."

This reverts commit 798871461c6cd8a8cca61f3d42d7808e5a17ce53.

Reason for revert: failed tests are not included in the failure_info -> failed_steps. That made it impossible to run Swarming tasks to detect flaky tests on Waterfall. https://screenshot.googleplex.com/06Y56be5rC7.png

Original change's description:
> [Findit] Merge Heuristic pipelines for test failures.
>
> And create a new analyze_test_failure_pipeline. Then remove the old analyze_build_failure_pipeline.
>
> Bug:  789385 
> Change-Id: I9109f6402ea7e9f220ec4d7d0edc9f6726337fa3
> Reviewed-on: https://chromium-review.googlesource.com/795016
> Commit-Queue: Chan Li <chanli@chromium.org>
> Reviewed-by: Shuotao Gao <stgao@chromium.org>

TBR=stgao@chromium.org,robertocn@chromium.org,chanli@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  792590 
Change-Id: I30099a6c1aed6a5da848859c613577a0e869eb75
Reviewed-on: https://chromium-review.googlesource.com/812965
Commit-Queue: Shuotao Gao <stgao@chromium.org>
Reviewed-by: Shuotao Gao <stgao@chromium.org>

[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/services/compile_failure/compile_failure_analysis.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/test/build_failure_analysis_pipelines_test.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/build_failure_analysis_pipelines.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/test/detect_first_test_failure_pipeline_test.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/detect_first_test_failure_pipeline.py
[delete] https://crrev.com/c32afe359052fd0e09c26a900203930d47012e25/appengine/findit/pipelines/test_failure/heuristic_analysis_for_test_pipeline.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/start_test_try_job_pipeline.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/test/start_test_try_job_pipeline_test.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/test/trigger_swarming_tasks_pipeline_test.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/process_swarming_tasks_result_pipeline.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/extract_signal_for_test_pipeline.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/test/process_swarming_tasks_result_pipeline_test.py
[delete] https://crrev.com/c32afe359052fd0e09c26a900203930d47012e25/appengine/findit/pipelines/test_failure/test/analyze_test_failure_pipeline_test.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/services/test_failure/test_failure_analysis.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/test/identify_test_failure_suspect_pipeline_test.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/trigger_swarming_tasks_pipeline.py
[rename] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/analyze_build_failure_pipeline.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/test/extract_signal_for_test_pipeline_test.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/test_failure/identify_test_failure_suspect_pipeline.py
[delete] https://crrev.com/c32afe359052fd0e09c26a900203930d47012e25/appengine/findit/pipelines/test_failure/test/heuristic_analysis_for_test_pipeline_test.py
[add] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/waterfall/test/analyze_build_failure_pipeline_test.py
[modify] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/services/test_failure/test/test_failure_analysis_test.py

Project Member

Comment 7 by bugdroid1@chromium.org, Dec 7 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/b255ab8c009ed1e65bc66433c64b5129ddee6f5b

commit b255ab8c009ed1e65bc66433c64b5129ddee6f5b
Author: Shuotao Gao <stgao@chromium.org>
Date: Thu Dec 07 07:14:12 2017

Revert "[Findit] Merge Heuristic pipelines for compile failures."

This reverts commit 2efbaf68ea79c0d695b83e62e024ea165b8d5736.

Reason for revert: A bug in this CL was fixed by https://chromium.googlesource.com/infra/infra/+/48367b9cb60c24cff737ec927e1ac36060f0d698, but that fix was reverted.

Original change's description:
> [Findit] Merge Heuristic pipelines for compile failures.
> 
> Bug:  789385 
> Change-Id: I85f9d9e5b320c79d3ef82e4e0ac9a5408ddb15c9
> Reviewed-on: https://chromium-review.googlesource.com/795185
> Commit-Queue: Chan Li <chanli@chromium.org>
> Reviewed-by: Shuotao Gao <stgao@chromium.org>

TBR=stgao@chromium.org,robertocn@chromium.org,chanli@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  792590 
Change-Id: I8d61b8755b1c44d56b7a124e2c94872dc40295bf
Reviewed-on: https://chromium-review.googlesource.com/813594
Reviewed-by: Shuotao Gao <stgao@chromium.org>
Commit-Queue: Shuotao Gao <stgao@chromium.org>

[add] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/detect_first_compile_failure_pipeline.py
[add] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/test/extract_signal_for_compile_pipeline_test.py
[modify] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/services/compile_failure/compile_failure_analysis.py
[modify] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/test/analyze_compile_failure_pipeline_test.py
[add] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/test/detect_first_compile_failure_pipeline_test.py
[add] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/test/identify_compile_failure_suspect_pipeline_test.py
[modify] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/services/compile_failure/test/compile_failure_analysis_test.py
[delete] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/compile_failure/test/heuristic_analysis_for_compile_pipeline_test.py
[modify] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/test/start_compile_try_job_pipeline_test.py
[add] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/identify_compile_failure_suspect_pipeline.py
[modify] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/start_compile_try_job_pipeline.py
[delete] https://crrev.com/0bfe44fde69bd16520c66399fc2f765f7549c27a/appengine/findit/pipelines/compile_failure/heuristic_analysis_for_compile_pipeline.py
[add] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/extract_signal_for_compile_pipeline.py
[modify] https://crrev.com/b255ab8c009ed1e65bc66433c64b5129ddee6f5b/appengine/findit/pipelines/compile_failure/analyze_compile_failure_pipeline.py

Comment 8 by st...@chromium.org, Dec 7 2017

NextAction: 2018-01-03
All reverts are done. Next step is for chanli@ to reland them with proper fixes.
The NextAction date has arrived: 2018-01-03
Status: Verified (was: Assigned)
All CLs have been relanded and test failure analyses are WAI.

Sign in to add a comment