Several telemetry_GpuTests.* are passing despite running no tests |
||||
Issue descriptionI found that the following are green, but are not actually running anything. The logs show two different failures that: telemetry_GpuTests.gpu_rasterization telemetry_GpuTests.memory_test These fail with something like: "run_gpu_integration_test.py gpu_rasterization" fails with: stdout:Cannot find test class with name matching gpu_rasterization telemetry_GpuTests.pixel telemetry_GpuTests.trace_test These fail with: AssertionError: /usr/local/telemetry/src/media/test/data does not exist. Stainless link showing recent results for two boards: https://stainless.corp.google.com/search?exclude_retried=true&first_date=2018-08-25&master_builder_name=&builder_name_number=&shard=&exclude_acts=true&builder_name=&master_builder_name_number=&owner=&retry=&exclude_cts=true&exclude_non_production=false&hostname=&board=&test=%5Etelemetry_GpuTests%5C.pixel%24&suite=&build=%5ER70%5C-11019%5C.0%5C.0%24&status=GOOD&reason=&waterfall=&exclude_not_run=false&last_date=2018-08-31&exclude_non_release=true&exclude_au=true&model=%5E(grunt%7Ceve)%24&view=list
,
Aug 31
It looks like gpu_rasterization and memory_test may be due to stale control files referencing old tests that have since been removed or renamed. pixel and trace_test look like they still exist, however.
,
Aug 31
Yes, these tests are stale. I have not been able to maintain them.
,
Aug 31
On the other Chromium bots, these tests are deployed via isolates. The dependencies of the Telemetry-based pixel tests, for example, are encompassed in this build target: https://cs.chromium.org/chromium/src/chrome/test/BUILD.gn?type=cs&q=telemetry_gpu_integration_test&sq=package:chromium&g=0&l=324 Not sure how the ChromeOS testing setup transfers files from the builder to the test machine, but if these files are transferred, then the tests should be able to run again.
,
Sep 5
Have these tests provided a valuable signal in the past before they became stale? Is it worthwhile to get these up and running again for Chrome OS? If we can fix bubble the errors from run_gpu_integration_test.py up to make the tests fail, it might provide enough visibility to avoid them going stale again in the future.
,
Sep 7
,
Sep 11
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b49706f8d370036791ccbbf27adaa26a67b6a7bf commit b49706f8d370036791ccbbf27adaa26a67b6a7bf Author: Drew Davenport <ddavenport@chromium.org> Date: Tue Sep 11 01:34:59 2018 telemetry: Update list of gpu integration tests Update list of tests in generate_controlfiles.py, remove stale control files, generate new control files. BUG=chromium:879738 TEST=generate control files Change-Id: I1f5cade3a9e313eaf7bdb405cf30b2a9cac6cd5b Reviewed-on: https://chromium-review.googlesource.com/1216884 Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> [copy] https://crrev.com/b49706f8d370036791ccbbf27adaa26a67b6a7bf/server/site_tests/telemetry_GpuTests/control.info_collection [modify] https://crrev.com/b49706f8d370036791ccbbf27adaa26a67b6a7bf/server/site_tests/telemetry_GpuTests/generate_controlfiles.py [delete] https://crrev.com/679dc3761169cb9d5bb9eeeaf23fb46f6d74ce4f/server/site_tests/telemetry_GpuTests/control.gpu_rasterization [copy] https://crrev.com/b49706f8d370036791ccbbf27adaa26a67b6a7bf/server/site_tests/telemetry_GpuTests/control.depth_capture [rename] https://crrev.com/b49706f8d370036791ccbbf27adaa26a67b6a7bf/server/site_tests/telemetry_GpuTests/control.noop_sleep
,
Sep 11
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/843959274f2fd35d6d9fd8b9f4faefba367e6cd3 commit 843959274f2fd35d6d9fd8b9f4faefba367e6cd3 Author: Drew Davenport <ddavenport@chromium.org> Date: Tue Sep 11 04:01:07 2018 telemetry: Raise error on failed gpu test Detect a failed gpu_integration_test from the exit code and raise an error so that the test failure bubbles up and is visible BUG=chromium:879738 TEST=test_that telemetry_GpuTests.gpu_rasterization fails as expected Change-Id: I19d4b1d59708fd775b47b0b4bffda9bfaa14b331 Reviewed-on: https://chromium-review.googlesource.com/1216883 Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Trybot-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/843959274f2fd35d6d9fd8b9f4faefba367e6cd3/server/cros/telemetry_runner.py
,
Sep 11
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/7873326f18fffb53bd3aebbd8516b3c7c3310e99 commit 7873326f18fffb53bd3aebbd8516b3c7c3310e99 Author: Drew Davenport <ddavenport@chromium.org> Date: Tue Sep 11 04:01:47 2018 chromeos-chrome: Add media/test/data to telemetry deps Add the contents of media/test/data to the telemetry_dep tarball. This directory has data needed by gpu integration tests. The data in the directory currently is 74M uncompressed: > du -sh /build/grunt/usr/local/telemetry/src/media/test/data/ 74M /build/grunt/usr/local/telemetry/src/media/test/data/ BUG=chromium:879738 TEST=test_that telemetry_GpuTests.pixel can run Change-Id: I8f312b19acaa91f5100821d4cb55dd6b8bb351e7 Reviewed-on: https://chromium-review.googlesource.com/1217024 Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/7873326f18fffb53bd3aebbd8516b3c7c3310e99/chromeos-base/chromeos-chrome/chromeos-chrome-9999.ebuild
,
Jan 16
(6 days ago)
,
Jan 16
(6 days ago)
|
||||
►
Sign in to add a comment |
||||
Comment 1 by nednguyen@chromium.org
, Aug 31