This is the tracking bug for porting the graphics drawElements Quality Program (graphics_dEQP) test suite from autotest to tast.
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/219838823e7c7e4ded55f9d414b393e253941b22 commit 219838823e7c7e4ded55f9d414b393e253941b22 Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Tue Jul 24 04:04:58 2018 Port the get_gles_version() function into a tast support package. This CL creates a support package for tast graphics tests and ports the get_gles_version() function (defined in autotest/files/client/cros/graphics/graphics_utils.py). Additional helper functions had to be ported as well. TEST=to be added in an additional patchset in this CL BUG=chromium:865107 Change-Id: Ie39a1fb45d094bf5282af226d60660068629c2eb Reviewed-on: https://chromium-review.googlesource.com/1144238 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [add] https://crrev.com/219838823e7c7e4ded55f9d414b393e253941b22/src/chromiumos/tast/local/graphics/utils_test.go [add] https://crrev.com/219838823e7c7e4ded55f9d414b393e253941b22/src/chromiumos/tast/local/graphics/utils.go
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/2bdca51125bca8352e97114a8dfaec50a1441c69 commit 2bdca51125bca8352e97114a8dfaec50a1441c69 Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Sat Jul 28 02:11:16 2018 Port the GraphicsApiHelper functionality to tast. This CL ports the functionality of the GraphicsApiHelper class defined in autotest/files/client/cros/graphics/graphics_utils.py to the tast graphics support package. The main functions that calling code should use are SupportedAPIs() and DEQPExecutable(). However, GLESVersion() and SupportsVulkanForDEQP() are also exported so that the calling code can get the necesary arguments to pass to the main functions. TEST=added unit tests to this CL BUG=chromium:865107 CQ-DEPEND=CL:1144238 Change-Id: I3826ec42bc70b11d40eee7c242e0238c130ba5de Reviewed-on: https://chromium-review.googlesource.com/1147150 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [modify] https://crrev.com/2bdca51125bca8352e97114a8dfaec50a1441c69/src/chromiumos/tast/local/graphics/utils_test.go [modify] https://crrev.com/2bdca51125bca8352e97114a8dfaec50a1441c69/src/chromiumos/tast/local/graphics/utils.go
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/7aed3d2ce114a74a7406f2702fe49f8c10e6e59f commit 7aed3d2ce114a74a7406f2702fe49f8c10e6e59f Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Sat Jul 28 07:54:33 2018 Add a helper function to get the right environment for DEQP binaries. This CL adds the DEQPEnvironment function to the graphics support package in tast. The function returns the appropriate environment variables to run DEQP tests. Also, a few details are fixed in the SupportsVulkanForDEQP function. TEST=added unit tests to this CL BUG=chromium:865107 CQ-DEPEND=CL:1147150 Change-Id: I9f28c66f0ebbcfa68f1f6a922cfbe8b2e5fae182 Reviewed-on: https://chromium-review.googlesource.com/1153633 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [modify] https://crrev.com/7aed3d2ce114a74a7406f2702fe49f8c10e6e59f/src/chromiumos/tast/local/graphics/utils_test.go [modify] https://crrev.com/7aed3d2ce114a74a7406f2702fe49f8c10e6e59f/src/chromiumos/tast/local/graphics/utils.go
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/222c0feed09a397e27185656d6e3c9661c7efe38 commit 222c0feed09a397e27185656d6e3c9661c7efe38 Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Fri Aug 03 04:50:16 2018 Add the ParseDEQPOutput function to the graphics support package. This CL adds the ParseDEQPOutput function to the graphics support package to parse log files generated by DEQP tests. The parsing is essentially a state machine, so for ease of review, please refer to the diagram at https://tinyurl.com/deqp-parsing-fsm. The DEQPOutcomeIsFailure function is also added. It's exported in case callers need to interpret the keys of the |stats| map returned by the ParseDEQPOutput. TEST=added unit tests to this CL BUG=chromium:865107 CQ-DEPEND=CL:1153633 Change-Id: I3c85209724f92ca2c2890dc0cbeb11dcc5bb869f Reviewed-on: https://chromium-review.googlesource.com/1159198 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBeginWithoutEnd1.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBadXMLNoResult.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildMultipleIncomplete.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/deqp_parser_test.go [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBadXMLNoStatusCode.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBeginWithoutEnd2.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBadXMLCasePathMismatch.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildMultipleWatchdogTimeout.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildSingleFail.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildSingleIncomplete.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBadXMLNoCasePath.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBadXMLManyResults.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildSingleWatchdogTimeout.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeTerminateWithoutBegin1.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeTerminateWithoutCauseNotLastLine1.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeEndWithoutBegin2.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeTerminateWithoutCauseNotLastLine2.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeTerminateWithoutCauseLastLine.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildSingleSuccess.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/deqp_parser.go [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeNoTestCases.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeTerminateWithoutBegin2.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBeginWithoutTestName.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/WildMultiplePassAndFail.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeEndWithoutBegin1.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeBadXMLIncomplete.log [add] https://crrev.com/222c0feed09a397e27185656d6e3c9661c7efe38/src/chromiumos/tast/local/graphics/testdata/deqp_parser/FakeEmpty.log
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/635baa8a090a3bbbb4b7412a66a6684e77af242d commit 635baa8a090a3bbbb4b7412a66a6684e77af242d Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Tue Oct 02 23:42:07 2018 Port set/get_dirty_writeback_centisecs to tast. This CL ports the set_dirty_writeback_centisecs() and get_dirty_writeback_centisecs() functions defined in autotest/files/client/bin/utils.py to tast. BUG=chromium:865107 TEST=manually ran the functions and inspected the effect on /proc/sys/vm/dirty_writeback_centisecs. Change-Id: I2608695032c70ab8998eb45d1a98f7868e29ae04 Reviewed-on: https://chromium-review.googlesource.com/1241574 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [modify] https://crrev.com/635baa8a090a3bbbb4b7412a66a6684e77af242d/src/chromiumos/tast/local/graphics/utils.go
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/bfe13e860ac09a0f371b2874251bae5c0da763a4 commit bfe13e860ac09a0f371b2874251bae5c0da763a4 Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Fri Nov 16 02:49:32 2018 Create the starter code to port graphics_dEQP tests to tast. This CL provides much of the initialization code to port the graphics drawElements Quality Program test suite from autotest to tast to run it in the pre-CQ (eventually). TEST=passes when running tast -verbose run <ip> graphics.DEQP BUG=chromium:865107 Change-Id: I450aeb9963a2c4159ee2a4021d7e4897734724d9 Reviewed-on: https://chromium-review.googlesource.com/1142040 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [add] https://crrev.com/bfe13e860ac09a0f371b2874251bae5c0da763a4/src/chromiumos/tast/local/bundles/cros/graphics/deqp.go
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/1ed86b165fdaf597a3678e2d38f61e71776fe9da commit 1ed86b165fdaf597a3678e2d38f61e71776fe9da Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Sat Dec 01 13:19:30 2018 Tast graphics.DEQP: add code to actually run DEQP tests. This CL adds code to run DEQP tests individually and report the results. TEST=passes all tests when running tast -verbose run <ip> graphics.DEQP on an eve. BUG=chromium:865107 CQ-DEPEND=CL:1142040 Change-Id: Ibf00c51369b12f10f861922e2f7f279c072a956a Reviewed-on: https://chromium-review.googlesource.com/1278338 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> Reviewed-by: Shuhei Takahashi <nya@chromium.org> [modify] https://crrev.com/1ed86b165fdaf597a3678e2d38f61e71776fe9da/src/chromiumos/tast/local/bundles/cros/graphics/deqp.go [modify] https://crrev.com/1ed86b165fdaf597a3678e2d38f61e71776fe9da/src/chromiumos/tast/local/graphics/deqp_parser.go [modify] https://crrev.com/1ed86b165fdaf597a3678e2d38f61e71776fe9da/src/chromiumos/tast/local/graphics/utils.go
Shouldn't I see results on stainless now? Can't find any. Do we still need to hook this up?
I think the test is still disabled: https://crrev.com/c/1278338/8/src/chromiumos/tast/local/bundles/cros/graphics/deqp.go#23 (If there was an earlier discussion about why, sorry; I forget.)
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/81296cbc802147dcbb7c33bf141ff2f9bfcdebca commit 81296cbc802147dcbb7c33bf141ff2f9bfcdebca Author: Andres Calderon Jaramillo <andrescj@chromium.org> Date: Fri Dec 14 03:28:08 2018 Enable graphics.DEQP Tast test in informational mode. BUG=chromium:865107 TEST=graphics.DEQP passes on an eve with 11381.0.0. Change-Id: Iaaecea0d397e392e5fd23af8619d7eb73173c6a2 Reviewed-on: https://chromium-review.googlesource.com/1370764 Commit-Ready: Andres Calderon Jaramillo <andrescj@chromium.org> Tested-by: Andres Calderon Jaramillo <andrescj@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [modify] https://crrev.com/81296cbc802147dcbb7c33bf141ff2f9bfcdebca/src/chromiumos/tast/local/bundles/cros/graphics/deqp.go
Looking good. We should watch it for a few more days and check log files: https://stainless.corp.google.com/search?view=matrix&row=model&col=build&days=10&suite=%5Ebvt%5C-perbuild%24&test=%5Etast%5C.graphics%5C.DEQP%24&exclude_cts=true&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false
Comment 1 by bugdroid1@chromium.org
, Jul 24