graphics_dEQP chromeperf Failure_* is mismatch with wmatrix |
||||
Issue descriptiongraphics_dEQP failures number in chromeperf is not correctly reflect the actual failures number. cros-hana 134116 failures https://chromeperf.appspot.com/report?sid=cf3c2796b1d1f15f19e161dc323f3128fc79d4622c9a3b400ccbd991cf6e2cda haha Failed: on rogue 234/126484 tests failed. https://wmatrix.googleplex.com/failures/unfiltered?platforms=hana&tests=graphics_dEQP.vk-master&days_back=10&builds=R61-9723.0.0&releases=tot&hide_missing=True
,
Jul 13 2017
,
Jul 14 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/836e0f1633f6e53ccdf65d72da2acff6d4d666f2 commit 836e0f1633f6e53ccdf65d72da2acff6d4d666f2 Author: Po-Hsien Wang <pwang@chromium.org> Date: Fri Jul 14 07:02:46 2017 graphics_dEQP.py: fix chromeperf failure records 1) Fix number mismatch from wmatrix & chromeperf 2) Use dedicate function to dispatch _run_tests_invidually & run_tests_hasty BUG= chromium:742599 TEST=test_that cave graphics_dEQP.vk-master Before: test_that cave graphics_dEQP.vk-master, cat /tmp/test_that_../.../result-charts.json "Failures_VK": { "summary": { "units": "count", "type": "scalar", "value": 134115.0, "improvement_direction": "down" } } After: test_that cave graphics_dEQP.vk-master, cat /tmp/test_that_../.../result-charts.json "Failures_VK": { "summary": { "units": "count", "type": "scalar", "value": 56.0, "improvement_direction": "down" } } Change-Id: I5390c86efd303cbf57c8bab60ec330ea5d2d4b7a Reviewed-on: https://chromium-review.googlesource.com/571407 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> [modify] https://crrev.com/836e0f1633f6e53ccdf65d72da2acff6d4d666f2/client/site_tests/graphics_dEQP/graphics_dEQP.py
,
Jul 15 2017
Seems numbers are match now. The following is some number retrieved from chromeperf and wmatrix. ----------------------------------------------------------------------------------------------------------- cave: (wondering why the number here is different from my local test(56)) chromeperf: 103 https://chromeperf.appspot.com/report?sid=899927e533461648078fdcb73209354adc3dae1f183980441647d3847af312a6 wmatrix: Failed: on skylake 103/134092 tests failed.https://wmatrix.googleplex.com/failures/unfiltered?platforms=cave&tests=graphics_dEQP.vk-master&days_back=10&builds=R61-9742.0.0&releases=tot&hide_missing=True ----------------------------------------------------------------------------------------------------------- asuka: chromeperf: 104 https://chromeperf.appspot.com/report?sid=c178737fa56c1eedd9fc87d885701c130650be5eff7bea8d6a8d2796b344e4c4 wmatrix: Failed: on skylake 104/134092 tests failed. https://wmatrix.googleplex.com/failures/unfiltered?platforms=asuka&tests=graphics_dEQP.vk-master&days_back=10&builds=R61-9743.0.0&releases=tot&hide_missing=True ----------------------------------------------------------------------------------------------------------- lars chromeperf: 104 https://chromeperf.appspot.com/report?sid=a90fade398431f91e075ccf4623dadf3ce106d1371774351fcbb71b73610ecfd wmatrix: Failed: on skylake 104/134092 tests failed. https://wmatrix.googleplex.com/failures/unfiltered?platforms=asuka&tests=graphics_dEQP.vk-master&days_back=10&builds=R61-9743.0.0&releases=tot&hide_missing=True -----------------------------------------------------------------------------------------------------------
,
Jul 17 2017
Looks good, thanks! |
||||
►
Sign in to add a comment |
||||
Comment 1 by pwang@chromium.org
, Jul 13 2017