GPU FYI waterfall logs taking up a good amount of disk space |
||||
Issue descriptionIs there any chance we can get the size of the logs from master.chromium.gpu.fyi down a bit? Here's what it looks like on the machine that hosts it: 49.3GiB [##########] /Win7_Release__NVIDIA_ 31.4GiB [###### ] /Linux_Release__NVIDIA_ 29.9GiB [###### ] /Win7_x64_Release__NVIDIA_ 17.5GiB [### ] /Win7_Release__ATI_ 16.0GiB [### ] /Win7_Debug__NVIDIA_ 14.0GiB [## ] /Win8_Release__NVIDIA_ 13.8GiB [## ] /Win8_Debug__NVIDIA_ 13.5GiB [## ] /Win7_x64_Debug__NVIDIA_ 13.2GiB [## ] /Win7_Debug__ATI_ .......etc Here's what the Win7_Release__NVIDIA_ bot looks like: 62.5MiB [##########] 23180-log-steps-stdio 62.5MiB [######### ] 23182-log-steps-stdio 62.4MiB [######### ] 23181-log-steps-stdio 62.4MiB [######### ] 23176-log-steps-stdio 62.4MiB [######### ] 23177-log-steps-stdio 62.4MiB [######### ] 23174-log-steps-stdio 62.4MiB [######### ] 23178-log-steps-stdio 62.4MiB [######### ] 23179-log-steps-stdio 62.4MiB [######### ] 23175-log-steps-stdio 61.8MiB [######### ] 23159-log-steps-stdio .......etc It looks like the angle_deqp_gle3_tests are pretty talkative: 24.1MiB [### ] 22843-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 22842-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 22839-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 22844-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 22840-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 22841-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 22778-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio .......etc In total, the master is taking up 305 GB of space, which isn't so bad on a 3TB machine, but every little bit helps. Would it feasible to start compressing the larger logs? If not, could we try making these tests a little less verbose? If there's anything that can be done to decrease disk usage, it would be greatly appreciated.
,
Mar 28 2016
There are just a ton of tests in angle_deqp_gle3_tests . Individually, they don't have that much log spew. I'm aware that the webgl2_conformance tests are pretty verbose, spewing tons of undesirable log output. Are those showing up as a problem and if so should we focus on them immediately? These logs expire, yes? The expiration isn't enough to keep the disk space under control?
,
Mar 28 2016
Yeah, any logs older than 8 days are deleted. We can try decreasing that horizon, but then we'd be losing even more recent history. Why don't we try compressing all logs over 10 MB? This would save a big chunk of space and would be totally transparent to anyone trying to view the logs from the waterfall page. This is also what I'm doing for perf: https://codereview.chromium.org/1831373004/ Regarding the webgl2_conformance tests, yeah, those are pretty fat as well: ..... 24.1MiB [### ] 23004-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 23002-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 24.1MiB [### ] 23003-log-angle_deqp_gles3_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.7MiB [# ] 23180-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.7MiB [# ] 23182-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.7MiB [# ] 23176-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.6MiB [# ] 23181-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.6MiB [# ] 23174-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.6MiB [# ] 23177-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.6MiB [# ] 23179-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.6MiB [# ] 23175-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.6MiB [# ] 23178-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.0MiB [# ] 23164-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.0MiB [# ] 23151-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.0MiB [# ] 23159-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.0MiB [# ] 23173-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.0MiB [# ] 23165-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio 10.0MiB [# ] 23153-log-webgl2_conformance_tests_on_NVIDIA_GPU_on_Windows_on_Windows-2008ServerR2-SP1-stdio .....
,
Mar 28 2016
Sure, please go ahead and compress these logs. We are aware the WebGL 2.0 logs (and even to a certain degree the WebGL 1.0 logs) are too verbose, but there are other higher-priority issues on our bug list right now.
,
Mar 28 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/tools/build.git/+/5cd97849762662fffbb1d7e925976054fa2d4c68 commit 5cd97849762662fffbb1d7e925976054fa2d4c68 Author: bpastene@chromium.org <bpastene@chromium.org> Date: Mon Mar 28 22:58:04 2016 Enable compression of any gpu.fyi logs greater than 10 MB. BUG= 598061 TBR=kbr@chromium.org Review URL: https://codereview.chromium.org/1841603002 git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/build@299517 0039d316-1c4b-4281-b951-d872f2087c98 [modify] https://crrev.com/5cd97849762662fffbb1d7e925976054fa2d4c68/masters/master.chromium.gpu.fyi/master.cfg
,
Aug 16 2016
Ben, what's the current situation here? Some work has been done recently to reduce the size of the WebGL 2.0 conformance tests' logs, and with the compression being done above, I'm hoping this can be marked as fixed. Please see: https://github.com/KhronosGroup/WebGL/pull/1892 https://github.com/KhronosGroup/WebGL/pull/1893 https://github.com/KhronosGroup/WebGL/pull/1894
,
Aug 16 2016
Yep, disk usage dropped back down to reasonable levels. Thanks for help on webgl tests. |
||||
►
Sign in to add a comment |
||||
Comment 1 by bpastene@chromium.org
, Mar 25 2016