New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 849184 link

Starred by 2 users
Status: Fixed
Owner:
nya@chromium.org
Closed: Jul 20
Cc:
█ dshi@chromium.org
akes...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Stop tgz'ing CTS/GTS test results

Project Member Reported by nya@chromium.org, Jun 4 2018 
In gs_offloader.py, we compress a directory with more than 500 files into .tgz. In 2015 when the feature was implemented such directories were rare, but after introduction of ARC++, CTS/GTS test results often get such large directories.

Since tgz'ed test results are difficult to navigate, let's tune the constant.

Comment 1 by nya@chromium.org, Jun 4 2018 

Some stats about # of files (after decompressing .tgz):

205018691-chromeos-test/ 686
205018715-chromeos-test/ 692
205018768-chromeos-test/ 458
205018788-chromeos-test/ 760
205018796-chromeos-test/ 475
205053708-chromeos-test/ 843
205053742-chromeos-test/ 820
205142471-chromeos-test/ 1338
205142744-chromeos-test/ 1477

Those jobs were selected from failed CTS tests.
https://stainless.corp.google.com/search?view=list&first_date=2018-06-02&last_date=2018-06-04&test=CTS%7CGTS&build=%5ER69%5C-10746%5C.0%5C.0%24&model=%5Eeve%24&status=FAIL&exclude_cts=false&exclude_not_run=false&exclude_non_release=true&exclude_au=true&exclude_acts=true&exclude_retried=true&exclude_non_production=false

Comment 2 by nya@chromium.org, Jun 4 2018 

I will set the new limit to 3000, which is 2x of the biggest value in #c1.
Project Member

Comment 3 by bugdroid1@chromium.org, Jun 5 2018 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/a9cc41e66222f71f7cd2a63817fc6e1e4df94c00

commit a9cc41e66222f71f7cd2a63817fc6e1e4df94c00
Author: Shuhei Takahashi <nya@chromium.org>
Date: Tue Jun 05 12:42:19 2018

gs_offloader: Increase _MAX_FILE_COUNT.

gs_offloader compresses test artifacts into .tgz if there are too
many files. This logic was added to reduce "unnecessary load in
drone" ( crbug.com/537756 ) in 2015, and the threshold was decided
at that time.

However today CTS/GTS test results often exceed the limit
especially when a test failed and wa retried several times.
We often encounter such archived test results when debugging
failed CTS/GTS runs, which is annoying.

This change increases the threshold so that usual CTS/GTS test
results are not compressed. Please see  crbug.com/849184  for
how the new limit was decided.

Also removes unused import to silence the lint.

BUG= chromium:849184 
TEST=None

Change-Id: I5a05f5eaebd6a6d8e7e1c972253dd37b808f8d6e
Reviewed-on: https://chromium-review.googlesource.com/1084511
Commit-Ready: Shuhei Takahashi <nya@chromium.org>
Tested-by: Shuhei Takahashi <nya@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>

[modify] https://crrev.com/a9cc41e66222f71f7cd2a63817fc6e1e4df94c00/site_utils/gs_offloader.py

Comment 4 by nya@chromium.org, Jul 20

Status: Fixed (was: Started)
►

Sign in to add a comment