There was a tree closure because a call to https://storage.googleapis.com/nativeclient-once/object/newlib_arm_bc_a708c95953a96292307ec8cbaa0c9db31caf81bb.tgz timed out but it was reachable within the same second.
Another indication that storage.googleapis.com is slow today, is this failure:
The context_lost_tests failed on chromium.gpu/Android Release (Nexus 5X):
https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Android%20Release%20%28Nexus%205X%29/13715
Log snippet:
15604 2018-12-11 11:00:08.070 E: internal failure: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Read timed out.
Traceback (most recent call last):
File "/b/swarming/swarming_bot.1.zip/client/run_isolated.py", line 734, in map_and_run
use_symlinks=data.use_symlinks)
File "/b/swarming/swarming_bot.1.zip/client/run_isolated.py", line 531, in fetch_and_map
use_symlinks=use_symlinks)
File "/b/swarming/swarming_bot.1.zip/client/isolateserver.py", line 1294, in fetch_isolated
digest = fetch_queue.wait()
File "/b/swarming/swarming_bot.1.zip/client/isolateserver.py", line 960, in wait
digest = self._channel.next()
File "/b/swarming/swarming_bot.1.zip/utils/threading_utils.py", line 377, in _task_executer
result = func(*args, **kwargs)
File "/b/swarming/swarming_bot.1.zip/client/isolateserver.py", line 865, in fetch
sink(verifier.run())
File "/b/swarming/swarming_bot.1.zip/client/local_caching.py", line 586, in write
size = file_write(path, content)
File "/b/swarming/swarming_bot.1.zip/client/local_caching.py", line 41, in file_write
for d in content_generator:
File "/b/swarming/swarming_bot.1.zip/client/isolateserver.py", line 1032, in run
for chunk in self.stream:
File "/b/swarming/swarming_bot.1.zip/client/isolateserver.py", line 262, in zip_decompress
for chunk in content_generator:
File "/b/swarming/swarming_bot.1.zip/client/isolate_storage.py", line 395, in fetch
for data in connection.iter_content(NET_IO_FILE_CHUNK):
File "/b/swarming/swarming_bot.1.zip/utils/net.py", line 691, in iter_content
raise TimeoutError(e)
TimeoutError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Read timed out.
HTTPSConnectionPool(host='storage.googleapis.com', port=443): Read timed out.
15604 2018-12-11 11:00:09.091 E: Failed to fetch 2694b83f37f00bc170fbb7bd3189ba4e911846ab: ("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer'))
Comment 1 by mar...@chromium.org
, Dec 11