New issue
Advanced search Search tips

Issue 752908 link

Starred by 1 user
Status: Duplicate
Merged: issue 865069
Owner: ----
Closed: Aug 7
Cc:
kjlubick@chromium.org
mar...@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Feature



Sign in to add a comment

Support > 16MB return in swarming.task.stdout

Project Member Reported by rmis...@google.com, Aug 7 2017 

For https://bugs.chromium.org/p/skia/issues/detail?id=6924 ("SVG analysis") I wanted to gather all stdout from triggered swarming tasks.
Unfortunately the output from those tasks is > 16 MB: https://chrome-swarming.appspot.com/task?id=37d21650886dfe10&refresh=10&show_raw=1&wide_logs=true
When I call the swarming.task.stdout API call I get truncated output back.

In https://cs.chromium.org/chromium/infra/luci/appengine/swarming/server/task_result.py?q=task_resul&sq=package:chromium&l=198
It says- "Eventually, we'll want to add support for chunked fetch"

This issue is asking for that support.

Comment 1 by mar...@chromium.org, Aug 7 2017

Cc: -maruel@google.com -kjlubick@google.com kjlubick@chromium.org
Status: Available (was: Untriaged)
Supporting this means proper chunked output support in the API (which is a good thing but not implemented yet) so that a client would download XX Kb at a time and would iterate from there. This requires some UI work too.

To be clear; I think it's a good idea, just need help with it to update:
- Update task.stdout to work fine with chunked download
- Web UI to do chunked download
- swarming.py CLI to do chunked download

One nice side effect is that it is effectively enabling support for live streaming of stdout.

Word of warning: we plan to eventually split stdout and stderr (issue 732802) and this will slightly get in the way but should not be a big deal.
Project Member

Comment 2 by sheriffbot@chromium.org, Aug 7

Labels: Hotlist-Recharge-Cold
Status: Untriaged (was: Available)
This issue has been Available for over a year. If it's no longer important or seems unlikely to be fixed, please consider closing it out. If it is important, please re-triage the issue.

Sorry for the inconvenience if the bug really should have been left as Available.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot

Comment 3 by mar...@chromium.org, Aug 7

Mergedinto: 865069
Status: Duplicate (was: Untriaged)
I'll do the reverse instead, middle-out compression.
►

Sign in to add a comment