This doesn't appear to be related to task_output_dir.

It looks as though task stdout disappeared from the stdout stream late on 2017-09-07 pacific time. I don't yet have a suspected cause. The timestamp for https://chromium.googlesource.com/chromium/tools/build/+/60ab2bba2366227984e6b434eaf95e6987a2bd7c is close, but nothing in that CL appears to be related despite the changes to the swarming recipe module.

This appears to be due to https://chromium.googlesource.com/infra/luci/client-py.git/+/5e8001d9a710121ce7a68efd0804430a34b4f9e4, which rolled into chromium in https://chromium-review.googlesource.com/c/chromium/src/+/656401. It looks like the default value was supposed to be keeping the previous behavior; that may not be working as intended given that the stdout disappeared from the console.

The first build in which this appeared on that Mac Retina bot is https://luci-milo.appspot.com/buildbot/chromium.perf/Mac%20Retina%20Perf/1104. That build's blamelist contains the DEPS roll mentioned in #2.

Assigning to tansell and removing from trooper queue given that this appears to be an unintentional side-effect of that CL. Feel free to re-add troopers if necessary.

I'm just about to get onto a plane flight and won't have time to look at this until Monday. 

If it is really my CL you should be able to tell by a change in the swarming output;

-      '+---------------------------------------------------------------------+',
-      '| Shard 0  https://localhost:1/user/task/10100                        |',
-      '+---------------------------------------------------------------------+',
+      '+------------------------------------------------------+',
+      '| Shard 0  https://localhost:1/user/task/10100         |',
+      '+------------------------------------------------------+',
       'Foo',
-      '+---------------------------------------------------------------------+',
-      '| End of shard 0  Pending: 6.0s  Duration: 1.0s  Bot: swarm6  Exit: 0 |',
-      '+---------------------------------------------------------------------+',
+      '+------------------------------------------------------+',
+      '| End of shard 0                                       |',
+      '|  Pending: 6.0s  Duration: 1.0s  Bot: swarm6  Exit: 0 |',
+      '+------------------------------------------------------+',
+      'Total duration: 1.0s',


or

+      '+------------------------------------------------------+',
+      '| Shard 0  https://localhost:1/user/task/10100         |',
+      '|  Pending: 6.0s  Duration: 1.0s  Bot: swarm6  Exit: 0 |',
+      '+------------------------------------------------------+',

Yep, that change in the swarming output is the point of this bug. The perf bots that Juan is asking about are seeing something like

+------------------------------------------------------------------------+
| Shard 0  https://chromium-swarm.appspot.com/user/task/38784e7b10cbae10 |
|  Pending: 9476.3s  Duration: 3543.0s  Bot: build30-b4  Exit: 1         |
+------------------------------------------------------------------------+

w/o stdout, despite not passing anything to --task-output-stdout (which should mean that the stdout gets sent to the console and the JSON, if I'm reading your patch correctly).

The default should 'all' -- Can you compare what happens when you run it locally with "--task-output-stdout=all" ?

I'm guessing there is an error in my logic at;

+  def convert_value(self, opt, value):
+    if value not in self.choices:
+      raise optparse.OptionValueError("%s must be one of %s not %r" % (
+          self.get_opt_string(), self.choices, value))
+    stdout_to = []
+    if value == 'all':
+      stdout_to = ['json', 'console']
+    elif value != 'none':
+      stdout_to = [value]
+    return stdout_to

Tansell, any updates? As far as I can see this is still happening.

Issue 774173 has been merged into this issue.

This is a significant regression and is preventing external contributors from seeing the results of their tryjobs.

It's Saturday in Sydney so let's see if it's something tansell can look at on Monday.

The NextAction date has arrived: 2017-10-15