New issue
Advanced search Search tips

Issue 682889 link

Starred by 1 user

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: ----


Participants' hotlists:
SoM-Stuff


Sign in to add a comment

read build logs from logdog instead of buildbot

Project Member Reported by seanmccullough@chromium.org, Jan 19 2017

Issue description

It's okay for this just to be for the appengine cron jobs. a-d doesn't need to support this.


 
Project Member

Comment 1 by bugdroid1@chromium.org, Jan 20 2017

Project Member

Comment 2 by bugdroid1@chromium.org, Jan 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/feeac1dcc3bff4847a04a98068c0d2c6721b1aef

commit feeac1dcc3bff4847a04a98068c0d2c6721b1aef
Author: Daniel Jacques <dnj@chromium.org>
Date: Fri Jan 20 17:57:23 2017

Revert "LogDog: emit bootstrap project/prefix as property."

This reverts commit e9033069637d0ec74082e924bcacb2e923febfe4.

Reason for revert:
Bug: Exception: Can't start step LogDog Bootstrap while in step remote_run_result.


Original change's description:
> LogDog: emit bootstrap project/prefix as property.
> 
> Update annotated_run and remote_run to emit a LogDog bootstrap step
> which attaches LogDog project and prefix build properties.
> 
> BUG=chromium:682889
> TEST=unit
> 
> Change-Id: I09df465706e2d788610f78712b5e7eb671bcc069
> Reviewed-on: https://chromium-review.googlesource.com/430264
> Reviewed-by: Erik Staab <estaab@chromium.org>
> Commit-Queue: Daniel Jacques <dnj@chromium.org>
> 

TBR=dnj@chromium.org,estaab@chromium.org,iannucci@chromium.org,seanmccullough@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:682889

Change-Id: I3347f64c2bbe3e105a7ae901b4bd1b7803d5e037
Reviewed-on: https://chromium-review.googlesource.com/430326
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>

[modify] https://crrev.com/feeac1dcc3bff4847a04a98068c0d2c6721b1aef/scripts/slave/annotated_run.py
[modify] https://crrev.com/feeac1dcc3bff4847a04a98068c0d2c6721b1aef/scripts/slave/logdog_bootstrap.py
[modify] https://crrev.com/feeac1dcc3bff4847a04a98068c0d2c6721b1aef/scripts/slave/remote_run.py
[modify] https://crrev.com/feeac1dcc3bff4847a04a98068c0d2c6721b1aef/scripts/slave/unittests/annotated_run_test.py
[modify] https://crrev.com/feeac1dcc3bff4847a04a98068c0d2c6721b1aef/scripts/slave/unittests/logdog_bootstrap_test.py

Comment 3 by d...@chromium.org, Jan 20 2017

annotated_run also hit: @@@STEP_CLOSED@@@
INFO:monitoring_utils:Executing command: ['/opt/infra-python/run.py', 'infra.tools.send_monitoring_event', '--event-mon-output-file', '/b/build/slave/ios-simulator/.recipe_runtime/tmpLF7UnW/build_data/log_request_proto', '--event-mon-run-type', 'file', '--event-mon-service-name', u'buildbot/master/master.tryserver.chromium.mac', '--build-event-build-name', u'ios-simulator', '--build-event-build-number', '140948', '--build-event-build-scheduling-time', '1484934711000', '--build-event-type', 'BUILD', '--event-mon-timestamp-kind', 'POINT', '--build-event-hostname', 'vm899-m4']
Ensuring CIPD client is up-to-date
GET https://chrome-infra-packages.appspot.com/_ah/api/repo/v1/instance/resolve?version=c111be343c692e5285113a6b1c999887adbb268e&package_name=infra%2Ftools%2Fcipd%2Fmac-amd64
CIPD client c111be343c692e5285113a6b1c999887adbb268e => c111be343c692e5285113a6b1c999887adbb268e
INFO:cipd:CIPD client [/b/build/slave/ios-simulator/.recipe_cipd/cipd] ensuring 2 package(s)...
INFO:cipd:Loading package: infra/tools/luci/logdog/butler/mac-amd64 @ git_revision:3c620f7e17e0e6da46ebf4e97d67dfbdcb1684ac
INFO:cipd:Loading package: infra/tools/luci/logdog/annotee/mac-amd64 @ git_revision:3c620f7e17e0e6da46ebf4e97d67dfbdcb1684ac
INFO:annotated_run:Bootstrapping through LogDog: ['/b/build/slave/ios-simulator/.recipe_cipd/logdog_butler', '-log-level', 'warning', '-project', 'chromium', '-prefix', 'bb/tryserver.chromium.mac/ios-simulator/140948', '-output', 'logdog,host="services-dot-luci-logdog.appspot.com"', '-coordinator-host', 'luci-logdog.appspot.com', '-tag', 'buildbot.master=tryserver.chromium.mac', '-tag', 'buildbot.builder=ios-simulator', '-tag', 'buildbot.buildnumber=140948', '-service-account-json', '/creds/service_accounts/service-account-luci-logdog-publisher.json', '-output-max-buffer-age', '30s', 'run', '-stdout', 'tee=stdout', '-stderr', 'tee=stderr', '-streamserver-uri', 'unix:/b/build/.recipe_runtime/tmpEDGhod/butler.sock', '--', '/b/build/slave/ios-simulator/.recipe_cipd/logdog_annotee', '-log-level', 'warning', '-project', 'chromium', '-butler-stream-server', 'unix:/b/build/.recipe_runtime/tmpEDGhod/butler.sock', '-logdog-host', 'luci-logdog.appspot.com', '-name-base', 'recipes', '-print-summary', '-tee', 'annotations,text', '-json-args-path', '/b/build/slave/ios-simulator/.recipe_runtime/tmpLF7UnW/logdog_annotee_cmd.json', '-result-path', '/b/build/slave/ios-simulator/.recipe_runtime/tmpLF7UnW/bootstrap_result.json']

@@@SEED_STEP LogDog Bootstrap@@@

@@@STEP_CURSOR LogDog Bootstrap@@@

@@@STEP_STARTED@@@

@@@SET_BUILD_PROPERTY@logdog_project@chromium@@@

@@@SET_BUILD_PROPERTY@logdog_prefix@bb/tryserver.chromium.mac/ios-simulator/140948@@@

@@@STEP_CURSOR LogDog Bootstrap@@@

@@@STEP_CLOSED@@@

remoteFailed: [Failure instance: Traceback: <type 'exceptions.ValueError'>: No JSON object could be decoded
/home/chrome-bot/buildbot/build/third_party/twisted_10_2/twisted/spread/pb.py:516:expressionReceived
/home/chrome-bot/buildbot/build/third_party/twisted_10_2/twisted/spread/pb.py:828:proto_message
/home/chrome-bot/buildbot/build/third_party/twisted_10_2/twisted/spread/pb.py:842:_recvMessage
/home/chrome-bot/buildbot/build/third_party/twisted_10_2/twisted/spread/flavors.py:114:remoteMessageReceived
--- <exception caught here> ---
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/process/buildstep.py:176:remote_update
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/process/metrics.py:81:wrapper
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/process/buildstep.py:362:remoteUpdate
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/process/buildstep.py:333:addStdout
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/status/logfile.py:458:addStdout
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/status/logfile.py:454:addEntry
/home/chrome-bot/buildbot/build/scripts/master/chromium_step.py:880:logChunk
/home/chrome-bot/buildbot/build/scripts/master/chromium_step.py:887:outReceived
/home/chrome-bot/buildbot/build/third_party/buildbot_8_4p1/buildbot/process/buildstep.py:455:outReceived
/home/chrome-bot/buildbot/build/third_party/twisted_10_2/twisted/protocols/basic.py:455:dataReceived
/home/chrome-bot/buildbot/build/scripts/master/chromium_step.py:872:outLineReceived
/home/chrome-bot/buildbot/build/scripts/master/chromium_step.py:1065:handleOutputLine
/home/chrome-bot/buildbot/build/scripts/common/annotator.py:324:MatchAnnotation
/home/chrome-bot/buildbot/build/scripts/master/chromium_step.py:1079:SET_BUILD_PROPERTY
/usr/lib/python2.7/json/__init__.py:326:loads
/usr/lib/python2.7/json/decoder.py:366:decode
/usr/lib/python2.7/json/decoder.py:384:raw_decode
]
Project Member

Comment 4 by bugdroid1@chromium.org, Jan 20 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/eb3156e777fc4fd653b437dd4fadaa5f0d63bdee

commit eb3156e777fc4fd653b437dd4fadaa5f0d63bdee
Author: Dan Jacques <dnj@chromium.org>
Date: Fri Jan 20 22:40:41 2017

LogDog: emit bootstrap project/prefix as property.

(Reland of e9033069637d0ec74082e924bcacb2e923febfe4)

Update annotated_run and remote_run to emit a LogDog bootstrap step
which attaches LogDog project and prefix build properties.

This fixes two observed bugs:
- "remote_run.py": Annotator doesn't support nested steps. We move the
  "remote_run_result" step to be an explicit final step rather than an
  encompassing one.
- General: build properties passed to annotator apparently have to be
  valid JSON.

BUG=chromium:682889
TEST=unit

Change-Id: I952308341b41e15254afc0db3a96eb65d4fae0af
Reviewed-on: https://chromium-review.googlesource.com/430827
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/eb3156e777fc4fd653b437dd4fadaa5f0d63bdee/scripts/slave/annotated_run.py
[modify] https://crrev.com/eb3156e777fc4fd653b437dd4fadaa5f0d63bdee/scripts/slave/logdog_bootstrap.py
[modify] https://crrev.com/eb3156e777fc4fd653b437dd4fadaa5f0d63bdee/scripts/slave/remote_run.py
[modify] https://crrev.com/eb3156e777fc4fd653b437dd4fadaa5f0d63bdee/scripts/slave/unittests/annotated_run_test.py
[modify] https://crrev.com/eb3156e777fc4fd653b437dd4fadaa5f0d63bdee/scripts/slave/unittests/logdog_bootstrap_test.py

Status: Started (was: Assigned)

Sign in to add a comment