protobuf_version:v3.6.1 causes problems for monorail |
||
Issue descriptionMonorail tests are failing after https://chromium.googlesource.com/infra/infra/+/caa0bd3d76fcc9ccf0c8d999fb1fd5d77da2f4a7 With: "TypeError: __init__() got an unexpected keyword argument 'serialized_options'" on linux after running "make prpc_proto" To reproduce, go to appengine/monorail and run 'maoe prpc_proto' and 'make test'.
,
Nov 19
,
Nov 19
Yeah. I tried rolling this. It is not sufficient.
,
Nov 19
ok
,
Nov 19
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/0b198138877d2e7670dcc56baee83034f0d1b472 commit 0b198138877d2e7670dcc56baee83034f0d1b472 Author: Marc-Antoine Ruel <maruel@chromium.org> Date: Mon Nov 19 21:00:56 2018 Roll infra/luci/ 473a850bc..af1fb93ec (83 commits) https://chromium.googlesource.com/infra/luci/luci-py/+log/473a850bc451..af1fb93ece58 $ git log 473a850bc..af1fb93ec --date=short --no-merges --format='%ad %ae %s' 2018-11-19 maruel protobuf: upgrade to 3.6.1 from 3.5.1 2018-11-19 akeshet [swarming] rename _maybe_pubsub_notify_via_tq 2018-11-19 akeshet swarming: plugin api: add slice_number cancellation reasons 2018-11-19 akeshet [swarming] add a dev target to Makefile for local testing 2018-11-19 jchinlee [client] Print time with tasks update message. Also increase frequency. 2018-11-19 kjlubick [swarming-ui] Address some feedback with sorting/aliases 2018-11-16 vadimsh [auth] Put additional info extracted from credentials into the auth state. 2018-11-16 bradhall Only allow repeated dimensions for 'caches' 2018-11-16 maruel Remove last references to rietveld 2018-11-15 akeshet Revert "[swarming] rename _maybe_pubsub_notify_via_tq" 2018-11-15 akeshet [swarming] rename _maybe_pubsub_notify_via_tq 2018-11-14 tikuta [isolate] Increase max_concurrent_requests 2018-11-13 akeshet [swarming] allow swarming bot to run under Chrome Remote Desktop 2018-11-12 vadimsh [swarming] Add 'swarming/bot_auth/success' metric. 2018-11-12 jbudorick swarming: process tasks in LIFO order. 2018-11-10 akeshet [swarming] add an enabled field to ExternalSchedulerConfig 2018-11-09 akeshet [swarming] use argparse in start_bot.py 2018-11-09 vadimsh [swarming] Make 'auth' field in BotGroup repeated. 2018-11-09 nodir [prpc] Add pRPC client in Python 2018-11-09 vadimsh [swarming] Preparation for making 'auth' in BotGroup proto repeated. 2018-11-08 kjlubick Fix small wording things 2018-11-07 akeshet [swarming] regenerate protos to include prpc bindings 2018-11-07 kjlubick Fix sorting bots by last_seen etc 2018-11-07 akeshet [swarming] create a representation of external scheduler in PoolConfig 2018-11-06 nodir [compile_proto] Pass --prpc-python_out 2018-11-06 kjlubick Make ui2 the default for some pages 2018-11-06 tikuta [client] Add --filepath-filter option to choose download files from isolate 2018-11-06 vadimsh [swarming] Add gce.signed_metadata_token(...) call. 2018-11-06 kjlubick Add support to cancel running tasks from UI 2018-11-06 benjaminwagner Add reboot_required to bot state. 2018-11-05 vadimsh [auth] A function to verify JWTs produced by Google auth backends. 2018-11-02 tikuta Revert "[client] Increase worker to download isolate files faster" 2018-11-01 tikuta [client] Increase worker to download isolate files faster 2018-11-01 akeshet [swarming] proto definition of external scheduler API 2018-11-01 vadimsh [isolate] Don't unexpectedly raise KeyError if the cache state file was lost. 2018-10-31 maruel swarming: add support for sorting by COMPLETED_TS 2018-10-31 akeshet [swarming] add proto definition for swarming tasks 2018-10-31 akeshet [swarming] add external scheduler options to pools.cfg 2018-10-31 maruel Fix a small error in d63fdb9a27d0e89 2018-10-31 kjlubick Make secret bytes less opaque 2018-10-31 smut [Machine Provider] Fix snapshot label match detection 2018-10-30 bpastene swarming: Roll py-adb to 412d35c842fb0917233da6fd2c6c94dfa3450593. 2018-10-30 akeshet [swarming] add "package swarming" to swarming protos 2018-10-29 kjlubick Add aliasing to new botlist 2018-10-25 kjlubick Mostly feature complete Web Components botlist 2018-10-23 smut [Swarming] Support repeatable MP dimensions, reject invalid ones 2018-10-23 smut [GCE Backend] Remove limit on number of instance templates 2018-10-23 smut [GCE Backend] Increase allowable number of instance templates 2018-10-19 smut [Swarming] Require disk_type, num_cpus, project for MP bots 2018-10-19 fmatenaar [auth] Deflake jwt minting cache test 2018-10-19 fmatenaar [auth] Fix for inconsistent result types across token minting functions 2018-10-19 bradhall Add optional-dimension argument to swarming 2018-10-18 nodir [config_service] Add service_access_group 2018-10-18 maruel [swarming] Report memory usage after each handler 2018-10-18 fmatenaar [auth] Migrate chrome infra python code to use IAM:generateAccessToken 2018-10-18 smut [Machine Provider] Advertise snapshot-related labels 2018-10-17 maruel [components] reorder utils.py 2018-10-15 maruel [swarming] fix frontend / backend split 2018-10-09 maruel [swarming] Remove enforcement of unique 'id' in TaskSlice's 2018-10-08 robertocn [run_isolated] Replace parameters in environment variables. 2018-10-05 bpastene swarming: Roll py-adb to pick up log fix. 2018-10-05 hinoka luci-config: Default home page to /#/q/ 2018-10-05 bpastene swarming: Roll py-adb. 2018-10-03 kjlubick Adds the first pieces of the swarming bot-list using WebComponents. 2018-10-03 kjlubick Add sort-toggle element 2018-10-02 kjlubick Update WebComponents deps 2018-09-28 vadimsh [swarming] Return more detailed errors on service account misconfiguration. 2018-09-28 hinoka Fix luci-config UI issues caused due to incompatable npm/bower versions 2018-09-28 kjlubick Add CIPD links to task-page UI 2018-09-28 borenet Fix symlink bug in file_path.set_read_only() 2018-09-28 vadimsh Workaround a bug in task queue's stub. 2018-09-27 hinoka Move opensearch link to index.html 2018-09-27 hinoka Link the opensearch document from the front page 2018-09-27 hinoka Add opensearch to luci-config UI 2018-09-25 kjlubick Fix bad div 2018-09-25 maruel [swarming] enable capacity to outlive the last bot 2018-09-25 kjlubick Fix expiration time for task slices 2018-09-24 maruel [client] improve NamedCache logging 2018-09-24 maruel [client] fix unit test on macOS 2018-09-24 maruel [swarming] enable indexing for BotEvent.dimensions_flat 2018-09-24 kjlubick Prevent soft refresh from changing the current slice 2018-09-24 kjlubick make it more clear when a deduped task was run 2018-09-24 kjlubick Add relative paths to repro commands Created with: roll-dep infra/luci R=vadimsh@chromium.org Bug: 906761 Change-Id: Ibdc25d849b969da65015800868bf7f03d6cbab0d Reviewed-on: https://chromium-review.googlesource.com/c/1343159 Reviewed-by: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org> Cr-Commit-Position: refs/heads/master@{#19075} [modify] https://crrev.com/0b198138877d2e7670dcc56baee83034f0d1b472/DEPS
,
Nov 19
https://chromium-review.googlesource.com/c/infra/infra/+/1342708 fixes monorail tests, but breaks some other tests :) I'm working on fixing them too.
,
Nov 19
infra/libs/buildbucket/proto/* can't find google/rpc/status.proto:
Exception occurred when listing tests. Aborting.
Traceback (most recent call last):
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/expect_tests/pipeline.py", line 521, in result_loop_single_context
gen_loop_process(*test_gen_args)
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/expect_tests/pipeline.py", line 335, in gen_loop_process
result_queue.put_nowait)
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/expect_tests/type_definitions.py", line 465, in gen_stage_loop
for test in tests:
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/expect_tests/pipeline.py", line 278, in generate_tests
gens = get_test_gens_package(testing_context, opts, subpath=subpath)
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/expect_tests/pipeline.py", line 243, in get_test_gens_package
mod = load_module(modname)
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/expect_tests/pipeline.py", line 212, in load_module
mod = __import__(modname)
File "/Users/vadimsh/infra-git/infra/appengine/findit/pipelines/compile_failure/test/revert_and_notify_compile_culprit_pipeline_test.py", line 12, in <module>
from pipelines.compile_failure import (
File "/Users/vadimsh/infra-git/infra/appengine/findit/pipelines/compile_failure/revert_and_notify_compile_culprit_pipeline.py", line 8, in <module>
from pipelines.create_revert_cl_pipeline import CreateRevertCLPipeline
File "/Users/vadimsh/infra-git/infra/appengine/findit/pipelines/create_revert_cl_pipeline.py", line 11, in <module>
from services import culprit_action
File "/Users/vadimsh/infra-git/infra/appengine/findit/services/culprit_action.py", line 20, in <module>
from services import ci_failure
File "/Users/vadimsh/infra-git/infra/appengine/findit/services/ci_failure.py", line 16, in <module>
from services import step_util
File "/Users/vadimsh/infra-git/infra/appengine/findit/services/step_util.py", line 15, in <module>
from common.waterfall import buildbucket_client
File "/Users/vadimsh/infra-git/infra/appengine/findit/common/waterfall/buildbucket_client.py", line 12, in <module>
from buildbucket_proto.rpc_pb2 import GetBuildRequest
File "/Users/vadimsh/infra-git/infra/appengine/findit/first_party/buildbucket_proto/rpc_pb2.py", line 37, in <module>
notification__pb2.DESCRIPTOR,
File "/Users/vadimsh/infra-git/infra/ENV/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 878, in __new__
return _message.default_pool.AddSerializedFile(serialized_pb)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "rpc.proto":
rpc.proto: Import "google/rpc/status.proto" has not been loaded.
buildbucket.v2.BatchResponse.Response.error: "google.rpc.Status" seems to be defined in "status.proto", which is not imported by "rpc.proto". To use it here, please add the necessary import.
Which is very odd, since generated code has
from google.rpc import status_pb2 as google_dot_rpc_dot_status__pb2
which in theory should load rpc.proto's descriptor.
Also 'which is not imported by "rpc.proto"' part is total lie: https://chromium.googlesource.com/infra/luci/luci-go/+/master/buildbucket/proto/rpc.proto#13
,
Nov 19
The following revision refers to this bug: https://chromium.googlesource.com/infra/luci/luci-py.git/+/430746cae9ef8c19c80fbe684be4d3b0ded41a3f commit 430746cae9ef8c19c80fbe684be4d3b0ded41a3f Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Nov 19 23:14:38 2018 [proto] Fix google/rpc/*_pb2.py, it has wrong proto paths in it. status.proto should be added to the registry as google/rpc/status.proto, otherwise importing modules (which import it as 'google/rpc/status.proto') can't find it. R=nodir@chromium.org, iannucci@chromium.org, maruel@chromium.org BUG= 906761 Change-Id: I22781b3745937ddc451d02341cb03cba64493a16 Reviewed-on: https://chromium-review.googlesource.com/c/1342945 Auto-Submit: Vadim Shtayura <vadimsh@chromium.org> Commit-Queue: Robbie Iannucci <iannucci@chromium.org> Reviewed-by: Robbie Iannucci <iannucci@chromium.org> [modify] https://crrev.com/430746cae9ef8c19c80fbe684be4d3b0ded41a3f/client/third_party/google/README.swarming [modify] https://crrev.com/430746cae9ef8c19c80fbe684be4d3b0ded41a3f/client/third_party/google/rpc/code_pb2.py [modify] https://crrev.com/430746cae9ef8c19c80fbe684be4d3b0ded41a3f/client/third_party/google/rpc/error_details_pb2.py [modify] https://crrev.com/430746cae9ef8c19c80fbe684be4d3b0ded41a3f/client/third_party/google/rpc/status_pb2.py
,
Nov 19
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/12cd5f8292dee67610b692fd141431f5a65c544f commit 12cd5f8292dee67610b692fd141431f5a65c544f Author: Vadim Shtayura <vadimsh@chromium.org> Date: Mon Nov 19 23:58:42 2018 Update protobuf lib to 3.6.1. Existing 3.5.1 is no longer compatible with code generated by protoc 3.6.1. Fix all the broken stuff. Also includes infra/luci DEPS roll to pick up _pb2.py change that mutually depends on protobuf lib change: https://chromium.googlesource.com/infra/luci/luci-py/+log/af1fb93ece58..430746cae9ef $ git log af1fb93ec..430746cae --date=short --no-merges --format='%ad %ae %s' 2018-11-19 vadimsh [proto] Fix google/rpc/*_pb2.py, it has wrong proto paths in it. 2018-11-19 iannucci [swarming] Move isolate and cipd server settings to pools.cfg. 2018-11-19 maruel [swarming] expand BotEvent.ALLOWED_EVENTS to be more readable R=maruel@chromium.org BUG= 906761 Change-Id: I41b96fc246f707647951e899348c4187b4b49e6c Reviewed-on: https://chromium-review.googlesource.com/c/1342708 Commit-Queue: Vadim Shtayura <vadimsh@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Cr-Commit-Position: refs/heads/master@{#19077} [modify] https://crrev.com/12cd5f8292dee67610b692fd141431f5a65c544f/infra/libs/buildbucket/swarming/test/flatten_swarmingcfg_test.py [modify] https://crrev.com/12cd5f8292dee67610b692fd141431f5a65c544f/bootstrap/deps.pyl [modify] https://crrev.com/12cd5f8292dee67610b692fd141431f5a65c544f/appengine/cr_rev/appengine_module/cr_rev/test/cr_rev_api_test.py [modify] https://crrev.com/12cd5f8292dee67610b692fd141431f5a65c544f/DEPS [modify] https://crrev.com/12cd5f8292dee67610b692fd141431f5a65c544f/infra/services/bugdroid/config_service.py
,
Nov 20
It should be working now. Requires gclient sync (to pick up 'infra/luci' roll) and gclient runhooks (to rebuild infra/ENV).
,
Nov 20
Thanks! Would it be possible to add a presubmit step to generate monorail proto files before testing? |
||
►
Sign in to add a comment |
||
Comment 1 by vadimsh@chromium.org
, Nov 19Owner: vadimsh@chromium.org
Status: Assigned (was: Untriaged)