New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 10 users

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Feature

Blocked on:
issue 842920

Blocking:
issue 695105
issue 832580



Sign in to add a comment
link

Issue 833946: design higher-level LUCI config

Reported by no...@chromium.org, Apr 17 2018 Project Member

Issue description

cr-buildbucket.cfg and luci-scheduler.cfg files are suffering from duplication inside each file and across the files. In particular, builder names are duplicated. In general, Buildbucket had to come up with "builder mixins" to mitigate repetition. There are other configs that also have builders: cq.cfg, and json files in
https://chromium.googlesource.com/chromium/src/+/a052727f16ac1becbd68deeb2fe5e4bb3c552b5b/testing/buildbot/chromium.mac.json

We need a higher-level config for LUCI services, where a client can define a builder once and specify all of its properties including scheduling and properties. New config format must encapsulation LUCI implementation details, such as the fact that buildbucket and luci-scheduler happen to be separate services.
 
Showing comments 39 - 138 of 138 Older

Comment 39 by bugdroid1@chromium.org, Nov 27

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/36fbecc328f6db9eec0047c4ad5bd6a8e69419c1

commit 36fbecc328f6db9eec0047c4ad5bd6a8e69419c1
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Nov 27 16:50:24 2018

[lucicfg] Move 'State' into its own file, add 'clear_state' for tests.

State will grow noticeably in the future. It is a bit more readable to keep it
in a dedicated file.

R=iannucci@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Ib60dd21143bd18407c35c99e35e762bd99787f89
Reviewed-on: https://chromium-review.googlesource.com/c/1351554
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/36fbecc328f6db9eec0047c4ad5bd6a8e69419c1/lucicfg/generator.go
[add] https://crrev.com/36fbecc328f6db9eec0047c4ad5bd6a8e69419c1/lucicfg/state.go
[modify] https://crrev.com/36fbecc328f6db9eec0047c4ad5bd6a8e69419c1/lucicfg/testdata/hi.star

Comment 41 by bugdroid1@chromium.org, Nov 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/2f37f58c54c9d3ba76f8c5f13b7e890a72e74f24

commit 2f37f58c54c9d3ba76f8c5f13b7e890a72e74f24
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Nov 29 18:12:34 2018

[starlark] Add 'stacktrace(skip=0)' builtin.

It captures a stack trace at the moment it was called and returns it as a value,
so it can be rendered at some later time.

Will be used to "remember" where various lucicfg rules were defined, so that
at a later time we can emit useful error messages.

R=iannucci@chromium.org
BUG=833946

Change-Id: I52d6cf694fc4a454c1534695a8f9ecc53e154c5c
Reviewed-on: https://chromium-review.googlesource.com/c/1354521
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[add] https://crrev.com/2f37f58c54c9d3ba76f8c5f13b7e890a72e74f24/starlark/builtins/stacktrace.go
[add] https://crrev.com/2f37f58c54c9d3ba76f8c5f13b7e890a72e74f24/starlark/builtins/stacktrace_test.go

Comment 42 by bugdroid1@chromium.org, Nov 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/8dbbd471c4ba648397b7b44423a33503ec463b4d

commit 8dbbd471c4ba648397b7b44423a33503ec463b4d
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Nov 29 22:21:55 2018

[lucicfg] Add an internal API for emitting errors with captured stack traces.

Also teach Starlark tests to declare they expect an error to happen.

R=iannucci@chromium.org
BUG=833946

Change-Id: If397c472600a973d2c3cf0b3db88f291c94cc7cb
Reviewed-on: https://chromium-review.googlesource.com/c/1354524
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/cmd/main.go
[add] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/errors.go
[modify] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/generator.go
[delete] https://crrev.com/e929ae24be82178d719cb82da40b7123c761d19d/lucicfg/generator_test.go
[delete] https://crrev.com/e929ae24be82178d719cb82da40b7123c761d19d/lucicfg/greetings.go
[modify] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/starlark/stdlib/builtins.star
[add] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/starlark/stdlib/internal/error.star
[modify] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/starlark_test.go
[modify] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/state.go
[add] https://crrev.com/8dbbd471c4ba648397b7b44423a33503ec463b4d/lucicfg/testdata/errors.star
[delete] https://crrev.com/e929ae24be82178d719cb82da40b7123c761d19d/lucicfg/testdata/hi.star

Comment 43 by bugdroid1@chromium.org, Nov 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/583d93c146cb0cce6785f40be89f33d7a7cb875a

commit 583d93c146cb0cce6785f40be89f33d7a7cb875a
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Nov 29 22:49:46 2018

[lucicfg] Low-level API for generating arbitrary configs.

This is an escape hatch for cases when a high-level declarative API is not
sufficient. It will also be used internally by the high-level generator itself.

Config generating functions are registered as callbacks to allow high-level
configs to be defined in a declarative way, where order of definitions is not
important, as long as the entire high-level config is consistent. To allow this,
we should execute all top-level Starlark code before doing any evaluation of
the state, thus a necessity for delaying generator calls via callbacks.

R=iannucci@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Ib62640a316fc1255de873defc521be7cf67665bd
Reviewed-on: https://chromium-review.googlesource.com/c/1352371
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/cmd/main.go
[add] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/configset.go
[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/errors.go
[add] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/genctx.go
[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/generator.go
[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/protos_test.go
[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/state.go
[add] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/testdata/config_set.star
[add] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/testproto/gen.go
[add] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/testproto/test.pb.go
[add] https://crrev.com/583d93c146cb0cce6785f40be89f33d7a7cb875a/lucicfg/testproto/test.proto

Comment 45 by bugdroid1@chromium.org, Dec 3

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4b0a053fadfba9ae294733dff376682956d6a369

commit 4b0a053fadfba9ae294733dff376682956d6a369
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Dec 03 19:14:56 2018

[starlark] Expose CaptureStacktrace as public Go function.

This would allow to capture stack traces in Go implementation of various builtin
methods.

R=nodir@chromium.org
BUG=833946

Change-Id: I7ccc5d6604697d046696d6cc7d464f9fccb9edf4
Reviewed-on: https://chromium-review.googlesource.com/c/1357256
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/4b0a053fadfba9ae294733dff376682956d6a369/starlark/builtins/stacktrace.go

Comment 46 by bugdroid1@chromium.org, Dec 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/7838e91a61d4bedab52cc2c4539d9f3273507e98

commit 7838e91a61d4bedab52cc2c4539d9f3273507e98
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Dec 04 01:35:22 2018

[lucicfg] Implement 'graph.key(typ1, id1, typ2, id2, ...)' internal API.

Config entities will be defined as named nodes in a DAG. graph.key(...) returns
an opaque label that identifies such a node.

We intern keys to make their identities (~address) match their values 1-to-1.
That allows them to be used as keys in Starlark dicts and Go maps right away,
without any additional code.

R=nodir@chromium.org
BUG=833946

Change-Id: I4db550081e331b2e4c49761ac581f0543814e213
Reviewed-on: https://chromium-review.googlesource.com/c/1357257
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/graph/doc.go
[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/graph/graph.go
[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/graph/key.go
[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/graph/starlark_test.go
[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/graph/testdata/key.star
[modify] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/starlark/assets.gen.go
[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/state.go
[add] https://crrev.com/7838e91a61d4bedab52cc2c4539d9f3273507e98/lucicfg/testdata/graph.star

Comment 47 by bugdroid1@chromium.org, Dec 4

Project Member

Comment 48 by bugdroid1@chromium.org, Dec 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/fa7cff5e3a65d834aafbc1228149bd1bdade0f90

commit fa7cff5e3a65d834aafbc1228149bd1bdade0f90
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Dec 04 21:52:22 2018

[lucicfg] Implement 'graph.add_edge' API.

Checks for cycles. Doesn't yet check for dangling edges.

R=nodir@chromium.org
BUG=833946

Change-Id: Ia1642181d97632ebaa96c87ae6b564e34471dd2f
Reviewed-on: https://chromium-review.googlesource.com/c/1361566
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/fa7cff5e3a65d834aafbc1228149bd1bdade0f90/lucicfg/graph/graph.go
[modify] https://crrev.com/fa7cff5e3a65d834aafbc1228149bd1bdade0f90/lucicfg/graph/node.go
[add] https://crrev.com/fa7cff5e3a65d834aafbc1228149bd1bdade0f90/lucicfg/graph/testdata/edge.star

Comment 49 by bugdroid1@chromium.org, Dec 5

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/08b8fa0527ebe6b2cc31f03b8720e745858e6bda

commit 08b8fa0527ebe6b2cc31f03b8720e745858e6bda
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Dec 05 00:47:14 2018

[lucicfg] Check that the graph has no dangling edges when finalizing it.

Also:
  1. Make all graph errors implement BacktracableError.
  2. Forbid querying non-finalized graphs.

R=nodir@chromium.org
BUG=833946

Change-Id: Ic912f5379a09cf12490fe3b16d547ad695e01639
Reviewed-on: https://chromium-review.googlesource.com/c/1361576
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/errors.go
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/generator.go
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/graph/graph.go
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/graph/node.go
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/graph/testdata/edge.star
[add] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/graph/testdata/finalization.star
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/graph/testdata/node.star
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/state.go
[add] https://crrev.com/08b8fa0527ebe6b2cc31f03b8720e745858e6bda/lucicfg/testdata/graph_dangling_edge.star

Comment 50 by bugdroid1@chromium.org, Dec 5

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0cc8989a28833d473520233ae2ffff4e6561f4fa

commit 0cc8989a28833d473520233ae2ffff4e6561f4fa
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Dec 05 20:47:28 2018

[lucicfg] Slightly improve error messages by removing redundant info.

Instead of reporting the full key (a list of (kind, id) pairs), we report only
the last pair. The stack trace should provide enough context to understand
what exactly the error refers to.

This reduces how many internal implementation details (e.g. composite keys) we
leak to the outside.

R=nodir@chromium.org
BUG=833946

Change-Id: I4227d5c015b037c498d63f4d4a3104d2aff5b91f
Reviewed-on: https://chromium-review.googlesource.com/c/1362398
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/0cc8989a28833d473520233ae2ffff4e6561f4fa/lucicfg/graph/graph.go
[modify] https://crrev.com/0cc8989a28833d473520233ae2ffff4e6561f4fa/lucicfg/graph/key.go
[modify] https://crrev.com/0cc8989a28833d473520233ae2ffff4e6561f4fa/lucicfg/graph/node.go
[modify] https://crrev.com/0cc8989a28833d473520233ae2ffff4e6561f4fa/lucicfg/graph/testdata/edge.star
[modify] https://crrev.com/0cc8989a28833d473520233ae2ffff4e6561f4fa/lucicfg/graph/testdata/node.star
[modify] https://crrev.com/0cc8989a28833d473520233ae2ffff4e6561f4fa/lucicfg/testdata/graph_dangling_edge.star

Comment 51 by bugdroid1@chromium.org, Dec 6

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0e40450f0980d1ae6aa1ab2150a065f4dd25a798

commit 0e40450f0980d1ae6aa1ab2150a065f4dd25a798
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Dec 06 21:28:50 2018

[lucicfg] Pattern matching for expected errors messages in tests.

Now we can declare that a test is expected to emit an error like:

Traceback (most recent call last):
  ...
  //testdata/errors_like.star:???: in func2
  //testdata/errors_like.star:???: in capture_stack
Error: ??? err

And '...' will be matched to zero or more arbitrary lines and '???' to a word.

This should make tests that examine errors much less brittle.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I2d978ff7df55d1291f92cee36bb6a7f7cc6bf4aa
Reviewed-on: https://chromium-review.googlesource.com/c/1365130
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/0e40450f0980d1ae6aa1ab2150a065f4dd25a798/lucicfg/starlark_test.go
[add] https://crrev.com/0e40450f0980d1ae6aa1ab2150a065f4dd25a798/lucicfg/testdata/errors_like.star

Comment 52 by bugdroid1@chromium.org, Dec 6

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/e10def5af7048a119f1c9b096564de4d4510fddd

commit e10def5af7048a119f1c9b096564de4d4510fddd
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Dec 06 21:30:40 2018

[starlark] Make starlarktest.RunTests discover tests recursively.

This will allow to group tests by package etc. instead of having a ton of files
in a single directory.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I814b65850b422f48d0040a159317dcb239e065cd
Reviewed-on: https://chromium-review.googlesource.com/c/1365131
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/e10def5af7048a119f1c9b096564de4d4510fddd/starlark/starlarktest/runner.go
[add] https://crrev.com/e10def5af7048a119f1c9b096564de4d4510fddd/starlark/starlarktest/testdata/d/works_too.star

Comment 53 by bugdroid1@chromium.org, Dec 7

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/004d0b87580c34886825603c346097c48a7229d1

commit 004d0b87580c34886825603c346097c48a7229d1
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Dec 07 00:07:30 2018

[lucicfg] Implement 'graph.children(...)' query.

Returns direct children of a node with given kind, sorted either by their key
(default), or the order of how they were linked to the parent node (will be
useful for Milo configs, where an order of builders in a builder group matters
for UI).

R=nodir@chromium.org
BUG=833946

Change-Id: I1da12a8174790268a97bc0018faf2e3c5c9e695b
Reviewed-on: https://chromium-review.googlesource.com/c/1363877
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/graph/graph.go
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/graph/key.go
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/graph/node.go
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/graph/starlark_test.go
[add] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/graph/testdata/children.star
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/graph/testdata/finalization.star
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/004d0b87580c34886825603c346097c48a7229d1/lucicfg/testdata/graph.star

Comment 54 by bugdroid1@chromium.org, Dec 7

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/df9b4818ec08df51f50efe1b61e34d00c644667d

commit df9b4818ec08df51f50efe1b61e34d00c644667d
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Dec 07 04:10:02 2018

[lucicfg] Setup basic structure of LUCI rules and generators.

Implement very simple subsets of core.project(), core.bucket() and
core.logdog() as an example.

In particular:
  * All rules (node definition statements) live in //internal/luci/rules/*.star,
    one per file for easier search.
  * //internal/luci/common.star defines keys for all LUCI nodes.
  * //internal/luci/generators.star implements all config generators.
  * //internal/luci/lib/*.star are various utilities.

R=nodir@chromium.org
BUG=833946

Change-Id: I05ca65676508cdafb99241dad05dbf9d00bb27a4
Reviewed-on: https://chromium-review.googlesource.com/c/1364750
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/cmd/main.go
[modify] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/graph/graph.go
[modify] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/protos.go
[modify] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/builtins.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/generator.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/common.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/generators.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/lib/service.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/rules/bucket.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/rules/logdog.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/starlark_test.go
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/errors/bucket_without_project.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/errors/bucket_without_service.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/errors/logdog_without_project.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/errors/project_redefinition.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/errors/project_without_name.star
[add] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/full_example.star
[rename] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/misc/config_set.star
[rename] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/misc/errors.star
[rename] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/misc/errors_like.star
[rename] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/misc/generator_api.star
[rename] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/misc/graph.star
[rename] https://crrev.com/df9b4818ec08df51f50efe1b61e34d00c644667d/lucicfg/testdata/misc/graph_dangling_edge.star

Comment 55 by bugdroid1@chromium.org, Dec 10

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/a55c8a7168eac483e5405111adb47b5a70c595f7

commit a55c8a7168eac483e5405111adb47b5a70c595f7
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Dec 10 23:50:40 2018

[lucicfg] Boilerplate for command line interface.

Follows current "best practices" of CLIs in luci-go:
  * All subcommands have -log-level and -json-output.
  * Subcommand that do RPCs consistently expose auth and service host options.
  * All Chrome Infra hardcoded stuff is replaceable (if only anyone wanted to
    replace it).
  * Has version string that also ends up in User-Agent strings.
  * Supports required positional arguments and required flags.

R=iannucci@chromium.org
BUG=833946

Change-Id: I792d416e21fe82220eda9df8eed49e729c70d40b
Reviewed-on: https://chromium-review.googlesource.com/c/1368813
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/hardcoded/chromeinfra/chromeinfra.go
[add] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/lucicfg/cli/app.go
[add] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/lucicfg/cli/base.go
[add] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/lucicfg/cli/generate.go
[add] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/lucicfg/cli/validate.go
[add] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/lucicfg/cmd/lucicfg/main.go
[delete] https://crrev.com/e0291a833256290b8551792e6750ec1e02a861c1/lucicfg/cmd/main.go
[add] https://crrev.com/a55c8a7168eac483e5405111adb47b5a70c595f7/lucicfg/version.go

Comment 56 by bugdroid1@chromium.org, Dec 12

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/2e46e5dee4bb0e3ce4367435865925348fde82bd

commit 2e46e5dee4bb0e3ce4367435865925348fde82bd
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Dec 12 00:02:01 2018

[starlark] Add genstruct(...) and ctor(...) builtins.

They can be used to create structs "tagged" with a symbol. It's not quite
like a typed struct (each individual object still can have arbitrary fields),
but pretty close:

   mystruct = genstruct("mystruct")
   s = mystruct(a=1, b=2)
   assert.eq(ctor(s), mystruct)
   assert.eq(ctor(1), None)
   assert.eq(ctor(struct(a=1, b=2)), "struct")

Can be used to add some sort of type safety to Starlark libraries. Note that
using type(strct) is insufficient, since it always returns "struct".

R=nodir@chromium.org, iannucci@chromium.org
BUG=833946

Change-Id: I65abc0594441367a7ecd3c0faa05c06499d61c30
Reviewed-on: https://chromium-review.googlesource.com/c/1370693
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/2e46e5dee4bb0e3ce4367435865925348fde82bd/starlark/builtins/struct.go
[add] https://crrev.com/2e46e5dee4bb0e3ce4367435865925348fde82bd/starlark/builtins/struct_test.go

Comment 57 by bugdroid1@chromium.org, Dec 12

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/6c166b858d7d555ba7474f1ceac79a8a5a060ce4

commit 6c166b858d7d555ba7474f1ceac79a8a5a060ce4
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Dec 12 02:20:50 2018

[lucicfg] Add @stdlib//internal/luci/lib/acl.star library.

It defines public API for acls:
  * A function that constructs an ACL entry: acl.entry(...).
  * Various initial LUCI roles, e.g acl.BUILDBUCKET_READER.

Also contains internal API used by the rules implementation:
  * aclimpl.validate_acls: checks format of acl.entry(...) definitions.
  * aclimpl.normalize_acls: expands, dedups and sorts ACLs.

More roles will be added when we add support for more services.

R=nodir@chromium.org, iannucci@chromium.org
BUG=833946

Change-Id: I9a41b3ffdc745b66138f73ede984aeff27984b17
Reviewed-on: https://chromium-review.googlesource.com/c/1370611
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/generator.go
[modify] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/starlark/stdlib/builtins.star
[add] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/starlark/stdlib/internal/luci/lib/acl.star
[modify] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[add] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/testdata/acl/entry.star
[add] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/testdata/acl/normalize_acls.star
[add] https://crrev.com/6c166b858d7d555ba7474f1ceac79a8a5a060ce4/lucicfg/testdata/acl/validate_acls.star

Comment 58 by bugdroid1@chromium.org, Dec 12

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/71109ce07933a26aa8ec91eda73856d3350c5e46

commit 71109ce07933a26aa8ec91eda73856d3350c5e46
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Dec 12 02:33:00 2018

[lucicfg] Add support for luci-config, logdog and buildbucket ACLs.

ACLs can be defined either on a project or on a bucket levels. ACLs defined on
the project level are inherited by all buckets.

Additionally, some ACLs (such as PROJECT_CONFIGS_READER, LOGDOG_*) can be
defined only on the project level, since LUCI doesn't support defining them on
a bucket level yet or they make no sense when defined on a bucket level.

R=nodir@chromium.org, iannucci@chromium.org
BUG=833946

Change-Id: Ib6175cb2cdc83aae7300c5e2a731882f6ecc4430
Reviewed-on: https://chromium-review.googlesource.com/c/1370612
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/71109ce07933a26aa8ec91eda73856d3350c5e46/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/71109ce07933a26aa8ec91eda73856d3350c5e46/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/71109ce07933a26aa8ec91eda73856d3350c5e46/lucicfg/starlark/stdlib/internal/luci/rules/bucket.star
[modify] https://crrev.com/71109ce07933a26aa8ec91eda73856d3350c5e46/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/71109ce07933a26aa8ec91eda73856d3350c5e46/lucicfg/testdata/full_example.star

Comment 59 by bugdroid1@chromium.org, Dec 20

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/82607619879e0e403f666fea49c137a2b8611f5c

commit 82607619879e0e403f666fea49c137a2b8611f5c
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Dec 20 01:15:48 2018

[lucicfg] Make graph.key objects introspectable through Starlark.

They now have attributes 'container', 'kind', 'id'. This would become useful
for implementing rules that behave differently depending on a kind of an object
they are related to.

For the key [(k1, i1), (k2, i2), (k3, i3)]:
  'container' is key [(k1, i1), (k2, i2)]
  'kind' is string 'k3'
  'id' is string 'i3'

Term "container" was chosen to avoid confusion with term "parent", already used
in the graph API to denote a different sort of relation.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I16ea3709b6a2adbff9a9d7f810d1080cd5c48f24
Reviewed-on: https://chromium-review.googlesource.com/c/1385666
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/82607619879e0e403f666fea49c137a2b8611f5c/lucicfg/graph/graph.go
[modify] https://crrev.com/82607619879e0e403f666fea49c137a2b8611f5c/lucicfg/graph/key.go
[add] https://crrev.com/82607619879e0e403f666fea49c137a2b8611f5c/lucicfg/graph/keyset.go
[modify] https://crrev.com/82607619879e0e403f666fea49c137a2b8611f5c/lucicfg/graph/node.go
[modify] https://crrev.com/82607619879e0e403f666fea49c137a2b8611f5c/lucicfg/graph/testdata/key.star

Comment 60 by bugdroid1@chromium.org, Dec 20

Project Member

Comment 61 by bugdroid1@chromium.org, Dec 20

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4749df1b10b6d6fb810a1d88021d6c84ff081b2d

commit 4749df1b10b6d6fb810a1d88021d6c84ff081b2d
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Dec 20 22:03:09 2018

[lucicfg] Show full key path in node.String().

This will be helpful for printing error messages involving ambiguities when
builder with a same name is defined in multiple buckets.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I91bde848d8a45a4fd745f1a040d7d153d6b69b73
Reviewed-on: https://chromium-review.googlesource.com/c/1387628
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/4749df1b10b6d6fb810a1d88021d6c84ff081b2d/lucicfg/graph/node.go
[modify] https://crrev.com/4749df1b10b6d6fb810a1d88021d6c84ff081b2d/lucicfg/graph/testdata/node.star

Comment 62 by sergeybe...@chromium.org, Dec 20

FYI, I'm starting a parallel effort for chromium/src side configs in issue 912367. Design doc is in the works, I expect to release it in January 2019.

There is a chance the two efforts will roll into a single larger project, since separating LUCI-only configs and chromium-specific configs still results in unnecessary duplication.

Comment 63 by bugdroid1@chromium.org, Dec 20

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/b889c4bdc18d45184d8a88a6eb02fc653f5c68d3

commit b889c4bdc18d45184d8a88a6eb02fc653f5c68d3
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Dec 20 23:19:04 2018

[lucicfg] Allow nodes to be marked as idempotent.

Idempotent nodes can be redeclared, but only if previous declaration was also
marked as idempotent and has same 'props'. This is useful to "auto-declare"
nodes of first usage.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I4d86c0681974d980cab8a39d3f5c8bccf35b9347
Reviewed-on: https://chromium-review.googlesource.com/c/1385673
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/b889c4bdc18d45184d8a88a6eb02fc653f5c68d3/lucicfg/graph/graph.go
[modify] https://crrev.com/b889c4bdc18d45184d8a88a6eb02fc653f5c68d3/lucicfg/graph/node.go
[modify] https://crrev.com/b889c4bdc18d45184d8a88a6eb02fc653f5c68d3/lucicfg/graph/testdata/node.star
[modify] https://crrev.com/b889c4bdc18d45184d8a88a6eb02fc653f5c68d3/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/b889c4bdc18d45184d8a88a6eb02fc653f5c68d3/lucicfg/starlark/stdlib/internal/graph.star

Comment 64 by bugdroid1@chromium.org, Dec 20

Project Member

Comment 65 by bugdroid1@chromium.org, Dec 21

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/d4f2cd56eee65b842128834e760da7632cf2ede0

commit d4f2cd56eee65b842128834e760da7632cf2ede0
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Dec 21 02:50:48 2018

[starlark] Teach 'fail' builtin to accept custom stack trace for errors.

This is useful in lucicfg to display stack traces of where nodes are
defined when discovering errors during the generation phase.

Unfortunately Starlark stringifies errors from builtins, so we have to pass
structured information in a roundabout way though thread locals.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I5d6da3b9765679df456fb599a604bc3424879599
Reviewed-on: https://chromium-review.googlesource.com/c/1387635
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/d4f2cd56eee65b842128834e760da7632cf2ede0/starlark/builtins/fail.go
[modify] https://crrev.com/d4f2cd56eee65b842128834e760da7632cf2ede0/starlark/builtins/fail_test.go

Comment 66 by bugdroid1@chromium.org, Dec 21

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/577e508664290fabf83a1a3093d8a4a51258aab7

commit 577e508664290fabf83a1a3093d8a4a51258aab7
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Dec 21 03:04:45 2018

[lucicfg] Display traces to badly defined nodes on errors in generators.

Also don't stop on the first error, keep running generator callbacks to collect
all errors. Generators should not depend on each other.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I40d57948465b0a58b9db8630ad3ae1f55b7b4698
Reviewed-on: https://chromium-review.googlesource.com/c/1387636
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/configset.go
[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/errors.go
[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/generator.go
[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/starlark_test.go
[modify] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/testdata/errors/bucket_without_service.star
[add] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/testdata/misc/fail_with_stack.star
[add] https://crrev.com/577e508664290fabf83a1a3093d8a4a51258aab7/lucicfg/testdata/misc/generator_errors.star

Comment 67 by bugdroid1@chromium.org, Dec 21

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/2c6bc8a7ee925d1b79982a177097237d0e57d6c2

commit 2c6bc8a7ee925d1b79982a177097237d0e57d6c2
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Dec 21 03:12:45 2018

[lucicfg] Deduplicate errors with identical backtraces.

May happen if different generators all discover exact same erroneous node in
the graph.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I6421e721d7b98406c62b1783e226d6315b2f4d84
Reviewed-on: https://chromium-review.googlesource.com/c/1387638
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/2c6bc8a7ee925d1b79982a177097237d0e57d6c2/lucicfg/generator.go
[modify] https://crrev.com/2c6bc8a7ee925d1b79982a177097237d0e57d6c2/lucicfg/state.go
[modify] https://crrev.com/2c6bc8a7ee925d1b79982a177097237d0e57d6c2/lucicfg/testdata/misc/generator_errors.star

Comment 70 by bugdroid1@chromium.org, Dec 27

Project Member

Comment 71 by bugdroid1@chromium.org, Jan 2

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0b841cfedf2076c9fb6e20af92562f53fc003ea9

commit 0b841cfedf2076c9fb6e20af92562f53fc003ea9
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 02 21:07:33 2019

[starlarkproto] Add support for map<k,v> fields.

Also improve error messages when assigning unexpected types to a proto-valued
fields.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: Ia648549d33dc74842fff2e0be063ee9802bdbd59
Reviewed-on: https://chromium-review.googlesource.com/c/1392589
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/conversions.go
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/default.go
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/doc.go
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/loader.go
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/loader_test.go
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testdata/ints.star
[add] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testdata/map_int.star
[add] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testdata/map_msg.star
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testdata/misc.star
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testdata/msg.star
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testdata/msg_repeated.star
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testprotos/test.pb.go
[modify] https://crrev.com/0b841cfedf2076c9fb6e20af92562f53fc003ea9/starlark/starlarkproto/testprotos/test.proto

Comment 72 by bugdroid1@chromium.org, Jan 2

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/aa907e8fbcd3aeb840ba9d5f9a2f810829ad1041

commit aa907e8fbcd3aeb840ba9d5f9a2f810829ad1041
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 02 21:15:33 2019

[lucicfg] Expose well-known protobuf types to Starlark code.

Some configs use them, e.g. Buildbucket uses google.protobuf.UInt32Value.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: Ia578386c4d128e35bfa393ba1446a9217e6312e3
Reviewed-on: https://chromium-review.googlesource.com/c/1392588
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/aa907e8fbcd3aeb840ba9d5f9a2f810829ad1041/lucicfg/protos.go

Comment 73 by bugdroid1@chromium.org, Jan 3

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/8f20b70edb7d57e1fb618b36fc51e5add64cb6fc

commit 8f20b70edb7d57e1fb618b36fc51e5add64cb6fc
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 03 22:35:30 2019

[lucicfg] Add a mode for tests in which error(...) immediately aborts execution.

Note that by default error(...) just collects the error without aborting the
script, all collected errors are reported at the end.

fail_on_errors() mode is useful for writing smallish unit tests that check error
detection logic.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I0ca06978871ec1733ba12d9104fb6675eec99467
Reviewed-on: https://chromium-review.googlesource.com/c/1394190
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/8f20b70edb7d57e1fb618b36fc51e5add64cb6fc/lucicfg/errors.go
[modify] https://crrev.com/8f20b70edb7d57e1fb618b36fc51e5add64cb6fc/lucicfg/state.go
[add] https://crrev.com/8f20b70edb7d57e1fb618b36fc51e5add64cb6fc/lucicfg/testdata/misc/fail_on_errors_mode.star

Comment 74 by bugdroid1@chromium.org, Jan 3

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0

commit 0ed1f105d7c6c668d1072cce1e63d6c6db8147a0
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 03 22:47:50 2019

[lucicfg] Add graph.sorted_nodes function, rename 'exec' order to 'def'.

Also clarify that graph.children(..., 'def') and graph.parents(..., 'def')
sort nodes according to the order edges where defined, regardless of the order
of definition of nodes themselves.

On the other hand, sorted_nodes(..., 'def') sorts nodes by the order they were
declared in the graph.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: If3ce2580c72ca3ccf99e0c55130c699a7b03abdc
Reviewed-on: https://chromium-review.googlesource.com/c/1394247
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/graph/graph.go
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/graph/node.go
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/graph/testdata/children.star
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/graph/testdata/descendants.star
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/graph/testdata/parents.star
[add] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/graph/testdata/sorting.star
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/0ed1f105d7c6c668d1072cce1e63d6c6db8147a0/lucicfg/testdata/misc/graph.star

Comment 75 by bugdroid1@chromium.org, Jan 3

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/dd8514e8c3a0e491ee942b3552b32653bec08b5d

commit dd8514e8c3a0e491ee942b3552b32653bec08b5d
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 03 23:44:31 2019

[lucicfg] Initial empty definition of core.builder rule.

There is an aspect that makes the implementation somewhat tricky: we want to
allow to refer to builders using their names alone (without specifying the
bucket name), if this is unambiguous. For cases when there are more than one
builder with a given name, the ambiguity can be resolved by using the full
builder name (e.g. "ci/linux builder").

This is solved by introduction of intermediate 'builder_ref' node:
  * Nodes that refer to builders have builder_refs as children.
  * A builder_ref has a builder as a child.
  * A single builder has two parent builder_ref's: one has bucket-scoped
    name (e.g. "ci/linux builder") and another is global (e.g. "linux builder").
  * Ambiguities manifest as builder_ref nodes with more than one child.

There's an internal API to follow such refs.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I1c126dd3f387a5915f4c49079b05acb1d95b45c8
Reviewed-on: https://chromium-review.googlesource.com/c/1390984
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/starlark/stdlib/internal/luci/common.star
[modify] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/starlark/stdlib/internal/luci/generators.star
[add] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/testdata/full_example.star
[add] https://crrev.com/dd8514e8c3a0e491ee942b3552b32653bec08b5d/lucicfg/testdata/misc/builder_refs.star

Comment 76 by bugdroid1@chromium.org, Jan 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/6a1d1ce6842335fe93e01e332664e9d60130ccb7

commit 6a1d1ce6842335fe93e01e332664e9d60130ccb7
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 04 20:54:46 2019

[lucicfg] Module to define durations in a way similar to go's time.Duration.

Will be used to get rid of awkward '_sec' in various '<something>_sec' fields
and also to transform stuff like '1800 # 30 min' into '30 * time.minute'.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I32abe93bbf68bff4d3dc95c31b55d4f19e93b9a9
Reviewed-on: https://chromium-review.googlesource.com/c/1395742
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[add] https://crrev.com/6a1d1ce6842335fe93e01e332664e9d60130ccb7/lucicfg/duration.go
[modify] https://crrev.com/6a1d1ce6842335fe93e01e332664e9d60130ccb7/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/6a1d1ce6842335fe93e01e332664e9d60130ccb7/lucicfg/starlark/stdlib/builtins.star
[add] https://crrev.com/6a1d1ce6842335fe93e01e332664e9d60130ccb7/lucicfg/starlark/stdlib/internal/time.star
[add] https://crrev.com/6a1d1ce6842335fe93e01e332664e9d60130ccb7/lucicfg/testdata/misc/time.star

Comment 77 by bugdroid1@chromium.org, Jan 4

Project Member

Comment 79 by bugdroid1@chromium.org, Jan 8

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/87686f6c1da41f45a3143646ac0f8f2b9d5f8964

commit 87686f6c1da41f45a3143646ac0f8f2b9d5f8964
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 08 01:46:59 2019

[lucicfg] Add validate.int(), validate.bool() and validate.str_dict().

validate.int will be used to validate e.g. swarming priority
validate.bool will be used to validate Toggle-typed buildbucket fields.
validate.str_dict will be used to validate e.g. dimensions and properties.

Adjust the interface of the validators slightly to allow None to be used as
a value for required=False fields. It is useful to propagate None as "hasn't
been specified" flag (or equivalently to represent Toggle.UNSET), instead of
replacing it with some default value right away.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Ifbdc9a740972803e7f6d4bb707b07bcd418fb74b
Reviewed-on: https://chromium-review.googlesource.com/c/1399142
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/87686f6c1da41f45a3143646ac0f8f2b9d5f8964/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/87686f6c1da41f45a3143646ac0f8f2b9d5f8964/lucicfg/starlark/stdlib/internal/luci/lib/service.star
[modify] https://crrev.com/87686f6c1da41f45a3143646ac0f8f2b9d5f8964/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[modify] https://crrev.com/87686f6c1da41f45a3143646ac0f8f2b9d5f8964/lucicfg/starlark/stdlib/internal/luci/rules/logdog.star
[modify] https://crrev.com/87686f6c1da41f45a3143646ac0f8f2b9d5f8964/lucicfg/testdata/misc/validators.star

Comment 80 by bugdroid1@chromium.org, Jan 8

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/4c84998eba52eb3a7efb27112a07adab7215bda2

commit 4c84998eba52eb3a7efb27112a07adab7215bda2
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 08 02:48:24 2019

[cipd] Add infra/tools/luci/lucicfg/${platform} package.

It will probably become part of depot_tools, so build it for all platforms
supported by depot_tools.

R=nodir@chromium.org
CC=garymm@google.com
BUG=833946

Change-Id: Iaf57032eddb514930799045230582fb53b5f9715
Reviewed-on: https://chromium-review.googlesource.com/c/1399321
Auto-Submit: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Cr-Commit-Position: refs/heads/master@{#19819}
[add] https://crrev.com/4c84998eba52eb3a7efb27112a07adab7215bda2/build/packages/lucicfg.yaml

Comment 81 by bugdroid1@chromium.org, Jan 8

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c

commit cb94fd9523b01c1b26c29b6408cb5262da0fbe7c
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 08 19:29:22 2019

[lucicfg] Add empty core.gitiles_poller, define 'triggers/triggered_by' rels.

It's a first instance of polymorphism in lucicfg: 'triggered_by' can refer to
either a builder or a gitiles_poller. As additional complication, we allow to
refer to builders via either their full names ("<bucket>/<name>") or short names
("<name>"), if the latter is non-ambiguous. Same for gitiles_poller names.

This is resolved by introduction of a new internal 'triggerer' node: it is an
entity that can be referred to in 'triggered_by' relation. Both 'builder' and
'gitiles_poller' (each) have two 'triggerer' nodes as children: a bucket-scoped
one and a global one.

This is very similar to 'builder_ref', except the direction of the relation is
reversed: a 'builder_ref' has associated builders as children, but 'triggerer'
has them as parents. This allows to catch cycles in triggering graphs.

For example, a builder that triggers itself produces the following cycle:
  core.builder_ref('b') ->
  core.builder('ci/b') ->
  core.triggerer('ci/b') ->
  core.builder_ref('b')

While such cycle doesn't necessarily mean an invalid configuration, it does
have a potential to introduce a positive feedback loop in runtime that can blow
everything up, so we forbid them.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I25b21a51fc2b355a8688182ccf55820c768ee46b
Reviewed-on: https://chromium-review.googlesource.com/c/1394188
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/starlark/stdlib/internal/luci/common.star
[modify] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[add] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[add] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/testdata/errors/ambiguous_references.star
[add] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/testdata/errors/poller_builder_clash.star
[modify] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/testdata/full_example.star
[add] https://crrev.com/cb94fd9523b01c1b26c29b6408cb5262da0fbe7c/lucicfg/testdata/misc/triggerer.star

Comment 82 by bugdroid1@chromium.org, Jan 8

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/80712f8b382bbdae855ec0d1d01f6d6170a94981

commit 80712f8b382bbdae855ec0d1d01f6d6170a94981
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 08 20:14:56 2019

[lucicfg] Make validate.duration() return duration instead of number of units.

Turns out it is more useful to keep 'duration' type until the very end.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I815b32813a752db9273091cc98bb4bc5cb7ed91f
Reviewed-on: https://chromium-review.googlesource.com/c/1401327
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/80712f8b382bbdae855ec0d1d01f6d6170a94981/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/80712f8b382bbdae855ec0d1d01f6d6170a94981/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[modify] https://crrev.com/80712f8b382bbdae855ec0d1d01f6d6170a94981/lucicfg/testdata/misc/validators.star

Comment 83 by bugdroid1@chromium.org, Jan 8

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/78b43ec434fc1fc1ae67023923cacd7c9884c7d2

commit 78b43ec434fc1fc1ae67023923cacd7c9884c7d2
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 08 20:29:37 2019

[lucicfg] Be careful when mixing assert.fails(...) and fail(...) builtin.

fail(...) builtin uses a thread local state (FailureCollector) to capture
structured information about an error (because Starlark stringifies all errors
returned by a builtin).

assert.fails(...) "catches" starlark errors, but it doesn't clear the captured
error in the thread local state. Fixing this would require forking assert
library, which I'd like to avoid.

So instead just don't use failure collector in tests that don't care about
extra information (like custom stack trace) attached to errors. In lucicfg only
tests that use "Expect errors" blocks care about custom stack traces, and they
generally do not use 'assert' library.

R=tandrii@chromium.org
BUG=833946

Change-Id: I06d9f8fe48c6720070141a4d2e2bcd3d9d88584a
Reviewed-on: https://chromium-review.googlesource.com/c/1397345
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/78b43ec434fc1fc1ae67023923cacd7c9884c7d2/lucicfg/configset.go
[modify] https://crrev.com/78b43ec434fc1fc1ae67023923cacd7c9884c7d2/lucicfg/generator.go
[modify] https://crrev.com/78b43ec434fc1fc1ae67023923cacd7c9884c7d2/lucicfg/starlark_test.go
[modify] https://crrev.com/78b43ec434fc1fc1ae67023923cacd7c9884c7d2/starlark/builtins/fail.go

Comment 84 by bugdroid1@chromium.org, Jan 9

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/73abe063c75c7926c68e9520154a169df4c2ad7d

commit 73abe063c75c7926c68e9520154a169df4c2ad7d
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 09 01:14:07 2019

[lucicfg] Add swarming helper module.

It defines swarming.cache(...) and swarming.dimension(...) structs that will be
used when defining builders.

  core.builder(
      ...
      caches = [
          swarming.cache('path', name='cache_name'),
          swarming.cache('name_same_as_path'),
          swarming.cache('warmish', wait_for_warm_cache=5*time.minute),
      ],
      ...
      dimensions = {
          'os': 'Linux',
          'expiring': swarming.dimension('value', expiration=5*time.minute),
          'many': ['val1', swarming.dimension('val2')],
      },
      ...
  )

Also defines a bunch of swarmingimpl.validate_X functions that will be used
by the builder rule implementation to validate parameters.

The documentation is mostly copied from Buildbucket's project_config.proto.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: Ie55c56af785ed0538fc7674c607bed0f699ec3a6
Reviewed-on: https://chromium-review.googlesource.com/c/1399992
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/73abe063c75c7926c68e9520154a169df4c2ad7d/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/73abe063c75c7926c68e9520154a169df4c2ad7d/lucicfg/starlark/stdlib/builtins.star
[add] https://crrev.com/73abe063c75c7926c68e9520154a169df4c2ad7d/lucicfg/starlark/stdlib/internal/luci/lib/swarming.star
[add] https://crrev.com/73abe063c75c7926c68e9520154a169df4c2ad7d/lucicfg/testdata/swarming/cache.star
[add] https://crrev.com/73abe063c75c7926c68e9520154a169df4c2ad7d/lucicfg/testdata/swarming/dimension.star
[add] https://crrev.com/73abe063c75c7926c68e9520154a169df4c2ad7d/lucicfg/testdata/swarming/tags.star

Comment 85 by bugdroid1@chromium.org, Jan 9

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/82c27dc23ccd959a4cdd974e26b0237f63337704

commit 82c27dc23ccd959a4cdd974e26b0237f63337704
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 09 22:54:37 2019

[lucicfg] Implement all fields of core.builder(...) except 'recipe'.

'recipe' field needs a definition of core.recipe(...) rule. It will be done
later.

The following proto fields we explicitly do not support:
  * All mixin related stuff.
  * 'category' (was never used)
  * 'auto_builder_dimension' (shouldn't be necessary with Starlark)
  * recipe.properties (we always use properties_j)

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: Iac635f6f28a946e2fb133f42c377f8b901c4a3e5
Reviewed-on: https://chromium-review.googlesource.com/c/1399994
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/82c27dc23ccd959a4cdd974e26b0237f63337704/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/82c27dc23ccd959a4cdd974e26b0237f63337704/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/82c27dc23ccd959a4cdd974e26b0237f63337704/lucicfg/starlark/stdlib/internal/luci/lib/swarming.star
[modify] https://crrev.com/82c27dc23ccd959a4cdd974e26b0237f63337704/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/82c27dc23ccd959a4cdd974e26b0237f63337704/lucicfg/testdata/full_example.star

Comment 86 by bugdroid1@chromium.org, Jan 9

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b

commit ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 09 23:18:17 2019

[lucicfg] Add basic core.recipe(...) rule.

Make 'recipe=...' be required when defining a builder.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: Iff91bd0d8a134036f9722b376d72b3749357b1a9
Reviewed-on: https://chromium-review.googlesource.com/c/1401474
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/starlark/stdlib/internal/luci/common.star
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[add] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/starlark/stdlib/internal/luci/rules/recipe.star
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/testdata/errors/ambiguous_references.star
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/testdata/errors/poller_builder_clash.star
[modify] https://crrev.com/ef45b74c00894b9c47e5b1df40b5cfc0f8b58f4b/lucicfg/testdata/full_example.star

Comment 87 by bugdroid1@chromium.org, Jan 10

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/51509a72b83b5c5167b213a69dda6f30701b03d9

commit 51509a72b83b5c5167b213a69dda6f30701b03d9
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 10 01:25:55 2019

[lucicfg] Add acl.SCHEDULER_* roles, rename Buildbucket roles for consistency.

Scheduler and Buildbucket use different names for the same kind of roles:
  Buildbucket's SCHEDULER ~= Scheduler's TRIGGERER
  Buildbucket's WRITER ~= Scheduler's OWNER

This is confusing. So we'll need to make a compromise somewhere: either use
Buildbucket-inspired roles or Scheduler-inspired ones (or pick something
completely unrelated).

Arguably, scheduler's ones (TRIGGERER, OWNER) are clearer than buildbucket's
ones (SCHEDULER, WRITER). For one, 'schedule' is a pretty overloaded term, and
SCHEDULER_SCHEDULER will look odd. Also Buildbucket's WRITER role doesn't convey
the powers it grants (e.g. ability to cancel all builds in the bucket).

So we now have:
  BUILDBUCKET_READER - can list buckets, read build properties
  BUILDBUCKET_TRIGGERER - can launch buildbucket tasks
  BUILDBUCKET_OWNER - full access to the bucket (e.g. clearing it)
  SCHEDULER_READER - can list scheduler job invocations
  SCHEDULER_TRIGGERER - can launch scheduler jobs by submitting triggers
  SCHEDULER_OWNER - can abort jobs

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I494a9af90714bf1739c91fdb6fa25167681be1e6
Reviewed-on: https://chromium-review.googlesource.com/c/1400850
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/starlark/stdlib/internal/luci/lib/acl.star
[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/testdata/acl/entry.star
[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/testdata/acl/normalize_acls.star
[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/testdata/acl/validate_acls.star
[modify] https://crrev.com/51509a72b83b5c5167b213a69dda6f30701b03d9/lucicfg/testdata/full_example.star

Comment 89 by bugdroid1@chromium.org, Jan 11

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/aa7c87510bda051d0f4b8d52d1ad08308ae7105e

commit aa7c87510bda051d0f4b8d52d1ad08308ae7105e
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 11 05:22:06 2019

[lucicfg] Implement (most of) scheduler.cfg generation.

Also slightly adjust codestyle in the buildbucket generator for consistency.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: Ie2db248c91421c85631527f63395723d43e395e2
Reviewed-on: https://chromium-review.googlesource.com/c/1404297
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/errors/ambiguous_references.star
[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/errors/bucket_without_service.star
[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/errors/poller_builder_clash.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/errors/scheduler_disambiguation_fail.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/errors/triggering_builder_no_account.star
[modify] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/full_example.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/scheduler/ambiguity_resolution.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/scheduler/builder_builder.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/scheduler/dedups_triggerer_acl.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/scheduler/many_to_many.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/scheduler/poller_builder.star
[add] https://crrev.com/aa7c87510bda051d0f4b8d52d1ad08308ae7105e/lucicfg/testdata/scheduler/single_poller.star

Comment 90 by bugdroid1@chromium.org, Jan 11

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4f9ab59fed2146929cd085c7946651bd2126e709

commit 4f9ab59fed2146929cd085c7946651bd2126e709
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 11 19:34:51 2019

[lucicfg] Make rule's arguments kwargs only and None by default.

Using positional args leads to some very confusing error messages like:

    core.builder(
        name = 'b',
        bucket = 'ci',
        service_account = 'noop@example.com',
    )

    Traceback (most recent call last):
      ...
    Error: function builder takes at least 3 positional arguments (3 given)

Instead we now get:

    Traceback (most recent call last):
      ...
    Error: missing required field "recipe"

Hopefully https://github.com/google/starlark-go/issues/61 will land someday
and we'll be able to say def f(*, a=None, b=None) to force all arguments to
be passed via keywords.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I96336bd6b849e55db0b0a2fa508786212c5652d6
Reviewed-on: https://chromium-review.googlesource.com/c/1406418
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/stdlib/internal/luci/rules/bucket.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/starlark/stdlib/internal/luci/rules/recipe.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/testdata/acl/entry.star
[add] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/testdata/errors/builder_without_recipe.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/testdata/errors/project_without_name.star
[modify] https://crrev.com/4f9ab59fed2146929cd085c7946651bd2126e709/lucicfg/testdata/misc/validators.star

Comment 91 by bugdroid1@chromium.org, Jan 11

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/3000bf43edfc8b220ba822311d24cbdfe46a0d72

commit 3000bf43edfc8b220ba822311d24cbdfe46a0d72
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 11 19:59:15 2019

[lucicfg] Finish gitiles_poller(...) definition.

I decided to omit 'triggering_policy' for it, since I've never seen custom
polices used with pollers. Can be added later if needed.

Also the validation rules are quite involved, so let the server do the thorough
validation and only validate types in lucicfg.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Icc56c5a3f2073dce51b3ad859e454ea36bef5382
Reviewed-on: https://chromium-review.googlesource.com/c/1406423
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/errors/ambiguous_references.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/errors/poller_builder_clash.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/errors/scheduler_disambiguation_fail.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/full_example.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/scheduler/ambiguity_resolution.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/scheduler/dedups_triggerer_acl.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/scheduler/many_to_many.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/scheduler/poller_builder.star
[modify] https://crrev.com/3000bf43edfc8b220ba822311d24cbdfe46a0d72/lucicfg/testdata/scheduler/single_poller.star

Comment 92 by bugdroid1@chromium.org, Jan 15

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/e22dfba782fbedc56bca4ed0818f2113cc3f914f

commit e22dfba782fbedc56bca4ed0818f2113cc3f914f
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 15 04:45:48 2019

[lucicfg] docgen: Add AST parser for the documentation generator.

It takes a Starlark's AST and reduces it only to structures important for the
documentation generation purposes, preserving their docstring and comments.

It works on a single file only. It will be used by a higher level parser that
understands relations between modules in a package, e.g. if a symbol is defined
in an internal module, but then publicly exposed through an alias in another
module, the documentation generator will be able to trace the references back
to the original definition, to grab a documentation from it.

R=tandrii@chromium.org
BUG=833946

Change-Id: Ic5ae1ac25978cb3964f20d3ae43bc3265fc7078c
Reviewed-on: https://chromium-review.googlesource.com/c/1407728
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[add] https://crrev.com/e22dfba782fbedc56bca4ed0818f2113cc3f914f/lucicfg/docgen/ast/parser.go
[add] https://crrev.com/e22dfba782fbedc56bca4ed0818f2113cc3f914f/lucicfg/docgen/ast/parser_test.go

Comment 93 by bugdroid1@chromium.org, Jan 15

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/7f11c87315815a78eb3554e4d5c76e059c350f60

commit 7f11c87315815a78eb3554e4d5c76e059c350f60
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 15 05:10:42 2019

[lucicfg] docgen: Add docstring parser.

It parses a docstring into a main body, a list of arguments and a list of
additional sections, like "Returns". All extracted strings are properly
deindented and cleaned up, ready to be inserted into e.g. markdown.

This is surprisingly non-trivial, complicated by careful handling of
indentation, since it is syntactically significant in this case.

R=tandrii@chromium.org
BUG=833946

Change-Id: I38c1f99cb71331e2f6617196415b4e914e15614e
Reviewed-on: https://chromium-review.googlesource.com/c/1408602
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[add] https://crrev.com/7f11c87315815a78eb3554e4d5c76e059c350f60/lucicfg/docgen/docstring/docstring.go
[add] https://crrev.com/7f11c87315815a78eb3554e4d5c76e059c350f60/lucicfg/docgen/docstring/docstring_test.go

Comment 94 by bugdroid1@chromium.org, Jan 15

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/44f46fb06a244d4a352071774427cbb03214216e

commit 44f46fb06a244d4a352071774427cbb03214216e
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 15 23:50:02 2019

[lucicfg] docgen: Understand 'load(...)' statements and aliasing.

This allows to discover AST nodes with definitions (and documentation!) of all
symbols exported by a module by following the chain of assignments, load(...)'s
and struct lookups.

R=tandrii@chromium.org
BUG=833946

Change-Id: Icea3dc4cffb8cc1cdcdfa13fe416d175d1d90849
Reviewed-on: https://chromium-review.googlesource.com/c/1407993
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[add] https://crrev.com/44f46fb06a244d4a352071774427cbb03214216e/lucicfg/docgen/symbols/loader.go
[add] https://crrev.com/44f46fb06a244d4a352071774427cbb03214216e/lucicfg/docgen/symbols/loader_test.go
[add] https://crrev.com/44f46fb06a244d4a352071774427cbb03214216e/lucicfg/docgen/symbols/symbols.go

Comment 95 by bugdroid1@chromium.org, Jan 17

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/786e5243fb201d489d44b0750adb1fac3975726e

commit 786e5243fb201d489d44b0750adb1fac3975726e
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 17 19:28:14 2019

[lucicfg] Add validate.float and validate.type.

Will be used to validate scheduler's TriggeringPolicy definitions.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I252daebcd5d4a624474601f2a72aae1520e6f589
Reviewed-on: https://chromium-review.googlesource.com/c/1417292
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/786e5243fb201d489d44b0750adb1fac3975726e/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/786e5243fb201d489d44b0750adb1fac3975726e/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[modify] https://crrev.com/786e5243fb201d489d44b0750adb1fac3975726e/lucicfg/testdata/misc/validators.star

Comment 96 by bugdroid1@chromium.org, Jan 17

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/1a0d955d7361e3c127006559f935fed350f3d6c1

commit 1a0d955d7361e3c127006559f935fed350f3d6c1
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 17 20:04:58 2019

[lucicfg] docgen: Add the actual generator.

It is a text generator based on text/template that has access to functions that
return structured information about symbols defined in the starlark code. This
allows templates to insert docs extracted from the source code at appropriate
places.

This CL has only very basic implementation, as a proof of concept. It needs
a lot more polish before it generates acceptably looking and useful docs.

R=tandrii@chromium.org
BUG=833946

Change-Id: I849ace37b62782f25ac2603d968016a709728f5a
Reviewed-on: https://chromium-review.googlesource.com/c/1408603
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[add] https://crrev.com/1a0d955d7361e3c127006559f935fed350f3d6c1/lucicfg/cmd/docgen/main.go
[add] https://crrev.com/1a0d955d7361e3c127006559f935fed350f3d6c1/lucicfg/doc/README.md
[add] https://crrev.com/1a0d955d7361e3c127006559f935fed350f3d6c1/lucicfg/doc/generate.go
[add] https://crrev.com/1a0d955d7361e3c127006559f935fed350f3d6c1/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/1a0d955d7361e3c127006559f935fed350f3d6c1/lucicfg/docgen/docstring/docstring.go
[add] https://crrev.com/1a0d955d7361e3c127006559f935fed350f3d6c1/lucicfg/docgen/generator.go

Comment 98 by bugdroid1@chromium.org, Jan 18

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/fc59bd114abd60c52ccad825779d9e1f2406fd5d

commit fc59bd114abd60c52ccad825779d9e1f2406fd5d
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 18 05:25:24 2019

[lucicfg] Cleanup docs a bit.

This also moves 'ctor' and 'genstruct' from the global namespace to __native__,
since they are a bit weird and I don't want to advertise them as public API.

R=tandrii@chromium.org
BUG=833946

Change-Id: If856001d4856f41096a1c04d3b89ceb7f904fa1d
Reviewed-on: https://chromium-review.googlesource.com/c/1413997
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/doc/README.md
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/generator.go
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/generator.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/lib/acl.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/lib/swarming.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/lib/validate.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/rules/bucket.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/rules/logdog.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/internal/luci/rules/recipe.star
[add] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/native_doc.star
[add] https://crrev.com/fc59bd114abd60c52ccad825779d9e1f2406fd5d/lucicfg/starlark/stdlib/proto_doc.star

Comment 99 by bugdroid1@chromium.org, Jan 18

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/37b4f2fcc80cf8075860412039614792ba2e0d93

commit 37b4f2fcc80cf8075860412039614792ba2e0d93
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 18 21:31:58 2019

[lucicfg] Add 'schedule' and 'triggering_policy' to core.builder(...).

Also improve the documentation regarding how batching happens, i.e. how git
commits transform into builds.

This concludes cr-buildbucket.cfg and luci-scheduler.cfg portion of lucicfg.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I4d58104565ca04ad2e9aeb779d269f81bdf37a36
Reviewed-on: https://chromium-review.googlesource.com/c/1419286
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/doc/README.md
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/starlark/stdlib/internal/luci/generators.star
[add] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/starlark/stdlib/internal/luci/lib/scheduler.star
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/37b4f2fcc80cf8075860412039614792ba2e0d93/lucicfg/testdata/full_example.star

Comment 100 by bugdroid1@chromium.org, Jan 19

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f

commit 40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sat Jan 19 03:13:53 2019

[lucicfg] Add a way to provide "meta" configuration.

This is configuration for the config generator itself. Stuff like where to
place files and how to call LUCI Config.

It can be set via core.meta(...) in *.star, or overriden through command line
flags.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I109d5be20b73b8e1f72fe7293fba94e9b7b5d2e9
Reviewed-on: https://chromium-review.googlesource.com/c/1420019
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/doc/README.md
[modify] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/doc/templates/README.mdt
[add] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/meta.go
[add] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/meta_test.go
[modify] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/starlark/stdlib/builtins.star
[add] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/starlark/stdlib/internal/meta.star
[modify] https://crrev.com/40266bdca3e0c5da0bfb4b4ffe2d6649c2c1f85f/lucicfg/state.go

Comment 101 by bugdroid1@chromium.org, Jan 22

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/42221c6b4351b26da2f4ad6910335d8391642192

commit 42221c6b4351b26da2f4ad6910335d8391642192
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 22 20:08:21 2019

[lucicfg] Add ConfigSet Go type, extract validation logic into it.

This code will be shared by multiple lucicfg subcommands.

Also fix printing of errors. Apparently errors.WalkLeaves completely skips
errors produced with errors.Reason(...).Err().

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Ie196e92af88f7f2d16680f23d9041f1772c54da8
Reviewed-on: https://chromium-review.googlesource.com/c/1423325
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/cli/base/base.go
[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/cli/cmds/validate/validate.go
[delete] https://crrev.com/73453766172cf2ba3a100ba6e54e1b735be70109/lucicfg/cli/cmds/validate/validate_test.go
[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/configset.go
[add] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/configset_test.go
[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/genctx.go
[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/generator.go
[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/starlark_test.go
[modify] https://crrev.com/42221c6b4351b26da2f4ad6910335d8391642192/lucicfg/state.go

Comment 102 by bugdroid, Jan 23

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/5256b7785e98410664e2696df9d4f5b52ae4dfb7

commit 5256b7785e98410664e2696df9d4f5b52ae4dfb7
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 23 05:21:15 2019

[lucicfg] Implement FindTrackedFiles function.

It will be used to figure out what existing files on disk are supposed to be
generated from *.star. See 'tracked_files' in meta.config(...) doc for more
information.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I74930c7ce871993b682a9dd4d0a4822a324f99fd
Reviewed-on: https://chromium-review.googlesource.com/c/1429189
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[add] https://crrev.com/5256b7785e98410664e2696df9d4f5b52ae4dfb7/lucicfg/tracked.go
[add] https://crrev.com/5256b7785e98410664e2696df9d4f5b52ae4dfb7/lucicfg/tracked_test.go

Comment 103 by bugdroid, Jan 23

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/5256b7785e98410664e2696df9d4f5b52ae4dfb7

commit 5256b7785e98410664e2696df9d4f5b52ae4dfb7
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 23 05:21:15 2019

[lucicfg] Implement FindTrackedFiles function.

It will be used to figure out what existing files on disk are supposed to be
generated from *.star. See 'tracked_files' in meta.config(...) doc for more
information.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I74930c7ce871993b682a9dd4d0a4822a324f99fd
Reviewed-on: https://chromium-review.googlesource.com/c/1429189
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[add] https://crrev.com/5256b7785e98410664e2696df9d4f5b52ae4dfb7/lucicfg/tracked.go
[add] https://crrev.com/5256b7785e98410664e2696df9d4f5b52ae4dfb7/lucicfg/tracked_test.go

Comment 104 by bugdroid, Jan 24

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/84873aa35aa338d819313d3af60b0ee6052da647

commit 84873aa35aa338d819313d3af60b0ee6052da647
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 24 01:45:47 2019

[lucicfg] Implement ConfigSet.Write.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I85978c7b1c3fe1ccea3c190bbe3b0140cc38b0fc
Reviewed-on: https://chromium-review.googlesource.com/c/1432641
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/84873aa35aa338d819313d3af60b0ee6052da647/lucicfg/configset.go
[modify] https://crrev.com/84873aa35aa338d819313d3af60b0ee6052da647/lucicfg/configset_test.go

Comment 106 by bugdroid, Jan 24

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/3897842737faad2053e6ba4c1b5c6b4c3590a605

commit 3897842737faad2053e6ba4c1b5c6b4c3590a605
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 24 19:19:04 2019

[lucicfg] Make FindTrackedFiles treat missing directories as empty.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Ica5f8772c7090692da5ae155cbbdab2eb58ba9ba
Reviewed-on: https://chromium-review.googlesource.com/c/1433577
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/3897842737faad2053e6ba4c1b5c6b4c3590a605/lucicfg/tracked.go
[modify] https://crrev.com/3897842737faad2053e6ba4c1b5c6b4c3590a605/lucicfg/tracked_test.go

Comment 107 by bugdroid, Jan 24

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/3fca1b70ed44e01fb5fba74c730562b2416b7fd2

commit 3fca1b70ed44e01fb5fba74c730562b2416b7fd2
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 24 19:24:51 2019

[lucicfg] Add few utility methods to ConfigSet.

They'll be used to produce some not-entirely-useless -json-output for 'generate'
subcommand.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I2cb2a30df0fb6b9743372809a972fe9465fc900e
Reviewed-on: https://chromium-review.googlesource.com/c/1433578
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/3fca1b70ed44e01fb5fba74c730562b2416b7fd2/lucicfg/configset.go
[modify] https://crrev.com/3fca1b70ed44e01fb5fba74c730562b2416b7fd2/lucicfg/configset_test.go

Comment 108 by bugdroid, Jan 24

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/7447fc21cf1c6db108b5c6aa178c01429d6a8032

commit 7447fc21cf1c6db108b5c6aa178c01429d6a8032
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 24 20:33:05 2019

[lucicfg] Improve -json-output.

Include version of lucicfg that produced it. List all errors if there were many.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Icceef7abc50c2a414f6adbe21dbc37e50d9e31c2
Reviewed-on: https://chromium-review.googlesource.com/c/1434942
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/7447fc21cf1c6db108b5c6aa178c01429d6a8032/lucicfg/cli/base/base.go
[add] https://crrev.com/7447fc21cf1c6db108b5c6aa178c01429d6a8032/lucicfg/cli/base/errors.go

Comment 109 by bugdroid, Jan 25

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645

commit 50f483922eda9e3bf3ae07db9b2d27bdf9d9e645
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Jan 25 00:10:48 2019

[lucicfg] Implement 'generate' and half of 'validate' subcommands.

'validate' is kept backward compatible with what we had already (validating
arbitrary, not necessary generated, config sets).

Brief explanation of supported modes:

$ lucicfg generate main.star
  Generates *.cfg files and puts them in the output directory set via
  meta.config(config_dir=...) or via -config-dir flag.

$ lucicfg generate main.star -validate
  Same as above, but also sends generated configs to LUCI Config for validation.

$ lucicfg validate <DIR> -config-set ...
  Read-only operation, can be used from presubmits. Takes files from <DIR> and
  sends them to LUCI Config for validation.

$ lucicfg validate main.star (not implemented yet)
  Read-only operation, can be used from presubmits. Generates *.cfg into memory,
  compares them to existing files on disk, failing if there's a diff. Otherwise
  sends generated configs to LUCI Config for validation.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I886f99c18ac22f0ebf06558b26837853ec092134
Reviewed-on: https://chromium-review.googlesource.com/c/1435656
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/cli/base/base.go
[add] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/cli/base/generate.go
[add] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/cli/base/validate.go
[modify] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/cli/cmds/generate/generate.go
[modify] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/cli/cmds/validate/validate.go
[modify] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/configset.go
[add] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/testdata/.gitignore
[modify] https://crrev.com/50f483922eda9e3bf3ae07db9b2d27bdf9d9e645/lucicfg/testdata/full_example.star

Comment 110 by bugdroid, Jan 26

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/da6c13ea0d6e49931caaa6473e0be1aa85f3af98

commit da6c13ea0d6e49931caaa6473e0be1aa85f3af98
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sat Jan 26 00:17:39 2019

[lucicfg] Finish 'lucicfg validate' subcommand.

It can now check for stale files on disk.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I43d86bba470d4776f5e7379a796062d934b38d53
Reviewed-on: https://chromium-review.googlesource.com/c/1436474
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/da6c13ea0d6e49931caaa6473e0be1aa85f3af98/lucicfg/cli/cmds/validate/validate.go
[modify] https://crrev.com/da6c13ea0d6e49931caaa6473e0be1aa85f3af98/lucicfg/configset.go

Comment 111 by bugdroid, Jan 26

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/150388ea78aa9c26d4d54f0deebf1875fc527ca6

commit 150388ea78aa9c26d4d54f0deebf1875fc527ca6
Author: Nodir Turakulov <nodir@google.com>
Date: Sat Jan 26 00:49:47 2019

[buildbucket] Validate Builder.swarming_host

Move swarming hostname validation from Swarming message level
to indivdual Builder level.

On config import, Swarming message level hostname is copied to builder_defaults
which then applied to all builders.

Bug: 833946
Change-Id: I4597df62fedd7d32e21ee7900b4544f4f1db65fe
Reviewed-on: https://chromium-review.googlesource.com/c/1437747
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#20239}
[modify] https://crrev.com/150388ea78aa9c26d4d54f0deebf1875fc527ca6/appengine/cr-buildbucket/swarming/test/swarmingcfg_test.py
[modify] https://crrev.com/150388ea78aa9c26d4d54f0deebf1875fc527ca6/appengine/cr-buildbucket/test/config_test.py
[modify] https://crrev.com/150388ea78aa9c26d4d54f0deebf1875fc527ca6/appengine/cr-buildbucket/swarming/swarmingcfg.py

Comment 112 by bugdroid, Jan 26

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/1291476038ceb3997d890e9c54013a0349b8e6fc

commit 1291476038ceb3997d890e9c54013a0349b8e6fc
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sat Jan 26 20:08:18 2019

[lucicfg] Add time.days_of_week helper.

time.days_of_week('Mon,Fri-Sun') == [1, 5, 6, 7]. It is useful for Swarming
configs. Decided to put it here in case it is useful for something else, the
cost of supporting it is negligible.

Also expose time.star module doc in README.md.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I2027eb71fe0b5849baebae87a054ab47c2a8efda
Reviewed-on: https://chromium-review.googlesource.com/c/1437893
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/doc/README.md
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/internal/luci/lib/acl.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/internal/luci/lib/scheduler.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/internal/luci/lib/service.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/internal/luci/lib/swarming.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/internal/meta.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/starlark/stdlib/internal/time.star
[modify] https://crrev.com/1291476038ceb3997d890e9c54013a0349b8e6fc/lucicfg/testdata/misc/time.star

Comment 113 by bugdroid, Jan 26

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4882b4f148996b25654fcc02c0bba1f37e8c8cad

commit 4882b4f148996b25654fcc02c0bba1f37e8c8cad
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sat Jan 26 20:19:28 2019

[lucicfg] Move validate.star from //internal/luci to just //internal.

There's nothing LUCI-specific in it.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I04785f6b28d7db190817ec449676acc740a8807c
Reviewed-on: https://chromium-review.googlesource.com/c/1438075
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/lib/acl.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/lib/scheduler.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/lib/service.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/lib/swarming.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/rules/bucket.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/rules/logdog.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/luci/rules/recipe.star
[rename] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/starlark/stdlib/internal/validate.star
[modify] https://crrev.com/4882b4f148996b25654fcc02c0bba1f37e8c8cad/lucicfg/testdata/misc/validators.star

Comment 114 by bugdroid, Jan 28

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/d6d65680977c72e711a3cedec6fab1b803a97bc8

commit d6d65680977c72e711a3cedec6fab1b803a97bc8
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Jan 28 20:41:49 2019

[lucicfg] Add 'exec' builtin.

It is like 'load', except it doesn't import any symbols and used specifically
for side-effects of executing the module.

Useful for splitting large scripts into a bunch of smaller ones.

Starlark's 'load' statement is inconvenient to use for this:
  * It always requires something to be imported, i.e. load("...") is invalid
    statement. Should be load("...", <something>). When executing for side
    effects there's often nothing to import.
  * Attempting to imitate go's "_" import with load("...", "_") fails, since
    Starlark refused to load symbols starting with '_'.
  * Attempting to use some other sentinel symbol fails, since Starlark insists
    on not repeating loaded symbol names, e.g. the following is forbidden:
      load("//mod1.star", "dummy")
      load("//mod2.star", "dummy")

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I0acf68ca2391f67b4f0bbac5cc0951db3ca73a27
Reviewed-on: https://chromium-review.googlesource.com/c/1438721
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/d6d65680977c72e711a3cedec6fab1b803a97bc8/lucicfg/generator.go
[modify] https://crrev.com/d6d65680977c72e711a3cedec6fab1b803a97bc8/starlark/interpreter/helpers_test.go
[modify] https://crrev.com/d6d65680977c72e711a3cedec6fab1b803a97bc8/starlark/interpreter/interpreter.go
[modify] https://crrev.com/d6d65680977c72e711a3cedec6fab1b803a97bc8/starlark/interpreter/interpreter_test.go

Comment 115 by bugdroid, Jan 28

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/4a7051f8dfd6ead9008a9821329f49247ae66326

commit 4a7051f8dfd6ead9008a9821329f49247ae66326
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Jan 28 20:59:45 2019

Roll infra/go/src/go.chromium.org/luci/ 7a8ca4593..d6d656809 (26 commits)

https://chromium.googlesource.com/infra/luci/luci-go/+log/7a8ca4593e55..d6d65680977c

$ git log 7a8ca4593..d6d656809 --date=short --no-merges --format='%ad %ae %s'
2019-01-28 vadimsh [lucicfg] Add 'exec' builtin.
2019-01-28 vadimsh [lucicfg] Add strutil.expand_int_set(...) internal API.
2019-01-28 hinoka [milo] Remove tabs for changes/properties
2019-01-26 vadimsh [lucicfg] Move validate.star from //internal/luci to just //internal.
2019-01-26 vadimsh [lucicfg] Add time.days_of_week helper.
2019-01-26 vadimsh [lucicfg] Let lucicfg know about Swarming config protos.
2019-01-26 vadimsh [lucicfg] Finish 'lucicfg validate' subcommand.
2019-01-25 tandrii [milo] make Gerrit queries faster.
2019-01-25 smut [GCE] Support minimum CPU platform
2019-01-25 vadimsh [lucicfg] Implement 'generate' and half of 'validate' subcommands.
2019-01-24 smut [GCE] Fetch and convert metadata from file
2019-01-24 vadimsh [lucicfg] Fix flaky test.
2019-01-24 hinoka [milo] Don't pass in empty dimensions (cores:) when querying swarming for pools
2019-01-24 nodir [buildbucket] Extend priority docs
2019-01-24 vadimsh [lucicfg] Improve -json-output.
2019-01-24 vadimsh [lucicfg] Add few utility methods to ConfigSet.
2019-01-24 vadimsh [lucicfg] Make FindTrackedFiles treat missing directories as empty.
2019-01-24 vadimsh [lucicfg] Use "projects/<name>" as config set name by default.
2019-01-24 vadimsh [lucicfg] Add Meta.WasTouched(...), call DetectTouchedFlags() lazily.
2019-01-24 smut [GCE] Install config import handler
2019-01-24 vadimsh [lucicfg] Implement ConfigSet.Write.
2019-01-24 hinoka [milo] Fix properties and blamelist in build.proto view
2019-01-23 smut [GCE] Handle drained VMs
2019-01-23 vadimsh [lucicfg] Implement FindTrackedFiles function.
2019-01-23 tandrii common: update doc wrt to deprecation of serviceAccountActor gcloud role.
2019-01-23 vadimsh [lucicfg] Remove overzealous validation.

Created with:
  roll-dep infra/go/src/go.chromium.org/luci

TBR=tandrii@chromium.org
BUG=833946

Change-Id: I2b5203646cfa86f3367e76107e51e86e33994f6e
Reviewed-on: https://chromium-review.googlesource.com/c/1440468
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#20258}
[modify] https://crrev.com/4a7051f8dfd6ead9008a9821329f49247ae66326/DEPS

Comment 116 by bugdroid, Jan 28

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/ae2c823757e68e1679afaf6d4e8b7a772e5df747

commit ae2c823757e68e1679afaf6d4e8b7a772e5df747
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Jan 28 23:41:59 2019

[lucicfg] Add meta.version(), bump the version to 1.0.0.

Can be used in end-user scripts to bark if lucicfg is older than required.
This will be useful in early stages of lucicfg deployment when luccifg is
still frequently changing.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Ib1f412a5d48d1e48639a02b9d61f3741feb2fc0d
Reviewed-on: https://chromium-review.googlesource.com/c/1440922
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/ae2c823757e68e1679afaf6d4e8b7a772e5df747/lucicfg/doc/README.md
[modify] https://crrev.com/ae2c823757e68e1679afaf6d4e8b7a772e5df747/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/ae2c823757e68e1679afaf6d4e8b7a772e5df747/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/ae2c823757e68e1679afaf6d4e8b7a772e5df747/lucicfg/starlark/stdlib/internal/meta.star
[add] https://crrev.com/ae2c823757e68e1679afaf6d4e8b7a772e5df747/lucicfg/testdata/misc/version.star
[modify] https://crrev.com/ae2c823757e68e1679afaf6d4e8b7a772e5df747/lucicfg/version.go

Comment 117 by bugdroid, Jan 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/753c1c5ea76d03a9bb65101a3dd829ab88d7c530

commit 753c1c5ea76d03a9bb65101a3dd829ab88d7c530
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 29 00:35:39 2019

[lucicfg] Rename 'meta' package to 'lucicfg'.

That way it can have more stuff directly related to lucicfg itself, not only
lucicfg's config.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I72b92723de3e87d2f76dc72f3832e10a8bc16eb0
Reviewed-on: https://chromium-review.googlesource.com/c/1441511
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/cli/base/base.go
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/cli/base/generate.go
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/cli/cmds/generate/generate.go
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/cli/cmds/validate/validate.go
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/doc/README.md
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/meta.go
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/starlark/stdlib/internal/luci/generators.star
[rename] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/starlark/stdlib/internal/lucicfg.star
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/testdata/full_example.star
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/testdata/misc/version.star
[modify] https://crrev.com/753c1c5ea76d03a9bb65101a3dd829ab88d7c530/lucicfg/version.go

Comment 118 by bugdroid, Jan 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/a0bb9730dd59e6a823c5730d6c081e788052990f

commit a0bb9730dd59e6a823c5730d6c081e788052990f
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 29 02:20:20 2019

[lucicfg] Rename 'core.generator' to 'lucicfg.generator'.

'core.*' has shaped up to be "core LUCI-domain model stuff". 'generator' is not
LUCI-specific at all, it can be used to generate arbitrary configs. So move it
to 'lucicfg', where the rest of generic lucicfg facilities are.

R=tandrii@chromium.org
BUG=833946

Change-Id: I3f8eec88007bb115c43bb88534510e4c69c844ab
Reviewed-on: https://chromium-review.googlesource.com/c/1441671
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/doc/README.md
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/external/crostesting/proto/config/source_tree_test_config.pb.go
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/starlark/stdlib/builtins.star
[delete] https://crrev.com/7dbff3549906966604c7c7c4ebe5ad45c9f70675/lucicfg/starlark/stdlib/internal/generator.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/starlark/stdlib/internal/lucicfg.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/testdata/misc/config_set.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/testdata/misc/generator_api.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/testdata/misc/generator_errors.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/testdata/misc/graph.star
[modify] https://crrev.com/a0bb9730dd59e6a823c5730d6c081e788052990f/lucicfg/testdata/misc/graph_dangling_edge.star

Comment 119 by bugdroid, Jan 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/e2281b0154c8e4e8ed60fd3881efdbde21c2b6aa

commit e2281b0154c8e4e8ed60fd3881efdbde21c2b6aa
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Jan 29 23:47:20 2019

[lucicfg] Use subcommands.Section as subcommands separator.

Something has changed in 'subcommands' and running "lucicfg -unknown" causes
a panic instead of reporting an error. Using subcommands.Section(...) serves
as a workaround.

R=tandrii@chromium.org
BUG=833946

Change-Id: I14a287e1822053599b4f1c0d68e522bfe1e59072
Reviewed-on: https://chromium-review.googlesource.com/c/1441676
Auto-Submit: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/e2281b0154c8e4e8ed60fd3881efdbde21c2b6aa/lucicfg/cli/app.go

Comment 120 by bugdroid, Jan 30

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/5b0cef35a03240bf1d1a72c24dbee7ff01753488

commit 5b0cef35a03240bf1d1a72c24dbee7ff01753488
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 30 00:28:45 2019

Roll infra/go/src/go.chromium.org/luci/ d6d656809..e2281b015 (8 commits)

https://chromium.googlesource.com/infra/luci/luci-go//+log/d6d65680977c..e2281b0154c8

$ git log d6d656809..e2281b015 --date=short --no-merges --format='%ad %ae %s'
2019-01-29 vadimsh [lucicfg] Use subcommands.Section as subcommands separator.
2019-01-29 hinoka [milo] Link to the builder page correctly for v2 build page
2019-01-29 vadimsh [lucicfg] Rename 'core.generator' to 'lucicfg.generator'.
2019-01-29 friedman [mmutex] Add \n to the end of printf
2019-01-29 vadimsh [lucicfg] Rename 'meta' package to 'lucicfg'.
2019-01-29 seanabraham [lucicfg] Create Chrome OS source tree test config proto.
2019-01-28 vadimsh [lucicfg] Add meta.version(), bump the version to 1.0.0.
2019-01-28 nodir [buildbucket] Document auto-added buildsets

Created with:
  roll-dep infra/go/src/go.chromium.org/luci

TBR=tandrii@chromium.org
BUG=833946

Change-Id: I8f37cfacd178484a44614283a7637c419776f93e
Reviewed-on: https://chromium-review.googlesource.com/c/1444534
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#20319}
[modify] https://crrev.com/5b0cef35a03240bf1d1a72c24dbee7ff01753488/DEPS

Comment 121 by bugdroid, Jan 30

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools/+/ef40cbac465300f920466b3e6f5b0edc13ed0e6c

commit ef40cbac465300f920466b3e6f5b0edc13ed0e6c
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Jan 30 02:39:08 2019

[cipd] Add lucicfg tool.

It knows how to interpret configuration files written in Starlark-based
DSL and produce a bunch of protobuf messages as a result.

Will be used to generate various infra configs. In particular, will be
used by infra developers (to update configs), by PRESUBMIT.py on dev
machines (to verify configs before uploading CLs) and by bots (to verify
configs before submitting CLs).

R=tandrii@chromium.org
BUG=833946

Change-Id: Iceec7d808ce180f7d4a341fab8b5ce11933c2a6b
Reviewed-on: https://chromium-review.googlesource.com/c/1444499
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/ef40cbac465300f920466b3e6f5b0edc13ed0e6c/cipd_manifest.versions
[add] https://crrev.com/ef40cbac465300f920466b3e6f5b0edc13ed0e6c/lucicfg
[modify] https://crrev.com/ef40cbac465300f920466b3e6f5b0edc13ed0e6c/cipd_manifest.txt
[add] https://crrev.com/ef40cbac465300f920466b3e6f5b0edc13ed0e6c/lucicfg.bat

Comment 122 by bugdroid, Jan 30

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9db1ad3b9853b79d21464a33698ff09a6c985bd0

commit 9db1ad3b9853b79d21464a33698ff09a6c985bd0
Author: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Date: Wed Jan 30 04:25:50 2019

Roll src/third_party/depot_tools 687c59d3a44e..ef40cbac4653 (1 commits)

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+log/687c59d3a44e..ef40cbac4653


git log 687c59d3a44e..ef40cbac4653 --date=short --no-merges --format='%ad %ae %s'
2019-01-30 vadimsh@chromium.org [cipd] Add lucicfg tool.


Created with:
  gclient setdep -r src/third_party/depot_tools@ef40cbac4653

The AutoRoll server is located here: https://autoroll.skia.org/r/depot-tools-chromium-autoroll

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.



BUG=chromium:833946
TBR=agable@chromium.org

Change-Id: I644f11afdb284c1bdd0b626548ef280b34bfcf1b
Reviewed-on: https://chromium-review.googlesource.com/c/1444764
Reviewed-by: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Commit-Queue: chromium-autoroll <chromium-autoroll@skia-public.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#627310}
[modify] https://crrev.com/9db1ad3b9853b79d21464a33698ff09a6c985bd0/DEPS

Comment 123 by bugdroid, Jan 31

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/2c550b41709760657c2bb95a942c5daa0b782078

commit 2c550b41709760657c2bb95a942c5daa0b782078
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Jan 31 00:06:36 2019

[starlark] Don't lose stack trace in error messages if exec(...) fails.

R=tandrii@chromium.org
BUG=833946

Change-Id: I525e5a3517c2c09a9c6ce96cd78329df01d9a67d
Reviewed-on: https://chromium-review.googlesource.com/c/1444569
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/2c550b41709760657c2bb95a942c5daa0b782078/starlark/interpreter/interpreter.go
[modify] https://crrev.com/2c550b41709760657c2bb95a942c5daa0b782078/starlark/interpreter/interpreter_test.go

Comment 124 by bugdroid, Feb 1

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/19758d5c8f157c17e32cbf15a9dd874a29baa645

commit 19758d5c8f157c17e32cbf15a9dd874a29baa645
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Feb 01 18:43:31 2019

[starlark] Limit exec's to be allowed only from other exec's.

If a module is loaded (via load(...)) it is a "library-like" module, it must not
do any execs.

R=tandrii@chromium.org
BUG=833946

Change-Id: I555a06cd65dfe0c8a8e3d23fb66b26560cdb2222
Reviewed-on: https://chromium-review.googlesource.com/c/1449307
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/19758d5c8f157c17e32cbf15a9dd874a29baa645/starlark/interpreter/helpers_test.go
[modify] https://crrev.com/19758d5c8f157c17e32cbf15a9dd874a29baa645/starlark/interpreter/interpreter.go
[modify] https://crrev.com/19758d5c8f157c17e32cbf15a9dd874a29baa645/starlark/interpreter/interpreter_test.go

Comment 125 by bugdroid, Feb 1

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/d9019d4804902761f3256f2a23225340069e015f

commit d9019d4804902761f3256f2a23225340069e015f
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Fri Feb 01 18:47:02 2019

[starlark] Print full stack trace when some deep load(...) fails.

This is similar to https://chromium-review.googlesource.com/c/1444569 but for
load(...).  It was harder to spot load(...) had the same problem because there's
usually not many errors in the top level of a loaded script (it is just a bunch
of function definitions).

R=tandrii@chromium.org
BUG=833946

Change-Id: I15996fc8da5da71954ec7426a0bf6ddec336ea68
Reviewed-on: https://chromium-review.googlesource.com/c/1449309
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/d9019d4804902761f3256f2a23225340069e015f/starlark/interpreter/interpreter.go
[modify] https://crrev.com/d9019d4804902761f3256f2a23225340069e015f/starlark/interpreter/interpreter_test.go
[modify] https://crrev.com/d9019d4804902761f3256f2a23225340069e015f/starlark/interpreter/loaders_test.go

Comment 127 by bugdroid, Feb 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/421318bc0495da7b0c6c79d25ed38a59dc95734b

commit 421318bc0495da7b0c6c79d25ed38a59dc95734b
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 04 20:11:49 2019

[starlark] Add PreExec/PostExec hooks, improve doc.

They'll be used to "open" and "close" scopes for lucicfg.var() variables.

Each 'exec'-ed module will get its own variables scope: all changes made to
variables inside a scope are "forgotten" when exiting the scope.

This will be used to define multiple configs of the same kind inside a single
lucicfg workspace, for example configs for multiple different Swarming servers.
They will execute in their own variables scope, not interfering with each
other.

R=tandrii@chromium.org
BUG=833946

Change-Id: Icd4076b28df3569bb7b4322725eab95de2df6a9d
Reviewed-on: https://chromium-review.googlesource.com/c/1450841
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/421318bc0495da7b0c6c79d25ed38a59dc95734b/starlark/interpreter/helpers_test.go
[modify] https://crrev.com/421318bc0495da7b0c6c79d25ed38a59dc95734b/starlark/interpreter/interpreter.go
[modify] https://crrev.com/421318bc0495da7b0c6c79d25ed38a59dc95734b/starlark/interpreter/interpreter_test.go

Comment 128 by bugdroid, Feb 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/4f458e7f03a67755e691904467ad65e4c91179f9

commit 4f458e7f03a67755e691904467ad65e4c91179f9
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 04 20:27:19 2019

[lucicfg] Implement lucicfg.var().

Vars can be used to implicitly propagate information down 'exec' call stack,
but not up or "sideways".

Useful for declaring module-level default values for fields instead of plumbing
them through all calls explicitly.

Subject to following rules:

1. Only modules that are invoked via 'exec' can set or get variables. Libraries
   (modules loaded via 'load') can only declare them. This is because libraries
   are loaded lazily (so essentially at unpredictable moments) and their module
   dict is cached and reused. The state inside library's module dict must not
   depend on whatever state variables had when the library was loaded.
2. Each 'exec' call opens a new scope for variables. Variables set within a
   scope are "forgotten" when exiting the scope. Variable value lookup start
   with the innermost scope, and goes up to the top scope. Thus variable values
   propagate down 'exec' call stack, but not up or "sideways".
3. Variables are set-only. If a variable was set in the current scope or any
   of parent scopes, assigning a new value to it is an error.

R=tandrii@chromium.org
BUG=833946

Change-Id: Idaf44866125c7cf8ef63247587b5b67eabf0782f
Reviewed-on: https://chromium-review.googlesource.com/c/1451101
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/doc/README.md
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/generator.go
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/starlark/stdlib/internal/lucicfg.star
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/starlark/stdlib/native_doc.star
[modify] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/state.go
[add] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/testdata/misc/support/shared_vars.star
[add] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/testdata/misc/support/uses_vars.star
[add] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/testdata/misc/vars.star
[add] https://crrev.com/4f458e7f03a67755e691904467ad65e4c91179f9/lucicfg/vars/vars.go

Comment 129 by bugdroid, Feb 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0599d2ae781317f088c8248f97bb9de38f2fc381

commit 0599d2ae781317f088c8248f97bb9de38f2fc381
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 04 21:22:49 2019

[lucicfg] Forbid modifying the graph as a side effect of a 'load'.

All loads must be pure. We now have 'exec's specifically for dirty evaluations.

This is theoretically a breaking change, but no one was relying on loads with
side effects.

R=tandrii@chromium.org
BUG=833946

Change-Id: I3684693a8c7c0a0d6cd01dd8e3271bc34655bb00
Reviewed-on: https://chromium-review.googlesource.com/c/1451669
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

[modify] https://crrev.com/0599d2ae781317f088c8248f97bb9de38f2fc381/lucicfg/configset.go
[modify] https://crrev.com/0599d2ae781317f088c8248f97bb9de38f2fc381/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/0599d2ae781317f088c8248f97bb9de38f2fc381/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/0599d2ae781317f088c8248f97bb9de38f2fc381/lucicfg/state.go
[modify] https://crrev.com/0599d2ae781317f088c8248f97bb9de38f2fc381/lucicfg/version.go

Comment 130 by bugdroid, Feb 4

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/e48c958d0748f019f963fb647e5894d630759901

commit e48c958d0748f019f963fb647e5894d630759901
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 04 22:43:50 2019

[lucicfg] Make var.get() auto-initialize the variable to the supplied default.

It means once var.get() is called for the first time, all subsequent calls
will observe the exact same value (either an explicitly set one or the default).

This is a measure to reduce "statefullness" of vars, meant to outlaw the
following case:

  lib.do_something()  # does something based on lib.var default value
  lib.var.set(123)
  lib.do_something()  # suddenly does something else, since lib.var is set now

Now lib.var.set(...) call will fail with a message explaining that it was
auto-initialized as a side effect of first 'get'.

R=iannucci@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I0efb87cc0833d3e74ac9e19741b27f83ea166476
Reviewed-on: https://chromium-review.googlesource.com/c/1453018
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

[modify] https://crrev.com/e48c958d0748f019f963fb647e5894d630759901/lucicfg/doc/README.md
[modify] https://crrev.com/e48c958d0748f019f963fb647e5894d630759901/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/e48c958d0748f019f963fb647e5894d630759901/lucicfg/starlark/stdlib/internal/lucicfg.star
[modify] https://crrev.com/e48c958d0748f019f963fb647e5894d630759901/lucicfg/state.go
[modify] https://crrev.com/e48c958d0748f019f963fb647e5894d630759901/lucicfg/testdata/misc/vars.star
[modify] https://crrev.com/e48c958d0748f019f963fb647e5894d630759901/lucicfg/vars/vars.go

Comment 131 by bugdroid, Feb 10

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30

commit 8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sun Feb 10 23:12:51 2019

[lucicfg] Stop returning graph.node from 'add_node'.

graph.node objects aren't really supposed to be used while the graph is under
construction.

This is a cleanup in preparation for making various declarations return
graph.key objects. It will be needed for Milo and CQ declarations, e.g. to allow

   milo.console(
       name = 'ci-console',
       title = 'CI Console',

       builder_entries = [
           milo.builder_entry_spec(
               builder = 'Linux CI builder',
               category = 'infra',
               short_name = 'lnx',
           ),
       ],
   )

Also remove 'go:generate goimports -w .' from crostesting protos' generate.go.
It causes non-null diff on *.pb.go even if nothing is touched.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: Icd6a8e2040afa93b3b5c87c189c81e7a9bdc5811
Reviewed-on: https://chromium-review.googlesource.com/c/1462509
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30/lucicfg/external/crostesting/proto/config/generate.go
[modify] https://crrev.com/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30/lucicfg/graph/graph.go
[modify] https://crrev.com/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30/lucicfg/graph/testdata/node.star
[modify] https://crrev.com/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/8059e6f2afcbc676ec1e5c9a6dbd6ad593fa6d30/lucicfg/version.go

Comment 132 by bugdroid, Feb 11

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3

commit d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 11 17:48:59 2019

[lucicfg] Put ACLs directly into Bucket proto instead of via AclSet.

R=nodir@chromium.org
BUG=833946

Change-Id: Iac2bce37be1e7cf627f48c4b4ee18941c6edac1b
Reviewed-on: https://chromium-review.googlesource.com/c/1464459
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/full_example.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/scheduler/ambiguity_resolution.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/scheduler/builder_builder.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/scheduler/dedups_triggerer_acl.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/scheduler/many_to_many.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/scheduler/poller_builder.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/testdata/scheduler/single_poller.star
[modify] https://crrev.com/d5e2961d0a9160dea897e87a98e67d5f6e7a2fd3/lucicfg/version.go

Comment 133 by bugdroid, Feb 11

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/848d87add663bb265a035d2a166728fce25c17e2

commit 848d87add663bb265a035d2a166728fce25c17e2
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 11 23:17:27 2019

[lucicfg] Allow calling rules inline right where they are referenced.

Rules now return graph.keyset(...) objects that are accepted by other rules.

This is now allowed (but subject to all existing constraints, e.g. name
uniqueness):

    core.builder(
        ...
        recipe = core.recipe(
            ...
        ),
    )

This is in preparation for adding Milo and CQ rules that use "annotated"
many-to-many relations (milo groups <-> builders, CQ ref sets <-> builders).
Each such annotated relation would be represented by a graph node. Requiring
all of them to be declared at the top scope leads to a lot of unnecessary
repetition.

R=tandrii@chromium.org, nodir@chromium.org
BUG=833946

Change-Id: I1bb742d7973ca7f3c419fa843e42ff0e6c965aa5
Reviewed-on: https://chromium-review.googlesource.com/c/1462540
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/doc/README.md
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/common.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/rules/bucket.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/rules/logdog.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/starlark/stdlib/internal/luci/rules/recipe.star
[add] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/testdata/errors/wrong_kind.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/testdata/full_example.star
[modify] https://crrev.com/848d87add663bb265a035d2a166728fce25c17e2/lucicfg/version.go

Comment 134 by bugdroid, Feb 11

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5

commit 2af4f06eb36d466fc1a36fafb24e48356c7c3bc5
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Mon Feb 11 23:51:39 2019

[lucicfg] Rename `core.*` to `luci.*`.

It is more descriptive and makes more sense when lucicfg is used to generate
other sorts of configs.

R=nodir@chromium.org
BUG=833946

Change-Id: I06e5eb9675c41b2098fdbc3d739aee67c0879e06
Reviewed-on: https://chromium-review.googlesource.com/c/1464902
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/doc/README.md
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/doc/templates/README.mdt
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/docgen/docstring/docstring_test.go
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/docgen/generator.go
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/graph/doc.go
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/meta.go
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/common.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/generators.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/lib/acl.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/lib/scheduler.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/lib/swarming.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/rules/builder.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/rules/gitiles_poller.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/luci/rules/recipe.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/starlark/stdlib/internal/lucicfg.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/ambiguous_references.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/bucket_without_project.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/bucket_without_service.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/builder_without_recipe.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/logdog_without_project.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/poller_builder_clash.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/project_redefinition.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/project_without_name.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/scheduler_disambiguation_fail.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/triggering_builder_no_account.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/errors/wrong_kind.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/full_example.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/misc/support/execed.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/misc/triggerer.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/scheduler/ambiguity_resolution.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/scheduler/builder_builder.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/scheduler/dedups_triggerer_acl.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/scheduler/many_to_many.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/scheduler/poller_builder.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/testdata/scheduler/single_poller.star
[modify] https://crrev.com/2af4f06eb36d466fc1a36fafb24e48356c7c3bc5/lucicfg/version.go

Comment 135 by bugdroid, Feb 12

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra/+/782dea371f97ff6e18d20da6915384531720bc8a

commit 782dea371f97ff6e18d20da6915384531720bc8a
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Tue Feb 12 00:52:24 2019

Roll infra/go/src/go.chromium.org/luci/ 4abe49150..ea41200b0 (5 commits)

https://chromium.googlesource.com/infra/luci/luci-go//+log/4abe491501b6..ea41200b060f

$ git log 4abe49150..ea41200b0 --date=short --no-merges --format='%ad %ae %s'
2019-02-12 vadimsh [swarming] Pick up bots.proto change, to use from lucicfg.
2019-02-11 vadimsh [lucicfg] Rename `core.*` to `luci.*`.
2019-02-11 vadimsh [lucicfg] Allow calling rules inline right where they are referenced.
2019-02-11 maruel [archiver] Reduce the number of properties on Archiver
2019-02-11 seanabraham crostesting: Add lots of missing fields to target test config proto.

Created with:
  roll-dep infra/go/src/go.chromium.org/luci

TBR=nodir@chromium.org, maruel@chromium.org
BUG=833946

Change-Id: Ia802188561c1527d032dcb87493463d93d8b08a2
Reviewed-on: https://chromium-review.googlesource.com/c/1465204
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Cr-Commit-Position: refs/heads/master@{#20595}
[modify] https://crrev.com/782dea371f97ff6e18d20da6915384531720bc8a/DEPS

Comment 136 by bugdroid, Feb 13 (5 days ago)

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/0dd93d57ff96fc6e525051477bdc19b86db87290

commit 0dd93d57ff96fc6e525051477bdc19b86db87290
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Wed Feb 13 19:16:31 2019

[starlark] Add Interpreter.LoadSource function.

It will be used to load text proto files (like Milo console header definitions)
stored alongside *.star code in lucicfg packages.

Also cleanup some comments and minor implementation details in Interpreter.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I94a2053dbcc776edfcca6b448371db40caa2c826
Reviewed-on: https://chromium-review.googlesource.com/c/1469267
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>

[modify] https://crrev.com/0dd93d57ff96fc6e525051477bdc19b86db87290/starlark/interpreter/interpreter.go
[modify] https://crrev.com/0dd93d57ff96fc6e525051477bdc19b86db87290/starlark/interpreter/interpreter_test.go

Comment 137 by bugdroid, Feb 14 (5 days ago)

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/ae701cae096d129766981bd97624e485d94a159c

commit ae701cae096d129766981bd97624e485d94a159c
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Thu Feb 14 01:22:21 2019

[lucicfg] Add `sequence.next(name)` internal API.

Will be used to generate unique keys for luci.console_view_entry(builder=...)
nodes when they are used inline as:

    luci.console_view(
        entries = [
            luci.console_view_entry(builder=...),
        ],
    )

Values passed to luci.console_view_entry in this case are not sufficient to
generate a unique enough key.

R=nodir@chromium.org, tandrii@chromium.org
BUG=833946

Change-Id: I2b421b30b03080108c90ae97e6346f98a025155d
Reviewed-on: https://chromium-review.googlesource.com/c/1471045
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/ae701cae096d129766981bd97624e485d94a159c/lucicfg/generator.go
[add] https://crrev.com/ae701cae096d129766981bd97624e485d94a159c/lucicfg/sequence.go
[modify] https://crrev.com/ae701cae096d129766981bd97624e485d94a159c/lucicfg/starlark/assets.gen.go
[add] https://crrev.com/ae701cae096d129766981bd97624e485d94a159c/lucicfg/starlark/stdlib/internal/sequence.star
[modify] https://crrev.com/ae701cae096d129766981bd97624e485d94a159c/lucicfg/state.go
[add] https://crrev.com/ae701cae096d129766981bd97624e485d94a159c/lucicfg/testdata/misc/sequence_next.star

Comment 138 by bugdroid, Feb 16 (3 days ago)

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/infra/luci/luci-go.git/+/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4

commit 147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4
Author: Vadim Shtayura <vadimsh@chromium.org>
Date: Sat Feb 16 02:15:11 2019

[lucicfg] Add support for Milo builder lists.

This introduces 3 new declarations:
    * luci.milo(...) for declaring UI-related project properties. Analogues to
      how luci.logdog(...) defines log-related properties.
    * luci.list_view(...) for declaring a builder list (which is internally
      a special kind of console).
    * luci.list_view_entry(...) for declaring an entry in the builder list. Its
      usage is not required (just passing a builder name works too), but it is
      still useful to declare that a builder belongs to a list _after_ the list
      has been defined. It is also the only way to add pure Buildbot builders
      to a view.

R=nodir@chromium.org
BUG=833946

Change-Id: Ia74ac211ecb5bdd3acf248812f93a7489e139c62
Reviewed-on: https://chromium-review.googlesource.com/c/1472090
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>

[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/doc/README.md
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/graph/node.go
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/graph/testdata/node.star
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/assets.gen.go
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/builtins.star
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/graph.star
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/luci/common.star
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/luci/generators.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/luci/rules/list_view.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/luci/rules/list_view_entry.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/luci/rules/milo.star
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/starlark/stdlib/internal/luci/rules/project.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/testdata/errors/list_view_dup_builder.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/testdata/errors/list_view_multiparent_entry.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/testdata/errors/list_view_orphan_entry.star
[add] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/testdata/errors/list_view_wrong_entry.star
[modify] https://crrev.com/147bb2c6d6e4b0fdd4fd0badd3ac4025d5de07a4/lucicfg/testdata/full_example.star
Showing comments 39 - 138 of 138 Older

Sign in to add a comment