Create an app to track progress of migration from Buildbot to LUCI
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/403e8f9e44bcecb291da6630e479796669f94d77 commit 403e8f9e44bcecb291da6630e479796669f94d77 Author: Nodir Turakulov <nodir@google.com> Date: Tue May 09 23:02:13 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/5110bd57e3b85e0089db39eddeafbe89b68bc20e commit 5110bd57e3b85e0089db39eddeafbe89b68bc20e Author: Nodir Turakulov <nodir@google.com> Date: Wed May 10 07:03:44 2017 luci-migration: initial boilerplate code derived from https://github.com/luci/luci-go/tree/master/scheduler/appengine Bug= 719734 Change-Id: If884f55366e2d5d76ab49728624866504c43c93f Change-Id: If884f55366e2d5d76ab49728624866504c43c93f Reviewed-on: https://chromium-review.googlesource.com/498017 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/luci-migration.infra_testing [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/templates/includes/base.html [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/fonts/glyphicons-halflings-regular.eot [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/js/bootstrap.min.js [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/fonts/glyphicons-halflings-regular.woff2 [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/queue.yaml [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/templates/pages/index.html [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/css/bootstrap.min.css [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/handler.go [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/fonts/glyphicons-halflings-regular.svg [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/fonts/glyphicons-halflings-regular.woff [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/jquery/jquery.min.js [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/static/bootstrap/fonts/glyphicons-halflings-regular.ttf [add] https://crrev.com/5110bd57e3b85e0089db39eddeafbe89b68bc20e/go/src/infra/appengine/luci-migration/app.yaml
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/262440860ec7e6ce68efda3a3e06e79b48b5d30b commit 262440860ec7e6ce68efda3a3e06e79b48b5d30b Author: Nodir Turakulov <nodir@google.com> Date: Wed May 10 07:16:16 2017
we need this to prove builders WAI partially because the app will retry failed builds to fight flakiness. We cannot prove LUCI builds are correct because of the flakiness; CQ does not retry LUCI builds, but Buildbot builds.
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/a510e15770699140b5eac1f682e472d39be6a4f7 commit a510e15770699140b5eac1f682e472d39be6a4f7 Author: Nodir Turakulov <nodir@google.com> Date: Tue May 16 02:12:11 2017 luci-migration: discover builders Add a cron job that, for each hardcoded master, fetches builder names and registers them in the datastore. Registration implies creating a Monorail issue and putting an entity. Do not bother about potential 10m cron timeout to register all builders on first run, but exit on 9m before creating an issue. Bug: 719734 Change-Id: I017e87fb75c751e512e161cbfc0069def352b741 Reviewed-on: https://chromium-review.googlesource.com/502330 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/luci-migration.infra_testing [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/monorail/monorailtest/monorailtest.go [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/cron.yaml [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/discovery.go [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/builders.go [modify] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/handler.go [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/monorail/monorailtest/monorailtest.infra_testing [modify] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/app.yaml [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/discovery_test.go [add] https://crrev.com/a510e15770699140b5eac1f682e472d39be6a4f7/go/src/infra/appengine/luci-migration/bugs.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/30ef5f8f141ccc09f9afc181534e3f11396f42b0 commit 30ef5f8f141ccc09f9afc181534e3f11396f42b0 Author: Nodir Turakulov <nodir@google.com> Date: Thu May 18 06:30:19 2017 luci-migration: put code to packages Put code to packages storage and discovery. Bug: 719734 Change-Id: I3cffe3fe3bd32d95611bef1b5c9adae30c99ebdc Reviewed-on: https://chromium-review.googlesource.com/507013 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [rename] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/discovery/discovery_test.go [rename] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/discovery/bugs.go [add] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/common/common.go [add] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/storage/builders.go [modify] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/luci-migration.infra_testing [delete] https://crrev.com/c7fe4f26aaad78dc02583874e1f9443458e7c0ae/go/src/infra/appengine/luci-migration/builders.go [add] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/storage/storage.infra_testing [modify] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/handler.go [add] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/discovery/discovery.infra_testing [add] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/common/common.infra_testing [rename] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/discovery/discovery.go [add] https://crrev.com/30ef5f8f141ccc09f9afc181534e3f11396f42b0/go/src/infra/appengine/luci-migration/storage/masters.go
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/b77d854431e82b219679f925ae9a3ee0069d6004 commit b77d854431e82b219679f925ae9a3ee0069d6004 Author: Nodir Turakulov <nodir@google.com> Date: Thu May 18 18:26:30 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/4f6a46b03bb5731287ecab9bf44fb990237ed8ee commit 4f6a46b03bb5731287ecab9bf44fb990237ed8ee Author: Nodir Turakulov <nodir@google.com> Date: Thu May 18 20:01:25 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53 commit 6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53 Author: Nodir Turakulov <nodir@google.com> Date: Fri May 19 07:13:22 2017 luci-migration: add app package GAE does not like the current organization of packages. Bug: 719734 Change-Id: Idd8b85185c9a7232c025a2eb89cfdf459b57771a Reviewed-on: https://chromium-review.googlesource.com/508872 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [rename] https://crrev.com/6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53/go/src/infra/appengine/luci-migration/app/handlers.go [rename] https://crrev.com/6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53/go/src/infra/appengine/luci-migration/app/queue.yaml [rename] https://crrev.com/6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53/go/src/infra/appengine/luci-migration/app/app.yaml [rename] https://crrev.com/6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53/go/src/infra/appengine/luci-migration/app/cron.yaml [rename] https://crrev.com/6039b1bfc9596fe4abfdad4b28f6f4aa55ae3a53/go/src/infra/appengine/luci-migration/app/app.infra_testing
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/28c5a0617156cc39b54d259576af68e41efaa27b commit 28c5a0617156cc39b54d259576af68e41efaa27b Author: Nodir Turakulov <nodir@google.com> Date: Sat May 20 00:07:36 2017 luci-migration: use luci-config use luci-config for hostnames and master list Bug: 719734 Change-Id: Ifda7020d33861ad257574f3c38e922fe81255bbb Reviewed-on: https://chromium-review.googlesource.com/508285 Reviewed-by: Daniel Jacques <dnj@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/discovery/discovery_test.go [modify] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/discovery/bugs.go [delete] https://crrev.com/1277be0e78e89e29724bf489c70459746c4ab371/go/src/infra/appengine/luci-migration/common/common.go [modify] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/storage/builders.go [modify] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/app/handlers.go [add] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/config/gen.go [modify] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/app/cron.yaml [rename] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/config/config.infra_testing [add] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/config/config.pb.go [add] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/config/config.proto [modify] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/discovery/discovery.go [add] https://crrev.com/28c5a0617156cc39b54d259576af68e41efaa27b/go/src/infra/appengine/luci-migration/config/config.go [delete] https://crrev.com/1277be0e78e89e29724bf489c70459746c4ab371/go/src/infra/appengine/luci-migration/storage/masters.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/214d8cfd4afa56e9fa74841e18eb1d9042bef7e6 commit 214d8cfd4afa56e9fa74841e18eb1d9042bef7e6 Author: Nodir Turakulov <nodir@google.com> Date: Sat May 20 00:34:17 2017 luci-migration: apply RVG to bugs for internal builders Bug: 719734 Change-Id: Ida4ed951d164f5b77665bd037d32640f13180441 Reviewed-on: https://chromium-review.googlesource.com/508238 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/214d8cfd4afa56e9fa74841e18eb1d9042bef7e6/go/src/infra/appengine/luci-migration/discovery/discovery_test.go [modify] https://crrev.com/214d8cfd4afa56e9fa74841e18eb1d9042bef7e6/go/src/infra/appengine/luci-migration/discovery/bugs.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/c86c3020283bb824894a927678acc4d7bbfb3335 commit c86c3020283bb824894a927678acc4d7bbfb3335 Author: Nodir Turakulov <nodir@google.com> Date: Sat May 20 02:39:17 2017 luci-migration: retry experimental LUCI builds Builds are flaky. To fight flakiness, CQ generally retries builds, but it does not retry experimental builds. We cannot make LUCI builds non-experimental before we are confident they WAI. We cannot compute builder correctness correctly without retries because of the flakiness. Retry experimental LUCI builds by listening to buildbucket PubSub and calling buildbucket.put API. Bug: 719734 Change-Id: If0ca353290d129af803152d23785a26a34cf8f74 Reviewed-on: https://chromium-review.googlesource.com/509748 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/c86c3020283bb824894a927678acc4d7bbfb3335/go/src/infra/appengine/luci-migration/app/handlers.go [add] https://crrev.com/c86c3020283bb824894a927678acc4d7bbfb3335/go/src/infra/appengine/luci-migration/flakiness/flakiness.infra_testing [add] https://crrev.com/c86c3020283bb824894a927678acc4d7bbfb3335/go/src/infra/appengine/luci-migration/flakiness/flakiness.go [add] https://crrev.com/c86c3020283bb824894a927678acc4d7bbfb3335/go/src/infra/appengine/luci-migration/flakiness/flakiness_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/3e8a8c4cf027cd601a8386b299fe15af02f53922 commit 3e8a8c4cf027cd601a8386b299fe15af02f53922 Author: Nodir Turakulov <nodir@google.com> Date: Sat May 20 02:56:37 2017 luci-migration: add SchedulingType.UNSET Add SchedulingType.UNSET so that we distinguish it from TRYJOBS Also temporarily disable registration of non-TRYJOB builders TBR=dnj@chromium.org Bug: 719734 Change-Id: I1812af291a569db9ffae6fdb6c1dbf359693c412 Reviewed-on: https://chromium-review.googlesource.com/510086 Reviewed-by: Nodir Turakulov <nodir@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/3e8a8c4cf027cd601a8386b299fe15af02f53922/go/src/infra/appengine/luci-migration/discovery/discovery.go [modify] https://crrev.com/3e8a8c4cf027cd601a8386b299fe15af02f53922/go/src/infra/appengine/luci-migration/config/config.proto
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/da526bb4f0c95c301baf4b1bd8ce82de822b40ba commit da526bb4f0c95c301baf4b1bd8ce82de822b40ba Author: Nodir Turakulov <nodir@google.com> Date: Sat May 20 04:28:10 2017 luci-migration: move templates and static dirs to app https://chromium-review.googlesource.com/c/508872/ should have put templates and static dirs to app dir too Bug: 719734 Change-Id: I7a93f2520cf2fe60bd3742a580f0b80c23a2f163 Reviewed-on: https://chromium-review.googlesource.com/509828 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/fonts/glyphicons-halflings-regular.ttf [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/templates/includes/base.html [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/css/bootstrap.min.css [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/fonts/glyphicons-halflings-regular.eot [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/fonts/glyphicons-halflings-regular.svg [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/jquery/jquery.min.js [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/fonts/glyphicons-halflings-regular.woff2 [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/fonts/glyphicons-halflings-regular.woff [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/templates/pages/index.html [rename] https://crrev.com/da526bb4f0c95c301baf4b1bd8ce82de822b40ba/go/src/infra/appengine/luci-migration/app/static/bootstrap/js/bootstrap.min.js
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/b2d9ee5c9083a669f0a709b510134c0af21c7e28 commit b2d9ee5c9083a669f0a709b510134c0af21c7e28 Author: Nodir Turakulov <nodir@google.com> Date: Mon May 22 15:57:02 2017 luci-migration: fix UNSET https://chromium-review.googlesource.com/c/510086/ was bad because 1) it does not include changes to config.pb.go 2) apparently, enum member names must be unique within proto package Bug: 719734 Change-Id: Id06af6f38652676bdf78b320e31ea9390b3d43f6 Reviewed-on: https://chromium-review.googlesource.com/510002 Reviewed-by: Daniel Jacques <dnj@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/b2d9ee5c9083a669f0a709b510134c0af21c7e28/go/src/infra/appengine/luci-migration/discovery/bugs.go [modify] https://crrev.com/b2d9ee5c9083a669f0a709b510134c0af21c7e28/go/src/infra/appengine/luci-migration/config/config.proto [modify] https://crrev.com/b2d9ee5c9083a669f0a709b510134c0af21c7e28/go/src/infra/appengine/luci-migration/config/config.pb.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/55cf4ac749b2ccccea52d88e0513628ac2f53ee7 commit 55cf4ac749b2ccccea52d88e0513628ac2f53ee7 Author: Nodir Turakulov <nodir@google.com> Date: Mon May 22 16:08:33 2017 luci-migration: record orig build id When retrying a potentially flaky build, record its original build id Bug: 719734 Change-Id: I041180cc411293dae7e2a46066637f8ca8d5d08f Reviewed-on: https://chromium-review.googlesource.com/509978 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/55cf4ac749b2ccccea52d88e0513628ac2f53ee7/go/src/infra/appengine/luci-migration/flakiness/flakiness_test.go [modify] https://crrev.com/55cf4ac749b2ccccea52d88e0513628ac2f53ee7/go/src/infra/appengine/luci-migration/flakiness/flakiness.go
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/4682829cb7d75d0e3df94e8fc93c07c24d4e245a commit 4682829cb7d75d0e3df94e8fc93c07c24d4e245a Author: Nodir Turakulov <nodir@google.com> Date: Tue May 23 19:36:59 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/377054897682262b05c6c4d2782cda48441bccd2 commit 377054897682262b05c6c4d2782cda48441bccd2 Author: Nodir Turakulov <nodir@google.com> Date: Thu Jun 01 02:27:32 2017 luci-migration: add pages define a protocol between analysis pipeline and HTML handlers: Builder.Migration field and BuilderMigrationDetails entity. Add HTML handlers for / /masters/:master/ /masters/:master/builders/:builder/ Bug: 719734 Change-Id: I3789a604d7da7730d6c0e67f81f8800645780095 Reviewed-on: https://chromium-review.googlesource.com/514422 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/templates/includes/base.html [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/common_test.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/index.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/builder.go [modify] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/handlers.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/master.go [delete] https://crrev.com/93241dee2acb6147f7b0ae1a43f41433128a502e/go/src/infra/appengine/luci-migration/storage/builders.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/templates/pages/builder.html [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/builder_test.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/index_test.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/storage/builder.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/master_test.go [modify] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/templates/pages/index.html [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/templates/pages/master.html [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/app/.gitignore [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/storage/migration.go [add] https://crrev.com/377054897682262b05c6c4d2782cda48441bccd2/go/src/infra/appengine/luci-migration/storage/builder_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/f2814499f904de8d63ba7856d857f402f3807eda commit f2814499f904de8d63ba7856d857f402f3807eda Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 02 19:41:25 2017 luci-migration: refactor config.proto Currently config.proto is organized around external services and it is unclear how a value is used. We want both dev and prod luci-migration apps to analyze same data, but it may be confusing to have prod cr-buildbucket to be specified in the luci-migration-dev. Simplify config.proto. A future CL will add "analysis" message. Both dev and prod apps will analyze prod builds on cr-buildbucket.appspot.com Bug: 719734 Change-Id: I902170b9a53a70dcad7b3abf07dec0b9bb83f8c5 Reviewed-on: https://chromium-review.googlesource.com/511582 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Daniel Jacques <dnj@chromium.org> [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/discovery/discovery_test.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/config/config.proto [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/app/index.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/app/handlers.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/app/master.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/app/index_test.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/app/master_test.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/config/config.pb.go [modify] https://crrev.com/f2814499f904de8d63ba7856d857f402f3807eda/go/src/infra/appengine/luci-migration/discovery/discovery.go
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/8616db114a7aeaea673b5f5b0cef4c11d4f01011 commit 8616db114a7aeaea673b5f5b0cef4c11d4f01011 Author: Nodir Turakulov <nodir@google.com> Date: Sat Jun 03 09:05:43 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/6d24e0f2d7772d9dd1edddf58e5233030688be84 commit 6d24e0f2d7772d9dd1edddf58e5233030688be84 Author: Nodir Turakulov <nodir@google.com> Date: Thu Jun 08 20:52:04 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/d001f9bb2dc0609a7fbe92ae947611fad4291350 commit d001f9bb2dc0609a7fbe92ae947611fad4291350 Author: Nodir Turakulov <nodir@google.com> Date: Thu Jun 08 22:11:55 2017 luci-migration: add build analysis Add a push task that compares Buildbot and LUCI builds and persists comparison report in Builder and BuilderMigrationDetails entities. Add a cron job that enqueues a task per not-yet-migrated builder. Bug: 719734 Change-Id: Ia464d1e80027ee21e691c0e2c7fdaae81cd5bfbd Reviewed-on: https://chromium-review.googlesource.com/514342 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/tryjobs.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/app/index_test.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/storage/builder.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/bbutil/bbutil_test.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/bbutil/search.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/group.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/app/handlers.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/app/cron.yaml [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/group_test.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/bbutil/bbutil.infra_testing [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/config/config.proto [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/flakiness/flakiness_test.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/discovery/discovery.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/analysis.infra_testing [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/flakiness/flakiness.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/storage/migration.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/app/builder_test.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/app/master_test.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/bbutil/bbutil.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/storage/builder_test.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/tryjobs_test.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/tryjobs_template.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/app/analysis.go [modify] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/config/config.pb.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/compare.go [add] https://crrev.com/d001f9bb2dc0609a7fbe92ae947611fad4291350/go/src/infra/appengine/luci-migration/analysis/compare_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90 commit 4e3262ba5aa5a86723b07e350c2e8abaebd8cf90 Author: Nodir Turakulov <nodir@google.com> Date: Thu Jun 08 22:49:08 2017 luci-migration: schedule LUCI builds Schedule a LUCI build when a Buildbot build completes. This way we will have consistent builds between Buildbot and LUCI that is important to evaluate correctness. Bug: 719734 Change-Id: I078812ddab5eaa58debb526216e1a58200b7cad5 Reviewed-on: https://chromium-review.googlesource.com/522842 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [delete] https://crrev.com/2c9a60b5a9638d21553f7dea8b33e284fe4f2e7c/go/src/infra/appengine/luci-migration/flakiness/flakiness.go [add] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/scheduling/scheduling_test.go [modify] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/app/templates/pages/builder.html [modify] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/app/handlers.go [modify] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/app/builder.go [modify] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/app/builder_test.go [modify] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/storage/builder.go [add] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/scheduling/scheduling.go [delete] https://crrev.com/2c9a60b5a9638d21553f7dea8b33e284fe4f2e7c/go/src/infra/appengine/luci-migration/flakiness/flakiness_test.go [modify] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/bbutil/bbutil.go [rename] https://crrev.com/4e3262ba5aa5a86723b07e350c2e8abaebd8cf90/go/src/infra/appengine/luci-migration/scheduling/scheduling.infra_testing
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/b0e08819bb8223ea562ce3871bdb3bd4944e29ed commit b0e08819bb8223ea562ce3871bdb3bd4944e29ed Author: Nodir Turakulov <nodir@google.com> Date: Thu Jun 08 23:52:21 2017
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/37408e0eb37f44c7ef6af28f9ea4d1e349936d48 commit 37408e0eb37f44c7ef6af28f9ea4d1e349936d48 Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 00:03:14 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/36ce9785d53e82fe36a2a7909fb7f14e7369ca04 commit 36ce9785d53e82fe36a2a7909fb7f14e7369ca04 Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 01:20:20 2017 luci-migration: remove confidence The confidence calculation wasn't correct. With the new scheduler/ package it won't be useful because we schedule LUCI builds right after Buildbot builds. R=dnj@chromium.org Bug: 719734 Change-Id: I164acf40560e9fa9f1619e626c7767751d64a6aa Reviewed-on: https://chromium-review.googlesource.com/528482 Reviewed-by: Daniel Jacques <dnj@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/36ce9785d53e82fe36a2a7909fb7f14e7369ca04/go/src/infra/appengine/luci-migration/storage/migration.go [modify] https://crrev.com/36ce9785d53e82fe36a2a7909fb7f14e7369ca04/go/src/infra/appengine/luci-migration/analysis/compare_test.go [modify] https://crrev.com/36ce9785d53e82fe36a2a7909fb7f14e7369ca04/go/src/infra/appengine/luci-migration/app/templates/pages/builder.html [modify] https://crrev.com/36ce9785d53e82fe36a2a7909fb7f14e7369ca04/go/src/infra/appengine/luci-migration/app/templates/pages/master.html [modify] https://crrev.com/36ce9785d53e82fe36a2a7909fb7f14e7369ca04/go/src/infra/appengine/luci-migration/analysis/compare.go
The following revision refers to this bug: https://chrome-internal.googlesource.com/infradata/config/+/d8bc8848c8cbe2691b1b283a151007dd0fc9714e commit d8bc8848c8cbe2691b1b283a151007dd0fc9714e Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 16:45:52 2017
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/f1286f369608eafa490aa7d023ec3f0a3999a37f commit f1286f369608eafa490aa7d023ec3f0a3999a37f Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 17:06:28 2017 luci-migration: fix fetcher.Fetch fetcher.Fetch returns context.Cancelled even if it was cancelled because we've found enough builds. As a result, the caller thinks that fetching failed and does not start analysis. Do not return context.Cancelled that was returned by fetchLUCIBuilds. TBR=tandrii@chromium.org BUG= 719734 Change-Id: I1d16a67c0416bcca8d123e13ef7bdd7acdc88301 Reviewed-on: https://chromium-review.googlesource.com/529644 Reviewed-by: Nodir Turakulov <nodir@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/f1286f369608eafa490aa7d023ec3f0a3999a37f/go/src/infra/appengine/luci-migration/analysis/tryjobs.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/cb0e4b4a7cad4307647f4f557718343cc44786b7 commit cb0e4b4a7cad4307647f4f557718343cc44786b7 Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 17:55:40 2017 luci-migration: use range input for percentage Use <input type="range"> instead of <input type="number"> for experiment percentage R=tandrii@chromium.org BUG= 719734 Change-Id: Ibda1f48741e7c06ef4e0458aa2760b08ee98b930 Reviewed-on: https://chromium-review.googlesource.com/529764 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/cb0e4b4a7cad4307647f4f557718343cc44786b7/go/src/infra/appengine/luci-migration/app/templates/pages/builder.html
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/168ca42d78167fa30620aa0f28f65938249cdd21 commit 168ca42d78167fa30620aa0f28f65938249cdd21 Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 19:11:33 2017 luci-migration: make bbutil.ParseTimestamp more precise Retrying ignores a failed build id there is another build that created_ts one microsecond later or more. However, this does not work because bbutil.ParseTimestamp ignores microsecons. Do not ignore them R=tandrii@chromium.org BUG= 719734 Change-Id: Id93f3974fd01d340d30328f41d81999d24d9ef2c Reviewed-on: https://chromium-review.googlesource.com/529610 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Erik Staab <estaab@chromium.org> [modify] https://crrev.com/168ca42d78167fa30620aa0f28f65938249cdd21/go/src/infra/appengine/luci-migration/bbutil/bbutil.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/ffffe0ed2a76d97c00dda049411fa79319c3ef76 commit ffffe0ed2a76d97c00dda049411fa79319c3ef76 Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 19:42:15 2017 luci-migration: fix format strings Fix invalid format strings and args R=tandrii@chromium.org, estaab@chromium.org BUG= 719734 Change-Id: I59b5efcc4a4ca1eeca0e72246c908a0d2d6490d3 Reviewed-on: https://chromium-review.googlesource.com/529905 Reviewed-by: Erik Staab <estaab@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/ffffe0ed2a76d97c00dda049411fa79319c3ef76/go/src/infra/appengine/luci-migration/app/handlers.go [modify] https://crrev.com/ffffe0ed2a76d97c00dda049411fa79319c3ef76/go/src/infra/appengine/luci-migration/discovery/discovery.go [modify] https://crrev.com/ffffe0ed2a76d97c00dda049411fa79319c3ef76/go/src/infra/appengine/luci-migration/scheduling/scheduling.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/ea979f05d1e3ee22a80a7d96d70e312a81c14523 commit ea979f05d1e3ee22a80a7d96d70e312a81c14523 Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 22:43:16 2017 luci-migration: check special group membership of writes R=estaab@chromium.org BUG= 719734 Change-Id: I53516471e83fcf4d395fce4f4b5304b6c9ea6dab Reviewed-on: https://chromium-review.googlesource.com/530127 Commit-Queue: Nodir Turakulov <nodir@chromium.org> Reviewed-by: Erik Staab <estaab@chromium.org> [modify] https://crrev.com/ea979f05d1e3ee22a80a7d96d70e312a81c14523/go/src/infra/appengine/luci-migration/app/builder_test.go [modify] https://crrev.com/ea979f05d1e3ee22a80a7d96d70e312a81c14523/go/src/infra/appengine/luci-migration/app/builder.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d commit 543157ce0dcc84a072cde3bdb7e89a17c8dbb78d Author: Nodir Turakulov <nodir@google.com> Date: Fri Jun 09 22:44:16 2017 luci-migration: use rel dates Super log absolute dates are hard to reason about. Use relative dates. R=estaab@chromium.org BUG= 719734 Change-Id: I327cba7b0f2716744baba1b8a5643d54dc0bbd18 Reviewed-on: https://chromium-review.googlesource.com/530029 Reviewed-by: Erik Staab <estaab@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/analysis/group.go [add] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/common/common.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/app/common_test.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/app/builder.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/app/handlers.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/analysis/tryjobs_template.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/app/templates/pages/builder.html [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/analysis/tryjobs.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/app/builder_test.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/analysis/compare.go [add] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/common/common.infra_testing [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/analysis/compare_test.go [modify] https://crrev.com/543157ce0dcc84a072cde3bdb7e89a17c8dbb78d/go/src/infra/appengine/luci-migration/bbutil/bbutil.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/1501e176436e32fedd2d7347db9b494e96dba09c commit 1501e176436e32fedd2d7347db9b494e96dba09c Author: Nodir Turakulov <nodir@google.com> Date: Sat Jun 10 04:18:22 2017 luci-migration: add README.md Document the procedure to migrate a try builder. R=estaab@chromium.org BUG= 719734 Change-Id: I5631ad203ddaf97265e121ea5377375420f4303b Reviewed-on: https://chromium-review.googlesource.com/530130 Reviewed-by: Erik Staab <estaab@chromium.org> Commit-Queue: Nodir Turakulov <nodir@chromium.org> [modify] https://crrev.com/1501e176436e32fedd2d7347db9b494e96dba09c/go/src/infra/appengine/luci-migration/analysis/tryjobs_template.go [modify] https://crrev.com/1501e176436e32fedd2d7347db9b494e96dba09c/go/src/infra/appengine/luci-migration/app/templates/includes/base.html [add] https://crrev.com/1501e176436e32fedd2d7347db9b494e96dba09c/go/src/infra/appengine/luci-migration/README.md [modify] https://crrev.com/1501e176436e32fedd2d7347db9b494e96dba09c/go/src/infra/appengine/luci-migration/app/templates/pages/builder.html
Comment 1 by bugdroid1@chromium.org
, May 9 2017