Design doc: go/quota-scheduler Demo: https://quota-scheduler-demo.googleplex.com/
Initial stack of CLs is under review at https://chromium-review.googlesource.com/c/infra/infra/+/1157204/ and related CLs.
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/508adda0144a99c1f010fe0947da23b852079a27 commit 508adda0144a99c1f010fe0947da23b852079a27 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Aug 13 21:15:21 2018 qscheduler: initial account and vector protos and logic Also, include various convenient vector helper methods and aliases. BUG= chromium:870537 TEST=unit tests Change-Id: Ie5d2bb2a667f4be43529b3066303f19fc195c7b9 Reviewed-on: https://chromium-review.googlesource.com/1157201 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/account/account_test.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/vector/vector_test.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/vector/vector.infra_testing [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/vector/vector.proto [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/vector/vector.pb.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/vector/vector.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/account/account.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/account/account.pb.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/account/gen.go [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/account/account.infra_testing [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/account/account.proto [add] https://crrev.com/508adda0144a99c1f010fe0947da23b852079a27/go/src/infra/qscheduler/qslib/types/vector/gen.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/fe229c3df6d58bbfd5fbf1fd1e9179db3e6999a0 commit fe229c3df6d58bbfd5fbf1fd1e9179db3e6999a0 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Aug 13 21:42:06 2018 qscheduler: add simple wrapper scripts for generate / test BUG= chromium:870537 TEST=unit tests Change-Id: I4e0b5dd0fd4647a408d1657cf7701356b4014c12 Reviewed-on: https://chromium-review.googlesource.com/1157205 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> [add] https://crrev.com/fe229c3df6d58bbfd5fbf1fd1e9179db3e6999a0/go/src/infra/qscheduler/qslib/generate.sh [add] https://crrev.com/fe229c3df6d58bbfd5fbf1fd1e9179db3e6999a0/go/src/infra/qscheduler/qslib/run_tests.sh
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/a57131ef49fd6ab612738834fda0c1f1bc2eef50 commit a57131ef49fd6ab612738834fda0c1f1bc2eef50 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Aug 13 21:53:41 2018 qscheduler: add protos for basic state and config model BUG= chromium:870537 TEST=unit tests Change-Id: Id436596a676b9e5cb561139302c02593f48fb2d1 Reviewed-on: https://chromium-review.googlesource.com/1157202 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Commit-Queue: Aviv Keshet <akeshet@chromium.org> [modify] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/account/account_test.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/task/task_test.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/task/task.pb.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/task/task.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/task/gen.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/types.pb.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/types.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/task/task.proto [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/worker.go [modify] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/vector/gen.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/types.proto [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/task/task.infra_testing [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/gen.go [add] https://crrev.com/a57131ef49fd6ab612738834fda0c1f1bc2eef50/go/src/infra/qscheduler/qslib/types/types.infra_testing
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/64d3cb6125902da141e1e13d38a7a47b902e19e5 commit 64d3cb6125902da141e1e13d38a7a47b902e19e5 Author: Aviv Keshet <akeshet@chromium.org> Date: Thu Aug 16 06:01:22 2018 qscheduler: initial implementation of prioritization logic BUG= chromium:870537 TEST=unit tests Change-Id: I31313b4302d230d420798554711fea8e1deda701 Reviewed-on: https://chromium-review.googlesource.com/1157203 Reviewed-by: smut <smut@google.com> Commit-Queue: Aviv Keshet <akeshet@chromium.org> [add] https://crrev.com/64d3cb6125902da141e1e13d38a7a47b902e19e5/go/src/infra/qscheduler/qslib/priority/prioritize_test.go [modify] https://crrev.com/64d3cb6125902da141e1e13d38a7a47b902e19e5/go/src/infra/qscheduler/qslib/types/account/account.go [add] https://crrev.com/64d3cb6125902da141e1e13d38a7a47b902e19e5/go/src/infra/qscheduler/qslib/priority/prioritize.go [modify] https://crrev.com/64d3cb6125902da141e1e13d38a7a47b902e19e5/go/src/infra/qscheduler/qslib/types/account/account_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/1a2dc3958d2251accaac30bd18302bf1a7762fb3 commit 1a2dc3958d2251accaac30bd18302bf1a7762fb3 Author: Aviv Keshet <akeshet@chromium.org> Date: Fri Aug 17 04:26:53 2018 qscheduler: initial implementation of scheduler logic BUG= chromium:870537 TEST=unit tests Change-Id: I218590ed807c62c4b2fe81ef2d45bd5a51ac29a2 Reviewed-on: https://chromium-review.googlesource.com/1157204 Reviewed-by: smut <smut@google.com> Commit-Queue: Aviv Keshet <akeshet@chromium.org> [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/mutaters/mutaters.proto [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/mutaters/mutaters.go [modify] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/priority/prioritize_test.go [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/mutaters/gen.go [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/mutaters/mutaters_test.go [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/mutaters/mutaters.pb.go [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/mutaters/mutaters.infra_testing [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/scheduler/scheduler.infra_testing [modify] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/priority/prioritize.go [modify] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/types/worker.go [add] https://crrev.com/1a2dc3958d2251accaac30bd18302bf1a7762fb3/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9 commit e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Aug 20 21:51:08 2018 qscheduler: normalize protos Prior to this change, a number of proto types in this project had a field specifying their own ID, but were also stored by parent objects in an ID-to-instance map. This is redundant, and is a potential source of inconsistent data, and was leading to extra meaningless boilerplate in test code. Eliminate this redundancy in the schema. BUG= chromium:870537 TEST=unit test Change-Id: I650559249aa53ed793b156d6087c5caaca9614b3 Reviewed-on: https://chromium-review.googlesource.com/1175556 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Aviv Keshet <akeshet@chromium.org> [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/types/task/task.pb.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/priority/prioritize_test.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/types/types.pb.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/mutaters/mutaters.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/mutaters/mutaters_test.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/types/task/task.proto [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [add] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/scheduler/helpers.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/types/types.proto [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/priority/prioritize.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/types/worker.go [modify] https://crrev.com/e0b1fbe93939f2ca7e0d8e11daf85f4a4ecfcae9/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/9927cb06cd545a5b33357ad823d6c0b865dfb0e3 commit 9927cb06cd545a5b33357ad823d6c0b865dfb0e3 Author: Aviv Keshet <akeshet@chromium.org> Date: Mon Aug 20 22:01:04 2018 qscheduler: add a scheduler preemption test case The lack of this case was uncovered in go tool cover inspection. BUG= chromium:870537 TEST=unit test Change-Id: I02b9cbe1fe297475177df0b3909ad1f023ef25cf Reviewed-on: https://chromium-review.googlesource.com/1176735 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: smut <smut@google.com> [modify] https://crrev.com/9927cb06cd545a5b33357ad823d6c0b865dfb0e3/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/9927cb06cd545a5b33357ad823d6c0b865dfb0e3/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/908b9b1042f1370685c0365a8f2885c9fba08b98 commit 908b9b1042f1370685c0365a8f2885c9fba08b98 Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Aug 21 00:57:11 2018 qscheduler: implement account updating behavior This change: - Implements the ability to calculate and update account balances based on workload. - Simplifies the way in which account maximum charge is configured, making it a single maximum charge time for all buckets rather than a distinct one per bucket. BUG= chromium:870537 TEST=unit test Change-Id: I87083ff400427506977fda6096a7ccdaaf067fe3 Reviewed-on: https://chromium-review.googlesource.com/1180264 Reviewed-by: smut <smut@google.com> Commit-Queue: Aviv Keshet <akeshet@chromium.org> [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/account/account_test.go [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/account/account.pb.go [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/types.pb.go [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/vector/vector.go [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/account/account.go [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/types.proto [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/types/account/account.proto [modify] https://crrev.com/908b9b1042f1370685c0365a8f2885c9fba08b98/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/b358c944bfe793ba66dcdd88bc41f9b1e1967b2a commit b358c944bfe793ba66dcdd88bc41f9b1e1967b2a Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Aug 21 01:18:20 2018 qscheduler: add a tutils utility package for converting time protos The standard ptypes library returns an error on proto parsing errors when converting to/from timestamp.Timestamp or duration.Duration. In most cases, callers would find it more convenient to simply panic on parsing errors, and allow calls to the conversion func to be inline with timestamp assignments etc. This implements a tiny convenience library for this purpose. BUG= chromium:870537 TEST=unit test Change-Id: I459879ce7e75139b483be4b09ff1c65d3df40fc3 Reviewed-on: https://chromium-review.googlesource.com/1180374 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Reviewed-by: smut <smut@google.com> [add] https://crrev.com/b358c944bfe793ba66dcdd88bc41f9b1e1967b2a/go/src/infra/qscheduler/qslib/tutils/tutils.go [add] https://crrev.com/b358c944bfe793ba66dcdd88bc41f9b1e1967b2a/go/src/infra/qscheduler/qslib/tutils/tutils.infra_testing [modify] https://crrev.com/b358c944bfe793ba66dcdd88bc41f9b1e1967b2a/go/src/infra/qscheduler/qslib/priority/prioritize_test.go [modify] https://crrev.com/b358c944bfe793ba66dcdd88bc41f9b1e1967b2a/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/b358c944bfe793ba66dcdd88bc41f9b1e1967b2a/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/b606a292d6da486fac90257fb1c37d4ae8d5b6af commit b606a292d6da486fac90257fb1c37d4ae8d5b6af Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Aug 21 01:43:42 2018 qscheduler: implement workload generators for simulation Workload generators create new task requests on a regular basis. They are configured based on: which account the tasks belong to, how many tasks to create, how often to create them, how long the tasks should run, and how often to move on to a new testing image. BUG= chromium:870537 TEST=unit test Change-Id: I4ab84057a85868cd8df344a842a1bf6d53a3fe4a Reviewed-on: https://chromium-review.googlesource.com/1180421 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: smut <smut@google.com> [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/config.pb.go [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/state.proto [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/generator.go [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/generator_test.go [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/simulation.infra_testing [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/gen.go [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/config.proto [add] https://crrev.com/b606a292d6da486fac90257fb1c37d4ae8d5b6af/go/src/infra/qscheduler/qslib/simulation/state.pb.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/3bc17be6c420dc642a6fea633d8f010db22dd4db commit 3bc17be6c420dc642a6fea633d8f010db22dd4db Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Aug 21 01:53:04 2018 qscheduler: use pretty.Compare instead of DeepEqual in tests where possible pretty.Compare does a good job of showing precisely what fields differ between an actual and expected struct, resulting in much easier to comprehend dumps when a test fails. Also, rename a few tests. BUG= chromium:870537 TEST=Verified that unit tests fail if expectations are modified, and that failures are informative and pinpoint the unexpected diff. Change-Id: I9d8ae39d5097cad873085d6e35ac479bb52ea4dc Reviewed-on: https://chromium-review.googlesource.com/1179068 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: smut <smut@google.com> [modify] https://crrev.com/3bc17be6c420dc642a6fea633d8f010db22dd4db/go/src/infra/qscheduler/qslib/simulation/generator_test.go [modify] https://crrev.com/3bc17be6c420dc642a6fea633d8f010db22dd4db/go/src/infra/qscheduler/qslib/mutaters/mutaters_test.go [modify] https://crrev.com/3bc17be6c420dc642a6fea633d8f010db22dd4db/go/src/infra/qscheduler/qslib/priority/prioritize_test.go [modify] https://crrev.com/3bc17be6c420dc642a6fea633d8f010db22dd4db/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/6d026d39c25b4d705c58485dc5dde4044a7be064 commit 6d026d39c25b4d705c58485dc5dde4044a7be064 Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Aug 21 02:07:57 2018 qscheduler: add a bunch of comments and todo cleanups BUG= chromium:870537 TEST=unit tests Change-Id: Ic73b191567e59a9382eca6476f008e0459011d5b Reviewed-on: https://chromium-review.googlesource.com/1176433 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: smut <smut@google.com> [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/types/account/account_test.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/types/task/task_test.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/types/vector/vector.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/mutaters/mutaters.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/types/vector/vector_test.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/scheduler/helpers.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/priority/prioritize_test.go [modify] https://crrev.com/6d026d39c25b4d705c58485dc5dde4044a7be064/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/5e8381bbf727c7cca1ff83adf1e1584063c5777c commit 5e8381bbf727c7cca1ff83adf1e1584063c5777c Author: Aviv Keshet <akeshet@chromium.org> Date: Fri Aug 24 18:46:18 2018 qscheduler: move mutaters to types/ and rename to mutators BUG= chromium:870537 TEST=unit test Change-Id: Ic9057e3f431648c0f480a3d233c60521828481c9 Reviewed-on: https://chromium-review.googlesource.com/1182290 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> Commit-Queue: Aviv Keshet <akeshet@chromium.org> [rename] https://crrev.com/5e8381bbf727c7cca1ff83adf1e1584063c5777c/go/src/infra/qscheduler/qslib/types/mutators.pb.go [delete] https://crrev.com/10c422253f5fbeddb057804651ae22af00e63b29/go/src/infra/qscheduler/qslib/mutaters/gen.go [rename] https://crrev.com/5e8381bbf727c7cca1ff83adf1e1584063c5777c/go/src/infra/qscheduler/qslib/types/mutators.proto [modify] https://crrev.com/5e8381bbf727c7cca1ff83adf1e1584063c5777c/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [rename] https://crrev.com/5e8381bbf727c7cca1ff83adf1e1584063c5777c/go/src/infra/qscheduler/qslib/types/mutators.go [modify] https://crrev.com/5e8381bbf727c7cca1ff83adf1e1584063c5777c/go/src/infra/qscheduler/qslib/scheduler/scheduler.go [rename] https://crrev.com/5e8381bbf727c7cca1ff83adf1e1584063c5777c/go/src/infra/qscheduler/qslib/types/mutators_test.go [delete] https://crrev.com/10c422253f5fbeddb057804651ae22af00e63b29/go/src/infra/qscheduler/qslib/mutaters/mutaters.infra_testing
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/345f0c99dbabce4ddb7199e349e3c00d54fb8a4b commit 345f0c99dbabce4ddb7199e349e3c00d54fb8a4b Author: Aviv Keshet <akeshet@chromium.org> Date: Fri Aug 24 19:09:06 2018 qscheduler: use types.NewWorker() in one place BUG= chromium:870537 TEST=unit test Change-Id: I9b08f400758e4440b37da6b6bc1eab036e8323cb Reviewed-on: https://chromium-review.googlesource.com/1180250 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: smut <smut@google.com> [modify] https://crrev.com/345f0c99dbabce4ddb7199e349e3c00d54fb8a4b/go/src/infra/qscheduler/qslib/types/mutators_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/22867afd66f31ec75dd1b7249ffafaf496384eed commit 22867afd66f31ec75dd1b7249ffafaf496384eed Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Aug 28 01:23:42 2018 quotascheduler: encapsulate scheduler logic This change moves the bulk of scheduler logic (which was formerly spread across several packages) into a single package, scheduler, and defines an exported type Scheduler with methods to interact with the quotascheduler algorithm. Details about the implementation of the algorithm are no longer exported. This CL does not introduce any behavioral changes. A few new methods that Scheduler will implement are not yet defined, because it is easier to first accomplish non-behavior-changing refactor in this CL, and behavior changes in a separate follow-up. BUG= chromium:870537 TEST=unit tests Change-Id: I7017b6f287855006a9134aa0c4a159adb8684baa Reviewed-on: https://chromium-review.googlesource.com/1189426 Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/prioritize_test.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/types.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/scheduler.proto [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/gen.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/mutators.proto [modify] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/types/vector/vector.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/prioritize.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/scheduler.pb.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/worker.go [modify] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/helpers.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/mutators.go [delete] https://crrev.com/c9cf39aefd043f6c2510d896e3220efe977e2313/go/src/infra/qscheduler/qslib/types/types.infra_testing [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/mutators.pb.go [rename] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/mutators_test.go [modify] https://crrev.com/22867afd66f31ec75dd1b7249ffafaf496384eed/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/6f12ac656c7f827edbd9a4e1829421bd0f01a644 commit 6f12ac656c7f827edbd9a4e1829421bd0f01a644 Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Aug 29 00:07:10 2018 qscheduler: unify mutaters into a single proto The Mutater interface is unnecessary; all that callers need is a type that describes tasks being assigned to workers. Create a new proto for this (Assignment) and drop unnecessary Mutator abstraction. Internally, we still define an apply method for Assignment, but this is mainly for convenience, and we do not export it. BUG= chromium:870537 TEST=unit tests Change-Id: I76025d1057c5be339a5584801d2ffa335ff8b36a Reviewed-on: https://chromium-review.googlesource.com/1189089 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org> [add] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/assignment.pb.go [rename] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/assignment_test.go [modify] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/types.go [add] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/assignment.go [delete] https://crrev.com/86eb5fbc086326f927ea49df57e7bd48d0d08a40/go/src/infra/qscheduler/qslib/scheduler/mutators.proto [add] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/assignment.proto [modify] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [delete] https://crrev.com/86eb5fbc086326f927ea49df57e7bd48d0d08a40/go/src/infra/qscheduler/qslib/scheduler/mutators.go [delete] https://crrev.com/86eb5fbc086326f927ea49df57e7bd48d0d08a40/go/src/infra/qscheduler/qslib/scheduler/mutators.pb.go [modify] https://crrev.com/6f12ac656c7f827edbd9a4e1829421bd0f01a644/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/f6c2c9665af02054c9e4e84749a36ccb081fe3a0 commit f6c2c9665af02054c9e4e84749a36ccb081fe3a0 Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Oct 03 19:17:39 2018 qscheduler: initial implementation of reconciler interface This CL puts down the scaffolding for the reconciler API and the initial implementation of its Reap method. BUG= chromium:870537 TEST=unit tests Change-Id: I9842dfc1eb55e78435135bec76a81e395457db8d Reviewed-on: https://chromium-review.googlesource.com/c/1194900 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Reviewed-by: smut <smut@google.com> Cr-Commit-Position: refs/heads/master@{#18033} [modify] https://crrev.com/f6c2c9665af02054c9e4e84749a36ccb081fe3a0/go/src/infra/qscheduler/qslib/scheduler/assignment.pb.go [add] https://crrev.com/f6c2c9665af02054c9e4e84749a36ccb081fe3a0/go/src/infra/qscheduler/qslib/reconciler/reconciler.infra_testing [modify] https://crrev.com/f6c2c9665af02054c9e4e84749a36ccb081fe3a0/go/src/infra/qscheduler/qslib/scheduler/assignment.proto [add] https://crrev.com/f6c2c9665af02054c9e4e84749a36ccb081fe3a0/go/src/infra/qscheduler/qslib/reconciler/reconciler_test.go [add] https://crrev.com/f6c2c9665af02054c9e4e84749a36ccb081fe3a0/go/src/infra/qscheduler/qslib/reconciler/reconciler.go [modify] https://crrev.com/f6c2c9665af02054c9e4e84749a36ccb081fe3a0/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/1a6936d929ea605aebf57a9d374908e350263c34 commit 1a6936d929ea605aebf57a9d374908e350263c34 Author: Aviv Keshet <akeshet@chromium.org> Date: Fri Oct 05 17:38:54 2018 qscheduler: implement Notify(NEW) and AddRequest in scheduler and reconciler This CL provides the scaffolding for the Notify endpoint of reconciler, and implements its behavior for adding new tasks to the queue. BUG= chromium:870537 TEST=Unit test added. Change-Id: I403caedc2f945eed9bd8d93dde32b62581957865 Reviewed-on: https://chromium-review.googlesource.com/c/1258584 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: smut <smut@google.com> Reviewed-by: Allen Li <ayatane@chromium.org> Cr-Commit-Position: refs/heads/master@{#18105} [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/scheduler/assignment.pb.go [add] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/reconciler/reconciler.pb.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/scheduler/assignment_test.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/scheduler/assignment.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/scheduler/assignment.proto [add] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/reconciler/reconciler.proto [add] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/reconciler/test_helpers.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/reconciler/reconciler_test.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/reconciler/reconciler.go [add] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/reconciler/gen.go [modify] https://crrev.com/1a6936d929ea605aebf57a9d374908e350263c34/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/f127402f56909a5db82d7cbfa86fc95edd489c8f commit f127402f56909a5db82d7cbfa86fc95edd489c8f Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Oct 16 21:15:40 2018 qscheduler: add a running request cache to State; reimplement Assignment This CL adds a cache for efficient lookup of requestID -> workerID for running requests, as well as methods for adding and removing running requests that correctly update this cache. This CL also moves the Assignment mutation methods, and reimplements it. This location is a better conceptual fit, and also allows them to use these internal cache-updating methods without needing access private methods from a different struct. BUG= chromium:870537 TEST=Unit tests Change-Id: I576bfe119de1107ab0beea91c99dbbcf48434810 Reviewed-on: https://chromium-review.googlesource.com/c/1279109 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Cr-Commit-Position: refs/heads/master@{#18370} [modify] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/assignment.pb.go [modify] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/state.proto [modify] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/types/vector/vector.go [delete] https://crrev.com/f2336c6c81a00554f7db114cb7f6591d8aa6241c/go/src/infra/qscheduler/qslib/scheduler/assignment.go [rename] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/state_test.go [modify] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/state.pb.go [add] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/state.go [modify] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/assignment.proto [modify] https://crrev.com/f127402f56909a5db82d7cbfa86fc95edd489c8f/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/484e8a166b860fcbf9a55ded8e80fe43c4b8874f commit 484e8a166b860fcbf9a55ded8e80fe43c4b8874f Author: Aviv Keshet <akeshet@chromium.org> Date: Thu Oct 18 00:59:48 2018 qscheduler: add and implement a NotifyRequest call to scheduler The NotifyRequest call to the scheduler is used to ensure that scheduler state is consistent with a request's state at a given time. This is a way to heal any inconsistencies that may result from dropped pubsub or lost Assignments between swarming and quotascheduler. This CL adds the concept of ConfirmedTime to Request and ConfirmedIdleTime to Worker, which represents the most recent time at which the state of that record was updated from the authoritative source (swarming). BUG= chromium:870537 TEST=Unit test added Change-Id: Ieb1a177ca1f868bd84450e2150603bde168f1baf Reviewed-on: https://chromium-review.googlesource.com/c/1262930 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Allen Li <ayatane@chromium.org> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org> Cr-Commit-Position: refs/heads/master@{#18421} [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/task.pb.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/worker.pb.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/state_test.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/state.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/task.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/worker.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/reconciler/test_helpers.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/worker.proto [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/reconciler/reconciler_test.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/reconciler/reconciler.go [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/task.proto [modify] https://crrev.com/484e8a166b860fcbf9a55ded8e80fe43c4b8874f/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/a3111605f5c17d3d7e8e5069e2c49a1deb198474 commit a3111605f5c17d3d7e8e5069e2c49a1deb198474 Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Oct 23 21:41:17 2018 qscheduler: add arguments to account.NewConfig BUG= chromium:870537 TEST=None Change-Id: I62889a8830ad8d78d19aa2bc34fe5d741168bed1 Reviewed-on: https://chromium-review.googlesource.com/c/1292763 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#18548} [modify] https://crrev.com/a3111605f5c17d3d7e8e5069e2c49a1deb198474/go/src/infra/qscheduler/qslib/scheduler/prioritize_test.go [modify] https://crrev.com/a3111605f5c17d3d7e8e5069e2c49a1deb198474/go/src/infra/qscheduler/qslib/types/account/account.go [modify] https://crrev.com/a3111605f5c17d3d7e8e5069e2c49a1deb198474/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/e22433b96085d07ffd78fe3352012ea5f2de3210 commit e22433b96085d07ffd78fe3352012ea5f2de3210 Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Oct 23 22:02:42 2018 qscheduler: add time as argument to Scheduler and State constructor BUG= chromium:870537 TEST=None Change-Id: If21a413f0e7939b1ef8b3d98adb6922d72ea79c3 Reviewed-on: https://chromium-review.googlesource.com/c/1292764 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#18549} [modify] https://crrev.com/e22433b96085d07ffd78fe3352012ea5f2de3210/go/src/infra/qscheduler/qslib/scheduler/state_test.go [modify] https://crrev.com/e22433b96085d07ffd78fe3352012ea5f2de3210/go/src/infra/qscheduler/qslib/scheduler/prioritize_test.go [modify] https://crrev.com/e22433b96085d07ffd78fe3352012ea5f2de3210/go/src/infra/qscheduler/qslib/scheduler/types.go [modify] https://crrev.com/e22433b96085d07ffd78fe3352012ea5f2de3210/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/e22433b96085d07ffd78fe3352012ea5f2de3210/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/c6679057418bdb6cd4a54b2e81d380c81f90a9da commit c6679057418bdb6cd4a54b2e81d380c81f90a9da Author: Aviv Keshet <akeshet@chromium.org> Date: Tue Oct 23 22:18:41 2018 qscheduler: add a NewRequest constructor This constructor makes tests and example code more readable. BUG= chromium:870537 TEST=None Change-Id: Iecf8efc33623a10b0dca1fa58f484f4e788eb447 Reviewed-on: https://chromium-review.googlesource.com/c/1292765 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#18550} [modify] https://crrev.com/c6679057418bdb6cd4a54b2e81d380c81f90a9da/go/src/infra/qscheduler/qslib/scheduler/state_test.go [modify] https://crrev.com/c6679057418bdb6cd4a54b2e81d380c81f90a9da/go/src/infra/qscheduler/qslib/scheduler/prioritize_test.go [modify] https://crrev.com/c6679057418bdb6cd4a54b2e81d380c81f90a9da/go/src/infra/qscheduler/qslib/scheduler/task.go [modify] https://crrev.com/c6679057418bdb6cd4a54b2e81d380c81f90a9da/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/89cfbf3446f778f4838871bea4b364d3a932a682 commit 89cfbf3446f778f4838871bea4b364d3a932a682 Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Oct 24 04:32:13 2018 qscheduler: treat same-instant NotifyRequest as forward-in-time Same-instance NotifyRequest calls are unlikely to occur in the wild, but they do occasionally occur when calling time.Now(). This has been observed to cause flakiness in the forthcoming doctest in https://chromium-review.googlesource.com/c/infra/infra/+/1292766 Treat same-instant notifications as though they were forward in time, to eliminate the flakiness of that test. BUG= chromium:870537 TEST=New unit tests added, verified to fail prior to fix. Change-Id: Ia9f5abbf223f6fd344d2e3d561555194e6bf1ae1 Reviewed-on: https://chromium-review.googlesource.com/c/1297522 Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Commit-Queue: Aviv Keshet <akeshet@chromium.org> Cr-Commit-Position: refs/heads/master@{#18553} [modify] https://crrev.com/89cfbf3446f778f4838871bea4b364d3a932a682/go/src/infra/qscheduler/qslib/scheduler/state_test.go [modify] https://crrev.com/89cfbf3446f778f4838871bea4b364d3a932a682/go/src/infra/qscheduler/qslib/scheduler/state.go [modify] https://crrev.com/89cfbf3446f778f4838871bea4b364d3a932a682/go/src/infra/qscheduler/qslib/scheduler/worker.go [modify] https://crrev.com/89cfbf3446f778f4838871bea4b364d3a932a682/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/9eb84a2ee99533a19e54f28a05e493430db020a3 commit 9eb84a2ee99533a19e54f28a05e493430db020a3 Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Oct 24 04:50:34 2018 qscheduler: improve scheduler package documentation Add a doctest with example usage for the scheduler package. BUG= chromium:870537 TEST=go doc -ex .; go test (with and without correct output) Change-Id: Ief440dfcd76cd0396baf7f115b99de9df5b9aca5 Reviewed-on: https://chromium-review.googlesource.com/c/1292766 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#18554} [add] https://crrev.com/9eb84a2ee99533a19e54f28a05e493430db020a3/go/src/infra/qscheduler/qslib/scheduler/doc_test.go [modify] https://crrev.com/9eb84a2ee99533a19e54f28a05e493430db020a3/go/src/infra/qscheduler/qslib/scheduler/task.go [modify] https://crrev.com/9eb84a2ee99533a19e54f28a05e493430db020a3/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/9eb84a2ee99533a19e54f28a05e493430db020a3/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/5ffe28ec9f7c2036749594ba3f2545971b371816 commit 5ffe28ec9f7c2036749594ba3f2545971b371816 Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Oct 24 07:44:16 2018 qscheduler: include timestamp in preemption assignments BUG= chromium:870537 TEST=fixed test case for preemption, new test passes. Change-Id: I27fdcc012e6758546b2dae8ce034d60792bbf20b Reviewed-on: https://chromium-review.googlesource.com/c/1294990 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#18555} [modify] https://crrev.com/5ffe28ec9f7c2036749594ba3f2545971b371816/go/src/infra/qscheduler/qslib/scheduler/scheduler_test.go [modify] https://crrev.com/5ffe28ec9f7c2036749594ba3f2545971b371816/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The following revision refers to this bug: https://chromium.googlesource.com/infra/infra/+/021706198d6dfe814b747517674a386bc48fa1ce commit 021706198d6dfe814b747517674a386bc48fa1ce Author: Aviv Keshet <akeshet@chromium.org> Date: Wed Oct 24 19:19:27 2018 qscheduler: overhaul reconciler.Notify and related tests/documentation - Integrate reconciler.Notify properly with scheduler.NotifyRequest, to allow it to resolve inconsistent or out-of-order notification messages. - Add proper unit tests for reconciler. - Add a proper doc_test / usage example. BUG= chromium:870537 TEST=New unittests added Change-Id: I9ef1e145c6f9ae3f33c50afe8da016de842275a1 Reviewed-on: https://chromium-review.googlesource.com/c/1294770 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org> Cr-Commit-Position: refs/heads/master@{#18562} [modify] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/reconciler/reconciler.pb.go [modify] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/scheduler/state.proto [add] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/reconciler/doc_test.go [modify] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/reconciler/reconciler.proto [delete] https://crrev.com/58fcf8a2adf2e79c90a10f750ea59d189e827bcc/go/src/infra/qscheduler/qslib/reconciler/test_helpers.go [modify] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/reconciler/reconciler_test.go [modify] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/reconciler/reconciler.go [modify] https://crrev.com/021706198d6dfe814b747517674a386bc48fa1ce/go/src/infra/qscheduler/qslib/scheduler/scheduler.go
The initial logic is in place. Now moving on the appengine-ification, and plugging in to swarming. Closing this bug.
Comment 1 by akes...@chromium.org
, Aug 3