ML Service: Add a real integration test |
|||||
Issue descriptionML Service has one integration autotest that checks the process is running. We should upgrade to one that actually gets a Mojo connection from it via D-Bus, to prove it's really alive.
,
Jul 11
This is not easy and probably needs to wait until we have a (real or test) client of ML service that we can drive in autotest.
,
Jul 16
Per rkc, there are several different ideas we can pursue here to test the daemon<->chromium connection: * Use a browsertest. Currently these are not being run by autotest as far as we know, but we could work to make one launched in an autotest environment. * Make a chrome://ml-service-internals page with a button called "Run tests" and hook into all the tests there. We can make an autotest that clicks the button. * We could test the daemon side of the bootstrap by making a ml_service_driver.cc or such that does the minimal steps to run Mojo have it send the Mojo invitation via D-Bus just like Chrome does.
,
Jul 16
,
Aug 31
,
Sep 5
For reference here are a couple of -internals/ pages that have UIs to do interesting test-like things: chrome://download-internals/ chrome://usb-internals/
,
Sep 17
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/d88bd7951b846a3adcffee52e1aa8a07ca1e785b commit d88bd7951b846a3adcffee52e1aa8a07ca1e785b Author: Michael Martis <martis@chromium.org> Date: Mon Sep 17 03:13:03 2018 chromeos-base/ml: Download and deploy simpler test model. This CL fetches and installs the simpler test model that is introduced in the CQ-DEPEND CL below. Since this model is now small enough to be deployed on all devices, we remove the notion of test / non-test models from the ebuild. BUG= chromium:836102 TEST=`ebuild-chell ml-9999.ebuild test` CQ-DEPEND=CL:1226750 Change-Id: I40812ff57057b5c5cd1614a509c7441910e58d2f Reviewed-on: https://chromium-review.googlesource.com/1226770 Commit-Ready: Michael Martis <martis@chromium.org> Tested-by: Michael Martis <martis@chromium.org> Reviewed-by: Andrew Moylan <amoylan@chromium.org> [modify] https://crrev.com/d88bd7951b846a3adcffee52e1aa8a07ca1e785b/chromeos-base/ml/ml-9999.ebuild [modify] https://crrev.com/d88bd7951b846a3adcffee52e1aa8a07ca1e785b/chromeos-base/ml/Manifest
,
Sep 17
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform2/+/c22823d41e6427df18e303521dfffc2a098ccad4 commit c22823d41e6427df18e303521dfffc2a098ccad4 Author: Michael Martis <martis@chromium.org> Date: Mon Sep 17 03:13:02 2018 ml: Replace test model with a much simpler one. This CL removes the large (30KB) tab discarder test model and replaces it with an existing, smaller (0.5KB) model that adds two scalars together. This lets us deploy the test model on all (i.e. non-test) images, simplifying our test setup. To do this, we remove the smaller model from the source tree and instead serve it from the distfiles mirror (see the CQ-DEPEND CL below). BUG= chromium:836102 TEST=`ebuild-chell ml-9999.ebuild test` CQ-DEPEND=CL:1226770 Change-Id: I985f5e74a2427d7f936dad50fde1f34acf4c272d Reviewed-on: https://chromium-review.googlesource.com/1226750 Commit-Ready: Michael Martis <martis@chromium.org> Tested-by: Michael Martis <martis@chromium.org> Reviewed-by: Andrew Moylan <amoylan@chromium.org> [add] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/test_utils.cc [modify] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/mojom/model.mojom [modify] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/model_impl_test.cc [modify] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/model_metadata.cc [modify] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/test_utils.h [modify] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/machine_learning_service_impl_test.cc [delete] https://crrev.com/dea98c10047db8950a9a7de122ddd03b6e3ff83e/ml/testdata/add.tflite [modify] https://crrev.com/c22823d41e6427df18e303521dfffc2a098ccad4/ml/ml.gyp
,
Sep 21
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/d3542c510099d4dfc42938a6f35ae18be5a45def commit d3542c510099d4dfc42938a6f35ae18be5a45def Author: Andrew Moylan <amoylan@chromium.org> Date: Fri Sep 21 03:26:19 2018 Add autotestPrivate.bootstrapMachineLearningService This function makes a basic call using the ML Service client library. Doing this triggers the ML Service daemon to be started (via D-Bus service activation) and the D-Bus -> Mojo IPC bootstrap to occur. The VoidCallback is invoked when the basic call (LoadModel on the test model) completes successfully. Tested: - Ran it with a Chrome OS Tast test (testing successful bootstrap) - Added a test to the test.js covering the case of failure to connect to ML Service Bug: 836102 Change-Id: I509eab4a518a933109095eec1ece811bc30e4f99 Reviewed-on: https://chromium-review.googlesource.com/1229878 Reviewed-by: Ben Wells <benwells@chromium.org> Reviewed-by: Achuith Bhandarkar <achuith@chromium.org> Reviewed-by: Joel Hockey <joelhockey@chromium.org> Commit-Queue: Andrew Moylan <amoylan@chromium.org> Cr-Commit-Position: refs/heads/master@{#593066} [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/chrome/browser/extensions/BUILD.gn [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/chrome/browser/extensions/api/autotest_private/autotest_private_api.cc [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/chrome/browser/extensions/api/autotest_private/autotest_private_api.h [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/chrome/common/extensions/api/autotest_private.idl [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/chrome/test/data/extensions/api_test/autotest_private/test.js [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/extensions/browser/extension_function_histogram_value.h [modify] https://crrev.com/d3542c510099d4dfc42938a6f35ae18be5a45def/tools/metrics/histograms/enums.xml
,
Sep 21
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/80342d60d43e6e686d167942caacb5b1736e5c3e commit 80342d60d43e6e686d167942caacb5b1736e5c3e Author: Andrew Moylan <amoylan@chromium.org> Date: Fri Sep 21 07:51:12 2018 tast-use-flags: Add ml_service to IUSE BUG= chromium:836102 TEST=none Change-Id: I8403fe9594310437a685262141f5e30bcd0b9517 Reviewed-on: https://chromium-review.googlesource.com/1237154 Commit-Ready: Andrew Moylan <amoylan@chromium.org> Tested-by: Andrew Moylan <amoylan@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [rename] https://crrev.com/80342d60d43e6e686d167942caacb5b1736e5c3e/chromeos-base/tast-use-flags/tast-use-flags-0.0.1-r7.ebuild [modify] https://crrev.com/80342d60d43e6e686d167942caacb5b1736e5c3e/chromeos-base/tast-use-flags/tast-use-flags-0.0.1.ebuild
,
Sep 21
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast/+/e739f2ed3d038b79b5f7e3988ede3329588bb1f4 commit e739f2ed3d038b79b5f7e3988ede3329588bb1f4 Author: Andrew Moylan <amoylan@chromium.org> Date: Fri Sep 21 07:51:13 2018 tast: Add ml_service software dep Add a "ml_service" software dep that corresponds to the "ml_service" USE flag. Not all boards will have ML Service. BUG= chromium:836102 TEST=none CQ-DEPEND=CL:1237154 Change-Id: Ifc2e6487fc2d0cb18f8ce53b1e6a71c29f42c106 Reviewed-on: https://chromium-review.googlesource.com/1237015 Commit-Ready: Andrew Moylan <amoylan@chromium.org> Tested-by: Andrew Moylan <amoylan@chromium.org> Reviewed-by: Andrew Moylan <amoylan@chromium.org> [modify] https://crrev.com/e739f2ed3d038b79b5f7e3988ede3329588bb1f4/src/chromiumos/cmd/local_test_runner/main.go [modify] https://crrev.com/e739f2ed3d038b79b5f7e3988ede3329588bb1f4/docs/test_dependencies.md
,
Sep 24
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/38257112b62f4b8c7a0f40caffb0aee412b0a25a commit 38257112b62f4b8c7a0f40caffb0aee412b0a25a Author: Andrew Moylan <amoylan@chromium.org> Date: Mon Sep 24 03:29:43 2018 autotest-tests-ml: Remove ML Service autotest This wasn't testing much, no longer passes now that ML Service starts on-demand instead of at startup, and is superceded by the Mojo bootstrap tast test platform.MlServiceBootstrap added in crrev.com/c/1229816. TEST=emerge-coral autotest-all BUG= chromium:836102 Change-Id: I1637d14e2ed24f06c27be4fa38212961093891e4 Reviewed-on: https://chromium-review.googlesource.com/1234074 Commit-Ready: Andrew Moylan <amoylan@chromium.org> Tested-by: Andrew Moylan <amoylan@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [delete] https://crrev.com/ee27220a38040b0bf85a4b8f2217b7685fcf2093/chromeos-base/autotest-tests-ml/autotest-tests-ml-9999.ebuild [rename] https://crrev.com/38257112b62f4b8c7a0f40caffb0aee412b0a25a/chromeos-base/autotest-all/autotest-all-0.0.1-r47.ebuild [modify] https://crrev.com/38257112b62f4b8c7a0f40caffb0aee412b0a25a/chromeos-base/autotest-all/autotest-all-0.0.1.ebuild [delete] https://crrev.com/ee27220a38040b0bf85a4b8f2217b7685fcf2093/chromeos-base/autotest-tests-ml/autotest-tests-ml-0.0.1-r2.ebuild
,
Sep 24
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/autotest/+/d2f706c8654ec6ef6c3221a719a5341965aa044a commit d2f706c8654ec6ef6c3221a719a5341965aa044a Author: Andrew Moylan <amoylan@chromium.org> Date: Mon Sep 24 13:49:55 2018 ml: Remove ML Service autotest This wasn't testing much, no longer passes now that ML Service starts on-demand instead of at startup, and is superceded by the Mojo bootstrap tast test platform.MlServiceBootstrap added in crrev.com/c/1229816. TEST=no CQ-DEPEND=CL:1234074 BUG= chromium:836102 Change-Id: I67d23a674ce82bedc7a90ccb870d45174023df0b Reviewed-on: https://chromium-review.googlesource.com/1234097 Commit-Ready: Andrew Moylan <amoylan@chromium.org> Tested-by: Andrew Moylan <amoylan@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> [modify] https://crrev.com/d2f706c8654ec6ef6c3221a719a5341965aa044a/site_utils/attribute_whitelist.txt [delete] https://crrev.com/7342ab73f254af331b02323dd65aaf72e9ce95e5/client/site_tests/ml_CheckMlProcesses/control [delete] https://crrev.com/7342ab73f254af331b02323dd65aaf72e9ce95e5/test_suites/control.ml_service
,
Sep 28
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/851be311649cc8b7a6bfe362f3c9d74fa32b6e57 commit 851be311649cc8b7a6bfe362f3c9d74fa32b6e57 Author: Andrew Moylan <amoylan@chromium.org> Date: Fri Sep 28 02:45:06 2018 tast-tests: Test Chrome->ML Service IPC bootstrap This test calls autotestPrivate.bootstrapMachineLearningService introduced in crrev.com/c/1229878 to make a simple call from Chrome to ML Service, which triggers the Mojo IPC bootstrap between the two processes. TEST=tast -verbose -checktestdeps always run $IP platform.MLServiceBootstrap BUG= chromium:836102 CQ-DEPEND=CL:1237015 Change-Id: I66e827560e85716ab23cda326de64a06e727a2ce Reviewed-on: https://chromium-review.googlesource.com/1229816 Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com> Tested-by: Andrew Moylan <amoylan@chromium.org> Reviewed-by: Dan Erat <derat@chromium.org> Reviewed-by: Andrew Moylan <amoylan@chromium.org> [add] https://crrev.com/851be311649cc8b7a6bfe362f3c9d74fa32b6e57/src/chromiumos/tast/local/bundles/cros/platform/ml_service_bootstrap.go
,
Oct 5
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by bugdroid1@chromium.org
, Jun 19 2018