derat@, is there any plan to do this in the near future? (If not, do you need help with this?)

I'm working on writing Tast tests for the new Demo Mode, and it's heavily related to enterprise enrollment. Should I start with migrating existing classes such as https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/client/cros/enterprise/enterprise_policy_base.py?type=cs&g=0 to Tast? Thanks!

Thanks for asking!

Our focus in Q4 is on rewriting Autotest tests that run as part of the Chrome OS CQ and PFQs, since that feels like the area that can benefit the most from faster, more-reliable, easier-to-debug tests. With that said, I also want to support new tests when it's possible.

Go conventions are pretty particular, so porting Autotest libraries to Tast is a fairly-involved process (both for code authors and for reviewers). We're also trying to avoid adding library code until it's needed by multiple tests, as it can be hard to design an interface before its use patterns are clear. If you're planning to start adding libraries (or even tests), I'd recommend trying to get feedback as early as possible, particularly if you haven't written much Go code in the past (sorry, I'm not sure if you have).

Is there a bug or document with more detail about the tests that you're working on? If you can point at any existing Autotest tests that have similar requirements to what your Demo Mode tests will need to do, that would be very helpful for getting an idea about how much work is needed.

I suspect that for this case, since we should probably have coverage soonish, we should probably just do the autotest. We should probably introduce a more basic enrollment tast test before looking into the more esoteric use cases like demo mode.

Thanks for the replies!

Re #4, briefly the things we want to test are:

1) Specific OOBE screens are shown during demo mode setup. It should be similar to the test in |chrome.go|, but show the demo mode setup pages instead of directly jumping to user login. (I think this part can be achieved with browser tests as well.)

2) The enrollment to the demo mode domain. We can add demo-mode specific tests to  existing autotest instead of Tast as achuith@ suggested in #5.

3) After demo session starts, certain Chrome apps and Android apps are installed and launched. 

IIUC, having auto tests or Tast tests for demo mode is for its ability to test end-to-end. +rkc@, michaelog@, if we could test the above three parts separately without autotests, do we still want to add them given the complexity outlined in #4?

I think our strategy has been to have a bunch of browser tests that provide code coverage and test the minutiae of individual features, and one autotest that runs through the whole flow end to end.

I agree with Achuith.

Use unit tests and browser tests instead of integration tests whenever possible -- see https://chromium.googlesource.com/chromiumos/docs/+/master/unit_tests.md#Prefer-unit-tests-to-functional-tests for some of the reasoning here. Only use integration tests (e.g. Autotest/Tast) to test interactions between different components.

And if you absolutely need to write a complicated enterprise integration test quickly, using Autotest is likely the best bet at the moment.

derat@, achuith@, thanks for the explanation!

I have an additional question: for the enterprise-related auto tests, e.g. [1]. Is there a way to see the args used to run the test?

The arg will determine whether the enrollment is real or fake ([2]). If real enrollment is still being tested, then there's no need for additional auto tests for demo mode because it's essentially just an enrollment.

Thanks!

[1] http://cautotest-prod/afe/#tab_id=view_job&object_id=262454739

[2] https://cs.corp.google.com/chromeos_public/src/third_party/autotest/files/client/cros/enterprise/enterprise_policy_base.py?type=cs&q=%22EnterprisePolicyTest%22+enterprise_policy_base&g=0&l=85

#9: I'm not an Autotest expert, but in the case of policy_URLWhitelist.notset_blocked, I'd think you'd want to look for a control.notset_blocked file in the autotest repository.

(If you have other Autotest questions, chromeos-infra-discuss@ is probably the right place to ask them.)