Test survey for Material Design Settings |
||||||||||||||||||||||
Issue description== Test Planning: > Does the feature needs manual testing? Yes, but it's already been going through a bunch for a year. > Feature implementation status (what is currently working and not working/known issues, all the code landed in canary, ready for testing etc.) Mostly done, hoping to ship in M59. Stuff still changing actively, though. > Link to Design Doc: https://www.chromium.org/developers/updating-webui-for-material-design/settings-material-design > Link to Test Plan: ( High level testcases,specific setup,flags etc) none > Are there any special points of interest from a testing perspective? Visual polish, extension and policy overrides. > Link to demos (if any): none > Targeted Platforms. All desktop platforms. > Is the work flow same as desktop and Mobile? (Answer if applicable to both Desktop and Mobile) N/A == Automation: > PS: Many new features get added to manual testing list every week; in order to maintain the same release frequency and accommodate new features testing, the test team may not give the same level of testing priority to your feature after it reaches Stable. So adding as much automated test coverage as possible is encouraged before you ship your feature to Stable. > What's automated? (Unit tests, API tests, Browser tests) Please provide links to automated tests. We have a bunch of unit, api, and browser tests. https://cs.chromium.org/chromium/src/chrome/test/data/webui/cr_elements/ https://cs.chromium.org/chromium/src/chrome/test/data/webui/settings/ https://cs.chromium.org/chromium/src/chrome/browser/ui/webui/settings/ https://cs.chromium.org/chromium/src/chrome/browser/extensions/api/settings_private/settings_private_apitest.cc?dr=CSs > What’s not automated that needs manual testing? Visual polish, translations. > List reasons for still needing manual testing. It's hard to test translations / visual polish in an automated way. == Feature-Component-Ownership Details: > What is the crbug label/component for the feature? Component: UI>Settings Labels: Proj-MaterialDesign-WebUI go/crsettings-bug, bit.ly/md-settings-bug > Are there any other team members or mailing lists that should be notified if issues are found? tbuckley == Stability: > Is there a mechanism to turn off the feature if it breaks product functionality or performance? Finch > List UMA stats and coverage to measure stability Settings.LoadCompletedTime Settings.LoadCompletedTime.MD Settings.LoadDocumentTime Settings.LoadDocumentTime.MD Settings.TimeUntilInteractive > It’s highly recommend always adding Stability.Counts and CrashExitCodes.* to your json > What are you finch studies and what % rollout are you requesting for each group MaterialDesignSettings, 50/50 for canary and dev so far. Possibly small % to beta in M58. > How are you monitoring the stability of the experiment groups 10 engineers testing daily (bunch more users and QA), 50/50 split in finch, automated crash reporting == Performance: > Any performance impact? Did the feature get tested on perf waterfall? Yes, but extensive work has been done. We run automated perf through a new perfomance team tool named "Lighthouse"[1] as well as have automated performance reporting from remote users via UMA.
,
Mar 13 2017
,
Mar 29 2017
+dchan for Chrome OS testing @Danny, do you know who should also look at this from the Chrome side?
,
Mar 29 2017
Is this a launch feature for chorme OS ?
,
Mar 30 2017
dchan@: it is not a chromeos-only feature, but the UI is seen on ChromeOS (as well as Windows, Linux, Mac).
,
Mar 30 2017
,
Mar 30 2017
+geetha, can you help out dbeam@
,
Mar 30 2017
@dbeam, India team will test the feature soon and update the status here in the bug. We will test the Settings page by enabling the flag "Enable Material Design settings" in chrome://flags.
,
Mar 30 2017
,
Mar 31 2017
Tested the feature MD-Settings on CROS Daisy-9416.0.0/59.0.3055.0 and Windows-7, Mac-10.12.3 & Linux Ubuntu-14.04 using Chrome canary 59.0.3057.0 with below steps 1.Opened Chrome and Enabled Material Design settings in chrome://flags. 2.Navigated chrome//settings. 3.Tested the UI for settings page, Chrome page ,Extensions and Bookmarks are displayed with out any issues on normal and Hidpi Machines. 4.UI displayed fine even after changing the language and for pop up bubbles. Not observed any major issues.Please find the attached screen shots for the tested features. Thanks.
,
Apr 3 2017
,
Apr 4 2017
sooo, can we call this fixed?
,
Apr 4 2017
,
Apr 5 2017
dbeam@ Could you please let us know any other steps to be performed from TE end. Please confirm to proceed further. Thanks in advance!!
,
Apr 5 2017
i can't think of anything else. every section, subpage, and horizontal feature is checked by SWEs, test engineers, automated tests, and users all the time.
,
Apr 6 2017
,
Apr 13 2017
I tested today on Kevin / 9455.0.0 / 59.0.3068.1 I setup three user accounts but none of them had MD settings by default. So I needed to enable the flag. Is there still a finch experiment? Here is what we found: https://bugs.chromium.org/p/chromium/issues/detail?id=710706 https://bugs.chromium.org/p/chromium/issues/detail?id=711126 https://bugs.chromium.org/p/chromium/issues/detail?id=711201 https://bugs.chromium.org/p/chromium/issues/detail?id=711202 https://bugs.chromium.org/p/chromium/issues/detail?id=711206 https://bugs.chromium.org/p/chromium/issues/detail?id=711208 https://bugs.chromium.org/p/chromium/issues/detail?id=711209 https://bugs.chromium.org/p/chromium/issues/detail?id=711210 https://bugs.chromium.org/p/chromium/issues/detail?id=711211 https://bugs.chromium.org/p/chromium/issues/detail?id=711212 https://bugs.chromium.org/p/chromium/issues/detail?id=711213 https://bugs.chromium.org/p/chromium/issues/detail?id=711214 https://bugs.chromium.org/p/chromium/issues/detail?id=711129 https://bugs.chromium.org/p/chromium/issues/detail?id=711130
,
Apr 13 2017
There is still a Finch experiment, it's enabled for 50% of dev/canary and was just enabled for 100% of Googlers. We will be going to 100% with beta/stable, but maintain the Finch config so we can easily disable it if needed.
,
Apr 13 2017
|
||||||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||||||
Comment 1 by dbeam@chromium.org
, Mar 13 2017