Test Survey for Photo Picker |
|||||
Issue description* Test Planning * 1. Does the feature need manual testing? Yes 2. Feature implementation status (what is currently working and not working/known issues, all the code landed in canary, ready for testing etc.) The code has landed in Canary and is ready for testing. 3. Link to Design Doc: https://docs.google.com/document/d/1oenFck8kfcWmrwBUCFv1Wi7-cJxEmlxbIx3jpjyZj3o/ 4. Link to Test Plan: (High level test cases, specific setup, flags etc) https://docs.google.com/document/d/1l9fljkcat4YwN60qMppWzgiU8H3U8Ic-sJ2Grmy38iA/ 5. Are there any special points of interest from a testing perspective? See 10). 6. Link to demos (if any): Probably easiest just to enable the feature and test it by searching for 'Photo' in about:flags and enable it, then go to a website that accepts photos and upload one. 7. Targeted Platforms. Android only. 8. Is the work flow same as desktop and Mobile? (Answer if applicable to both Desktop and Mobile) Not applicable. * Automation * 9. What's automated? (Unit tests, API tests, Browser tests) Please provide links to automated tests. The basic functionality is automated: Creating the dialog, selecting nothing or selecting a few photos, validating correct selection on close. Both single-selection and multi-selection is tested. https://cs.chromium.org/chromium/src/chrome/android/javatests/src/org/chromium/chrome/browser/photo_picker/PhotoPickerDialogTest.java?q=photopickerdialogtest+package:%5Echromium$&l=1 There are tests also regarding when to show the dialog (it is a Photo Picker, so no reason to show it when website accepts .txt files only, for example). See testPhotoPickerLaunchAndMimeTypes() here: https://cs.chromium.org/chromium/src/chrome/android/javatests/src/org/chromium/chrome/browser/SelectFileDialogTest.java 10. What’s not automated that needs manual testing? Do the image thumbnail look acceptable? (not too compressed/grainy, not too zoomed in, etc) For common image-accepting websites (Twitter, Facebook, etc), does it work end to end? Does it adversely affect other applications running on the device? Is it performant on a low-end (svelte) device? Is it buttery-smooth on a high-end device? Is it janky? Does it work on older Android OS-es? 11. List reasons for still needing manual testing. Some of the criteria is subjective, I guess. * Feature-Component-Ownership Details * 12. What is the crbug label/component for the feature? http://crbug.com/656015 13. Are there any other team members or mailing lists that should be notified if issues are found? No. * Stability * 14. Is there a mechanism to turn off the feature if it breaks product functionality or performance? Currently behind a flag (#enable-new-photo-picker in about:flags). 15. List UMA stats and coverage to measure stability, It’s highly recommend always adding Stability.Counts and CrashExitCodes.* to your json The only UMA stat currently captured is Android.SelectFileDialogScope (which tracks number of requests for images/videos/both/other). Planning on capturing decode success rate and speed as well as speed of various bitmap handling actions used. 16. What are you finch studies and what % rollout are you requesting for each group. No Finch study as of yet. 17. How are you monitoring the stability of the experiment groups. No experiment group at the moment. * Performance: * 18. Any performance impact? Did the feature get tested on perf waterfall? There should not be a performance impact, as this is a user-requested dialog. There's more of a concern on memory pressure on low-end devices.
,
Jun 9 2017
Assigning to rsgavara. Please let us know if you need anything else
,
Jun 12 2017
Thanks for the details! We will start testing this feature on M61.
,
Jul 5 2017
Friendly ping! We're still aiming to launch this in 61 so any feedback would be great.
,
Jul 13 2017
Testing is in progress, so far we encountered only https://bugs.chromium.org/p/chromium/issues/detail?id=737226, Will update testing details here once we are done.
,
Jul 17 2017
Great, thanks. We plan to request launch approval tomorrow, FYI. We will probably ask for it to be granted conditionally on wrapping up with you in the next week or so. Thanks
,
Jul 24 2017
Feature looks good. Here are the test results https://docs.google.com/spreadsheets/d/1F34vgDNIeiVoJfRnq6wVqVNq-xYQQvQhKzQn2gS6N0U/edit#gid=0 |
|||||
►
Sign in to add a comment |
|||||
Comment 1 by finnur@chromium.org
, May 31 2017