There are more than one contributors to the code now, it runs on at least 2 platforms, and is being open-sourced. Let's write some tests!

Upping the priority, as I'm starting to contribute to the code, and I really miss the tests.

As a general rule, I personally like to ensure 100% line and branch coverage for Python, simply because it's not type-safe and silly typos tend to crash otherwise parseable code. But any tests are better than no tests.

This work will happen after Franky is open-sourced, and I'm happy to contribute!

Yes!!!!!!!!

FTR, previous (unfinished) attempt to add some tests to Franky by bjoyce@:

https://chromereviews.googleplex.com/457567014/
https://chromereviews.googleplex.com/461127013/

Sergey I thought you owned this effort :) Shouldn't this be assigned to you?

I'm grabbing the bug for the time being, as I plan to start writing tests very soon.

In addition, running unit tests will require custom Python packages like coverage, and other packages Franky depends on. Rather than expecting each developer to install those on a system, I want the repo to use a hermetic virtualenv. We have a new tool called 'vpython' just for that.

The plan is to add vpython to depot_tools (which is required to work with the repo anyway), and implement all the developer workflow assuming vpython is on the path.

Making the bug public, now that Franky is open-sourced.

Python dependencies needed for Franky tests (for vpython):

appdirs==1.4.3
Appium-Python-Client==0.24
google-api-python-client==1.6.2
httplib2==0.10.3
oauth2client==4.0.0
packaging==16.8
pyasn1==0.2.3
pyasn1-modules==0.0.8
pyparsing==2.2.0
rsa==3.4.2
selenium==3.4.0
six==1.10.0
uritemplate==3.0.0

dnj@ uploaded all the wheels - thank you!

One diff:

selenium==3.4.1

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46

commit 5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Thu May 04 21:55:45 2017

Franky: add a test runner for sources.

BUG=623965
TBR=cmasso@google.com, lindsayw@chromium.org, bjoyce@google.com

Change-Id: Idc04f6d0e6a4ee4a2308d8f344356e17cf658f3d

[modify] https://crrev.com/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46/PRESUBMIT.py
[add] https://crrev.com/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46/test.py
[rename] https://crrev.com/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46/franky/test_data_reader_test.py
[add] https://crrev.com/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46/test.py.vpython
[add] https://crrev.com/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46/franky/coveragerc
[modify] https://crrev.com/5c90793c25f96ec1ab3f19fb8b73be0b5ac74d46/.gitignore

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/20826d76a50c7c19893a53b97721d191508edaf2

commit 20826d76a50c7c19893a53b97721d191508edaf2
Author: Sergey Berezin <sergeyberezin@google.com>
Date: Wed May 17 00:06:45 2017

Franky: add unittests for actions.elements

Additionally:
- test.py - disable coverage for partial test runs;
- add all Franky dependencies to vpython spec.

BUG=623965
TBR=cmasso@chromium.org

Change-Id: I102f7a5fce94c0ffa3863de601edfbce6acf18fd
Reviewed-on: https://chromium-review.googlesource.com/501249
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>
Reviewed-by: Mike Baxley <baxley@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>

[add] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/element_test.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/menu.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/tap.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/test.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/onboarding.py
[add] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/fixpath.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/scroll.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/settings.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/coveragerc
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/.gitignore
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/__init__.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/element.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/step.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/test.py.vpython
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/system.py
[modify] https://crrev.com/20826d76a50c7c19893a53b97721d191508edaf2/franky/actions/omnibox.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/2001af1a14faddfc4a6d94d26776432a6e642b96

commit 2001af1a14faddfc4a6d94d26776432a6e642b96
Author: Benjamin Joyce <bjoyce@google.com>
Date: Tue Jun 06 18:25:49 2017

Add Unit test for step.py.

Bug:623965
Change-Id: Id19490f1607b735b6a25a390d77a79ad67a00b39
Reviewed-on: https://chromium-review.googlesource.com/521731
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/2001af1a14faddfc4a6d94d26776432a6e642b96/franky/coveragerc
[add] https://crrev.com/2001af1a14faddfc4a6d94d26776432a6e642b96/franky/step_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/e22e54a50dab9ee2d6fddd51826f993526bee014

commit e22e54a50dab9ee2d6fddd51826f993526bee014
Author: Benjamin Joyce <bjoyce@google.com>
Date: Tue Jun 06 18:44:10 2017

Add unit tests for input.py

Bug:623965
Change-Id: Idedf200fb897a4e74e0846ddaca765b94909961e
Reviewed-on: https://chromium-review.googlesource.com/523027
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[add] https://crrev.com/e22e54a50dab9ee2d6fddd51826f993526bee014/franky/actions/input_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/6571a2e7a7f1593471f1d2f3cf2d5c7e320ca651

commit 6571a2e7a7f1593471f1d2f3cf2d5c7e320ca651
Author: Benjamin Joyce <bjoyce@google.com>
Date: Mon Jun 12 19:07:15 2017

Add unit tests for device_info.

Bug:623965
Change-Id: I74522deb51b9740c55b2b4266c3a220c8abda7cb
Reviewed-on: https://chromium-review.googlesource.com/527384
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/6571a2e7a7f1593471f1d2f3cf2d5c7e320ca651/franky/coveragerc
[add] https://crrev.com/6571a2e7a7f1593471f1d2f3cf2d5c7e320ca651/franky/device_info_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/a94657acef5c8d236b65dd498e002200c8b6b38f

commit a94657acef5c8d236b65dd498e002200c8b6b38f
Author: Benjamin Joyce <bjoyce@google.com>
Date: Thu Jun 15 21:38:03 2017

Add unit tests for app_info.

Bug:623965
Change-Id: Ic6ade26dab0cea6966fa2fb318aa3d80c213919b
Reviewed-on: https://chromium-review.googlesource.com/531785
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/a94657acef5c8d236b65dd498e002200c8b6b38f/franky/coveragerc
[add] https://crrev.com/a94657acef5c8d236b65dd498e002200c8b6b38f/franky/app_info_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/576095288c65a519a3df4d70d666d7fd048e88d3

commit 576095288c65a519a3df4d70d666d7fd048e88d3
Author: Benjamin Joyce <bjoyce@google.com>
Date: Mon Aug 14 20:31:06 2017

Add menu test coverage.

Bug:623965
Change-Id: I48b5521f522d4e5dff57146c1b3f0f4471f8f00e
Reviewed-on: https://chromium-review.googlesource.com/612721
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/576095288c65a519a3df4d70d666d7fd048e88d3/franky/coveragerc
[add] https://crrev.com/576095288c65a519a3df4d70d666d7fd048e88d3/franky/actions/menu_test.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/5394924acbcfc2f18a2db5b11840869404197e98

commit 5394924acbcfc2f18a2db5b11840869404197e98
Author: Benjamin Joyce <bjoyce@google.com>
Date: Mon Jan 08 19:47:03 2018

Data Saver screen appears on some devices or regions, so press Next button if it appears.
Add unit tests.

Bug:799093,623965
Change-Id: Ib80bbc22500faeb11d30a283887608ba315ecad5
Reviewed-on: https://chromium-review.googlesource.com/851083
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[add] https://crrev.com/5394924acbcfc2f18a2db5b11840869404197e98/franky/actions/onboarding_test.py
[modify] https://crrev.com/5394924acbcfc2f18a2db5b11840869404197e98/franky/actions/onboarding.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/34e5a5802649625cabd609557c9e53c2d15c3031

commit 34e5a5802649625cabd609557c9e53c2d15c3031
Author: Benjamin Joyce <bjoyce@google.com>
Date: Wed Jan 10 23:06:01 2018

Add ability to delay based on objects being there or not being there.

Bug:799151, 623965
Change-Id: I769d280c1d40799a237a96e20623bda85b712a58
Reviewed-on: https://chromium-review.googlesource.com/853356
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>

[modify] https://crrev.com/34e5a5802649625cabd609557c9e53c2d15c3031/franky/actions/timeout.py
[modify] https://crrev.com/34e5a5802649625cabd609557c9e53c2d15c3031/franky/coveragerc
[modify] https://crrev.com/34e5a5802649625cabd609557c9e53c2d15c3031/franky/protos/actions_pb2.py
[add] https://crrev.com/34e5a5802649625cabd609557c9e53c2d15c3031/franky/actions/timeout_test.py
[modify] https://crrev.com/34e5a5802649625cabd609557c9e53c2d15c3031/franky/protos/actions.proto

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/92fc500307a4c127abfde375cd5042aee2bd2640

commit 92fc500307a4c127abfde375cd5042aee2bd2640
Author: Ben Joyce <bjoyce@google.com>
Date: Tue Apr 10 01:40:15 2018

Add runner unit test cases.

Bug:623965
Change-Id: I95efa009aefb326468f693d0896f4b3196784ae9
Reviewed-on: https://chromium-review.googlesource.com/1000423
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>
Commit-Queue: Benjamin Joyce <bjoyce@google.com>

[add] https://crrev.com/92fc500307a4c127abfde375cd5042aee2bd2640/franky/runner_test.py
[modify] https://crrev.com/92fc500307a4c127abfde375cd5042aee2bd2640/franky/coveragerc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src/tools/franky/+/641797db348005e4326a15bdacea946384a6cdfb

commit 641797db348005e4326a15bdacea946384a6cdfb
Author: Ben Joyce <bjoyce@google.com>
Date: Wed Jul 11 18:41:35 2018

Add unit tests for input formatter, timeout util. Increase menu_test coverage.

Bug:623965
Change-Id: I1a8c3e418bb5e101f971f3f16fc88767c97f728a
Reviewed-on: https://chromium-review.googlesource.com/1128458
Commit-Queue: Benjamin Joyce <bjoyce@google.com>
Reviewed-by: Lindsay Pasricha <lindsayw@chromium.org>

[modify] https://crrev.com/641797db348005e4326a15bdacea946384a6cdfb/franky/actions/menu_test.py
[add] https://crrev.com/641797db348005e4326a15bdacea946384a6cdfb/franky/appium_util/input_formatter_test.py
[add] https://crrev.com/641797db348005e4326a15bdacea946384a6cdfb/franky/appium_util/timeout_util_test.py