New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 787287 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

Is yaml package missing on lab servers

Project Member Reported by rohi...@chromium.org, Nov 21 2017

Issue description

https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/757128

Pre-submit is failing with no module name py yaml error, but I see a lot of py files in ChromeOS codebase using yaml so my assumption was yaml is well supported in our infra.

Is yaml package missing on lab servers? 

11/19 11:15:34.616 ERROR|   logging_manager:0626| tko parser:     File "/usr/local/autotest/server/cros/tradefed_test.py", line 43, in <module>
11/19 11:15:34.617 ERROR|   logging_manager:0626| 
11/19 11:15:34.617 ERROR|   logging_manager:0626| tko parser:       from autotest_lib.server.cros import cts_expected_failure_parser
11/19 11:15:34.618 ERROR|   logging_manager:0626| 
11/19 11:15:34.618 ERROR|   logging_manager:0626| tko parser:     File "/usr/local/autotest/server/cros/cts_expected_failure_parser.py", line 6, in <module>
11/19 11:15:34.619 ERROR|   logging_manager:0626| 
11/19 11:15:34.619 ERROR|   logging_manager:0626| tko parser:       import yaml
11/19 11:15:34.620 ERROR|   logging_manager:0626| 
11/19 11:15:34.620 ERROR|   logging_manager:0626| tko parser:   ImportError: No module named yaml
 

Comment 1 by nxia@chromium.org, Nov 22 2017

Cc: nxia@chromium.org pprabhu@chromium.org dshi@chromium.org
Owner: rohi...@chromium.org
No autotest code imports yaml, except the skylab code which is run inside the GAE instances.

+ dshi@, in order use yaml, should it be added to side-packages?
Cc: rohi...@chromium.org
Owner: dshi@chromium.org
Assigning to Dan for his inputs.


Comment 3 by dshi@chromium.org, Nov 28 2017

The failure is in tko parser, it won't run in ssp.
The only way to fix this is to add yaml to site-packages.
Let me know if there is a similar fix/bug I can use for this fix.

If there isn't any, we might need some help from infra :-)

Comment 5 by dshi@chromium.org, Nov 28 2017

Adding package to site-packages? The only module you need to touch is utils/external_packages.py, do a git log on that module, and you can see how each module was added. should be straightforward.
I see some of pyaml modules already in the mirror.

gsutil ls gs://chromeos-mirror/gentoo/distfiles/*pyaml*
gs://chromeos-mirror/gentoo/distfiles/pyaml-14.05.7.tar.gz
gs://chromeos-mirror/gentoo/distfiles/pyaml-15.5.7.tar.gz
gs://chromeos-mirror/gentoo/distfiles/pyaml-15.6.3.tar.gz
gs://chromeos-mirror/gentoo/distfiles/pyaml-15.8.2.tar.gz

I am thinking to copy version 3.10 which is relatively latest. 

- Copy the tar.gz file to gs://chromeos-mirror/gentoo/distfiles/
- Any specific way to generate 'hex_sum'?

Comment 7 by dshi@chromium.org, Nov 28 2017

Cc: vapier@chromium.org
I don't know about gs://chromeos-mirror
+vapier 

For hex_sum, you can just run build_external and let it print out the hex sum it's expecting.

Comment 8 by vapier@chromium.org, Nov 29 2017

Labels: -Restrict-View-Google
you're looking at the wrong name for the source tarballs:
$ gsutil ls gs://chromeos-mirror/gentoo/distfiles/PyYAML*
gs://chromeos-mirror/gentoo/distfiles/PyYAML-3.08.tar.gz
gs://chromeos-mirror/gentoo/distfiles/PyYAML-3.09.tar.gz
gs://chromeos-mirror/gentoo/distfiles/PyYAML-3.10.tar.gz
gs://chromeos-mirror/gentoo/distfiles/PyYAML-3.11.tar.gz
gs://chromeos-mirror/gentoo/distfiles/PyYAML-3.12.tar.gz
Thanks, vapier@ !

https://chromium-review.googlesource.com/c/chromiumos/third_party/autotest/+/798029

When running build_externals.py, every time it installs PyYAML. Not sure if that is expected behavior? Or am I missing anything obvious in my changes?
Project Member

Comment 10 by bugdroid1@chromium.org, Dec 1 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/third_party/autotest/+/b822a0c34a929a0e2cdedff76bff8b10b1d183ae

commit b822a0c34a929a0e2cdedff76bff8b10b1d183ae
Author: Rohit Makasana <rohitbm@chromium.org>
Date: Fri Dec 01 22:54:44 2017

Add PyYAML in site package.

BUG= chromium:787287 
TEST=ran ./build_externals.py

Change-Id: I11c7c6d15c4556d667f8561ba994d9e8fb1b30e1
Reviewed-on: https://chromium-review.googlesource.com/798029
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>

[modify] https://crrev.com/b822a0c34a929a0e2cdedff76bff8b10b1d183ae/utils/external_packages.py

Once this change is pushed to the lab, I will retry pushing my CTS test CL.
Owner: rohi...@chromium.org
This CL just got pushed to the production. 

Will try submitting my CLs now.
Status: Fixed (was: Assigned)
Cc: dchan@chromium.org
Status: Verified (was: Fixed)

Sign in to add a comment