New issue
Advanced search Search tips

Issue 638824 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

recipe_simulation_test.py train occurs AssertionError related to google.protobuf.__path__

Project Member Reported by tikuta@chromium.org, Aug 18 2016

Issue description

$ ./unittests/recipe_simulation_test.py train
Traceback (most recent call last):
  File "/usr/local/google/home/tikuta/infra/build/scripts/slave/.recipe_deps/recipe_engine/recipes.py", line 22, in <module>
    from recipe_engine import env
  File "/usr/local/google/home/tikuta/infra/build/scripts/slave/.recipe_deps/recipe_engine/recipe_engine/env.py", line 68, in <module>
    os.path.realpath(google.protobuf.__path__[0]))
AssertionError
To create a coverage report, run:
    ./unittests/recipe_simulation_test.py train --html_report PATH

This is caused by https://codereview.chromium.org/2154943002/
Could you fix this by checking all elements of google.protobuf.__path__?


 

Comment 1 by d...@chromium.org, Aug 18 2016

Owner: ----
So I don't think that CL caused the issue. All that CL did was use the realpath instead of whatever path the system settled on. The assert is still present in the previous version.

Checking *all* elements of __path__ is self-defeating, as the entire point of the assertion is to assert that specifically the recipe engine's vendored version is the one that was loaded, and that would be in path element [0].

This vendoring check works on all of our buildslaves and every developer system that I've seen so far. I am unable to reproduce this locally.

Without any additional details, I don't think I (or anyone else) can proceed with identifying or fixing the issue. We need to know:
1) What your google.protobuf.__path__ value is.
2) Any ideas why your system might be doing something different from everyone else's.

(Please don't assign owners for components like this. It gives the false sense that someone has actually looked at the bug, and that can hinder triage. If you want someone to be on the bug, just CC them.)

Comment 2 by tikuta@chromium.org, Aug 19 2016

I modified env.py as follows and ran train.

 import google.protobuf
+print 'THIRD_PARTY', THIRD_PARTY
+print 'google.protobuf.__path__', google.protobuf.__path__
 assert (os.path.realpath(THIRD_PARTY) in
         os.path.realpath(google.protobuf.__path__[0]))

$ ./unittests/recipe_simulation_test.py train
THIRD_PARTY /usr/local/google/home/tikuta/infra/build/scripts/slave/.recipe_deps/recipe_engine/recipe_engine/third_party
google.protobuf.__path__ ['/usr/local/google/home/tikuta/.linuxbrew/lib/python2.7/site-packages/google/protobuf', '/usr/local/google/home/tikuta/infra/build/scripts/slave/.recipe_deps/recipe_engine/recipe_engine/third_party/google/protobuf', '/usr/local/google/home/tikuta/.linuxbrew/Cellar/python/2.7.12_1/lib/python2.7/site-packages/google/protobuf']
Traceback (most recent call last):
  File "/usr/local/google/home/tikuta/infra/build/scripts/slave/.recipe_deps/recipe_engine/recipes.py", line 22, in <module>
    from recipe_engine import env
  File "/usr/local/google/home/tikuta/infra/build/scripts/slave/.recipe_deps/recipe_engine/recipe_engine/env.py", line 70, in <module>
    os.path.realpath(google.protobuf.__path__[0]))
AssertionError
To create a coverage report, run:
    ./unittests/recipe_simulation_test.py train --html_report PATH


When I stop to use python from brew, this is not reproduced.
But, did you intend to have assert if someone use python and protobuf library from brew?

(Sorry for assigning you to this issue.)

Comment 3 by d...@chromium.org, Aug 19 2016

Labels: -Pri-1 Pri-3
That's interesting. The recipe engine's import *must* be first, in [0] (it's [1] right now). What you're seeing here is the recipe engine importing your local protobuf version instead of its vendored. This is problematic, since the protobuf package name alone doesn't differentiate between proto2 and proto3 libraries or their respective minor versions, hence why we vendor it in the first place.

I don't have the bandwidth right now to try and recreate your homebrew environment and see why it's importing your local one. If you want to dive into it, since you already have the environment set up, a patch would be happily accepted. I'm going to decrease the priority since you have a workaround.
Status: Available (was: Untriaged)

Comment 5 by tikuta@chromium.org, Aug 22 2016

Owner: tikuta@chromium.org
Project Member

Comment 6 by bugdroid1@chromium.org, Aug 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/37ac639b5795bbfdaed0e000d8d32211d12f0fe2

commit 37ac639b5795bbfdaed0e000d8d32211d12f0fe2
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 23 19:02:55 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

recipe_engine:
  https://crrev.com/772978c9b350224f62085c258408a02dcd97986c Remove preloaded 'google' module (tikuta@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 638824 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2274673002

[modify] https://crrev.com/37ac639b5795bbfdaed0e000d8d32211d12f0fe2/infra/config/recipes.cfg

Project Member

Comment 7 by bugdroid1@chromium.org, Aug 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build.git/+/84ea9f11905aeb0d78be8c92c3f54149dfb4382a

commit 84ea9f11905aeb0d78be8c92c3f54149dfb4382a
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 23 19:14:04 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

depot_tools:
  https://crrev.com/9249e01b702c6fe86f089ae0a8769a0f673b4a93 git cl archive: Add "--dry-run" and "--notags" flags. (kmarshall@chromium.org)
  https://crrev.com/37ac639b5795bbfdaed0e000d8d32211d12f0fe2 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
recipe_engine:
  https://crrev.com/772978c9b350224f62085c258408a02dcd97986c Remove preloaded 'google' module (tikuta@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 638824 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2272583004

[modify] https://crrev.com/84ea9f11905aeb0d78be8c92c3f54149dfb4382a/infra/config/recipes.cfg

Project Member

Comment 8 by bugdroid1@chromium.org, Aug 23 2016

Project Member

Comment 9 by bugdroid1@chromium.org, Aug 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/infra/infra.git/+/accb77f1b753f91ee9f5ed83f98114523b5fb085

commit accb77f1b753f91ee9f5ed83f98114523b5fb085
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 23 19:35:09 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/84ea9f11905aeb0d78be8c92c3f54149dfb4382a Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
depot_tools:
  https://crrev.com/9249e01b702c6fe86f089ae0a8769a0f673b4a93 git cl archive: Add "--dry-run" and "--notags" flags. (kmarshall@chromium.org)
  https://crrev.com/37ac639b5795bbfdaed0e000d8d32211d12f0fe2 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
recipe_engine:
  https://crrev.com/772978c9b350224f62085c258408a02dcd97986c Remove preloaded 'google' module (tikuta@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG= 638824 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2272803002

[modify] https://crrev.com/accb77f1b753f91ee9f5ed83f98114523b5fb085/infra/config/recipes.cfg

Project Member

Comment 10 by bugdroid1@chromium.org, Aug 23 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/da54cec72f53ebfe9b67853375e02d3a8d1bc2de

commit da54cec72f53ebfe9b67853375e02d3a8d1bc2de
Author: recipe-roller <recipe-roller@chromium.org>
Date: Tue Aug 23 23:16:41 2016

Roll recipe dependencies (trivial).

This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

build:
  https://crrev.com/8c41d5fe109c2af1e7aeeb04455e47c1ccea1a46 Removing Shapiro, Welch's and Anderson-Darling stats tests from the code that compares samples. (robertocn@chromium.org)
  https://crrev.com/b8bb5bdccf448b62d492d58c23441b69d1e43026 Making android bisect not nest waiting steps (to avoid long names). (robertocn@chromium.org)
  https://crrev.com/84ea9f11905aeb0d78be8c92c3f54149dfb4382a Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
depot_tools:
  https://crrev.com/9249e01b702c6fe86f089ae0a8769a0f673b4a93 git cl archive: Add "--dry-run" and "--notags" flags. (kmarshall@chromium.org)
  https://crrev.com/37ac639b5795bbfdaed0e000d8d32211d12f0fe2 Roll recipe dependencies (trivial). (recipe-roller@chromium.org)
recipe_engine:
  https://crrev.com/772978c9b350224f62085c258408a02dcd97986c Remove preloaded 'google' module (tikuta@chromium.org)

TBR=martiniss@chromium.org,phajdan.jr@chromium.org
BUG=639813,chromium:636486, 638824 

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2272813002
Cr-Commit-Position: refs/heads/master@{#413888}

[modify] https://crrev.com/da54cec72f53ebfe9b67853375e02d3a8d1bc2de/infra/config/recipes.cfg

Status: Fixed (was: Available)

Sign in to add a comment