New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 832283 link

Starred by 7 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug


Show other hotlists

Hotlists containing this issue:
Chromium-Packagers


Sign in to add a comment

Build from official tarball broken: missing third_party/blink/tools/blinkpy/common

Project Member Reported by jose.dap...@lge.com, Apr 12 2018

Issue description

Chrome Version: 67.0.3393.4

What steps will reproduce the problem?
(1) Try to build Chromium official tarball as obtained from https://commondatastorage.googleapis.com/chromium-browser-official/chromium-67.0.3393.4.tar.xz

What is the expected result?

Build is successful

What happens instead?

It fails to build because the tarball does not include the required files:

ninja: error: '../../../chromium-67.0.3393.4/third_party/blink/tools/blinkpy/common/name_style_converter.py', needed by 'obj/third_party/blink/renderer/bindings/core/v8/bindings_core_impl_generated.inputdeps.stamp', missing and no known rule to make it
WARNING: exit code 1 from a shell command.
ERROR: Function failed: do_compile (log file is located at /home/dape/Development/rpi/poky-pyro/build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/chromium-x11-dev/67.0.3393.4-r0/temp/log.do_compile.22174)
/

If I check the tarball, third_party/blink/tools/blinkpy/common is empty.
 
Cc: tkent@chromium.org thomasanderson@chromium.org raphael....@intel.com
Cc: vamshi.kommuri@chromium.org
 Issue 831239  has been merged into this issue.
Components: -Infra>Client>Chrome
Owner: tkent@chromium.org
Status: Assigned (was: Untriaged)
Appears to be fallout from https://chromium.googlesource.com/chromium/tools/build/+/2a1e887493fdea46926ce3efbf53ad15cc5f708d

This is also breaking https://ci.chromium.org/buildbot/chromium.infra.cron/Build%20From%20Tarball/

Comment 4 by tkent@chromium.org, Apr 12 2018

Status: Started (was: Assigned)
Can anyone add CodeReview+1 to https://chromium-review.googlesource.com/c/chromium/tools/build/+/1011482 ?  I'm not a committer of this repository.

Project Member

Comment 5 by bugdroid1@chromium.org, Apr 13 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/0a90b16fc705ee38b659620709f71ad6bd0b9de5

commit 0a90b16fc705ee38b659620709f71ad6bd0b9de5
Author: Kent Tamura <tkent@chromium.org>
Date: Fri Apr 13 03:10:21 2018

Revert "export_tarball: Exclude third_party/blink/tools/ and third_party/blink/web_tests"

This reverts commit 2a1e887493fdea46926ce3efbf53ad15cc5f708d.

Reason for revert:  crbug.com/832283  Chromium build depends on third_party/blink/tools/blinkpy/common/.

Original change's description:
> export_tarball: Exclude third_party/blink/tools/ and third_party/blink/web_tests
>
> as well as third_party/WebKit/Tools and third_party/WebKit/LayoutTests.
> This is a preparation to move tools and tests in third_party/WebKit to
> third_party/blink.
>
> Bug:  829695 ,  829697 
> Change-Id: I46ee0d05b62979053033ce8f2b1144c7e19aae25
> Reviewed-on: https://chromium-review.googlesource.com/1004077
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Commit-Queue: Kent Tamura <tkent@chromium.org>

TBR=dpranke@chromium.org,tkent@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  829695 ,  829697 ,  832283 
Change-Id: Ifeec896351c20b58bb609521ac6c0ac9f5fe0802
Reviewed-on: https://chromium-review.googlesource.com/1011482
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>

[modify] https://crrev.com/0a90b16fc705ee38b659620709f71ad6bd0b9de5/scripts/slave/recipe_modules/chromium/resources/export_tarball.py

Comment 6 by tkent@chromium.org, Apr 13 2018

Status: Fixed (was: Started)
Summary: Build from official tarball broken: missing third_party/blink/tools/blinkpy/common (was: Build from official tarball broken: missing third_party/blink/tools/common)
This should be fixed since the next tarball.

Comment 7 by tkent@chromium.org, Apr 17 2018

Components: Blink>Infra
Status: Started (was: Fixed)
I'm removing blink/tools dependency from build steps.

Project Member

Comment 8 by bugdroid1@chromium.org, Apr 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0

commit 968e7691f3b246d7cfcfcc7ce2b99eef145c9da0
Author: Kent Tamura <tkent@chromium.org>
Date: Tue Apr 17 23:05:57 2018

Move name_style_converter.py from blink/tools to renderer/build/scripts/.

We'd like to provide smaller tarballs with which one can build Chrome,
and such tarballs exclude //third_party/WebKit/Tools/Script. We're
moving //third_party/WebKit/Tools/Scripts to //third_party/blink/tools,
and would like to exclude //third_party/blink/tools as well. However,
some build scripts use name_style_converter.py in the directory.

This CL moves name_style_converter.py out from
//third_party/blink/tools, and enables to exclude
//third_party/blink/tools.

test-webkitpy includes //third_party/blink/renderer/build/scripts. So
the following existing tests run on buildbots in addition to
name_style_converter_test.py:
- in_file_unittest.py
- minimize_css_unittest.py
This CL fixes test failures of in_file_unittest.py.

Bug:  832283 
Change-Id: I7d3b3524daa59291eaf7f323818a7b0e25163179
Reviewed-on: https://chromium-review.googlesource.com/1013768
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Kent Tamura <tkent@chromium.org>
Cr-Commit-Position: refs/heads/master@{#551514}
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/BUILD.gn
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/WebKit/Tools/Scripts/test-webkitpy
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/bindings/scripts/code_generator_web_agent_api.py
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/bindings/scripts/scripts.gni
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/bindings/scripts/utilities.py
[add] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/build/scripts/blinkbuild/__init__.py
[rename] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/build/scripts/blinkbuild/name_style_converter.py
[rename] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/build/scripts/blinkbuild/name_style_converter_test.py
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/build/scripts/in_file_unittest.py
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/renderer/build/scripts/json5_generator.py
[add] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/tools/blinkpy/README.md
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/tools/move_blink_source.py
[modify] https://crrev.com/968e7691f3b246d7cfcfcc7ce2b99eef145c9da0/third_party/blink/tools/plan_blink_move.py

Project Member

Comment 9 by bugdroid1@chromium.org, Apr 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/859b13e2a724573556dfd8c7cd2f4587c4d584b2

commit 859b13e2a724573556dfd8c7cd2f4587c4d584b2
Author: Kent Tamura <tkent@chromium.org>
Date: Tue Apr 17 23:22:44 2018

Reland "export_tarball: Exclude third_party/blink/tools/ and third_party/blink/web_tests"

as well as third_party/WebKit/Tools and third_party/WebKit/LayoutTests.
This is a preparation to move tools and tests in third_party/WebKit to
third_party/blink.

No difference from the reverted 2a1e887493fdea46926ce3efbf53ad15cc5f708d.
We removed blink/tools/ build dependency.

Bug:  829695 ,  829697 ,  832283 
Change-Id: I82222641e50b3652d3705fc60213d7da21867e4d
Reviewed-on: https://chromium-review.googlesource.com/1013855
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Kent Tamura <tkent@chromium.org>

[modify] https://crrev.com/859b13e2a724573556dfd8c7cd2f4587c4d584b2/scripts/slave/recipe_modules/chromium/resources/export_tarball.py

Comment 10 by tkent@chromium.org, Apr 17 2018

Status: Fixed (was: Started)

Comment 11 by ajha@chromium.org, Apr 18 2018

 Issue 834155  has been merged into this issue.

Comment 12 by tkent@chromium.org, Apr 18 2018

Cc: dpranke@chromium.org
Status: Started (was: Fixed)
> Reland "export_tarball: Exclude third_party/blink/tools/ and third_party/blink/web_tests"

Maybe I was wrong. We can exclude third_party/blink/tools only for very recent versions.

Per https://groups.google.com/a/chromium.org/d/msg/chromium-packagers/So-ojMYOQdI/riamnwzdCQAJ and the ensuing discussion, would it be possible to revert that CL again and regenerate the tarball? Right now all packagers need to manually add the missing files to their builds in order to build the M66 stable tarball.
Project Member

Comment 14 by bugdroid1@chromium.org, Apr 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/38678e7431139248918afd926cd010c6e4a32ef9

commit 38678e7431139248918afd926cd010c6e4a32ef9
Author: Kent Tamura <tkent@chromium.org>
Date: Wed Apr 18 17:49:09 2018

export_tarball: Exclude third_party/blink/tools only for M68+.

This script is used for creating tarballs for branches too, and M66 and M67
depend on third_party/blink/tools.

Bug:  832283 
Change-Id: I4a94241df7aba1a48bcdda709815549310429bf1
Reviewed-on: https://chromium-review.googlesource.com/1016141
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>

[modify] https://crrev.com/38678e7431139248918afd926cd010c6e4a32ef9/scripts/slave/recipe_modules/chromium/resources/export_tarball.py

Comment 15 by tkent@chromium.org, Apr 19 2018

 Issue 834141  has been merged into this issue.
The CL from comment #14 broke the generation of M68 tarballs. From https://ci.chromium.org/buildbot/chromium.infra.cron/publish_tarball/10593:

Traceback (most recent call last):
  File "/b/rr/tmpvNs_Bt/rw/recipes/.recipe_deps/build/scripts/slave/recipe_modules/chromium/resources/export_tarball.py", line 230, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/b/rr/tmpvNs_Bt/rw/recipes/.recipe_deps/build/scripts/slave/recipe_modules/chromium/resources/export_tarball.py", line 185, in main
    nonessential_dirs.add('third_party/blink/tools')
AttributeError: 'tuple' object has no attribute 'add'

Comment 17 by tkent@chromium.org, Apr 19 2018

>     nonessential_dirs.add('third_party/blink/tools')
> AttributeError: 'tuple' object has no attribute 'add'

Oops! I'm working on a fix for this.

Project Member

Comment 18 by bugdroid1@chromium.org, Apr 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/6a5b75fe06098442399e86e2dccbcd3005d34391

commit 6a5b75fe06098442399e86e2dccbcd3005d34391
Author: Kent Tamura <tkent@chromium.org>
Date: Thu Apr 19 20:24:27 2018

export_tarball: Fix a bug for M68+

nonessential_dirs was a tuple, which is immutable. This CL changes
nonessential_dirs and TEST_DIRS sets.

Bug:  832283 
Change-Id: Ia07143ee4b880ccd24cfcb6dbc3037f8ab14a238
Reviewed-on: https://chromium-review.googlesource.com/1018662
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>

[modify] https://crrev.com/6a5b75fe06098442399e86e2dccbcd3005d34391/scripts/slave/recipe_modules/chromium/resources/export_tarball.py

Ping re. comment #13.

Comment 20 by tkent@chromium.org, Apr 27 2018

Status: Fixed (was: Started)
Now chromium-66.0.3359.139*.tar.xz is available, and I don't think we should update the old tarball.

Sign in to add a comment