New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 736322 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 17
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug
Build-Toolchain

Blocked on:
issue 745353

Blocking:
issue 884766
issue 649672
issue 898728



Sign in to add a comment

dev-lang/python:3*: uprev to 3.5.3+

Project Member Reported by djkurtz@chromium.org, Jun 23 2017

Issue description

portage-stable currently installs stock dev-lang/python-3.3.5-r1.

Upstream gentoo no longer even includes python:3.3.  The latest stable is 3.4.5.

Fixes for OpenSSL 1.1 are first found in v3.5.3, and carry into the v3.6 branch:

598894ff48  Issue #26470 : Port ssl and hashlib module to OpenSSL 1.1.0.

Note: python-3 was only added to the chroot for "sigrok" ( issue 383429 ), and AFAICT, that is still the only thing that uses it.

Are these packages still required?
Is anybody using them?

They don't seem to build at present:
  sudo PYTHON_TARGETS="python2_7 python3_3" emerge python-exec
  sudo emerge libsigrok libsigrokdecode sigrok-cli

>>> Preparing source in /var/tmp/portage/sci-libs/libsigrokdecode-0.3.0-r19/work/libsigrokdecode-0.3.0 ...
sed: can't read configure: No such file or directory
 * ERROR: sci-libs/libsigrokdecode-0.3.0-r19::chromiumos failed (prepare phase):
 *   (no error message)
 *
 * Call stack:
 *     ebuild.sh, line  93:  Called src_prepare
 *   environment, line 5009:  Called die
 * The specific snippet of code:
 *       sed -i -e '/build_runtc=/s:yes:no:' configure || die
 * 

 
Blockedon: 745353
python updates (2.7.10 -> 2.7.13 & 3.3 -> 3.6) are interdependent due to shared dependencies (eselect-python & python-exec), so must go in together.

CLs:
https://chromium-review.googlesource.com/580218 chromeos-base/dev-install: Do not install /usr/local/bin/python symlink        
https://chromium-review.googlesource.com/580219 media-libs/vulkan-loader: Add support for python 3.6        
https://chromium-review.googlesource.com/580220 virtual/target-chromium-os-sdk: Update python:3 RDEPEND to 3.6        
https://chromium-review.googlesource.com/580221 dev-lang/python: update to 2.7.13        
https://chromium-review.googlesource.com/580450 BACKPORT: python-utils-r1.eclass: Enable python3_6 target        
https://chromium-review.googlesource.com/580451 eselect-python, python-exec, python3: Uprev to latest from gentoo        
https://chromium-review.googlesource.com/574040 eselect-python: Move from app-admin to app-eselect        

Project Member

Comment 3 by bugdroid1@chromium.org, Jul 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/fbe4aece95869343c4e4d4814cbb5ee54930fa48

commit fbe4aece95869343c4e4d4814cbb5ee54930fa48
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Sat Jul 22 09:46:02 2017

make_chroot: Do not update python

CL:176050 ("Support Python 2.7 in chroot bootstrap process.") forced
a python update after the portage upgrade when making the chroot.

At the time (2013-11-07), the BOOTSTRAP was "2013.01.30", which used
python 2.6, so this python update was needed to let parallel_emerge and
other chromite utilities use python 2.7 features.

CL:222090 ("sdk: bootstrap from newer Gentoo stage3") (2014-09-24) updated
the BOOSTRAP to its current version "2014.09.18" (yes, its almost 3 years
old).  This version already includes python 2.7.7 in it, so the
make_chroot time explicit python update is no longer required.  Instead,
the python update can be deferred to the update_chroot section, which
updates the entire sdk with full dependency resolution.

The motivation for removing this python update during make_chroot is that
in 1Q-2015 upstream gentoo moved eselect-python from app-admin to
app-eselect.

The "2014.09.18" stage3 still had app-admin/eselect-python, and therefore
its python2 and python3 ebuilds both depend on this location.  If we
move eselect-python to app-eselect and update the two python ebuilds,
but only emerge python2, the emerge is blocked due to the existing python3
still pulling in the old app-admin/eselect-python.

Also remove an old comment about the python2.* paths only existing after
the python update, since those paths do exist in the current BOOTSTRAP.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>

BUG=chromium:736313, chromium:736322 
TEST=cros_sdk --replace
TEST=cros_sdk --replace --bootstrap
 => make_chroot completes successfully in both cases
TEST=./setup_board --board=hana
TEST=./build_packages --board=hana
TEST=./build_image --board=hana test
TEST=test_that suite:bvt-cq

Change-Id: Ifa122ca72af3aa9df5a64be9c800e8ecf74a9cbf
Reviewed-on: https://chromium-review.googlesource.com/581307
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/fbe4aece95869343c4e4d4814cbb5ee54930fa48/sdk_lib/make_chroot.sh

Project Member

Comment 4 by bugdroid1@chromium.org, Jul 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/d82154343946adb4fc081358cb0c0db5606c633e

commit d82154343946adb4fc081358cb0c0db5606c633e
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Sat Jul 22 09:46:01 2017

profile.bashrc: Update for python 3.6

The python-utils-r1.eclass:python_wrapper_setup installs a stub program named
`python2` when PYTHON_COMPAT only lists python-3 programs.  It's designed to
catch bad packages that still use `python2` even though we said not to.

In Chromium OS though, the compiler sysroot_wrapper.hardened still uses
"python2" in its shebang.  With the shim in place, all calls to the
compiler binaries (clang, gcc, etc) will fail when emerging a python-3
only package with the mysterious message:

   python2 is not supported by python3.3

Conveniently, the function cros_post_pkg_setup_python_eclass_hack
removes the python2 shim such that the sysroot_wrapper works as
expected.

Now that we are updating to python3.6, update this helper function to
clear the new directory.  While we are at it, make it backwards compatible
and future proof with a glob (which only works if the string isn't quoted).

BUG= chromium:736322 
TEST=emerge-hana vulkan-loader
TEST=./build_packages --board=soraka
TEST=./build_packages --board=hana

Change-Id: I45178328433f366865a84da7e3df9bd5fcb5bf6f
Reviewed-on: https://chromium-review.googlesource.com/580217
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/d82154343946adb4fc081358cb0c0db5606c633e/profiles/base/profile.bashrc

Blockedon: 747810
Blockedon: -747810
Project Member

Comment 7 by bugdroid1@chromium.org, Jul 24 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/95e8c6f33ad5b52bf637883df27ff00fa55fbd29

commit 95e8c6f33ad5b52bf637883df27ff00fa55fbd29
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Mon Jul 24 17:40:11 2017

media-libs/vulkan-loader: Add support for python 3.6

As we are updating python 3 in the build from 3.3 -> 3.6, add python3_6
to PYTHON_COMPAT.

Also create a symlink, and rev it.

BUG= chromium:736322 
TEST=with python3_6 installed in the cros_sdk
  emerge-hana vulkan-loader
CQ-DEPEND=CL:580450

Change-Id: If69b76bcf1d384fc2562bdf6a2efdbf2f3e8da94
Reviewed-on: https://chromium-review.googlesource.com/580219
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[add] https://crrev.com/95e8c6f33ad5b52bf637883df27ff00fa55fbd29/media-libs/vulkan-loader/vulkan-loader-1.0.39.0-r3.ebuild
[rename] https://crrev.com/95e8c6f33ad5b52bf637883df27ff00fa55fbd29/media-libs/vulkan-loader/vulkan-loader-1.0.39.0.ebuild

Project Member

Comment 8 by bugdroid1@chromium.org, Jul 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/5124484160c8477bcc0236297cdb7801b0a15cdc

commit 5124484160c8477bcc0236297cdb7801b0a15cdc
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Thu Jul 27 11:22:44 2017

make_chroot: Update python before portage

The 2014.09.18 stage3 uses app-admin/eselect-python, but in 2015-Q1 it
moved to app-eselect/eselect-python. Since portage depends on python-exec,
and python-exec depends on eselect-python, update python and related
packages before portage.

This can be removed once the bootstrap has updated to include
app-eselect/eselect-python (ie, anything newer than 2015-Q1).

We must also immediately set python2 as default immediately after updating
python and before emerging portage because python3 is the default.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>

BUG= chromium:736322 
TEST=cros_sdk --replace --bootstrap
TEST=cros_sdk --replace

Change-Id: I4e30cfe2fcc8ea9ebd03060d2d8bc643480c4ec9
Reviewed-on: https://chromium-review.googlesource.com/583973
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/5124484160c8477bcc0236297cdb7801b0a15cdc/sdk_lib/make_chroot.sh

Owner: athilenius@chromium.org
Project Member

Comment 10 by bugdroid1@chromium.org, Jan 10 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a

commit 01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a
Author: Alec Thilenius <athilenius@chromium.org>
Date: Wed Jan 10 23:44:42 2018

Add Python 3.4.7

Added Python 3.4.7 and eselect.

BUG= chromium:736322 
TEST=sudo emerg dev-lang/python:3.4

Change-Id: I515d35c32ae0f5ff96c71e4ef0854278b5111dec
Reviewed-on: https://chromium-review.googlesource.com/852445
Commit-Ready: Alec Thilenius <athilenius@google.com>
Tested-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/metadata.xml
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/app-eselect/eselect-python/eselect-python-20140125-r1.ebuild
[modify] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/app-eselect/eselect-python/Manifest
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/files/3.4-getentropy-linux.patch
[modify] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/app-eselect/eselect-python/metadata.xml
[modify] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/Manifest
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/files/python-3.4.3-ncurses-pkg-config.patch
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/files/python-3.4.5-cross.patch
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/app-eselect/eselect-python/files/eselect-python-20140125-pythonX.Y-config.patch
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/python-3.4.7.ebuild
[add] https://crrev.com/01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a/dev-lang/python/files/3.6-disable-nis.patch

Project Member

Comment 11 by bugdroid1@chromium.org, Jan 10 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/0dee9b6e7be820ebdddecbd043534aa93d70bdbb

commit 0dee9b6e7be820ebdddecbd043534aa93d70bdbb
Author: Alec Thilenius <athilenius@chromium.org>
Date: Wed Jan 10 23:44:34 2018

Allow multiple versions of Python 3

Allow versions 3.3-3.6 of python to satisfy SDK target.

BUG= chromium:736322 
TEST=sudo emerge target-chromium-os-sdk

Change-Id: Id482333fc6bfcbc803386bd63efc848959ab0b64
Reviewed-on: https://chromium-review.googlesource.com/852444
Commit-Ready: Alec Thilenius <athilenius@google.com>
Tested-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[rename] https://crrev.com/0dee9b6e7be820ebdddecbd043534aa93d70bdbb/virtual/target-chromium-os-sdk/target-chromium-os-sdk-1-r95.ebuild
[modify] https://crrev.com/0dee9b6e7be820ebdddecbd043534aa93d70bdbb/virtual/target-chromium-os-sdk/target-chromium-os-sdk-1.ebuild

Project Member

Comment 12 by bugdroid1@chromium.org, Jan 11 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/63d13a2db24fd852f251d00911ed4d2e7faf0b65

commit 63d13a2db24fd852f251d00911ed4d2e7faf0b65
Author: Alec Thilenius <athilenius@chromium.org>
Date: Thu Jan 11 08:20:46 2018

pyparsing: Allow install with Python 3.4, 3.5, 3.6

Added python 3.4-3.6 to the allowed python versions for pyparsing.

BUG= chromium:736322 
TEST=sudo emerge dev-python/pyparsing

Change-Id: Ib2fbc07ed03cc5807fb78b65955b3482bb405a39
Reviewed-on: https://chromium-review.googlesource.com/861146
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/63d13a2db24fd852f251d00911ed4d2e7faf0b65/dev-python/pyparsing/pyparsing-2.0.1.ebuild

Project Member

Comment 13 by bugdroid1@chromium.org, Jan 12 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/20a735e003f373f878b83d43bc4ecfdfcfc50bc0

commit 20a735e003f373f878b83d43bc4ecfdfcfc50bc0
Author: Jason Clinton <jclinton@chromium.org>
Date: Fri Jan 12 06:22:35 2018

Revert "Add Python 3.4.7"

This reverts commit 01b5a4ebb859fc573fd83e1c2a8d10daed7eab5a.

Reason for revert: file conflict.

Original change's description:
> Add Python 3.4.7
>
> Added Python 3.4.7 and eselect.
>
> BUG= chromium:736322 
> TEST=sudo emerg dev-lang/python:3.4
>
> Change-Id: I515d35c32ae0f5ff96c71e4ef0854278b5111dec
> Reviewed-on: https://chromium-review.googlesource.com/852445
> Commit-Ready: Alec Thilenius <athilenius@google.com>
> Tested-by: Alec Thilenius <athilenius@google.com>
> Reviewed-by: Jason Clinton <jclinton@chromium.org>

Bug:  chromium:736322 
Change-Id: I1413c68b899551f28058077635c2cb6ec0f2ca21
Reviewed-on: https://chromium-review.googlesource.com/861423
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Alec Thilenius <athilenius@google.com>

[modify] https://crrev.com/20a735e003f373f878b83d43bc4ecfdfcfc50bc0/dev-lang/python/metadata.xml
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/app-eselect/eselect-python/eselect-python-20140125-r1.ebuild
[modify] https://crrev.com/20a735e003f373f878b83d43bc4ecfdfcfc50bc0/app-eselect/eselect-python/Manifest
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/dev-lang/python/files/3.4-getentropy-linux.patch
[modify] https://crrev.com/20a735e003f373f878b83d43bc4ecfdfcfc50bc0/app-eselect/eselect-python/metadata.xml
[modify] https://crrev.com/20a735e003f373f878b83d43bc4ecfdfcfc50bc0/dev-lang/python/Manifest
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/dev-lang/python/files/python-3.4.3-ncurses-pkg-config.patch
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/dev-lang/python/files/python-3.4.5-cross.patch
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/app-eselect/eselect-python/files/eselect-python-20140125-pythonX.Y-config.patch
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/dev-lang/python/python-3.4.7.ebuild
[delete] https://crrev.com/739fcf67840732b0f2e3e9e4d8dbff3da1633fae/dev-lang/python/files/3.6-disable-nis.patch

Project Member

Comment 14 by bugdroid1@chromium.org, Jan 18 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/da7135ebe22e5929d78367164090d3e1280ddae0

commit da7135ebe22e5929d78367164090d3e1280ddae0
Author: Alec Thilenius <athilenius@chromium.org>
Date: Thu Jan 18 22:44:37 2018

Add Python 3.4.7

Added Python 3.4.7 and eselect. Added rdepend to remove
app-admin/eselect which broke the sdk builder before.

BUG= chromium:736322 
TEST=sudo emerg dev-lang/python:3.4

Change-Id: I2b181c867c7bea81a4b932474976fc828b5cc45f
Reviewed-on: https://chromium-review.googlesource.com/871042
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/metadata.xml
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/app-eselect/eselect-python/eselect-python-20140125-r1.ebuild
[modify] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/app-eselect/eselect-python/Manifest
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/files/3.4-getentropy-linux.patch
[modify] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/app-eselect/eselect-python/metadata.xml
[modify] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/Manifest
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/files/python-3.4.3-ncurses-pkg-config.patch
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/files/python-3.4.5-cross.patch
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/app-eselect/eselect-python/files/eselect-python-20140125-pythonX.Y-config.patch
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/python-3.4.7.ebuild
[add] https://crrev.com/da7135ebe22e5929d78367164090d3e1280ddae0/dev-lang/python/files/3.6-disable-nis.patch

Updating directly to 3.6 was too difficult so we are currently working our way through stepwise updating to 3.4, 3.5, and then 3.6.

Now that 3.4 is in Portage, the next step would normally be to update PYTHON_TARGETS to PYTHON_TARGETS="python2_7 python3_4". This should cause all python packages that inherit from python-r1 to install site-packages for both 2.9 and 3.4. This works for the host: after changing this, we can ./update_chroot and ~120 packages get rebuilt with support for both successfully.

However, this breaks boards. The reason is that setup_board attempts to update cross-compilers by calling:
crossdev -P --oneshot -P --getbinpkg -P --usepkgonly --without-headers -t arm-none-eabi --binutils '[stable]' --gcc '[stable]' --libc '[stable]' --ex-gdb

This uses --usepkgonly to perform installation of gcc, libc, binutils, and gdb. Because gdb hasn't get gotten a binary package with PYTHON_TARGETS set to enable 3.4, the USE flag's aren't compatible and it fails: https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromiumos.tryserver%2Fno_vmtest_pre_cq%2F209341%2F%2B%2Frecipes%2Fsteps%2FSetupBoard%2F0%2Fstdout .

A work-around is to enable PYTHON_TARGETS for the host only until the updated binary packages are uploaded. This is done in: https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/875263 . Then, come back and try again once we have binary packages. 
That didn't work either: https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromiumos.tryserver%2Fno_vmtest_pre_cq%2F209964%2F%2B%2Frecipes%2Fsteps%2FSetupBoard%2F0%2Fstdout

So, we have a chicken-and-egg problem here. I think we're going to have to kill --usepkgonly for gdb. That doesn't seem that bad: in the worst case--if needed--compiling gdb is quick. We'll leave --usepkgonly enabled for the other 3 things, though.

Project Member

Comment 17 by bugdroid1@chromium.org, Jan 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/998cda1b8e1a72c66779b7d5aa8fa5cdbbd099ea

commit 998cda1b8e1a72c66779b7d5aa8fa5cdbbd099ea
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Sat Jan 20 04:35:45 2018

update_payload: Only depend on backports-lzma for python2_7

This dependency provides a backport of a Python 3.3 module so we don't
need it if we are building for newer Python versions.

BUG= chromium:795837 , chromium:736322 
TEST=./update_chroot with following CL adding 3.4 to PYTHON_TARGETS

Change-Id: I7803ff7a85b407ab548cc23392660fd06ef91649
Reviewed-on: https://chromium-review.googlesource.com/875260
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Lann Martin <lannm@chromium.org>

[modify] https://crrev.com/998cda1b8e1a72c66779b7d5aa8fa5cdbbd099ea/chromeos-base/update_payload/update_payload-9999.ebuild

Project Member

Comment 18 by bugdroid1@chromium.org, Jan 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/78d28d9a3c950ba5f110e265707c6b71559150a8

commit 78d28d9a3c950ba5f110e265707c6b71559150a8
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Sat Jan 20 17:03:12 2018

make_chroot.sh: Don't specify explicit highest Python ver

This will match against whichever slot is the highest.

$ sudo emerge -v --pretend python:2.7 python

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-lang/python-2.7.10-r2:2.7::chromiumos  USE="gdbm
hardened ipv6 ncurses readline sqlite ssl threads wide-unicode xml
-berkdb -build -doc -examples -tk -wininst" 11,977 KiB
[ebuild   R    ] dev-lang/python-3.4.7:3.4/3.4m  USE="gdbm hardened ipv6
ncurses readline sqlite ssl threads xml -build -examples -libressl -tk
-wininst" 0 KiB

Total: 2 packages (2 reinstalls), Size of downloads: 11,977 KiB

BUG= chromium:804038 , chromium:736322 
TEST=./update_chroot succeeds

Change-Id: Ica83ca8a122b69480d0369e6099e58812cdfc11a
Reviewed-on: https://chromium-review.googlesource.com/877405
Commit-Queue: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Trybot-Ready: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Alec Thilenius <athilenius@google.com>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/78d28d9a3c950ba5f110e265707c6b71559150a8/sdk_lib/make_chroot.sh

Project Member

Comment 19 by bugdroid1@chromium.org, Jan 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/f5683a0b8c22accd144d6bb0a5eff525e9942ca0

commit f5683a0b8c22accd144d6bb0a5eff525e9942ca0
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Jan 25 00:29:17 2018

sdk/package.use: Temporarily disable GDB's Python bindings

This is preventing upgrading Python versions from 3.3 to 3.4 because the
toolchain attempts to install GDB from binary-only packages which don't
have a matching USE PYTHON_TARGETS during the upgrade. This will take
Python out of the equation, temporarily.

BUG= chromium:736322 
TEST=./update_chroot succeeds and emerge gdb succeeds

Change-Id: I26f5a14d0f976f735668b0ce05a6b2bb640f83a5
Reviewed-on: https://chromium-review.googlesource.com/881828
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Lann Martin <lannm@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/f5683a0b8c22accd144d6bb0a5eff525e9942ca0/profiles/default/linux/package.use

Project Member

Comment 20 by bugdroid1@chromium.org, Jan 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/643edf440e66be3932a5cc107a1821930bdeb548

commit 643edf440e66be3932a5cc107a1821930bdeb548
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Thu Jan 25 00:29:17 2018

sys-devel/gdb: Uprev GDB to binary-rebuild without Python

We'll uprev one more time after this to turn Python back on.

BUG= chromium:736322 
TEST=./update_chroot and emerge gdb succeeds

Change-Id: Ic4f45cb1efc5b22d9a9649a5d7fdd6a809f44f57
Reviewed-on: https://chromium-review.googlesource.com/881950
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Lann Martin <lannm@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[rename] https://crrev.com/643edf440e66be3932a5cc107a1821930bdeb548/sys-devel/gdb/gdb-8.0.1.20171030-r2.ebuild

The attempt to disable Python for -r2 didn't work. The change that was merged resulted in this USE state at ToT:
http://cs/chromeos_public/src/third_party/chromiumos-overlay/profiles/default/linux/package.use?l=73&rcl=acb9a73aa0750cc44e8fb8333c81a320cf020daf

Yet, when looking at the state on a local build, we see:
$ equery uses cross-arm-none-eabi/gdb-8.0.1.20171030-r2
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for cross-arm-none-eabi/gdb-8.0.1.20171030-r2:
 U I
 + + client                         : Install the main "gdb" program (most people want this)
 - - lzma                           : Support for LZMA (de)compression algorithm
 - - mounted_sources                : <unknown>
 - - multitarget                    : Support all known targets in one gdb binary
 + + nls                            : Add Native Language Support (using gettext - GNU locale utilities)
 - - python                         : Enable support for the new internal scripting language, as well as extended pretty printers
 + + python_single_target_python2_7 : Build for Python 2.7 only
 - - python_single_target_python3_4 : <unknown>
 - - python_single_target_python3_5 : <unknown>
 - - python_single_target_python3_6 : <unknown>
 + + python_targets_python2_7       : Build with Python 2.7
 + + python_targets_python3_4       : Build with Python 3.4
 - - python_targets_python3_5       : <unknown>
 - - python_targets_python3_6       : <unknown>
 + + server                         : Install the "gdbserver" program (useful for embedded/remote targets)
 - - test                           : Workaround to pull in packages needed to run with FEATURES=test. Portage-2.1.2 handles this internally, so don't set it
                                      in make.conf/package.use anymore
 - - vanilla                        : Do not add extra patches which change default behaviour; DO NOT USE THIS ON A GLOBAL SCALE as the severity of the
                                      meaning changes drastically
 - - xml                            : Add support for XML files


When looking at a PreCQ attempt, we see the same failure as before:
https://logs.chromium.org/v/?s=chromeos%2Fbb%2Fchromiumos.tryserver%2Fno_vmtest_pre_cq%2F218813%2F%2B%2Frecipes%2Fsteps%2FSetupBoard%2F0%2Fstdout

Calculating dependencies  ... done!
!!! The following binary packages have been ignored due to non matching USE:
    =cross-arm-none-eabi/gdb-8.0.1.20171030-r2 -python_targets_python3_4
NOTE: The --binpkg-respect-use=n option will prevent emerge
      from ignoring these binary packages if possible.
      Using --binpkg-respect-use=y will silence this warning.
!!! All ebuilds that could satisfy "cross-arm-none-eabi/gdb" have been masked.
!!! One of the following masked packages is required to complete your request:
- cross-arm-none-eabi/gdb-8.0.1.20171030-r2::crossdev (masked by: use flag configuration mismatch)
Root cause seems to be that the ebuild's USE flags are still set even when USE=-python. It looks like the only way to work around this is to rip "python-single-r1" out of the gdb ebuild temporarily.

the USE=-python flag has been disabled.  the PYTHON_*TARGETS should be ignored when USE=-python at the ebuild level, but the binpkg might be throwing a wrench in it.

you can update package.use to also explicitly disable these targets.
sys-devel/gdb -python_targets_python3_4 -python_targets_python3_3 ...
Project Member

Comment 24 by bugdroid1@chromium.org, Feb 9 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/91acde5c72b1a0ddf6dc6da7a74a1fb914a01433

commit 91acde5c72b1a0ddf6dc6da7a74a1fb914a01433
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 09 07:41:33 2018

gdb: Uprev ebuild to pick up newer USE blocks

We tried to use package.use to disable all Python USE flag
incompatibility on binary packages but that didn't work. We've expanded
the list to include the Python eclass variables.

BUG= chromium:736322 
TEST=sudo emerge sys-devel/gdb; ./update_chroot succeeds

Change-Id: Ife27c1f246ffadae021e80fdb8fbeb2bbf6111b2
Reviewed-on: https://chromium-review.googlesource.com/905083
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[rename] https://crrev.com/91acde5c72b1a0ddf6dc6da7a74a1fb914a01433/sys-devel/gdb/gdb-8.0.1.20171030-r3.ebuild

Project Member

Comment 25 by bugdroid1@chromium.org, Feb 9 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/b7f2a365e06b3c2632681ee1278151c8598d0761

commit b7f2a365e06b3c2632681ee1278151c8598d0761
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 09 07:41:32 2018

profiles/default/linux/package.use: Extend gdb python block

This didn't work as expected so we're expanding this and taking into
account the Python eclass USE flags.

BUG= chromium:736322 
TEST=sudo emerge gdb works as expected

Change-Id: I04eecdecdce1e51582908a648646d50ecad94a2c
Reviewed-on: https://chromium-review.googlesource.com/905084
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/b7f2a365e06b3c2632681ee1278151c8598d0761/profiles/default/linux/package.use

The second attempt at a Python USE block for GDB didn't work (after waiting for the SDK to have a new release). Is it possible that the profiles/targets/sdk/package.use is needed to affect the binpkgs that are being created? Or is this perhaps somehow related to https://bugs.chromium.org/p/chromium/issues/detail?id=809312 in that we are not deterministically getting binpkgs from the same location?

Project Member

Comment 27 by bugdroid1@chromium.org, Feb 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/9f598caec2bba89360a0e86b2a6f818c7c6d228f

commit 9f598caec2bba89360a0e86b2a6f818c7c6d228f
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Fri Feb 23 11:36:49 2018

profiles/.../package.use: Migrate block from default to SDK

During unpacking of the SDK, it seems that the SDK profile is taking
higher precedence.

BUG= chromium:736322 
TEST=sudo emerge sys-devel/gdb; ./update_chroot succeeds

Change-Id: I4974ae46533484bf4d455c38621807609b3410d1
Reviewed-on: https://chromium-review.googlesource.com/931754
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/9f598caec2bba89360a0e86b2a6f818c7c6d228f/profiles/targets/sdk/package.use
[modify] https://crrev.com/9f598caec2bba89360a0e86b2a6f818c7c6d228f/profiles/default/linux/package.use

Cc: bcf@chromium.org llozano@chromium.org
 Issue 817570  has been merged into this issue.
Cc: cmt...@chromium.org
Components: Tools>ChromeOS-Toolchain
What I think are the last two changes for enabling Python 3.4 in the SDK are out for review:
https://chromium-review.googlesource.com/c/chromiumos/overlays/portage-stable/+/966495
https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/875263

After that, we'll just need to do the same make.default change for the general profile to open it up for all boards.

Project Member

Comment 31 by bugdroid1@chromium.org, Mar 21 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/e9bc948cd49e65ebc72bd0c4f99f4f5a0703b8d2

commit e9bc948cd49e65ebc72bd0c4f99f4f5a0703b8d2
Author: Mike Frysinger <vapier@chromium.org>
Date: Wed Mar 21 02:30:13 2018

python: disable build overrides for the sdk itself

We deployed this var override so we can cross-compile python packages,
but it's not playing well with multitarget python code (since this func
runs once, but the python probes multiple versions).  Drop it for the
sdk since the reason for the workaround isn't necessary, and we want to
enable multiple versions of python there.

We'll need to revisit this if we want to do multiple versions of python
in the board, but we also need to do custom patches on python 3.x first.

BUG= chromium:736322 
TEST=`emerge dbus-python` works correctly for the sdk

Change-Id: Ie24782028f2a00700cc4bb5c28f2010d0d14ad7d
Reviewed-on: https://chromium-review.googlesource.com/969903
Commit-Ready: Mike Frysinger <vapier@chromium.org>
Tested-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>

[modify] https://crrev.com/e9bc948cd49e65ebc72bd0c4f99f4f5a0703b8d2/profiles/base/profile.bashrc

Project Member

Comment 33 by bugdroid1@chromium.org, Mar 24 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/1e84c33ad6323a2b112d9a99484b4dfd87c80cfa

commit 1e84c33ad6323a2b112d9a99484b4dfd87c80cfa
Author: Jason D. Clinton <jclinton@chromium.org>
Date: Sat Mar 24 19:31:31 2018

make.defaults: Add python3_4 to PYTHON_TARGETS

This will cause ebuilds that support multiple Python versions to try
building for both python2_7 and python3_4. Ebuilds that support a single
version will continue to use python2_7.

BUG= chromium:736322 
TEST=./update_chroot; verified that ~122 packages were rebuilt with
python3_4 support.
CQ-DEPEND=CL:978466

Change-Id: I9a1121d441164c97d4ee32e21a55bb44580b8556
Reviewed-on: https://chromium-review.googlesource.com/875263
Commit-Ready: Jason Clinton <jclinton@chromium.org>
Tested-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Jason Clinton <jclinton@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/1e84c33ad6323a2b112d9a99484b4dfd87c80cfa/profiles/targets/sdk/make.defaults

PYTHON_TARGETS now has Python 3.4 enabled in the SDK which means that any package which targets the host chroot and has python-r1.eclass multi-Python support will compile Python 2.7 and 3.4 extensions. 

PYTHON_TARGETS changes is not yet enabled for any board: we think that there's significant churn related to cross-compiling to ARM/ARM64 remaining. Individual boards that wish to turn it on can do so and see if it works for them.

In the meantime, Meson can be built for the SDK. There's a CL out to enable that: https://chromium-review.googlesource.com/c/chromiumos/overlays/chromiumos-overlay/+/871977 .

We should get Python 3.4 up and happy on the boards before proceeding to the next step: Python 3.5.

can we re-enable python support in gdb now ?
Project Member

Comment 36 by bugdroid1@chromium.org, May 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/23b50aa4e47d0360bbd6ce89e21272f1c887723c

commit 23b50aa4e47d0360bbd6ce89e21272f1c887723c
Author: Andrew Bresticker <abrestic@waymo.com>
Date: Wed May 23 19:50:46 2018

dev-lang/python: Move python-3.4.7 to chromiumos-overlay

python-3.4.7 does not currently cross-compile. Move the ebuild over to
chromiumos-overlay so that we can hack on it.

BUG= chromium:736322 
TEST=sudo emerge =dev-lang/python-3.4.7

Change-Id: I2420898b193c054882eb6ae372c50732526bc8f5
Signed-off-by: Andrew Bresticker <abrestic@waymo.com>
Reviewed-on: https://chromium-review.googlesource.com/1069250
Commit-Ready: Andrew Bresticker <abrestic@chromium.org>
Tested-by: Andrew Bresticker <abrestic@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[add] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/files/3.4-getentropy-linux.patch
[modify] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/Manifest
[add] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/files/python-3.4.3-ncurses-pkg-config.patch
[modify] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/metadata.xml
[add] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/files/python-3.4.5-cross.patch
[add] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/python-3.4.7.ebuild
[add] https://crrev.com/23b50aa4e47d0360bbd6ce89e21272f1c887723c/dev-lang/python/files/3.6-disable-nis.patch

Project Member

Comment 37 by bugdroid1@chromium.org, May 23 2018

Project Member

Comment 38 by bugdroid1@chromium.org, May 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/6d39acc9c376001ed31de368ee2d31c75fd3fc83

commit 6d39acc9c376001ed31de368ee2d31c75fd3fc83
Author: Andrew Bresticker <abrestic@waymo.com>
Date: Fri May 25 19:28:22 2018

dev-lang/python: Patches for cross-compiling python-3.4.7

The changes necessary to cross-compile python-3.4.7 are similar to that
for python-2.7.10, specifically:
  - bring forward the "ChromiumOS specific changes" from the python-2.7.10
    ebuild, including the sed script for Lib/site.py
  - rebase the cross-h2py, cross-hack-compiler, cross-setup-sysroot, and
    ldshared patches on python-3.4.7 without conflicts
  - rebase the cross-distutils patch on python-3.4.7, but dropping the part
    patching _init_posix() since that no longer applies
  - drop the unique-semaphore-name patch since it has been fixed upstream

With these changes it is now possible to cross-compile python-3.4.7, at
least for amd64 targets.

BUG= chromium:736322 
TEST=emerge-metis =dev-lang/python-3.4.7

Change-Id: Id3820a36845dccbbcf7b8e6b45d477014542ba48
Signed-off-by: Andrew Bresticker <abrestic@waymo.com>
Reviewed-on: https://chromium-review.googlesource.com/1069696
Commit-Ready: Andrew Bresticker <abrestic@chromium.org>
Tested-by: Andrew Bresticker <abrestic@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[add] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/files/python-3.4.7-cross-setup-sysroot.patch
[add] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/python-3.4.7-r1.ebuild
[add] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/files/python-3.4.7-cross-hack-compiler.patch
[add] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/files/python-3.4.7-cross-distutils.patch
[add] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/files/python-3.4.7-cross-h2py.patch
[modify] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/python-3.4.7.ebuild
[add] https://crrev.com/6d39acc9c376001ed31de368ee2d31c75fd3fc83/dev-lang/python/files/python-3.4.7-ldshared.patch

Status: Fixed (was: Started)
I believe this is done? It's been a long time since I've looked at cros-sdk stuff.
Blocking: 884766
we made it to python-3.4.  i've filed a new bug to get us to python-3.6.
Blocking: 898728
Is this bug still open?  If it is, python3.5 would help build gfx libs significantly faster.
Cc: chadversary@chromium.org
I'm confused. The bug is closed; it's title is "uprev to 3.5.3+"; but the chroot does not yet have 3.5. What's up?
i think the team ran out of steam after the troubles they saw with just 3.3->3.4 transition.  lets just pivot to issue 884766 and see if we can't do a 3.4->3.6 jump.
Project Member

Comment 45 by bugdroid1@chromium.org, Jan 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/09ad4fa9ad8764094fd75875b5a5e1788a9fcc39

commit 09ad4fa9ad8764094fd75875b5a5e1788a9fcc39
Author: Chris McDonald <cjmcdonald@chromium.org>
Date: Wed Jan 16 09:46:56 2019

chromiumos-overlay: Delete old workaround code for gdb python

These packages don't exist in the tree anymore, so we definitely don't
need a workaround for a bug involving them.

BUG= chromium:736322 
TEST=`emerge sys-devel/gdb`

Change-Id: Iaceb5bf3f15f6c4f07af1d45510a03f61b229645
Reviewed-on: https://chromium-review.googlesource.com/1412449
Commit-Ready: Chris McDonald <cjmcdonald@chromium.org>
Tested-by: Chris McDonald <cjmcdonald@chromium.org>
Reviewed-by: Chirantan Ekbote <chirantan@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/09ad4fa9ad8764094fd75875b5a5e1788a9fcc39/profiles/targets/sdk/package.use

Sign in to add a comment