New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 736313 link

Starred by 1 user

Issue metadata

Status: Started
Owner:
Last visit > 30 days ago
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug

Blocking:
issue 649672



Sign in to add a comment

Uprev dev-lang/python:2.7 to 2.7.13

Project Member Reported by djkurtz@chromium.org, Jun 23 2017

Issue description

chromiumos-overlay currently has a custom modified version of python-2.7.10.

Upstream python 2.7.13 is the latest in the 2.7 series, and includes fixes for OpenSSL 1.1:

c2fc7c4f53  Issue #26470 : Port ssl and hashlib module to OpenSSL 1.1.0.
 
One complication for updating python 2.7 to 2.7.13 is the upstream patch [0] added in 2.7.10-r2 [1], but never applied by chromium's 2.7.10 ebuild.

[0] https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-lang/python/files/python-2.7.10-cross-compile-warn-test.patch

[1] https://gitweb.gentoo.org/repo/gentoo.git/commit/dev-lang/python?id=d3f5509e59f54cf3f9b27886aa508ba3e2c6e0b0

This was added to address https://bugs.python.org/issue25397.

However, it doesn't quite work ;-) - because the original configure check only approximately worked in the first place.

The first hunk of that patch changes the CFLAGS used during the "Check whether GCC supports PyArg_ParseTuple format" from  "-Werror -Wformat" to "-Werror=format".  This restricts the test to compile fail iff a "-Wformat" warning is generated when building the code; however other warnings are permitted:

void f(char*,...)__attribute((format(PyArg_ParseTuple, 1, 2)));

In fact, it the actual GCC warning generated by this code when "PyArg_ParseTuple" is not a supported attribute argument is "-Wignored-attributes", not "-Wformat":

x86_64-pc-linux-gnu-clang -pthread -c -fno-strict-aliasing -OPT:Olimit=0 -O2 -pipe -fwrapv -DNDEBUG   -I. -IInclude -I/var/tmp/portage/dev-lang/python-2.7.13-r2/work/Python-2.7.13/Include  -fPIC -DPy_BUILD_CORE -o Python/symtable.o /var/tmp/portage/dev-lang/python-2.7.13-r2/work/Python-2.7.13/Python/symtable.c
In file included from /var/tmp/portage/dev-lang/python-2.7.13-r2/work/Python-2.7.13/Modules/symtablemodule.c:1:
In file included from /var/tmp/portage/dev-lang/python-2.7.13-r2/work/Python-2.7.13/Include/Python.h:126:
/var/tmp/portage/dev-lang/python-2.7.13-r2/work/Python-2.7.13/Include/modsupport.h:27:65: warning: 'format' attribute argument not
      supported: PyArg_ParseTuple [-Wignored-attributes]
PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
                                                                ^
/var/tmp/portage/dev-lang/python-2.7.13-r2/work/Python-2.7.13/Include/pyport.h:917:57: note: expanded from macro 'Py_FORMAT_PARSETUPLE'
#define Py_FORMAT_PARSETUPLE(func,p1,p2) __attribute__((format(func,p1,p2)))
                                                        ^

Note: we actually build python WITHOUT -Werror, though, so building python-2.7.13 appears to work.  The problem manifests itself later when building a C python module with CFLAGS=-Werror, for example:

$ emerge-hana chromeos-base/autotest-deps-p2p
...
* INFO:root:setup myfaketest.
DEBUG:root:Running 'make -j48 '
DEBUG:root:[stdout] python setup.py build
DEBUG:root:[stdout] running build
DEBUG:root:[stdout] running build_py
DEBUG:root:[stdout] creating build
DEBUG:root:[stdout] creating build/lib.linux-x86_64-2.7
DEBUG:root:[stdout] creating build/lib.linux-x86_64-2.7/lansim
DEBUG:root:[stdout] copying py/__init__.py -> build/lib.linux-x86_64-2.7/lansim
DEBUG:root:[stdout] copying py/host.py -> build/lib.linux-x86_64-2.7/lansim
DEBUG:root:[stdout] copying py/simulator.py -> build/lib.linux-x86_64-2.7/lansim
DEBUG:root:[stdout] copying py/tools.py -> build/lib.linux-x86_64-2.7/lansim
DEBUG:root:[stdout] copying py/tuntap.py -> build/lib.linux-x86_64-2.7/lansim
DEBUG:root:[stdout] running build_ext
ERROR:root:[stderr] warning: build_py: byte-compiling is disabled, skipping.
ERROR:root:[stderr] 
DEBUG:root:[stdout] building 'pyiftun' extension
DEBUG:root:[stdout] creating build/temp.linux-x86_64-2.7
DEBUG:root:[stdout] armv7a-cros-linux-gnueabi-clang -O2 -O2 -pipe -march=armv8-a+crc -mtune=cortex-a57.cortex-a53 -mfpu=crypto-neon-fp-armv8 -mfloat-abi=hard -g -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -clang-syntax -fPIC -I/build/hana/usr/include/python2.7 -c pyiftun.c -o build/temp.linux-x86_64-2.7/pyiftun.o -O2 -Wall -Werror
ERROR:root:[stderr] In file included from pyiftun.c:14:
ERROR:root:[stderr] In file included from ../../../../../../../../../../usr/include/python2.7/Python.h:126:
ERROR:root:[stderr] ../../../../../../../../../../usr/include/python2.7/modsupport.h:27:65: error: 'format' attribute argument not supported: PyArg_ParseTuple [-Werror,-Wignored-attributes]
ERROR:root:[stderr] PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
ERROR:root:[stderr]                                                                 ^
ERROR:root:[stderr] ../../../../../../../../../../usr/include/python2.7/pyport.h:917:57: note: expanded from macro 'Py_FORMAT_PARSETUPLE'
ERROR:root:[stderr] #define Py_FORMAT_PARSETUPLE(func,p1,p2) __attribute__((format(func,p1,p2)))
ERROR:root:[stderr]                                                         ^
ERROR:root:[stderr] 1 error generated.
ERROR:root:[stderr] error: command 'armv7a-cros-linux-gnueabi-clang' failed with exit status 1
ERROR:root:[stderr] make: *** [Makefile:4: all] Error 1
ERROR:root:Command <make -j48 > failed, rc=2, Command returned non-zero exit status

python updates (2.7.10 -> 2.7.13 & 3.3 -> 3.6) are interdependent due to shared dependencies (eselect-python & python-exec), so must go in together.

CLs:
https://chromium-review.googlesource.com/580218 chromeos-base/dev-install: Do not install /usr/local/bin/python symlink        
https://chromium-review.googlesource.com/580219 media-libs/vulkan-loader: Add support for python 3.6        
https://chromium-review.googlesource.com/580220 virtual/target-chromium-os-sdk: Update python:3 RDEPEND to 3.6        
https://chromium-review.googlesource.com/580221 dev-lang/python: update to 2.7.13        
https://chromium-review.googlesource.com/580450 BACKPORT: python-utils-r1.eclass: Enable python3_6 target        
https://chromium-review.googlesource.com/580451 eselect-python, python-exec, python3: Uprev to latest from gentoo        
https://chromium-review.googlesource.com/574040 eselect-python: Move from app-admin to app-eselect        

Project Member

Comment 3 by bugdroid1@chromium.org, Jul 22 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/crosutils/+/fbe4aece95869343c4e4d4814cbb5ee54930fa48

commit fbe4aece95869343c4e4d4814cbb5ee54930fa48
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Sat Jul 22 09:46:02 2017

make_chroot: Do not update python

CL:176050 ("Support Python 2.7 in chroot bootstrap process.") forced
a python update after the portage upgrade when making the chroot.

At the time (2013-11-07), the BOOTSTRAP was "2013.01.30", which used
python 2.6, so this python update was needed to let parallel_emerge and
other chromite utilities use python 2.7 features.

CL:222090 ("sdk: bootstrap from newer Gentoo stage3") (2014-09-24) updated
the BOOSTRAP to its current version "2014.09.18" (yes, its almost 3 years
old).  This version already includes python 2.7.7 in it, so the
make_chroot time explicit python update is no longer required.  Instead,
the python update can be deferred to the update_chroot section, which
updates the entire sdk with full dependency resolution.

The motivation for removing this python update during make_chroot is that
in 1Q-2015 upstream gentoo moved eselect-python from app-admin to
app-eselect.

The "2014.09.18" stage3 still had app-admin/eselect-python, and therefore
its python2 and python3 ebuilds both depend on this location.  If we
move eselect-python to app-eselect and update the two python ebuilds,
but only emerge python2, the emerge is blocked due to the existing python3
still pulling in the old app-admin/eselect-python.

Also remove an old comment about the python2.* paths only existing after
the python update, since those paths do exist in the current BOOTSTRAP.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>

BUG=chromium:736313, chromium:736322 
TEST=cros_sdk --replace
TEST=cros_sdk --replace --bootstrap
 => make_chroot completes successfully in both cases
TEST=./setup_board --board=hana
TEST=./build_packages --board=hana
TEST=./build_image --board=hana test
TEST=test_that suite:bvt-cq

Change-Id: Ifa122ca72af3aa9df5a64be9c800e8ecf74a9cbf
Reviewed-on: https://chromium-review.googlesource.com/581307
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[modify] https://crrev.com/fbe4aece95869343c4e4d4814cbb5ee54930fa48/sdk_lib/make_chroot.sh

Blockedon: 747810
Blockedon: -747810
Project Member

Comment 6 by bugdroid1@chromium.org, Jul 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/d47b292a9f4e85795baf8a02fbd133c6540d4c2d

commit d47b292a9f4e85795baf8a02fbd133c6540d4c2d
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Thu Jul 27 17:28:23 2017

dev-lang/python: Use eselect-python from app-eselect

In 1Q-2015 upstream gentoo moved eselect-python from app-admin to
app-eselect.  We are now doing the same in our portage-stable repo, so
adjust corresponding chromiumos-overlay depenendencies.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>

BUG=chromium:736313
TEST=emerge-soraka python
  => Python 2.7 builds ok
CQ-DEPEND=CL:574040,CL:583973

Change-Id: Ifb5d19dcf8ee24981a1881c234b1ebd46ced8de9
Reviewed-on: https://chromium-review.googlesource.com/573885
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[rename] https://crrev.com/d47b292a9f4e85795baf8a02fbd133c6540d4c2d/dev-lang/python/python-2.7.10-r2.ebuild
[modify] https://crrev.com/d47b292a9f4e85795baf8a02fbd133c6540d4c2d/dev-lang/python/python-2.7.10.ebuild

Project Member

Comment 7 by bugdroid1@chromium.org, Jul 27 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/1cbf7c903b52abc77004206400ea95aec43b9d1f

commit 1cbf7c903b52abc77004206400ea95aec43b9d1f
Author: Daniel Kurtz <djkurtz@chromium.org>
Date: Thu Jul 27 17:28:24 2017

eselect-python: Move from app-admin to app-eselect

In 1Q-2015 upstream gentoo moved eselect-python from app-admin to
app-eselect.  Do the same now to our portage-stable repo.

Note 1: We add an additional blocker here to force the move for incremental
builds where app-admin/eselect-python is already installed.

Note 2: Also update the only existing dependency, in python-3, and revbump
its ebuild.

The only other eselect-python DEPEND in portage-stable, in
eclass/python.eclass, was already using app-eselect/ ...

The CQ-DEPEND'ed CL:573885 in chromiumos-overlay CL fixes up the
eselect-python DEPEND in the python2.7 ebuild.

Signed-off-by: Daniel Kurtz <djkurtz@chromium.org>

BUG=chromium:736313
TEST=In chroot w/ app-admin/eselect-python installed:
  sudo emerge eselect-python
   => app-admin/eselect-python is uninstalled and
      app-eselect/eselect-python is installed
TEST=In chroot w/ app-admin/eselect-python installed:
  ./build_packages --board=soraka
  => uninstalls app-admin/eselect-python,
     installs app-eselect/eselect-python, and
     builds packages without conflicts
CQ-DEPEND=CL:573885,CL:581307,CL:583973

Change-Id: I0ac2da28e425dd31c16756f17729fa988e6374b5
Reviewed-on: https://chromium-review.googlesource.com/574040
Commit-Ready: Daniel Kurtz <djkurtz@chromium.org>
Tested-by: Daniel Kurtz <djkurtz@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[rename] https://crrev.com/1cbf7c903b52abc77004206400ea95aec43b9d1f/app-eselect/eselect-python/Manifest
[rename] https://crrev.com/1cbf7c903b52abc77004206400ea95aec43b9d1f/app-eselect/eselect-python/eselect-python-20140125.ebuild
[rename] https://crrev.com/1cbf7c903b52abc77004206400ea95aec43b9d1f/app-eselect/eselect-python/metadata.xml
[rename] https://crrev.com/1cbf7c903b52abc77004206400ea95aec43b9d1f/dev-lang/python/python-3.3.5-r2.ebuild

Sign in to add a comment