New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 698478 link

Starred by 2 users

Issue metadata

Status: Verified
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

[aarch64 only] component_UpdateFlash fails to download component from omaha

Project Member Reported by ihf@chromium.org, Mar 4 2017

Issue description

component_UpdateFlash always fails on elm, hana, kevin, oak with
Failed: download-omaha-component.

The one thing these boards have in common is 64 kernel and 32 bit user space. We have 32 bit arm Flash and serve it correctly. A wild guess would be that we request 64 bit arm binaries from Omaha here and it correctly fails to serve them to us. I just can't find where we get the architecture from.

I don't think this is important for M-57.
 
I was not aware that we ship 64 bit arm chromebooks. Is this a new thing?
Cc: waff...@chromium.org
The architecture ought to be the architecture of Chrome, not the platform. Can you capture a request/response pair using chrome://net-internals? Look for traffic to https://clients2.google.com/service/update2, and you'll have to check "Include the actual bytes sent/received" in the Capture tab to get the XML request/response bodies.

Alternatively, if you can launch one of the boards with the flag --component-updater=url-source=http://foo.bar:port and can set up netcat or something at foo.bar on port, you can use that to grab the request, which I can translate into a response.

Comment 4 by ihf@chromium.org, Mar 6 2017

It is a new-ish thing (last half year). They are mixed 64 bit kernel and 32 bit user space as ARM Flash is only available in 32 bit. I should have an oak somewhere to check the response.
Cc: vsu...@chromium.org hsiangc@chromium.org

Comment 6 by ihf@chromium.org, Apr 1 2017

I finally managed to setup a build. Nothing obvious yet. More next week.

Comment 7 by ihf@chromium.org, Apr 4 2017

Status: Started (was: Untriaged)
Some observations.

The chromium architecture is properly identified in 
chrome/common/pepper_flash.cc
components/update_client/update_query_params.cc

Strangely about:components doesn't recognize the built-in Flash. about:flash does though. Also about:components does manage to update the CRLSet, so it is not fully dead.

Comment 8 by ihf@chromium.org, Apr 4 2017

correct architecture of chrome = arm (32 bit)

Comment 9 by ihf@chromium.org, Apr 5 2017

It looks like Omaha responds serves Flash based on the OS architecture, not the Chrome architecture. <os platform="Chrome OS" arch="arm" gets us a download, <os platform="Chrome OS" arch="aarch64" does not.

The question is, should Omaha know that Flash needs the Chrome architecture, not OS architecture. I guess it must have known this at some point, as Windows, Linux etc. were mixed. But maybe it doesn't, as we now have our own app id and hence the old check doesn't work anymore?

Alternatively I could upload the arm binary also for aarch64 OS. That could fix this without a Omaha server change, but while we never expect to get a true aarch64 binary that still seems a little iffy.


--- a/components/update_client/utils.cc
+++ b/components/update_client/utils.cc
@@ -156,7 +156,7 @@ std::string BuildProtocolRequest(
   base::StringAppendF(
       &request, "<os platform=\"%s\" arch=\"%s\"",
       os_long_name.c_str(),                                    // "platform"
-      base::SysInfo().OperatingSystemArchitecture().c_str());  // "arch"
+      "arm");   // was "aarch64"


request=<?xml version="1.0" encoding="UTF-8"?><request protocol="3.0" version="chrome-59.0.3056.0" prodversion="59.0.3056.0" requestid="{d722a39a-9720-4ffc-b30f-a828e0377254}" lang="en-US" updaterchannel="unknown" prodchannel="unknown" os="cros" arch="arm" nacl_arch="arm"><hw physmemory="4"/><os platform="Chrome OS" arch="arm" version="3.18.0-14507-gc0b917c"/><app appid="ckjlcfmdbdglblbjglepgnoekdnkoklc" version="0.0.0.0" nextversion="25.0.0.148"><event eventtype="3" eventresult="1" nextfp="1.61276eab2b85d268ecf1dd5e35b637236b376ab0532e94bbae19ccea9ed38bff"/><event eventtype="14" eventresult="1" downloader="direct" url="http://redirector.gvt1.com/edgedl/release2/f4Nk_vJYev0/25.0.0.148_chromeos_arm32_PepperFlashPlayer.crx" downloaded="12668528" total="12668528" download_time_ms="607"/></app></request>

Comment 10 by ihf@chromium.org, Apr 5 2017

Cc: songbj@chromium.org
I checked the Omaha sources but could not find any 32 bit browser on 64 bit OS special casing for Chrome. Maybe that never existed or already got removed. Adding songbj in case she knows, but this can wait for waffles to return.

I ran component_UpdateFlash autotest 10 times overnight on oak with the change in 9 and it passed 9 times with 1 unrelated flake. So once Omaha servers arm Flash to aarch64 devices everything should work just fine.

Comment 11 by ihf@chromium.org, Apr 25 2017

Joshua, could you please take a look? It seems Omaha needs to change its logic to serve Flash binaries based on the Chrome bits, not the OS bits.
Thanks for the ping.

The config is sensitive to both the arch of the underlying platform and of Chrome (transmitted in <os arch> and <request arch>, respectively). The issue is that the code doesn't recognize "aarch64" as a legitimate system architecture (this was simply an oversight in the original implementation). The fix should be easy, but just to make sure:

1 • Will it work for you if we assert that arm64 == aarch64, or do you need to maintain the distinction for any reason? (I'm pretty sure LLVM at least treats them as almost identical...)
2 • Do we need to recognize aarch32 as an alias for "arm" (which we treat as 32-bit) / "armv7"?

Comment 13 by ihf@chromium.org, Apr 25 2017

Cc: vapier@chromium.org adlr@chromium.org llozano@chromium.org
+some people who might be interested in component updates and compiler architecture

I think 1) is fine, at least for Flash purposes. As for 2) I don't know. It sounds better, but maybe Luis or Mike have an insight.


arm64 is the same as aarch64.  arm64 is simply the marketing name that ARM tries to get people to use instead of the underlying ISA name of aarch64.  there is no harm in equating them everywhere.  unless ARM pulls something stupid in the future and tries renaming things :).

i don't think aarch32 shows up anywhere, so i wouldn't worry about it.  equating arm and armv7 is a bit dicey in general (since there is armv[45678]), but it probably doesn't matter for our purposes since we have no Chrome OS device today that is older than armv7 and most likely never will.  and if we don't do any diff builds between armv7 and armv8, it's fine.

Comment 15 by ihf@chromium.org, Apr 29 2017

Owner: waff...@chromium.org
Status: Verified (was: Started)
Joshua, thanks for fixing! bob, elm, hana, kevin and oak have passed the test now in the lab.

Comment 16 by ihf@chromium.org, Apr 29 2017

Owner: ihf@chromium.org
Status: Started (was: Verified)
I am very confused though. It might be Intel got broken. No idea how. Maybe it just had a bad day. I will look at the test again on Monday.
I just looked at the test and there is an awful lot of yellow right now and all weekend. I can try and repro tomorrow.
Sorry I was looking at platform_ImageLoaderServer. Not the same test, so I have a separate issue to look into.

Comment 19 by ihf@chromium.org, May 1 2017

The M58&M59 branches are green, which means Omaha works.

There is a double regression, Intel on 9503, ARM now on 9505 (which was a Chrome uprev, there was also a recent toolchain uprev). We need a new issue for this.
If its only broken on ToT I think this is the imageloader crash seen in the platform_ImageLoaderServer tests. 

kernel: [  131.021179] init: imageloader main process (3637) killed by ABRT signal

My guess is that one of the recent imageloader CLs to land is missing a syscall in the seccomp filter.
Greg has created  issue 717030  to track the imageloader autotest failure

Comment 22 by ihf@chromium.org, May 8 2017

Status: Verified (was: Started)
I need to add expectations for veyron_rialto and guado/moblab which don't have Flash. Otherwise green.
Labels: -Restrict-View-Google

Sign in to add a comment