MOV link

https://drive.google.com/open?id=1_sDN9CNMBOE4m853dRrjpb91pgC91P2e

As requested by kbleicher@, marking this as stable blocker. 

Yes, requested the label to ease evaluation.

Can we get this confirmed / triaged / assigned?

Does this reproduce for all ARC++ apps (if not, the app in question might be useful information)?

I can see some crash reports from ARC:
1a1a0da26d97b8d3, de5af5e5b70dd981
both complaining about a non protected broadcast

Sergey, do you have any ideas about Kiosk start up?
In the log message, I found several Chrome sig11 errors. Are they related?

That's hard to guest what's happening on Android side without logcat logs. Are they available anywhere?
Also, it would be great to add "vmodule=arc_kiosk*=3" into /etc/chrome_dev.conf to enable enhanced logging about starting ARC++ Kiosk.

From the video it looks like the Android kiosk app was started and because of that the "Initializing application" splash screen was closed, but the app's UI didn't appear.

What is the app that is expected to be launched?

I assume this isn't limited to Veyron-Fievel?  Do we have a list of impacted board(s) or is this assumed for all that support ARC++?  

I will test on other veyron devices to see If I am able to reproduce this issue.

Passing to poromov@ since this ARC kiosk.

As hidehiko@ pointed out in #14, looks like chrome crashed in sig11, which might be the cause of the black screen. Not sure where we crashed tho.

In the mean time, logcat would be helpful. When you repro the problem, ssh into the device and run

android-sh -c 'logcat'

to collect the logcat log.

Any luck with the logcat? Could you run the command while the device is in the black screen?

Minor update of the command, add a "-d" so that logcat exits after dumping.
i.e.
  android-sh -c 'logcat -d'

Quick update.

I am unable to reproduce the issue consistently at this time. Out of 6-7 times I was able to successfully reproduce this issue once time morning.
 

Below are the logs for the one time I was able to reproduce the issue. I was able edit the conf file with the following line  "vmodule=arc_kiosk*=3" before capturing the logs. see attached

Deleted: debug-logs_20180206-103104.tgz 84.5 KB

debug-logs_20180206-103104.tgz 84.5 KB Download

I was able to reproduce this Issue on the Veyron-Tiger device.

I was not able to grab the android logs due to the following shell statements.

find: '/run/containers/android/' : no such file or container
/usr/sbin/android-sh: Container PID file not found, is the container running?

This means the android container is not running, probably due to chrome crash.

Any luck getting a coredump for the chrome crash? See https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/debugging-tips#TOC-Enabling-core-dumps

Crash from this morning

Deleted: core.chrome.1421.tar 7.7 MB

core.chrome.1421.tar 7.7 MB Download

another core dump

https://drive.google.com/file/d/1R2xFiDXC2Wu-Mpw0S8brKbDsr6rA1uKC/view?usp=sharing

It could be possible to get android logs until the crash by running "android -sh logcat" from the very beginning of the session - as splash screen is shown.

Also, inside the session Chrome logs "arc_kiosk*" logs into /home/chronos/user/log that is missing in all attached archives.

vaandres@, could you also add the board name and chromeos versions (platform version) for the crash dump. We need that info to know which symbol file to use to look at the coredump.

Board Name - Veyron Tiger
Chrome OS 10176.68.0 64.0.3282.144

attached are the log files from /home/chronos/user/log


I will try and get the android logs.

Deleted: userLog.tar 50.0 KB

userLog.tar 50.0 KB Download

vaandres@, are those two dumps the only ones you got? Do you see others under /var/coredumps after seeing the black screen?

The twos crashes are gpu process SIGABRT, probably caused by the chrome browser crash. Not the root cause.

Can this be related to https://crash.corp.google.com/browse?q=reportid=%271a1a0da26d97b8d3%27  (the logs from the bug report indicate the device uploaded this report, alongside identical report with ID de5af5e5b70dd981)

Yes there are more logs, I am pulling them out and will post a download link when they are available 

attached are all the logs inside the pulled from the following directory. The files are quite large

/mnt/stateful_partition/encrypted/var/coredumps

https://drive.google.com/file/d/1P_g_T2SeudSn2P_Ue6HotNI18zRO3M0T/view?usp=sharing

Thanks for coredumps. The big ones (400+M) are for the browser process. The crash stack are the same. However, the crash happens during chrome shutdown when destructing a exo::ShellSurface (probably due to android side problem). We need to figure out why chrome decides to shutdown. And only chrome user log and android logcat would help.

vaandres@, could you add "--disable-logging-redirect" to /etc/chrome_dev.conf to disable chrome user log redirection and try repro? This switch makes chrome to dump all logs to /var/log/chrome and hopefully it will give us some hints. For the android logcat, can you run "android-sh -c 'logcat'" while sitting on the login screen then manually start the android kiosk app to repro?


+oshima for for ClientControlledState::HandleTransitionEvents crash during shutdown.
e.g.
core.chrome.1164:
====
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0xed652016 in ?? ()
[Current thread is 1 (LWP 1164)]
(gdb) bt
#0  0xed652016 in ?? ()
#1  0x0ab1eca8 in ash::wm::ClientControlledState::HandleTransitionEvents (this=0x11bea6c0, window_state=0x129ddaf0, event=0xbe997b2c)
    at ../../../../../../../home/chrome-bot/chrome_root/src/ash/wm/client_controlled_state.cc:48
#2  0x0ab38f32 in ash::wm::WindowState::OnWMEvent (event=0xd5b529c <vtable for ash::wm::WMEvent+8>, this=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/ash/wm/window_state.cc:275
#3  ash::wm::WindowState::Restore (this=0x129ddaf0) at ../../../../../../../home/chrome-bot/chrome_root/src/ash/wm/window_state.cc:239
#4  0x0a1ffc9a in aura::Window::RemoveChildImpl (this=0xf8433c0, child=0xf7d2c00, new_parent=0x0) at ../../../../../../../home/chrome-bot/chrome_root/src/ui/aura/window.cc:845
#5  0x0a1feb38 in aura::Window::RemoveChild (this=0xf8433c0, child=0xf7d2c00) at ../../../../../../../home/chrome-bot/chrome_root/src/ui/aura/window.cc:408
#6  0x0a1fe87c in aura::Window::~Window (this=0xf7d2c00) at ../../../../../../../home/chrome-bot/chrome_root/src/ui/aura/window.cc:125
#7  0x0a1febf6 in aura::Window::~Window (this=0x12996ae0) at ../../../../../../../home/chrome-bot/chrome_root/src/ui/aura/window.cc:80
#8  0x0a486caa in views::Widget::CloseNow (this=0x1004aa90) at ../../../../../../../home/chrome-bot/chrome_root/src/ui/views/widget/widget.cc:607
#9  0x08a67464 in exo::ShellSurfaceBase::~ShellSurfaceBase (this=0x103c9180) at ../../../../../../../home/chrome-bot/chrome_root/src/components/exo/shell_surface_base.cc:315
#10 0x08a5d576 in exo::ClientControlledShellSurface::~ClientControlledShellSurface (this=0x12996ae0)
    at ../../../../../../../home/chrome-bot/chrome_root/src/components/exo/client_controlled_shell_surface.cc:184
#11 0x0abbcba0 in destroy_resource (element=0x128d5a40, data=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/wayland/src/src/wayland-server.c:677
#12 0x0abbbe74 in for_each_helper (entries=0x1206bbf0, func=<optimized out>, data=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/wayland/src/src/wayland-util.c:374
#13 wl_map_for_each (map=0x1206bbf0, func=<optimized out>, data=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/wayland/src/src/wayland-util.c:387
#14 0x0abbcbea in wl_client_destroy (client=0x1206bbd0) at ../../../../../../../home/chrome-bot/chrome_root/src/third_party/wayland/src/src/wayland-server.c:834
#15 0x08a709ea in exo::wayland::WlDisplayDeleter::operator() (this=<optimized out>, display=0xf548000)
    at ../../../../../../../home/chrome-bot/chrome_root/src/components/exo/wayland/scoped_wl.cc:20
#16 0x08a71256 in std::__1::unique_ptr<wl_display, exo::wayland::WlDisplayDeleter>::reset (this=0x12996ae0, __p=0x1) at /usr/bin/../include/c++/v1/memory:2608
#17 std::__1::unique_ptr<wl_display, exo::wayland::WlDisplayDeleter>::~unique_ptr (this=0x12996ae0) at /usr/bin/../include/c++/v1/memory:2576
#18 exo::wayland::Server::~Server (this=0xf89c7f8) at ../../../../../../../home/chrome-bot/chrome_root/src/components/exo/wayland/server.cc:4559
#19 0x0b7fba46 in std::__1::default_delete<exo::wayland::Server>::operator() (__ptr=0x12996ae0, this=<optimized out>) at /usr/bin/../include/c++/v1/memory:2399
#20 std::__1::unique_ptr<exo::wayland::Server, std::__1::default_delete<exo::wayland::Server> >::reset (__p=0x0, this=<optimized out>) at /usr/bin/../include/c++/v1/memory:2608
#21 ExoParts::~ExoParts (this=0xfaec3c0) at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/exo_parts.cc:152
#22 0x0acfbf0c in std::__1::default_delete<ExoParts>::operator() (__ptr=0x12996ae0, this=<optimized out>) at /usr/bin/../include/c++/v1/memory:2399
#23 std::__1::unique_ptr<ExoParts, std::__1::default_delete<ExoParts> >::reset (this=0xf492904, __p=0x0) at /usr/bin/../include/c++/v1/memory:2608
#24 ChromeBrowserMainExtraPartsAsh::PostMainMessageLoopRun (this=0xf4928c0)
    at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/ui/views/ash/chrome_browser_main_extra_parts_ash.cc:176
#25 0x092b9142 in ChromeBrowserMainParts::PostMainMessageLoopRun (this=0xf493c80)
    at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/chrome_browser_main.cc:1965
#26 0x08b527c0 in chromeos::ChromeBrowserMainPartsChromeos::PostMainMessageLoopRun (this=0xf493c80)
    at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/chromeos/chrome_browser_main_chromeos.cc:1212
#27 0x084dfc18 in content::BrowserMainLoop::ShutdownThreadsAndCleanUp (this=0xf487f00)
    at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/browser_main_loop.cc:1238
#28 0x084e1b50 in content::BrowserMainRunnerImpl::Shutdown (this=0xf484af0) at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/browser_main_runner.cc:200
#29 0x084dcb74 in content::BrowserMain (parameters=...) at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/browser_main.cc:48
#30 0x092aaec8 in content::ContentMainRunnerImpl::Run (this=0xf485f30) at ../../../../../../../home/chrome-bot/chrome_root/src/content/app/content_main_runner.cc:705
#31 0x092b0ca6 in service_manager::Main (params=...) at ../../../../../../../home/chrome-bot/chrome_root/src/services/service_manager/embedder/main.cc:456
#32 0x092a9fc0 in content::ContentMain (params=...) at ../../../../../../../home/chrome-bot/chrome_root/src/content/app/content_main.cc:19
#33 0x08088428 in ChromeMain (argc=<optimized out>, argv=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/app/chrome_main.cc:130
#34 0xb623f8a0 in __libc_start_main () from ./images/dir_3/lib/libc.so.6
#35 0x0c848fa8 in _start () at /usr/bin/../include/c++/v1/stdexcept:228

+oshima for real, for ClientControlledState::HandleTransitionEvents crash [1] during shutdown, which is probably a different problem from ARC kiosk not running.

[1] https://cs.chromium.org/chromium/src/ash/wm/client_controlled_state.cc?rcl=348bbca831edfbcea84934decaca02c10e1a0c85&l=48

Here is the logcat I was able to pull

Deleted: logcat.txt 447 KB

logcat.txt 447 KB View Download

there's some android logs under /var/log (but not logcat).
There might be some helpful signs in dmesg or /var/log/android.kmsg

The are CloudDPC errors in logcat from #35 similar to http://b/72087201 - lots of "clouddpc: [PolicyReapplyJobService] Failed to complete policy re-application"

Wen, are M64 builds also affected by that issue?

Hello all,

Attached you will find some logs for Veyron Tiger running Chrome OS 10176.68.0 64.0.3282.144

inside the folder varLog you will find all the files located inside veyron tiger's /var/log/ directory

I was able to initialize the android log cat with all the flags you requested

Deleted: logcat.txt 181 KB

logcat.txt 181 KB View Download

Deleted: debug-logs_20180207-171327.tgz 355 KB

debug-logs_20180207-171327.tgz 355 KB Download

Deleted: varLog.tar 2.4 MB

varLog.tar 2.4 MB Download

Right, it looks to be fixed in b/72087201#comment9. The fix isn't in M64. I do a cherry-pick for it.

I see that M64 has been on the stable channel. Is it too late to CP?

M64 ist at 10% for a handful of boards. Given that we have this stable blocker, we should consider stopping the rollout.

Wen, when does the CloudDPC crash happen? For all managed users or only in the presence of certain policies? Does the managed session recover, or is ARC gone for good?

Commenting on an email thread as well.  The stable push excludes ARC++ enabled boards due to this bug.  I've been monitoring and pulling those boards from staging.

Please tag this bug for a merge request if a CL is ready to be merged, along with some verbiage re: anticipated risk.  I'll likely approve soon, since I'd like to build an RC right away to capture ARC++ boards for a new round of stable.

Thanks

b/72087201 merge approved for M64.  Hoping this happens soon so I can kick off a build... Thanks

Do we just need to pick back https://googleplex-android-review.git.corp.google.com/#/c/platform/vendor/partner_gms/+/3579708/ ?

And do we also need to update clouddpc on 65?

Wen just submitted cr/184992402, which creates a new CloudDPC build for M64. Once we have a signed build of that, we need to drop it into the CrOS M64 branch.

I believe b/72087201#comment26 is the CP for M65, so we should be good there. Wen can confirm.

That comment looks like a critique CL that may be related, but I don't think can directly impact Chrome OS, unless there is a corresponding CL to Android/Chrome/CrOS?

The CP in b/72087201#comment26 is for M64. I'm releasing the APK to ARC++ now.

It's a two-step process:
* Fix the bug in CloudDPC
* Drop the new version of CloudDPC into the CrOS branch

Per #49, this is almost done for M64 now. Wen, do we need to do the same for M65?

The fixes were CPed into M65 a couple of days ago. So we don't need to CP them for M65 again.

The CloudDPC apk with the fix is merged into M64 now. See https://googleplex-android-review.git.corp.google.com/c/platform/vendor/partner_gms/+/3585027.

Please verify.

I noted that yesterday and kicked off a Chrome OS build that includes it.  I'm building a new stable RC that includes:

https://android-build.googleplex.com/builds/4594274/branches/git_nyc-mr1-arc-m64/targets/cheets_arm-user/cls?end=4581494

So this will be in the next Stable Chrome OS RC

I am not able to reproduce this issue on Veyron Tiger on Chrome os 	10176.69.0 64.0.3282.157

I am seeing still seeing this issue on Veyron-Fievel on Chrome os 10176.69.0 64.0.3282.157

MOV link
https://drive.google.com/file/d/1LIPVUTFOhopnFtwaNSZ65sN4xOU-4ekC/view?usp=sharing

coreDumps

https://drive.google.com/file/d/1qpJ3T3xpRN0SuPNbOfvou6J-UTbQDrFe/view?usp=sharing

Deleted: logcat5.txt 175 KB

logcat5.txt 175 KB View Download

Deleted: logcat6.txt 177 KB

logcat6.txt 177 KB View Download

Deleted: debug-logs_20180209-095258.tgz 345 KB

debug-logs_20180209-095258.tgz 345 KB Download

I can see that there are a number of mojo connection failures from different processes in the bug report. I feel that it might not relate to CloudDPC this time as the one we saw in M65 was introduced by a change only in M65 and the mojo connection failure there was only from CloudDPC itself. 

Bartosz, would you please take a look from Chrome side? 

Per #55 the RC I'm currently building (10176.69.0 64.0.3282.157) may still be ARC++-blocked?  Can we verify?

+Luis to coordinate the revert required

I believe this is a problem with the recent arcbridge update. CloudDPC uses a prebuilt library from google3/third_party/java/android_libs/arc/arcbridge. This was updated for M65 and is no longer backwards-compatible with M64 and earlier. Your CloudDPC build picked up this new library and hence, does not work.

I actually foresaw that as a risk, see the the thread "Updating CloudDPC dependencies for ARC++ P":

"The only caveat is that in the very unlikely event of an emergency bug fix for M64- involving CloudDPC, we will have to revert (2) temporarily so that we can build a CloudDPC compatible with M64-."

Here, (2) refers to https://critique.corp.google.com/#review/179942696

It is 8.30pm here and I need to run, but I think the steps should be relatively straightforward:

A) Stop all CloudDPC auto-builders as we are about to pull a required library from under their feet.
B) Revert arcbridge in google3 to its state in M64 (reverting (2) plus any additional steps Luis may think of)
C) Respin CloudDPC for M64, picking up the right version of arcbrige this time
D) Drop the new CloudDPC build into CrOS M64
E) Test
F) Revert (B), returning arcbridge in google3 to its current state
G) Restart CloudDPC auto-builders

Not very hard, but a lot of coordination as the weekend rolls on and the EMEA night begins.

I have stopped all CloudDPC auto-releaser as mentioned in A)

Sounds like we verified as not being fixed per the way I'm reading #58.  Can we clearly confirm? It impacts testing and deployment planning.  Thx

Also, is there any way to establish an ETA?  Is the plan outlined in #58?

Re:comment 60, there are two different bugs here. The first one has fixed. Comment#58 is another bug. 

Is the 2nd issue in #58 still a blocker?  Assume it is since the other fix fixed one board but not the other?  Please advise if that's not the case; we blocked CTS testing and need to resume.

I don't see two bugs here. We have one bug and the merge at #52 fixes it. The problem is that once we try to rebuild CloudDPC with the fix in it, we get a broken binary. #58 outlines the steps required to get a working binary.

+CC alexchau

I don't think CP causes problems, as Rapid created a separated branch for each release. Therefore, when CloudDPC M64 branch was created, the mojo lib changes weren't included. The CP shouldn't introduce those changes to the M64 build either.

you can see that, the CL changing the mojo lib isn't in the CloudDPC M64 build here https://rapid.corp.google.com/#/candidate-by-cl/android_for_work_clouddpc_for_arc/179707494.
Also, if that's the cause, why only one board is affected?

I never worked on google3 code much. So branching creates a branch of all google3, including all dependencies? Cool. In that case, the mojo uprev is not at fault. But something is causing a log of mojo failures.

Yes, that's correct. I suspect that might be something else causes mojo failures.

Looking at the logs again, something unrelated to CloudDPC seems to be broken. We get a Wayland failure, followed by a Wayland restart and a cascading mojo failure across numerous subsystems

https://bugs.chromium.org/p/chromium/issues/attachmentText?aid=324211#1214
https://bugs.chromium.org/p/chromium/issues/attachmentText?aid=324212#1225

Something is going wrong with code ARC++ itself. Luis, can you help?

This is a P0 blocker for the stable push and I need a clear answer.  Did last week's merge fix the problem for the Chrome OS stable push?  Are we clear to push ARC++ enabled boards?  Or is the release still compromised?

It's not clear if we had a good push that resulted in a new issue that's not a blocker, or if that merge never succeeded.

If we have a new non-blocking issue please track that in a new bug.

Thanks

I can only say that the bug initially reported should be fixed.

As Bartosz said, the new findings may relate to the wayland service, which is out of my scope. So I'll let Bartosz, Luis, and the QA team to give you the further decision.

The logs we are seeing look nothing like the original bug, so I presume the P0 is fixed. *However* the logs in #55 are massive crashes. Andres, is the device (especially a kiosk session) usable after these crashes? My understanding of #55 is no, it's not usable.

So we fixed one bug but it seems to have uncovered another but that is equally breaking kiosk sessions. Wen (CloudDPC) and I (enterprise) cannot help here. Someone from core ARC++ needs to look at this. Luis, you are the mojo expert. Do you understand what is going on?

We can track this in its own bug for clarity, but it will have to be a P0 stable blocker as well, because kiosk is still completely broken.

bartfab@

The device is only usable after the user reboots the device, and cancels the auto launch of the specified application. The user is able to use public sessions, guest mode and initialize chrome applications.

new bug

https://bugs.chromium.org/p/chromium/issues/detail?id=811398

The mojo error is probably caused by chrome crash.

The crash in #74 is ARC kiosk related, in ArcRobotAuthCodeFetcher::Fetch:

  fetch_request_job_->SetDMToken(client->dm_token());

Suspect client->dm_token() somehow returns a bad string ref.

From core.chrome.1015:
====
Program terminated with signal SIGSEGV, Segmentation fault.
#0  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long (this=0x34) at /usr/bin/../include/c++/v1/string:1226
1226            {return bool(__r_.first().__s.__size_ & __short_mask);}
[Current thread is 1 (LWP 1015)]
(gdb) bt
#0  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long (this=0x34) at /usr/bin/../include/c++/v1/string:1226
#1  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__get_pointer (this=<optimized out>) at /usr/bin/../include/c++/v1/string:1320
#2  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::data (this=<optimized out>) at /usr/bin/../include/c++/v1/string:1134
#3  std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::operator= (this=0xe6b1020, __str=...) at /usr/bin/../include/c++/v1/string:2047
#4  policy::DeviceManagementRequestJob::SetDMToken (this=0xe6b1000, dm_token=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/components/policy/core/common/cloud/device_management_service.cc:479
#5  0x05e97396 in arc::ArcRobotAuthCodeFetcher::Fetch(base::RepeatingCallback<void (bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)> const&) (this=0xfd05450, callback=...) at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/chromeos/arc/auth/arc_robot_auth_code_fetcher.cc:54
#6  0x05e95a64 in arc::ArcAuthService::RequestAccountInfo (this=0xfc653e0, initial_signin=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/chromeos/arc/auth/arc_auth_service.cc:247
#7  0x05e153ba in arc::mojom::AuthHostStubDispatch::Accept (impl=0xfc653e4, message=0xbe9eb988) at gen/components/arc/common/auth.mojom.cc:361
#8  0x09b35500 in mojo::internal::MultiplexRouter::ProcessIncomingMessage (this=0xd6ad540, message_wrapper=0xbe9eba50, client_call_behavior=<optimized out>, 
    current_task_runner=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/mojo/public/cpp/bindings/lib/multiplex_router.cc:880
#9  0x09b35294 in mojo::internal::MultiplexRouter::Accept (this=0xd6ad540, message=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/mojo/public/cpp/bindings/lib/multiplex_router.cc:604
#10 0x09b347e6 in mojo::Connector::ReadSingleMessage (this=0xd6ad570, read_result=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/mojo/public/cpp/bindings/lib/connector.cc:440
#11 mojo::Connector::ReadAllAvailableMessages (this=0xd6ad570) at ../../../../../../../home/chrome-bot/chrome_root/src/mojo/public/cpp/bindings/lib/connector.cc:469
#12 0x09b360ae in base::RepeatingCallback<void (unsigned int, mojo::HandleSignalsState const&)>::Run(unsigned int, mojo::HandleSignalsState const&) const & (args=..., args=..., 
    this=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/base/callback.h:94
#13 mojo::SimpleWatcher::OnHandleReady (this=0xfc16e40, watch_id=<optimized out>, result=0, state=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/mojo/public/cpp/system/simple_watcher.cc:276
#14 0x09b29718 in base::OnceCallback<void ()>::Run() && (this=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/base/callback.h:65
#15 base::debug::TaskAnnotator::RunTask (this=0xcbb2dc8, queue_function=0x9ef8bc1 "MessageLoop::PostTask", pending_task=0xbe9ebcb8)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/debug/task_annotator.cc:55
#16 0x09b29f48 in base::MessageLoop::RunTask (this=0xcbb2f20, pending_task=0xbe9ebcb8)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_loop.cc:391
#17 0x09b2a582 in base::MessageLoop::DeferOrRunPendingTask (this=<optimized out>, pending_task=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_loop.cc:403
#18 base::MessageLoop::DoWork (this=0xcbb2f20) at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_loop.cc:447
#19 0x09b2ab7c in base::MessagePumpLibevent::Run (this=0xcb9f030, delegate=0xcbb2f20)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_pump_libevent.cc:220
#20 0x0689e540 in base::RunLoop::Run (this=0xbe9ebe78) at ../../../../../../../home/chrome-bot/chrome_root/src/base/run_loop.cc:114
#21 0x06645f30 in ChromeBrowserMainParts::MainMessageLoopRun (this=<optimized out>, result_code=0xcb6ef0c)
    at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/chrome_browser_main.cc:1939
#22 0x058705ea in content::BrowserMainLoop::RunMainMessageLoopParts (this=0xcb6ef00)
    at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/browser_main_loop.cc:1199
#23 0x05872664 in content::BrowserMainRunnerImpl::Run (this=0xcb6baf0) at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/browser_main_runner.cc:140
#24 0x0586d7d6 in content::BrowserMain (parameters=...) at ../../../../../../../home/chrome-bot/chrome_root/src/content/browser/browser_main.cc:46
#25 0x06637ea8 in content::ContentMainRunnerImpl::Run (this=0xcb6cf30) at ../../../../../../../home/chrome-bot/chrome_root/src/content/app/content_main_runner.cc:705
#26 0x0663dc22 in service_manager::Main (params=...) at ../../../../../../../home/chrome-bot/chrome_root/src/services/service_manager/embedder/main.cc:456
#27 0x06636fa0 in content::ContentMain (params=...) at ../../../../../../../home/chrome-bot/chrome_root/src/content/app/content_main.cc:19
#28 0x0541a068 in ChromeMain (argc=<optimized out>, argv=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/app/chrome_main.cc:130
#29 0xb67188a0 in __libc_start_main () from ./image/dir_3/lib/libc.so.6
#30 0x09bc70e0 in _start () at ../../../../../../../home/chrome-bot/chrome_root/src/base/message_loop/message_pump_libevent.cc:262

Marking this verified according to #c72. 
1 out of the 2 bugs has been fixed.

This has been forked into https://bugs.chromium.org/p/chromium/issues/detail?id=811398. 
Please continue discussions in the new bug.

[Auto-generated comment by a script] We noticed that this issue is targeted for M-65; it appears the fix may have landed after branch point, meaning a merge might be required. Please confirm if a merge is required here - if so add Merge-Request-65 label, otherwise remove Merge-TBD label. Thanks.

re comment#77: I removed Merge-TBD label as the fix is already in the M65 build.

Sorry for the delay, was OOO on Friday and just caught up with stuff.

Going forward, if there's any problem during the M64 timeframe, I don't think reverting that change is feasible. That would require reverting a _huge_ set of changes in Android, since that includes a roll in libchrome :( We'll have to make forward fixes instead since that's going to be way less risky.