New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 843275 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Sep 18
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug
Build-Toolchain



Sign in to add a comment

Support gdb + gdbserver with Simple Chrome

Project Member Reported by steve...@chromium.org, May 15 2018

Issue description

Currently our support for gdb + gdbserver in Simple Chrome is not great and possibly broken. Recently I tried to follow the instructions here:

https://chromium.googlesource.com/chromiumos/docs/+/master/simple_chrome_workflow.md#Remote-GDB

I ran into the following problems:

1. We don't seem to have an environment variable for the correct gdb; we should fix that since otherwise it's a bit of a pain to locate. (We shouldn't try to support /usr/bin/gdb; there is a reason why we include the toolchain in .cros_cache).

2. The toolchain provided gdb doesn't support python, so we should include instructions to add python support or fix the instructions to not use python.

3. When I ran 'target remote 100.xxx.yy.zz:1234' (after I ran '/sbin/iptables -A INPUT -p tcp --dport 1234 -j ACCEPT' on the device) I got the following error, even though I can ping and ssh to the device:
00.xxx.yy.zz:1234: No route to host

We should investigate and fix the problems above and update the documentation.

Discussion thread:
https://groups.google.com/a/google.com/d/msgid/chromeos-chatty-eng/CANj_64E_1Z-marej3E%3D_yAsT-hQ%2B-MCKhLqUYmpmJS9RurC3LA%40mail.gmail.com.

 

Comment 1 Deleted

Comment 2 by cmtm@chromium.org, May 15 2018

This bug is for the same issue.
https://b.corp.google.com/issues/79586143

Comment 3 Deleted

Owner: yunlian@chromium.org
Status: Assigned (was: Untriaged)
I am not marking this as duplicate of 7958143 because the questions raised are a little different.
But assigning to the same person to take a look. 
crbug.com is better for tracking chrome/cros related issues anyway.

 Issue #3  is google specific however so I will ask about that in the internal bug.

For the python support issue, I think we can re-renable that for now. See  crbug.com/736322 

I will look at the two other issues.
Can you try cros_gdb inside simple chrome?
It seems that cros_gdb works inside chromiumos-chroot.
If it also works in simple chrome, I will modify the document.
I tried cros_gdb with Simple Chrome and got the following results:

(sdk samus 10681.0.0) ~/Work/chrome/src (master) $ cros_gdb
10:14:47: ERROR: Cannot determine which board to use. Please specify the with --board flag.
(sdk samus 10681.0.0) ~/Work/chrome/src (master) $ cros_gdb --board=samus
cros_gdb: Unhandled exception:
Traceback (most recent call last):
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/bin/cros_gdb", line 169, in <module>
    DoMain()
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/bin/cros_gdb", line 165, in DoMain
    commandline.ScriptWrapperMain(FindTarget)
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/lib/commandline.py", line 911, in ScriptWrapperMain
    ret = target(argv[1:])
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/scripts/cros_gdb.py", line 594, in main
    gdb.Run()
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/scripts/cros_gdb.py", line 425, in Run
    self.VerifyAndFinishInitialization(None)
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/scripts/cros_gdb.py", line 131, in VerifyAndFinishInitialization
    self.cross_gdb = self.GetCrossGdb()
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/scripts/cros_gdb.py", line 338, in GetCrossGdb
    toolchains = toolchain.GetToolchainsForBoard(self.board)
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/lib/toolchain.py", line 60, in GetToolchainsForBoard
    buildroot=buildroot)
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/lib/portage_util.py", line 206, in FindOverlays
    overlays = _ListOverlays(board=board, buildroot=buildroot)
  File "/usr/local/google/home/stevenjb/Work/chrome/src/third_party/chromite/lib/portage_util.py", line 188, in _ListOverlays
    raise MissingOverlayException('board overlay not found: %s' % board)
chromite.lib.portage_util.MissingOverlayException: board overlay not found: samus


I get the same behavior for eve and caroline.




Okey, it seems currently cros_gdb is not supported in simple chrome. I will work on that.

That would be great, thanks!

Proposed a workaround in the bugizer, I will continue to work on the cros_gdb
Project Member

Comment 12 by bugdroid1@chromium.org, May 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/fdd7e08cde5d02ec43ff91e5a3a6a2fd22035269

commit fdd7e08cde5d02ec43ff91e5a3a6a2fd22035269
Author: Yunlian Jiang <yunlian@chromium.org>
Date: Wed May 23 04:57:19 2018

cros_gdb: support simple chrome

This makes cros_gdb work in simple chrome work flow.

BUG= chromium:843275 
TEST=cros_gdb --board samus --remote ${ip} --attach browser
     works inside simple chrome chrome-sdk.

Change-Id: Id911494ecae1cdd7b6de733bf7ef0d815f47e240
Reviewed-on: https://chromium-review.googlesource.com/1068133
Commit-Ready: Yunlian Jiang <yunlian@chromium.org>
Tested-by: Yunlian Jiang <yunlian@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>

[modify] https://crrev.com/fdd7e08cde5d02ec43ff91e5a3a6a2fd22035269/scripts/cros_gdb.py

Project Member

Comment 13 by bugdroid1@chromium.org, May 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8d9c0f04b870d604359890237495d1ac5091c740

commit 8d9c0f04b870d604359890237495d1ac5091c740
Author: chromite-chromium-autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Wed May 23 07:13:03 2018

Roll src/third_party/chromite/ 07dfd789c..fdd7e08cd (5 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/07dfd789cb28..fdd7e08cde5d

$ git log 07dfd789c..fdd7e08cd --date=short --no-merges --format='%ad %ae %s'
2018-05-21 yunlian cros_gdb: support simple chrome
2018-05-21 bmgordon chroot_lib: Remove ChrootManager
2018-05-21 ayatane sysmon: Refactor prod_metrics: Remove _LoggingSink
2018-05-21 ayatane sysmon: Refactor prod_metrics: Remove _AtestSource
2018-05-21 ayatane sysmon: Refactor prod_metrics: Remove _ProdHostReporter

Created with:
  roll-dep src/third_party/chromite
BUG= chromium:843275 ,chromium:845174,chromium:None,chromium:None,chromium:None


The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.


TBR=chrome-os-gardeners@chromium.org

Change-Id: I32201b36652a22d6f98c4c470ae1319206badc92
Reviewed-on: https://chromium-review.googlesource.com/1069979
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#560986}
[modify] https://crrev.com/8d9c0f04b870d604359890237495d1ac5091c740/DEPS

The latest chrome checkout should contain the working cros_gdb, could you please try that can let me know if there is any issues?

Comment 15 by spang@chromium.org, Jun 15 2018

It still does not work.

gdbserver consistently gets the wrong base address when attaching to an existing invocation chrome, and attaching to an existing process is the only practical way to debug since chrome is started by session_manager.

Attaching using "gdb" directly on the target seess to get the correct base address, but does not have any symbols available.

This is typically what gdb via CrOS "gdbserver" typically thinks the memory layout looks like:

Symbols from "/ssd/src/chromium/src/out_eve/Release/chrome".
Remote serial target in gdb-specific protocol:
Debugging a target over a serial line.
        While running this, GDB does not access memory from...
Local exec file:
        `/ssd/src/chromium/src/out_eve/Release/chrome', file type elf64-x86-64.
        Entry point: 0x77c00
        0x0000000000000270 - 0x000000000000028c is .interp
        0x000000000000028c - 0x00000000000002ac is .note.ABI-tag
        0x00000000000002ac - 0x00000000000002d0 is .note.gnu.build-id
        0x00000000000002d0 - 0x0000000000007410 is .dynsym
        0x0000000000007410 - 0x000000000000c613 is .dynstr
        0x000000000000c618 - 0x000000000000c784 is .gnu.hash
        0x000000000000c784 - 0x000000000000d0f4 is .gnu.version
        0x000000000000d0f4 - 0x000000000000d4a4 is .gnu.version_r
        0x000000000000d4a8 - 0x0000000000050f48 is .rela.dyn
        0x0000000000050f48 - 0x000000000006c460 is .relr.dyn
        0x000000000006c460 - 0x0000000000072820 is .rela.plt
        0x0000000000072820 - 0x0000000000072844 is .init
        0x0000000000072850 - 0x0000000000076ae0 is .plt
        0x0000000000076b00 - 0x0000000009421656 is .text
        0x0000000009421660 - 0x0000000009421ebf is malloc_hook
        0x0000000009421ec0 - 0x0000000009422f1b is google_malloc
        0x0000000009422f1c - 0x0000000009422f2a is .fini
        0x0000000009423000 - 0x000000000a2b77c4 is .rodata
        0x000000000a2b77c8 - 0x000000000a2b81cc is .eh_frame
        0x000000000a2b81cc - 0x000000000a2b8340 is .eh_frame_hdr
        0x000000000a2b9780 - 0x000000000a2b9790 is .tdata
        0x000000000a2b9790 - 0x000000000a2b979c is .tbss
        0x000000000a2b9790 - 0x000000000a2b97a0 is .ctors
        0x000000000a2b97a0 - 0x000000000a2b97b0 is .dtors
        0x000000000a2b97b0 - 0x000000000a2b97b8 is .jcr
        0x000000000a2b97c0 - 0x000000000a94e820 is .data.rel.ro
        0x000000000a94e820 - 0x000000000a94e868 is .init_array
        0x000000000a94e868 - 0x000000000a94ec08 is .dynamic
        0x000000000a94ec08 - 0x000000000a952ea0 is .got
        0x000000000a952ea0 - 0x000000000a954ff8 is .got.plt
        0x000000000a955000 - 0x000000000a955000 is .tm_clone_table
        0x000000000a955000 - 0x000000000a9eae50 is .data
        0x000000000a9eae60 - 0x000000000ac573ec is .bss
        0x00007fff339dd120 - 0x00007fff339dd15c is .hash in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd160 - 0x00007fff339dd1a8 is .gnu.hash in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd1a8 - 0x00007fff339dd298 is .dynsym in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd298 - 0x00007fff339dd2f6 is .dynstr in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd2f6 - 0x00007fff339dd30a is .gnu.version in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd310 - 0x00007fff339dd348 is .gnu.version_d in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd348 - 0x00007fff339dd478 is .dynamic in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd478 - 0x00007fff339dd7b8 is .rodata in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd7b8 - 0x00007fff339dd7f4 is .note in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd7f4 - 0x00007fff339dd820 is .eh_frame_hdr in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd820 - 0x00007fff339dd898 is .eh_frame in system-supplied DSO at 0x7fff339dd000
        0x00007fff339dd8a0 - 0x00007fff339ddc44 is .text in system-supplied DSO at 0x7fff339dd000
        0x00007fff339ddc44 - 0x00007fff339ddc92 is .altinstructions in system-supplied DSO at 0x7fff339dd000
        0x00007fff339ddc92 - 0x00007fff339ddca4 is .altinstr_replacement in system-supplied DSO at 0x7fff339dd000

Comment 16 by spang@chromium.org, Jun 15 2018

(hopefully it was clear, but the reason this memory layout is wrong is that it's assuming that "chrome" is loaded at address zero, but it isn't)


Comment 17 by spang@chromium.org, Jun 15 2018

Also FWIW we've gotten exactly the same steps to work with binaries other than "chrome". There seems to be something particular to the "chrome" binary itself that prevents gdb from determining the load address.

Comment 18 by derat@chromium.org, Jun 15 2018

Cc: -derat@chromium.org
I use the command below to copy "chrome" to DUT
 deploy_chrome --build-dir=out_samus/Release --to=${IP} --target-dir=/usr/local/chrome --mount-dir=/opt/google/chrome --nostrip

And after that, I use 
cros_gdb --board samus --remote 172.16.243.167 --attach browser

I can use bt to print backtrace

It seems fine except it complains that it cannot find .dwo files. (I will fix that soon).

Could you please give me the instructions on how to reproduce this issue?
Project Member

Comment 20 by bugdroid1@chromium.org, Jun 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/chromite/+/e51d6a52987f5f3a642811c13b68538de6b21451

commit e51d6a52987f5f3a642811c13b68538de6b21451
Author: Yunlian Jiang <yunlian@google.com>
Date: Tue Jun 19 17:54:34 2018

cros_gdb: set debug-file-directory for simple chrome.

This sets debug-file-directory for simple chrome workflow so
that it can read the .dwo files.

BUG= chromium:843275 
TEST=cros_gdb --attach browser; 'bt'
     stack trace is shown.

Change-Id: I99b259e85b01ac19ecfa977a4db62d4f8969ac69
Reviewed-on: https://chromium-review.googlesource.com/1105180
Commit-Ready: Yunlian Jiang <yunlian@chromium.org>
Tested-by: Yunlian Jiang <yunlian@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>

[modify] https://crrev.com/e51d6a52987f5f3a642811c13b68538de6b21451/scripts/cros_gdb.py

Project Member

Comment 21 by bugdroid1@chromium.org, Jun 19 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/84e07da8461b4b76da32670f411417a117a8340c

commit 84e07da8461b4b76da32670f411417a117a8340c
Author: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Date: Tue Jun 19 23:46:40 2018

Roll src/third_party/chromite efd58885b78d..9cccc5ce4fb9 (7 commits)

https://chromium.googlesource.com/chromiumos/chromite.git/+log/efd58885b78d..9cccc5ce4fb9


git log efd58885b78d..9cccc5ce4fb9 --date=short --no-merges --format='%ad %ae %s'
2018-06-19 ayatane@chromium.org sysmon: Ignore chromeos3-* devservers
2018-06-19 dgarrett@google.com gclient: Stop using implicit git_cache_dir.
2018-06-19 lhchavez@google.com Add 'signed' image type examples / docs to cros flash
2018-06-19 dgarrett@google.com chromeos_config: Move toolchain builders to swarming.
2018-06-19 dgarrett@google.com build_status: Remove config_lib.UseBuildbucketScheduler.
2018-06-19 yunlian@google.com cros_gdb: set debug-file-directory for simple chrome.
2018-06-19 vapier@chromium.org pylintrc: convert warnings list into readable ids


Created with:
  gclient setdep -r src/third_party/chromite@9cccc5ce4fb9

The AutoRoll server is located here: https://chromite-chromium-roll.skia.org

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.



BUG= chromium:853866 , chromium:853391 ,chromium:852017, chromium:853969 , chromium:853025 , chromium:843275 ,chromium:None
TBR=chrome-os-gardeners@chromium.org

Change-Id: I38b6b86c7271b9ca0ec2148faa144816e012ac81
Reviewed-on: https://chromium-review.googlesource.com/1107006
Reviewed-by: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Commit-Queue: Chromite Chromium Autoroll <chromite-chromium-autoroll@skia-buildbots.google.com.iam.gserviceaccount.com>
Cr-Commit-Position: refs/heads/master@{#568648}
[modify] https://crrev.com/84e07da8461b4b76da32670f411417a117a8340c/DEPS

Project Member

Comment 22 by bugdroid1@chromium.org, Sep 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/a357dfccbbae85bdc636c22cfabd59383a7699dd

commit a357dfccbbae85bdc636c22cfabd59383a7699dd
Author: Michael Spang <spang@chromium.org>
Date: Fri Sep 14 06:56:40 2018

sys-devel/gdb: Fix debugging of stripped PIE executables with padded PT_TLS

Certain PIE executables produced by gold cannot be debugged by gdb after
being stripped. GDB requires program headers of PIE executables to match,
and those checks may fail due to adjustments made during stripping.

One case of this occurs because strip recomputes the memsz of PT_TLS and
does not add alignment, while gold does. This is another variant of PR
11786, so apply the same fix of relaxing the program header matching.

BUG= chromium:843275 
TEST=Remote debug chrome binary built with Chrome OS SDK (go/simplechrome)

Change-Id: Ib9247595edec63c0275e4853350593d5ff152e46
Reviewed-on: https://chromium-review.googlesource.com/1205377
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Michael Spang <spang@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>

[modify] https://crrev.com/a357dfccbbae85bdc636c22cfabd59383a7699dd/sys-devel/gdb/Manifest
[rename] https://crrev.com/a357dfccbbae85bdc636c22cfabd59383a7699dd/sys-devel/gdb/gdb-8.0.1.20180820.ebuild
[delete] https://crrev.com/d758d60cee51a8891a05998e52ff02d64d08f26b/sys-devel/gdb/gdb-8.0.1.20180501-r1.ebuild
[add] https://crrev.com/a357dfccbbae85bdc636c22cfabd59383a7699dd/sys-devel/gdb/gdb-8.0.1.20180820-r1.ebuild

Status: Fixed (was: Assigned)

Sign in to add a comment