New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 813824 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug

Blocked on:
issue 857028

Blocking:
issue 651354



Sign in to add a comment

EVACUATE_COPY step is sometimes using as little as 25% CPU (all workers idle)

Project Member Reported by gab@chromium.org, Feb 20 2018

Issue description

Project Member

Comment 1 by bugdroid1@chromium.org, Feb 20 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/f64a87c45a3070088eaaba1162ffbabdadacb928

commit f64a87c45a3070088eaaba1162ffbabdadacb928
Author: Gabriel Charette <gab@chromium.org>
Date: Tue Feb 20 19:07:06 2018

[v8] Add finer grain trace events to EVACUATE_COPY step.

This is in an attempt to diagnose why it's seen as a highly idle step
in some traces, e.g.
https://docs.google.com/document/d/1bdlWAWeP3j2yo2DYfeok6URqFCrt57yx-nucGMybGGQ/edit#heading=h.1baa99zf8jc0
https://drive.google.com/file/d/1kLfH1dM5TOJbiD2v_KJsxWXakpd2k-7q/view

R=mlippautz@chromium.org

Bug: chromium:813824
Change-Id: I743d9b7430a3970751398b4c5a4d569fbcae2cbe
Reviewed-on: https://chromium-review.googlesource.com/926844
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Gabriel Charette <gab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51400}
[modify] https://crrev.com/f64a87c45a3070088eaaba1162ffbabdadacb928/src/heap/mark-compact.cc

Project Member

Comment 2 by bugdroid1@chromium.org, Feb 21 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/2c1b6d383e5891aca3c2ddd451790fad0e286219

commit 2c1b6d383e5891aca3c2ddd451790fad0e286219
Author: Gabriel Charette <gab@chromium.org>
Date: Wed Feb 21 18:50:12 2018

[v8] Even finer grain trace events for EVACUATE_COPY step.

Fine grain events are showing that LiveObjectVisitor::VisitBlackObjectsNoFail
is the suspiciously slow step, breaking it down further.
https://docs.google.com/document/d/1bdlWAWeP3j2yo2DYfeok6URqFCrt57yx-nucGMybGGQ/edit#heading=h.97pqg2eosnw8
https://drive.google.com/file/d/1FKZHkXtWK2fZCk_3PClPcbpzKilnbuOr/view

R=mlippautz@chromium.org

Bug: chromium:813824
Change-Id: Idcebcbb268d3e00f5f1e304c66564b4b9c85a064
Reviewed-on: https://chromium-review.googlesource.com/929161
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Commit-Queue: Gabriel Charette <gab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51447}
[modify] https://crrev.com/2c1b6d383e5891aca3c2ddd451790fad0e286219/src/heap/mark-compact.cc

Comment 3 by gab@chromium.org, Feb 22 2018

As expected these trace events (even the first set) regresses these stats on their own (when v8.gc tracing is active that is) [1]

Looks like VisitBlackObjectsNoFail -> Visit() can be problematic as well as LiveObjectRange<kBlackObjects>++.

Still waiting for next batch of trace events to come in to diagnose further.

[1] https://chromeperf.appspot.com/report?sid=44032e8e4b0384fa037af66ba7cdb2b5944521647e2555f18b6157f42d688a3a

Comment 4 by gab@chromium.org, Feb 23 2018

Cc: simonhatch@chromium.org jgruber@chromium.org
 Issue 814800  has been merged into this issue.

Comment 5 by gab@chromium.org, Feb 23 2018

Cc: -jgruber@chromium.org
@simonhatch: in this issue I'm adding trace events so that I can't extract fine grain details from traces @ https://chromeperf.appspot.com/report?sid=44032e8e4b0384fa037af66ba7cdb2b5944521647e2555f18b6157f42d688a3a

While I can easily "View traces from this run", I can't seem to find a way to download the JSON trace file? Is there a way?

I also tried to "Request a Debug trace" with extra tracing categories but I don't see where those land..? It said job submitted with a link to https://chromeperf.appspot.com/job/14b2e120440000 but that 404s for me (at the moment of submission..).
Project Member

Comment 6 by bugdroid1@chromium.org, Feb 23 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/3856d8d57d6f2798277182f20cd3a9810eae56ea

commit 3856d8d57d6f2798277182f20cd3a9810eae56ea
Author: Gabriel Charette <gab@chromium.org>
Date: Fri Feb 23 10:49:16 2018

Revert "[v8] Add finer grain trace events to EVACUATE_COPY step."

This reverts commit f64a87c45a3070088eaaba1162ffbabdadacb928.

Revert "[v8] Even finer grain trace events for EVACUATE_COPY step."

This reverts commit 2c1b6d383e5891aca3c2ddd451790fad0e286219.

Traces have been gathered by the bots. I'm still analyzing the results
but let's revert these for now to soothe the poor bots which are
running with v8.gc tracing enabled and regressing like crazy
because of this  crbug.com/814800 .

R=hpayer@chromium.org

Bug:  chromium:814800 , chromium:813824
Change-Id: If97c1ff5d14de18414c3b8d65ecb3aaf4f1bb32d
Reviewed-on: https://chromium-review.googlesource.com/934131
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Gabriel Charette <gab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51502}
[modify] https://crrev.com/3856d8d57d6f2798277182f20cd3a9810eae56ea/src/heap/mark-compact.cc

re: #c5

To clarify, you don't want the html file but the original trace itself? If that's the case, I have no idea. I could look into it for you if that's what you're looking for.

Yeah sorry about that! Had a version that wasn't generating the url names properly and had to roll it back.

Your job is at: https://pinpoint-dot-chromeperf.appspot.com/job/14b2e120440000. Looks like it's still pending so there may be a lot of jobs running at the moment.


Comment 8 by gab@chromium.org, Feb 23 2018

Thanks for the link to the job.

Yes the original trace is what I'm after (surprised there isn't a Save button in the tracing UI it brings up). Thanks!
Cc: benjhayden@chromium.org
+benjhayden Any idea if there is actually a save button somewhere in the trace ui for getting at the original trace?
Project Member

Comment 10 by bugdroid1@chromium.org, Feb 24 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/8bd7a2a8ccea4c1915c9b5cc162ccd777e1e528b

commit 8bd7a2a8ccea4c1915c9b5cc162ccd777e1e528b
Author: Gabriel Charette <gab@chromium.org>
Date: Sat Feb 24 22:27:28 2018

Reland top-level breakdown trace events for EVACUATE_COPY.

I think those are safe to keep and add useful information to highlight
the type of evacuate copy step being performed.

This is a prerequisite for https://chromium-review.googlesource.com/c/v8/v8/+/934263
as it will drop tracing individual work items and will
leave it to top-level item trace events like these to
highlight individual work item's work.

Will reland after reverting them all however to confirm the re-adding
these doesn't cause regressions as the fine grain trace events
initially did.

R=hpayer@chromium.org

Bug: chromium:813824
Change-Id: I8e53b880ce0aeae08afa7a2c0f36a455f645bcfa
Reviewed-on: https://chromium-review.googlesource.com/934248
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Commit-Queue: Gabriel Charette <gab@chromium.org>
Cr-Commit-Position: refs/heads/master@{#51543}
[modify] https://crrev.com/8bd7a2a8ccea4c1915c9b5cc162ccd777e1e528b/src/heap/mark-compact.cc

#9: That's a vulcanized trace, so it doesn't contain the x-timeline-view-buttons that contains the Save button. However, there is a script to extract the trace json from the vulcanized html file: catapult/tracing/bin/html2trace

Comment 13 by gab@chromium.org, May 4 2018

Cc: gab@chromium.org
Owner: ----
Status: Untriaged (was: Started)
I will not realistically keep investigating this now that my Chrome Envoy is complete.
Status: Available (was: Untriaged)

Comment 15 by gab@chromium.org, Jun 27 2018

Blockedon: 857028

Sign in to add a comment