New issue
Advanced search Search tips

Issue 755260 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner: ----
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

blktrace on nvme has unexpected behavior

Project Member Reported by asavery@chromium.org, Aug 14 2017

Issue description

What steps will reproduce the problem?
(1) Run blktrace on nvme device
(2) Run blkparse
(3) If you run btt it shows device overhead greater than 100% for Q2G phase

What is the expected result?
Device overhead should be the percentage of time spent in each phase of the IO, and so it should not be greater than 100% for any phase

What happens instead?
Q2G has a device overhead greater than 100% and the average time for Q2C is less than the average time for Q2G, which doesn't make sense. Maybe we are missing a trace point somewhere.

I ran blktrace on a fizz with an nvme device and on a fizz with a sata device, and I am attaching the blkparse output from both of these. I am also attaching the avg output from btt to show the problem with overheads.
 
nvme_blkparse
2.1 MB View Download
sata_blkparse
475 KB View Download
nvme_btt_sh.avg
3.1 KB Download
sata_btt_sh.avg
3.4 KB Download
I am attaching the blkparse binary output files.
nvme_blkparse.bin
1.4 MB Download
sata_blkparse.bin
382 KB Download
Works for me, I don't see the Q2G error.

Tested on eve, (R62-9840.0.0)
blktrace -d /dev/nvme0n1 -D /tmp -o log_gwen
[Use a /tmp as it is a different block device, avoiding trace noise)

Copy traces locally:
 scp root@172.22.30.188:/tmp/log_gwen.blktrace.* ~/chromium/chroot/tmp

Process traces locally:
blkparse -d /tmp/log_gwen.bin2 -o /tmp/log_gwen.txt2 -i /tmp/log_gwen
and
btt -Avi /tmp/log_gwen.bin2

==================== Device Overhead ====================

       DEV |       Q2G       G2I       Q2M       I2D       D2C
---------- | --------- --------- --------- --------- ---------
 (259,  0) |   0.2980%  12.2966%   0.0000%   0.6598%  86.2907%
---------- | --------- --------- --------- --------- ---------
   Overall |   0.2980%  12.2966%   0.0000%   0.6598%  86.2907%

log_gwen.bin2
3.5 MB Download
log_gwen.txt2
5.2 MB Download
Reproduced the issue. btt is not understanding blktrace trace properly. 
Tested on ubnutun version (10yr old), blktrace 1.1.0 (2yr old) and ToT
Sent a feedback to owner: 
http://marc.info/?l=linux-btrace&m=150294139027472&w=2

Still investigating. btt mismatch IOs with similar LBA.
Components: Speed>Tracing

Comment 5 Deleted

Comment 6 Deleted

Project Member

Comment 8 by bugdroid1@chromium.org, Jan 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/portage-stable/+/e6a31ea7686c6d40b8d518013a41f314f28389cc

commit e6a31ea7686c6d40b8d518013a41f314f28389cc
Author: Gwendal Grignou <gwendal@chromium.org>
Date: Fri Jan 11 21:32:37 2019

blktrace: upgraded package to upstream

Upgraded sys-block/blktrace to version 1.2.0 on amd64

1.2.0 include a fix for btt when IOs overlapping

BUG= chromium:755260 
TEST=Using trace from
https://bugs.chromium.org/p/chromium/issues/detail?id=755260#c1
that btt is working fine.
Ran blktrace on eve.

Change-Id: If1324068caf2734253cec3653906aee4b04386c4
Signed-off-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/1406252
Reviewed-by: Mike Frysinger <vapier@chromium.org>

[delete] https://crrev.com/4aaff007e89c2fcd6b613d4c7d996aa1274c17d2/sys-block/blktrace/files/blktrace-1.0.1-ldflags-parallel-make.patch
[add] https://crrev.com/e6a31ea7686c6d40b8d518013a41f314f28389cc/sys-block/blktrace/files/blktrace-1.2.0-overlapping-io-stats.patch
[add] https://crrev.com/e6a31ea7686c6d40b8d518013a41f314f28389cc/metadata/md5-cache/sys-block/blktrace-1.2.0
[add] https://crrev.com/e6a31ea7686c6d40b8d518013a41f314f28389cc/sys-block/blktrace/files/blktrace-1.2.0-parallel-build.patch
[delete] https://crrev.com/4aaff007e89c2fcd6b613d4c7d996aa1274c17d2/metadata/md5-cache/sys-block/blktrace-1.1.0
[delete] https://crrev.com/4aaff007e89c2fcd6b613d4c7d996aa1274c17d2/sys-block/blktrace/blktrace-1.1.0.ebuild
[modify] https://crrev.com/e6a31ea7686c6d40b8d518013a41f314f28389cc/sys-block/blktrace/Manifest
[add] https://crrev.com/e6a31ea7686c6d40b8d518013a41f314f28389cc/sys-block/blktrace/blktrace-1.2.0.ebuild
[add] https://crrev.com/e6a31ea7686c6d40b8d518013a41f314f28389cc/sys-block/blktrace/files/blktrace-1.2.0-ldflags.patch

Sign in to add a comment