New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 4 users

Issue metadata

Status: Started
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: All
Pri: 3
Type: Bug



Sign in to add a comment
link

Issue 568024: [Navigation & Resource Timing]: responseStart is not spec compliant. Should record time of first byte from response (Header's first byte received)

Reported by kenjibaheux@chromium.org, Dec 9 2015 Project Member

Issue description

"[responseStart] must return the time immediately after the user agent receives the first byte of the response from the server...".

https://w3c.github.io/navigation-timing/#widl-PerformanceNavigationTiming-responseStart
 

Comment 1 by csharrison@chromium.org, Dec 9 2015

Cc: csharrison@chromium.org

Comment 2 by igrigo...@chromium.org, Dec 9 2015

Cc: igrigo...@chromium.org
This applies to Resource Timing as well.

Comment 3 by kenjibaheux@chromium.org, Dec 15 2015

Summary: Navigation&Resource timing: responseStart is not spec compliant. Should record time of first byte from response (Header's first byte received) (was: Navigation timing: responseStart is not spec compliant. Should record time of first byte from response (Header's first byte received))

Comment 4 by igrigo...@chromium.org, Jul 15 2016

Labels: Hotlist-PerformanceAPIs

Comment 5 by igrigo...@chromium.org, Jul 22 2016

Charles: can you clarify what is our current behavior?

Comment 6 by csharrison@chromium.org, Jul 22 2016

Cc: mmenke@chromium.org
Components: Internals>Network
We use the time the headers were fully received and processed:
https://cs.chromium.org/chromium/src/net/url_request/url_request_http_job.cc?rcl=1469200610&l=1014

I don't know URLRequestHTTPJob very well, so I'm not quite sure what that means *precisely* or why it would be difficult to change. +mmenke if it is reasonable to add metrics for the true first bytes received. Would the timestamp be significantly different? Also adding Internals>Network because this change will likely end up plumbing through //net.

Comment 7 by mmenke@chromium.org, Jul 22 2016

URLRequestHTTPJob measures the time when the last byte of headers reached the top layer of the stack (After writing them to cache, if necessary).  If we want when we receive the first byte, we'll need to hook it up to HttpStreamParser, and the QUIC/H2 equivanents...And if we wanted to be strictly correct, if H2 reads only a partial frame, it would have to record the time there, and then apply that time to the header response, if that frame is the header frame for a request.  I'm not sure if QUIC can receive partial frames or not.  For HttpStreamParser, we allow up to 3 bytes of cruft between responses, which presumably we'd need to exclude.

What does it mean for server auth?  Is the first byte the first byte of the first challenge, or the first byte of the actual headers?  What about 100 continue responses?  Do those really count, or do we care about the actual headers for the response?  Per spec, those could be read before send completes (Though I guess neither HAR files nor navigation/resource timing care about send complete?)

Comment 8 by mmenke@chromium.org, Jul 22 2016

I'm assuming it's ok to only record the time after SSL decoding, and the QUIC equivalent (Though may be able to record the pre-decrypt time on the QUIC path, not familiar with it).  If not, I propose open warfare between the Chrome network stack team and Ilya.  :)

Comment 9 by igrigo...@chromium.org, Jul 22 2016

/me ducks and runs for cover.. :-)

Re, time after SSL decoding: yeah, I think that makes sense.
Re, partial frame: don't have a strong opinion on this. I think it would be reasonable to keep it as time when the first full frame is received and ready to be processed by the client. Sounds like that'll also keep the implementation simple(r).
Re, auth: hmm, good question. We should clarify this in the spec.. I'm leaning towards "first HTTP response byte", would that make sense? That would also mean that first byte would be captured at receipt of 100 continue.

Comment 10 by mmenke@chromium.org, Jul 22 2016

I'm not sure that the 100 continue counts as part of the response, though - it's not a part of the response to the request, and exists to be sent before the server even sees the full request (Including body).

Don't get me wrong, if I implemented it as simply as possible without thinking too much about it, I'd include the 100 header and not the auth header, because that's how things are architected, and the 100 is at least somewhat sent in response to the final HTTP request for the resource, I'm just not sure that's the most consistent option.

Comment 11 by mmenke@chromium.org, Jul 22 2016

Labels: -Pri-2 Pri-3

Comment 13 by igrigo...@chromium.org, Sep 20 2016

Matt, would love to hear your thoughts on discussion in: 
https://github.com/w3c/resource-timing/issues/63#issuecomment-248320269

Comment 14 by tdres...@chromium.org, Aug 14 2017

Summary: [Navigation & Resource Timing]: responseStart is not spec compliant. Should record time of first byte from response (Header's first byte received) (was: Navigation&Resource timing: responseStart is not spec compliant. Should record time of first byte from response (Header's first byte received))

Comment 15 by tdres...@chromium.org, Jan 4 2018

Components: -Blink>PerformanceAPIs Blink>PerformanceAPIs>ResourceTiming Blink>PerformanceAPIs>NavigationTiming

Comment 16 by maxlg@chromium.org, Apr 23 2018

What's the status of this bug now?

Comment 17 by tdres...@chromium.org, May 7 2018

Matt: the spec (https://w3c.github.io/resource-timing/) has now been updated to make this more clear.

In particular:
"The time immediately after the user agent's HTTP parser receives the first byte of the response"

Who is the right owner for exposing this signal?

Comment 18 by mmenke@chromium.org, May 7 2018

Still not sure what that means for 100 responses or auth challenges.

Regardless, for H2, it's bnc@, for QUIC, any one on the QUIC team, for HTTP, no one, really.

Comment 19 by tdres...@chromium.org, May 7 2018

Ilya, any thoughts on on #10?

Comment 20 by igrigo...@google.com, May 7 2018

I believe we resolved that in: https://github.com/w3c/resource-timing/issues/63#issuecomment-260058020

> Re: auth challenges / continue, etc. Current definition in L2 ties responseStart to "last non-redirect fetch", which implies that it's the receipt of decoded header data of the last fetch. It sounds like this is consistent with what we want to keep; no changes needed.

Comment 21 by mmenke@chromium.org, May 7 2018

"For HTTP/1 this would be receipt of all header lines." - that sounds different from what comment #17 says.

Comment 22 by mmenke@chromium.org, May 7 2018

Oh, guess there was disagreement on that point, but the other point in that same comment stands.

Comment 23 by mmenke@chromium.org, Jun 6 2018

Labels: Network-Triaged

Comment 24 by npm@chromium.org, Sep 25

Cc: tommckee@chromium.org
Hmm no one has taken ownership of this bug. +tommckee@ in case you're interested in taking this

Comment 25 by acomminos@fb.com, Nov 22

Cc: acomminos@fb.com
Owner: acomminos@fb.com
Status: Started (was: Available)

Comment 26 by yoavweiss@chromium.org, Nov 22

acomminos@ - Thanks for starting work on this. Would be great to also cover it with WPTs that can (at least partially) close https://github.com/w3c/resource-timing/issues/87

Let me know if you need help with reviews/pointers/process

Comment 27 by acomminos@fb.com, Nov 25

Sure thing, adding those tests sounds like a great idea.

Comment 28 by bugdroid1@chromium.org, Nov 29

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/28a2e63ab77610e0ab30964a0e9f47e5aa4490aa

commit 28a2e63ab77610e0ab30964a0e9f47e5aa4490aa
Author: Andrew Comminos <acomminos@fb.com>
Date: Thu Nov 29 22:36:02 2018

Add web platform test for verifying first byte timing of responseStart

Verfies that the UA measures the time before the end of header parsing
in responseStart by simulating a delay after writing the status line and
flushing to the client. Chrome fails this test for now, but Firefox
passes.

Bug: 568024
Change-Id: I61569c6a5b0d8ea2f3a06cb37a3d1ed36c06d40f
Reviewed-on: https://chromium-review.googlesource.com/c/1354645
Commit-Queue: Andrew Comminos <acomminos@fb.com>
Reviewed-by: Nicolás Peña Moreno <npm@chromium.org>
Reviewed-by: Yoav Weiss <yoavweiss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#612412}
[modify] https://crrev.com/28a2e63ab77610e0ab30964a0e9f47e5aa4490aa/third_party/blink/web_tests/external/wpt/resource-timing/SyntheticResponse.py
[modify] https://crrev.com/28a2e63ab77610e0ab30964a0e9f47e5aa4490aa/third_party/blink/web_tests/external/wpt/resource-timing/resource-timing.html
[modify] https://crrev.com/28a2e63ab77610e0ab30964a0e9f47e5aa4490aa/third_party/blink/web_tests/external/wpt/resource-timing/resource-timing.js

Comment 29 by bugdroid1@chromium.org, Dec 12

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/8bb5e943bef43144a9ba8d47b5a6ccb993ce1bee

commit 8bb5e943bef43144a9ba8d47b5a6ccb993ce1bee
Author: Andrew Comminos <acomminos@fb.com>
Date: Wed Dec 12 11:26:59 2018

Add WPT to verify that 1XX responses are used to provide responseStart timing

Verifies that the UA sets responseStart to the time that an
initial 1XX response was received by returning {100, delay, 200} and
testing that the delay is not included in responseStart.

Bug: 568024
Change-Id: I7295e6478a95abe1b6154ddf1feab764f596a5ec
Reviewed-on: https://chromium-review.googlesource.com/c/1372359
Reviewed-by: Yoav Weiss <yoavweiss@chromium.org>
Commit-Queue: Yoav Weiss <yoavweiss@chromium.org>
Cr-Commit-Position: refs/heads/master@{#615853}
[modify] https://crrev.com/8bb5e943bef43144a9ba8d47b5a6ccb993ce1bee/third_party/blink/web_tests/external/wpt/resource-timing/SyntheticResponse.py
[modify] https://crrev.com/8bb5e943bef43144a9ba8d47b5a6ccb993ce1bee/third_party/blink/web_tests/external/wpt/resource-timing/resource-timing.js

Comment 30 by bugdroid1@chromium.org, Dec 13

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4

commit 5822269484ec9a3e2b11380c6a53ecc1de3ce6a4
Author: Andrew Comminos <acomminos@fb.com>
Date: Thu Dec 13 02:26:15 2018

Report the TTFB of SPDY headers as the time that the BufferedSpdyFramer received header metadata

Currently, we report `recv_first_byte_time` to a SpdyStream as the time
after all headers have been parsed. In the case of large headers that
span multiple packets, this is often incorrect. Use the time that the
BufferedSpdyFramer received the start of the HEADERS packet instead.

Bug: 568024
Change-Id: Ic5549acd380e70c8c073d3f1d6ef9243b31135d8
Reviewed-on: https://chromium-review.googlesource.com/c/1372471
Commit-Queue: Andrew Comminos <acomminos@fb.com>
Reviewed-by: Bence Béky <bnc@chromium.org>
Cr-Commit-Position: refs/heads/master@{#616168}
[modify] https://crrev.com/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4/net/spdy/buffered_spdy_framer.cc
[modify] https://crrev.com/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4/net/spdy/buffered_spdy_framer.h
[modify] https://crrev.com/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4/net/spdy/buffered_spdy_framer_unittest.cc
[modify] https://crrev.com/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4/net/spdy/spdy_session.cc
[modify] https://crrev.com/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4/net/spdy/spdy_session.h
[modify] https://crrev.com/5822269484ec9a3e2b11380c6a53ecc1de3ce6a4/net/spdy/spdy_test_util_common.cc

Comment 31 by bugdroid1@chromium.org, Dec 14

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9

commit 1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9
Author: Andrew Comminos <acomminos@fb.com>
Date: Fri Dec 14 05:22:38 2018

Implement 'responseStart' performance timing as time-to-first-byte for HTTP/1.1 and HTTP/2

For HTTP/1.1, use the time immediately after the first packet from the
header was read by the parser. This behaviour is verified by the test
wpt/resource-timing/resource-timing.html.

For HTTP/2, use `recv_first_byte_time` reported by the SpdyStream.

For cached responses, use the time after load, but before parsing the
headers of the cached entry.

Bug: 568024
Change-Id: I4a6215596be38b1deb84a56924522ef7925a94d4
Reviewed-on: https://chromium-review.googlesource.com/c/1356234
Commit-Queue: Andrew Comminos <acomminos@fb.com>
Reviewed-by: Matt Falkenhagen <falken@chromium.org>
Reviewed-by: Kinuko Yasuda <kinuko@chromium.org>
Reviewed-by: Matt Menke <mmenke@chromium.org>
Reviewed-by: Nicolás Peña Moreno <npm@chromium.org>
Cr-Commit-Position: refs/heads/master@{#616589}
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/browser/service_worker/service_worker_navigation_loader.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/browser/service_worker/service_worker_navigation_loader_unittest.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/browser/service_worker/service_worker_url_request_job.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/common/resource_timing_info.h
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/public/common/load_timing_info.mojom
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/public/common/load_timing_info_struct_traits.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/public/common/load_timing_info_struct_traits.h
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/renderer/loader/resource_dispatcher.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/renderer/loader/web_url_loader_impl.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/renderer/resource_timing_info_conversions.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/renderer/service_worker/service_worker_subresource_loader.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/content/renderer/service_worker/service_worker_subresource_loader_unittest.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/base/load_timing_info.h
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/http/http_basic_stream.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/http/http_cache_transaction.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/http/http_cache_transaction.h
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/http/http_network_transaction_unittest.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/http/http_stream_parser.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/http/http_stream_parser.h
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/spdy/spdy_stream.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/url_request/url_request.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/url_request/url_request_redirect_job.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/net/url_request/url_request_unittest.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/services/network/public/cpp/net_ipc_param_traits.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/third_party/blink/public/platform/web_url_load_timing.h
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/third_party/blink/renderer/core/timing/performance_resource_timing.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/third_party/blink/renderer/core/timing/performance_timing.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/third_party/blink/renderer/platform/exported/web_url_load_timing.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/third_party/blink/renderer/platform/loader/fetch/resource_load_timing.cc
[modify] https://crrev.com/1f2ff1cc0d5b42db366c3f809cd54aed90d6ddc9/third_party/blink/renderer/platform/loader/fetch/resource_load_timing.h

Comment 32 by bugdroid1@chromium.org, Dec 16

Project Member
The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/beaac007c5d1af9418305242050460d1f2ab33e6

commit beaac007c5d1af9418305242050460d1f2ab33e6
Author: Andrew Comminos <acomminos@fb.com>
Date: Sun Dec 16 23:12:35 2018

Verify TTFB in resource-timing.html WPT by measuring that the delay occurs between responseStart and responseEnd

When simulating a delay after a statusline, measure that the delay is
not included in the TTFB by measuring that it occurred between
responseStart and responseEnd rather than requestStart and
responseStart. This makes the tests less flaky when subject to shaky
network and thread scheduling conditions, as might be present using
Firefox's chaos mode.

Bug: 568024
Change-Id: I0356f6256af6bc87fbb14860ed05472526c03e5d
Reviewed-on: https://chromium-review.googlesource.com/c/1377725
Reviewed-by: Yoav Weiss <yoav@yoav.ws>
Commit-Queue: Andrew Comminos <acomminos@fb.com>
Cr-Commit-Position: refs/heads/master@{#617021}
[modify] https://crrev.com/beaac007c5d1af9418305242050460d1f2ab33e6/third_party/blink/web_tests/external/wpt/resource-timing/resource-timing.js

Sign in to add a comment