New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 674635 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: May 2017
Cc:
vasilvv@chromium.org
rch@chromium.org
█ danzh@chromium.org
█ fayang@chromium.org
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

Quic EndToEndTest.LargePostEarlyResponse flaky on chromecast devices

Project Member Reported by mbjorge@google.com, Dec 15 2016 
As of recently (seems like early Dec. 2016, I will look throught he history and see if I can find a more exact time) the LargePostEarlyResponse test  fro net_unittests (https://cs.chromium.org/chromium/src/net/tools/quic/end_to_end_test.cc?q=LargePostEarlyResponse&sq=package:chromium&l=2536) has been flaky when run on devices.

Looking at device logs, it appears that pretty much all the LargePostEarlyResponse/# tests fail the first time, with errors like:

[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/0
[2262:2262:1215/020042.662576:1735301129:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92b1600
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/0, where GetParam() = { server_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_36 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: RENO disable_hpack_dynamic_table: 0 force_hol_blocking: 1 use_cheap_stateless_reject: 1 } (148 ms)
[12985/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/0 (148 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/1
[2262:2262:1215/020042.810182:1735448731:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92b5e00
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/1, where GetParam() = { server_supported_versions: QUIC_VERSION_35 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_35 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: RENO disable_hpack_dynamic_table: 0 force_hol_blocking: 1 use_cheap_stateless_reject: 1 } (148 ms)
[12986/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/1 (148 ms)
[12987/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/2 (175 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/3
[2262:2262:1215/020043.145327:1735783876:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92ab000
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/3, where GetParam() = { server_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_36 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: TBBR disable_hpack_dynamic_table: 0 force_hol_blocking: 1 use_cheap_stateless_reject: 1 } (158 ms)
[12988/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/3 (158 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/4
[2262:2262:1215/020043.293874:1735932421:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92ab800
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/4, where GetParam() = { server_supported_versions: QUIC_VERSION_35 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_35 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: TBBR disable_hpack_dynamic_table: 0 force_hol_blocking: 1 use_cheap_stateless_reject: 1 } (149 ms)
[12989/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/4 (149 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/5
[2262:2262:1215/020043.442140:1736080688:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92ab400
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/5, where GetParam() = { server_supported_versions: QUIC_VERSION_34 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_34 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: TBBR disable_hpack_dynamic_table: 0 force_hol_blocking: 1 use_cheap_stateless_reject: 1 } (147 ms)
[12990/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/5 (147 ms)
[12991/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/6 (131 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/7
[2262:2262:1215/020043.734873:1736373428:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb91baa00
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/7, where GetParam() = { server_supported_versions: QUIC_VERSION_35 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_35 client_supports_stateless_rejects: 0 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: QBIC disable_hpack_dynamic_table: 0 force_hol_blocking: 0 use_cheap_stateless_reject: 0 } (133 ms)
[12992/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/7 (133 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/8
[2262:2262:1215/020043.852648:1736491198:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92a2a00
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/8, where GetParam() = { server_supported_versions: QUIC_VERSION_34 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_34 client_supports_stateless_rejects: 0 server_uses_stateless_rejects_if_peer_supported: 1 congestion_control_tag: QBIC disable_hpack_dynamic_table: 0 force_hol_blocking: 0 use_cheap_stateless_reject: 0 } (115 ms)
[12993/13562] EndToEndTests/EndToEndTest.LargePostEarlyResponse/8 (115 ms)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/9
[2262:2262:1215/020043.967741:1736606291:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb92a9800
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/9, where GetParam() = { server_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_36 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 0 congestion_control_tag: QBIC disable_hpack_dynamic_table: 0 force_hol_blocking: 0 use_cheap_stateless_reject: 0 } (115 ms)


And then on subsequent retries, some will pass and some will fail again. And then some more retries, and some more pass and a few still fail.

Retrying 1 test (retry #3)
[ RUN      ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/9
[2679:2679:1215/020219.243663:1831882214:INFO:quic_server.cc(124)] Listening on 127.0.0.1:0
../../net/tools/quic/end_to_end_test.cc:2467: Failure
Value of: nullptr
  Actual: 4-byte object <00-00 00-00>
Expected: stream
Which is: 0xb82df600
[  FAILED  ] EndToEndTests/EndToEndTest.LargePostEarlyResponse/9, where GetParam() = { server_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 client_supported_versions: QUIC_VERSION_36,QUIC_VERSION_35,QUIC_VERSION_34 negotiated_version: QUIC_VERSION_36 client_supports_stateless_rejects: 1 server_uses_stateless_rejects_if_peer_supported: 0 congestion_control_tag: QBIC disable_hpack_dynamic_table: 0 force_hol_blocking: 0 use_cheap_stateless_reject: 0 } (261 ms)

Even on runs that do ultimately pass, it's only because of retries. The first time around, almost all the variants (LargePostEarlyResponse/1, .. LargePostEarlyResponse/26) fail but due to how the wind is blowing they manage to succeed before gtest gives up on retries

Comment 1 by rch@chromium.org, Dec 15 2016 

Which trybot can we reproduce this on?

Comment 2 by mbjorge@google.com, Dec 15 2016 

I'm not sure if this is happening on any of the chromium trybots. We're seeing this when we run the net_unittests on the physical chromecast devices in our internal test infra.

Comment 3 by mbjorge@google.com, Dec 15 2016 

Looking at the test it seems like:

  // The stream is reset by server's reset stream.
  EXPECT_EQ(stream, nullptr);

is not actually happening for some reason

Comment 4 by mbjorge@chromium.org, May 15 2017

Status: WontFix (was: Untriaged)
Obsolete, these tests have been deleted

Comment 5 by rch@chromium.org, May 16 2017 

Which tests have been deleted? I think EndToEndTest.LargePostEarlyResponse is still around.

Comment 6 by mbjorge@chromium.org, May 16 2017 

I can't find them with codesearch: https://cs.chromium.org/search/?q=LargePostEarlyResponse+package:%5Echromium$&type=cs

Looks like they were removed with the last QUIC roll on May 12th: https://codereview.chromium.org/2875333002

Comment 7 by rch@chromium.org, May 16 2017 

Hah! Indeed. Good point :)
►

Sign in to add a comment