The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/510b09b90253a027bf2d33ad2b517b61e51a8fd9

commit 510b09b90253a027bf2d33ad2b517b61e51a8fd9
Author: Makoto Shimazu <shimazu@chromium.org>
Date: Wed Nov 01 04:43:56 2017

Disable an ExtensionWebRequestApiTest on Mac/Linux

ExtensionWebRequestApiTest.WebRequestUnloadImmediately in
renderer_side_navigation_browser_tests is flaky on Linux/Mac. This disables it.

TEST=browser_tests --disable-browser-side-navigation --gtest_filter=ExtensionWebRequestApiTest.WebRequestUnloadImmediately
TBR=battre@chromium.org

Bug: 780369
Change-Id: I61c32c1d35de888298d8344d0d367044ece64d39
Reviewed-on: https://chromium-review.googlesource.com/748661
Commit-Queue: Makoto Shimazu <shimazu@chromium.org>
Reviewed-by: Makoto Shimazu <shimazu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#513073}
[modify] https://crrev.com/510b09b90253a027bf2d33ad2b517b61e51a8fd9/chrome/browser/extensions/api/web_request/web_request_apitest.cc

93442:96515:1031/213616.737535:WARNING:embedded_test_server.cc(228)] Request not handled. Returning 404: /favicon.ico
[93442:775:1031/213616.755571:INFO:CONSOLE(0)] "[FAIL] insertMultipleOriginFramesAndImmediatelyRemoveFrames: API Test Error in insertMultipleOriginFramesAndImmediatelyRemoveFrames
Actual: [{"error":"net::ERR_ABORTED","frameId":1,"fromCache":false,"initiator":"http://hostname-of-main-frame:54376","method":"GET","parentFrameId":0,"tabId":12,"type":"sub_frame"},{"error":"net::ERR_ABORTED","frameId":-1,"fromCache":false,"initiator":"http://hostname-of-main-frame:54376","method":"GET","parentFrameId":-1,"tabId":-1,"type":"sub_frame"}]
Expected: [{"method":"GET","frameId":1,"parentFrameId":0,"tabId":12,"type":"sub_frame","fromCache":false,"initiator":"http://hostname-of-main-frame:54376","error":"net::ERR_ABORTED"},{"method":"GET","frameId":2,"parentFrameId":0,"tabId":12,"type":"sub_frame","fromCache":false,"initiator":"http://hostname-of-main-frame:54376","error":"net::ERR_ABORTED"}]
Error
    at Object.handleRequest (extensions::binding:64:27)
    at Object.<anonymous> (extensions::binding:374:32)
    at Object.<anonymous> (extensions::test:225:20)
    at Object.handleRequest (extensions::binding:64:27)
    at Object.<anonymous> (extensions::binding:374:32)
    at chrome-extension://chbigaineimkgiedpecafcaejhjcdebm/test_unload6.js:66:19
    at listener (chrome-extension://chbigaineimkgiedpecafcaejhjcdebm/test_unload6.js:26:7)
    at EventImpl.dispatchToListener (extensions::event_bindings:403:22)
    at Event.publicClassPrototype.(anonymous function) [as dispatchToListener] (extensions::utils:138:26)
    at EventImpl.dispatch_ (extensions::event_bindings:387:35)", source: chrome-extension://chbigaineimkgiedpecafcaejhjcdebm/test_unload.html?6 (0)
[93442:775:1031/213617.083502:INFO:CONSOLE(0)] "[FAIL] insertCrossOriginFrameAndImmediatelyRemoveFrame: API Test Error in insertCrossOriginFrameAndImmediatelyRemoveFrame
Actual: [{"error":"net::ERR_ABORTED","frameId":2,"fromCache":false,"initiator":"http://hostname-of-main-frame:54376","method":"GET","parentFrameId":0,"tabId":12,"type":"sub_frame","url":"http://origin-different-from-other-frames:54376/slow?10"}]
Expected: [{"method":"GET","url":"http://origin-different-from-other-frames:54376/slow?10","frameId":3,"parentFrameId":0,"tabId":12,"type":"sub_frame","fromCache":false,"initiator":"http://hostname-of-main-frame:54376","error":"net::ERR_ABORTED"}]
Error
    at Object.handleRequest (extensions::binding:64:27)
    at Object.<anonymous> (extensions::binding:374:32)
    at Object.<anonymous> (extensions::test:225:20)
    at Object.handleRequest (extensions::binding:64:27)
    at Object.<anonymous> (extensions::binding:374:32)
    at chrome-extension://chbigaineimkgiedpecafcaejhjcdebm/test_unload6.js:111:19
    at listener (chrome-extension://chbigaineimkgiedpecafcaejhjcdebm/test_unload6.js:26:7)
    at EventImpl.dispatchToListener (extensions::event_bindings:403:22)
    at Event.publicClassPrototype.(anonymous function) [as dispatchToListener] (extensions::utils:138:26)
    at EventImpl.dispatch_ (extensions::event_bindings:387:35)", source: chrome-extension://chbigaineimkgiedpecafcaejhjcdebm/test_unload.html?6 (0)
../../chrome/browser/extensions/api/web_request/web_request_apitest.cc:312: Failure
Value of: RunExtensionSubtest("webrequest", "test_unload.html?6")
  Actual: false
Expected: true
Failed 2 of 2 tests

The disagreement is on the frameId

Rob, would you mind taking a look at this? I think that you added the test.

The log from comment 2 shows two failures. Filtering only the relevant keys (frameId,parentFrameId,tabId), the results are:

insertMultipleOriginFramesAndImmediatelyRemoveFrames
Actual:   [{"frameId":1,"parentFrameId":0,"tabId":12},{"frameId":-1,"parentFrameId":-1,"tabId":-1}]
Expected: [{"frameId":1,"parentFrameId":0,"tabId":12},{"frameId":2 ,"parentFrameId":0 ,"tabId":12}]

insertCrossOriginFrameAndImmediatelyRemoveFrame
Actual:   [{"frameId":2,"parentFrameId":0,"tabId":12}]
Expected: [{"frameId":3,"parentFrameId":0,"tabId":12}]

The second failure is a consequence of the first failure. In the webRequest test framework, frameIds are normalized, known (i.e. non-negative) frameIds are mapped to a positive (incrementing) surrogate ID.
So the only issue here is the first failure, i.e. the IDs of the second frame cannot be determined.
The test is at https://chromium.googlesource.com/chromium/src/+/1e2fc173eaaf9d32e3b40814a99d3084aee252b2/chrome/test/data/extensions/api_test/webrequest/test_unload6.js#48
and basically adds two frames (both from different origins) and immediately removes the frames upon insertion.

IDs are provided through the ExtensionApiFrameIdMap API.
The triggers for the webRequest events (onErrorOccurred in this case) originate from the IO thread. The ID mapping is based on information at the UI thread, so a thread hop is involved.


The expected chronological order of method calls is:
ExtensionWebContentsObserver::RenderFrameCreated -> ExtensionApiFrameIdMap::CacheFrameData
URLRequest::NotifyRequestCompleted -> ... -> WebRequestEventDetails::DetermineFrameDataOnIO -> ExtensionApiFrameIdMap::GetFrameDataOnIO -> (uses cached frame data)
ExtensionWebContentsObserver::RenderFrameDeleted -> ExtensionApiFrameIdMap::RemoveFrameData

I conjecture that the the ID mapping is deleted before the request for the ID in the WebRequestEventDetails is fulfilled, i.e. ExtensionWebContentsObserver::RenderFrameDeleted is called before URLRequest::NotifyRequestCompleted.

If this is true, then there is not much that we can do (short of adding an artificial timer before deleting frame IDs).
Dominic, do you know anyone who can confirm or reject my conjecture?