------------
Background
------------

Currently net-log dumps contain these two main types of data:
  (1) NetLog events
  (2) tabInfo (aka polled events)

The central concept is the NetLog event stream, (1). These are emitted by the C++ code in response to things happening in the network stack.

The tabInfo is something of a hack to expose interesting data on chrome://net-internals. It works by dumping the current state of objects in a URLRequestContext on demand, rather than plumbing through notifications of changes to the various sub-objects. This is convenient as it is much simpler to write a dumper than to emit change events, making it very easy to simply throw random data into logs or visible on chrome://net-internals.

The tabInfo data is what powers tabs like chrome://net-internals/#sockets and #httpCache.

When viewing realtime data on chrome://net-internals, the tabInfo is implemented by polling the browser every 5 seconds and having it dump the URLRequestContext's state to JSON. This is initiated from javascript:

https://chromium.googlesource.com/chromium/src/+/8ec930192089b68778b8b5ab7deab7299ca1cf83/chrome/browser/resources/net_internals/browser_bridge.js#40

And ultimately serviced by the browser:

https://chromium.googlesource.com/chromium/src/+/98e96a7390e63333cd5e1a5128ef17e489cbfbb4/net/log/net_log_util.cc#320

When chrome://net-internals is open, it compares the polled data for differences and notifies the tabs when a change has occurred so they can repaint themselves with the displayed information. So this basically works (albeit inefficiently) when viewing live data on chrome://net-internals.

However, when logging to a file (either chrome://net-export, or --log-net-log=FILENAME), only a single snapshot of the tabInfo state is written at the very end of logging:

https://chromium.googlesource.com/chromium/src/+/a9850e17ec8a7e5e754d2b804ea1a93ff2352310/net/log/write_to_file_net_log_observer.cc#83

This has problems:

 (1) If there were interesting changes in tabInfo those cannot be seen in the log files. For long running logs, this diminishes the usefulness of this data. That said, a good portion of the data is static in nature though, so this isn't usually a limiting factor when doing post mortem debugging (in my experience).

 (2) This model presents problems for  issue 472706  and issue 472699. These bugs re-imagine the on-disk NetLog as being the canonical way to dump and view NetLog data. However with the way that NetLog files are currently written, where tabInfo is snapshotted once at the end, it would not be possible to build a realtime NetLog viewer solely by watching a NetLog file. At least not for the tabInfo data. This presents a problem in transitioning the data viewers for things like #sockets and #dns out of chrome://net-internals.

----------
Proposal
----------

I thought I had already filed this somewhere, but couldn't find it, so here goes again.

A solution to this problem, is to do away with "tabInfo" and instead export this data directly as events.

This has the advantage of saving how "tabInfo" values change over time, making it possible to eventually reflect this in the UI when loading postmortem log files. It also simplifies building a "realtime" NetLog viewer simply by watching the on-disk file as it is being written. Since the on-disk log file isn't blind to the tabInfo data as it changes.

The "pure" way to accomplish this is to re-write all the code that dumps state to tabInfo and have it emit NetLog events instead. In practice though I don't think that is a workable approach -- there is a lot of code to change, and it is also inconvenient to do this for much of the state (if it were simple to do they probably would have been written as events in the first place!)

Instead, I recommend accomplishing this the same way that chrome://net-internals does today. Namely, polling values on some interval O(seconds), caching the previously polled value, and emitting change events when something has changed. The mechanism would be a bit different in that it would be implemented in C++ rather than javascript -- probably as part of WriteToFileNetLogObserver. It would also need to formalize an event type for this information, and update the viewer's Javacript to look for the data under those events rather than tabInfo.

There are two notable downsides to this approach:
  (a) Larger log files.
  (b) Adds memory to the browser process during a capture. Whereas in chrome://net-internals model the memory bloat was in a renderer process.

The extent of both these problems can be tweaked through the implementation, so I don't see them as blockers. For instance "larger log files" is just a matter of testing and measuring the impact, and tuning the particularly noisy tabInfos to be smaller.