Text log requires excessive memory in logdog |
|||
Issue descriptionAndrii reported that viewing a 3MB text log on logdog took 451MB memory with a 108MB heap. The equivalent tab on buildbot took 67MB. https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Ftryserver.chromium.linux%2Flinux_chromium_asan_rel_ng%2F257003%2F%2B%2Frecipes%2Fsteps%2Fcompile__with_patch_%2F0%2Fstdout vs https://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_asan_rel_ng/builds/257003/steps/compile%20%28with%20patch%29/logs/stdio I would expect us to be closer to 130MB if we continue to include the metadata.
,
Nov 9 2017
FWIW, this is not surprising at all. BuildBot streams rendered HTML and renders it. LogDog hosts a non-trivial rendering application including RPCs, authentication, logic, caching, and frameworks (e.g., Polymer). Its base memory requirement will be notably larger for sure. After logs are rendered, their raw data is made candidate for garbage collection; only the rendered HTML should remain. Consequently, its per-log memory usage should scale similarly with BuildBot. Both applications have a base memory cost and a per-log cost. The per-log costs should be similar, since they both render logs similarly. The base cost for LogDog is expected to be much higher, and this is WAI.
,
Nov 9 2017
450MB memory for 3MB text is a huge overhead, whether it is per log or the base one. LogDog webpage isn't the only webpage opened at a time and until every machine ships with TB of RAM, I think 450MB ram requirement is not acceptable. However, good news is that today, the same page is just 100MB, which is better than buildbot was. So, I think this has actually greatly improved in last year. |
|||
►
Sign in to add a comment |
|||
Comment 1 by sheriffbot@chromium.org
, Nov 9 2017Status: Untriaged (was: Available)