The data structure that is using this schema needs to be memory-mapped, so this really must be "no-serialization serialization".

My impression was that Mojo does not support this, but it's worth double-checking. Who would be the right person to talk to?

Regarding not depending on external libraries and writing custom serialization code: we originally did consider that, but ultimately decided against it.

Flatbuffers gives us an IDL compiler to avoid writing boilerplate by hand, and otherwise it is really tiny: less than 1500 additional lines of code shipped with Chrome. Essentially we would have needed to reimplement most of that for our use case anyway.

No build failures after #16821: https://build.chromium.org/p/chromium.chrome/builders/Google%20Chrome%20Win?numbuilds=200

Seems like a flake indeed.

Well, then we should figure out why this flaked and fix it. Compiles shouldn't be flaky.

While I share your concerns, given that this step failed exactly once one one builder over a period of more than 6 months, the logs don't tell us anything useful, and the state on the builder was already destroyed, I don't see what we could really do at this point. Of course, if the problem comes up again we will be sure to investigate.

Does that sounds reasonable?

flatc.exe failed today on Win x64 Build bot.

https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium%2FWin_x64%2F13581%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout


FAILED: gen/components/url_pattern_index/flat/url_pattern_index_generated.h 
C:/b/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/gn_run_binary.py flatc.exe -c --keep-prefix -o gen/components/url_pattern_index/flat -I ../../ ../../components/url_pattern_index/flat/url_pattern_index.fbs
flatc.exe failed with exit code -1073741819
[8956/53616] ACTION //components/subresource_filter/core/common/flat:indexed_ruleset_gen(//build/toolchain/win:win_clang_x64)
FAILED: gen/components/subresource_filter/core/common/flat/indexed_ruleset_generated.h 
C:/b/depot_tools/win_tools-2_7_6_bin/python/bin/python.exe ../../build/gn_run_binary.py flatc.exe -c --keep-prefix -o gen/components/subresource_filter/core/common/flat -I ../../ ../../components/subresource_filter/core/common/flat/indexed_ruleset.fbs
flatc.exe failed with exit code -1073741819

Is it a flake again?

+Wouter, do you know what the -1073741819 exit code could mean?

The code is meant to only ever exit with 0 or 1, and there's no actual error message.

-1073741819 is 0xC0000005 which is an access violation error code. Not sure why it is giving that as return code.. is it running in some kind of sandbox?

I've never seen flatc give an access violation on parsing or generating code, so this is pretty serious.

I'm back in the office on Monday, when I can debug this. If you can get me the exact files to run it with that cause that that be even be better (though I suppose I can just pull the entirety of chromium)

Hi Wouter,

The files are here (https://chromium.googlesource.com/chromium/src.git/+/master/components/url_pattern_index/flat/) and here (https://chromium.googlesource.com/chromium/src.git/+/master/components/subresource_filter/core/common/flat/). They are built separately.

Yes, this probably is running in some sandbox, although I don't know details. Actually, I guess the python script is the one returning such a code (https://cs.chromium.org/chromium/src/build/gn_run_binary.py), likely subprocess.call().

I doubt you will be able to reproduce the issue, it flakes once in 3-6 months.

So can this be a problem external to flatc? truncated input files? running out of memory? what are typical sources for flakyness here?

I don't think we run commands that run as part of the build in a sandbox. The most common cause of flakiness is programs being buggy.

If it's a bug in flatc it would have to be non-deterministic problem like an unitialized variable or something (flatc is entirely single-threaded). I'll have a look anyway.

I'm tracking progress on finding this problem here: https://b.corp.google.com/u/0/issues/64612522
As predicted, so far not getting far finding anything.

This just started happening on my development workstation and happens every time I try to launch a build.

[984/8854] ACTION //extensions/browser/api/declarative_net...extension_ruleset_gen(//build/toolchain/win:win_clang_x86)
FAILED: gen/extensions/browser/api/declarative_net_request/flat/extension_ruleset_generated.h
C:/python_27_amd64/files/python.exe ../../build/gn_run_binary.py flatc.exe -c --keep-prefix -o gen/extensions/browser/api/declarative_net_request/flat -I ../../ ../../extensions/browser/api/declarative_net_request/flat/extension_ruleset.fbs
flatc.exe failed with exit code -1073740791
[989/8854] ACTION //components/url_pattern_index/flat:url_pattern_index_gen(//build/toolchain/win:win_clang_x86)
FAILED: gen/components/url_pattern_index/flat/url_pattern_index_generated.h
C:/python_27_amd64/files/python.exe ../../build/gn_run_binary.py flatc.exe -c --keep-prefix -o gen/components/url_pattern_index/flat -I ../../ ../../components/url_pattern_index/flat/url_pattern_index.fbs
flatc.exe failed with exit code -1073740791
[996/8854] ACTION //components/subresource_filter/core/com...t:indexed_ruleset_gen(//build/toolchain/win:win_clang_x86)
FAILED: gen/components/subresource_filter/core/common/flat/indexed_ruleset_generated.h
C:/python_27_amd64/files/python.exe ../../build/gn_run_binary.py flatc.exe -c --keep-prefix -o gen/components/subresource_filter/core/common/flat -I ../../ ../../components/subresource_filter/core/common/flat/indexed_ruleset.fbs
flatc.exe failed with exit code -1073740791

brucedawson@ was investigating a flaky build failure on Windows where the workaround was to flush to disk periodically during the build process. I can't find the bug; could it be related?

I'm going to try to work around this by purging flatc.exe and rebuilding it.

Deleting and manually rebuilding flatc.exe allowed the build to complete.

That's  Issue 644525 . I agree this here looks like a dupe.

Agreed, looks like flatc.exe is the victim here, tentatively closing this, and blame it on  Issue 644525 .

It's almost certainly  bug 644525  so I'm changing this to a duplicate of that bug.

The good news is that  bug 644525  is now better understood and we believe we can create a workaround.

Thanks a lot for investigating this, Bruce!