Issue metadata
Sign in to add a comment
|
testing/libfuzzer/archive_corpus.py should either err on dupe files, or be silent |
||||||||||||||||||||||
Issue descriptionI got this in a local build (I didn't `gclient sync` which might be the cause): [5525/46701] ACTION //third_party/WebKit/Source/core:html_preload_scanner_fuzzer_seed_corpus(//build/toolchain/linux:clang_x64) ../../testing/libfuzzer/archive_corpus.py:35: UserWarning: Duplicate name: 'document-write-then-close.html' z.write(corpus_file, os.path.basename(corpus_file)) ../../testing/libfuzzer/archive_corpus.py:35: UserWarning: Duplicate name: 'write-script-waiting-for-style-crash.html' z.write(corpus_file, os.path.basename(corpus_file)) ../../testing/libfuzzer/archive_corpus.py:35: UserWarning: Duplicate name: 'badentity.xhtml' z.write(corpus_file, os.path.basename(corpus_file)) ../../testing/libfuzzer/archive_corpus.py:35: UserWarning: Duplicate name: 'document-reload-with-failed-deferred-scripts.html' z.write(corpus_file, os.path.basename(corpus_file)) [5526/46701] ACTION //third_party/WebKit/Source/core:stylesheet_contents_fuzzer_seed_corpus(//build/toolchain/linux:clang_x64) ../../testing/libfuzzer/archive_corpus.py:35: UserWarning: Duplicate name: 'box.png' z.write(corpus_file, os.path.basename(corpus_file)) ../../testing/libfuzzer/archive_corpus.py:35: UserWarning: Duplicate name: 'background-image.html' z.write(corpus_file, os.path.basename(corpus_file)) We want our builds to be silent. This means that things are either important enough to be a hard error, or if this isn't really a problem, the tool should just shut up.
,
Oct 7 2016
This seems to be a feature of zipfile module, we can use warnings module to silent this: https://github.com/log2timeline/plaso/issues/8#issuecomment-147937769
,
Oct 7 2016
Should be fine, since zipfile uses it:
...
def _writecheck(self, zinfo):
"""Check for errors before writing a file to the archive."""
if zinfo.filename in self.NameToInfo:
import warnings
warnings.warn('Duplicate name: %r' % zinfo.filename, stacklevel=3)
...
or we have to switch to another way of creating an archive.
,
Nov 21 2016
,
Nov 21 2016
,
Nov 22 2016
After discussion in the CL comments (https://codereview.chromium.org/2520983002/), we've decided to interrupt the build in case of a warning happened.
,
Nov 22 2016
,
Nov 29 2016
I understood what's the problem. We recursively parse seed_corpus directory, then add all files to the archive. Files in sub-directories of seed_corpus dir can have same names -> warning.
,
Dec 2 2016
I noticed about 100 of these while building 56.0.2924.10 today. chrome/tools/build/make_zip.py:45: UserWarning: Duplicate name: 'icudtl.dat' output.write(os.path.join(build_dir, f), f) chrome/tools/build/make_zip.py:45: UserWarning: Duplicate name: 'natives_blob.bin' output.write(os.path.join(build_dir, f), f) chrome/tools/build/make_zip.py:45: UserWarning: Duplicate name: 'resources/inspector/toolbox.js' output.write(os.path.join(build_dir, f), f)
,
Dec 2 2016
pdknsk@, the fix hasn't been landed yet, but it's on its way: https://codereview.chromium.org/2520983002/
,
Dec 2 2016
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/0aa34793db987eec0518583d2a03f72ce9be8f4a commit 0aa34793db987eec0518583d2a03f72ce9be8f4a Author: mmoroz <mmoroz@chromium.org> Date: Fri Dec 02 10:07:59 2016 [libfuzzer] Turn warnings from zipfile into errors while archiving seed corpus. R=aizatsky@chromium.org, inferno@chromium.org, ochang@chromium.org, thakis@chromium.org BUG= 653920 Review-Url: https://codereview.chromium.org/2520983002 Cr-Commit-Position: refs/heads/master@{#435903} [modify] https://crrev.com/0aa34793db987eec0518583d2a03f72ce9be8f4a/testing/libfuzzer/archive_corpus.py
,
Dec 2 2016
,
Dec 7 2016
It's a different script. https://bugs.chromium.org/p/chromium/issues/detail?id=672060
,
Dec 9 2016
Good catch! Thanks for noticing that. |
|||||||||||||||||||||||
►
Sign in to add a comment |
|||||||||||||||||||||||
Comment 1 by mmoroz@chromium.org
, Oct 7 2016