lint_test_expectations.py UnicodeDecodeError while reading swap file in FlagExpectations |
|||
Issue description
What steps will reproduce the problem?
(1) Open a file in third_party/WebKit/LayoutTests/FlagExpectations/ in a text editor which creates swap files in the same directory
e.g. '$ vim third_party/WebKit/LayoutTests/FlagExpectations/README.txt'
(2) Run third_party/blink/tools/lint_test_expectations.py
While attempting to read '.README.txt.swp', we get an unhandled exception:
UnicodeDecodeError raised: 'utf8' codec can't decode byte 0xde in position 16: invalid continuation byte
Traceback (most recent call last):
File "/usr/local/google/home/mcnee/code/chromium/src/third_party/blink/tools/blinkpy/web_tests/lint_test_expectations.py", line 178, in main
exit_status = run_checks(host, options)
File "/usr/local/google/home/mcnee/code/chromium/src/third_party/blink/tools/blinkpy/web_tests/lint_test_expectations.py", line 132, in run_checks
failures.extend(lint(host, options))
File "/usr/local/google/home/mcnee/code/chromium/src/third_party/blink/tools/blinkpy/web_tests/lint_test_expectations.py", line 56, in lint
expectations_dict = port_to_lint.all_expectations_dict()
File "/usr/local/google/home/mcnee/code/chromium/src/third_party/blink/tools/blinkpy/web_tests/port/base.py", line 1406, in all_expectations_dict
expectations[path] = self._filesystem.read_text_file(path)
File "/usr/local/google/home/mcnee/code/chromium/src/third_party/blink/tools/blinkpy/common/system/filesystem.py", line 254, in read_text_file
return f.read()
File "/usr/local/google/home/mcnee/.vpython-root/216dab/lib/python2.7/codecs.py", line 686, in read
return self.reader.read(size)
File "/usr/local/google/home/mcnee/.vpython-root/216dab/lib/python2.7/codecs.py", line 492, in read
newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xde in position 16: invalid continuation byte
,
May 10 2018
Is this new or did you assign it to me because I touched this file last?
,
May 10 2018
If I understand the bug, this seems like a bug in your text editor. I don't think it makes sense to add complexity to the script to avoid .swp files.
,
May 10 2018
It looks like vim supports something called backupdir where you can change the directory these files are stored in. Is that an option for you?
,
May 10 2018
Sorry, I should have mentioned it was based on your introduction of all_expectations_dict here: https://chromium.googlesource.com/chromium/src/+/788f3061e6fb55c35fdccfc42085a1b612b99223
,
May 10 2018
Re #3 and #4: Once someone figures out that the swp files are the culprit, they can simply close the buffer and the swp file will be removed. Since it's not entirely clear what the cause is from the trace, it would be nice if we, say, handle the exception to alert the user of an unreadable file in the directory and then either re-raise the exception or continue to the next file. That way there's no need for the code to identify and avoid swp files specifically.
,
May 10 2018
Yeah, I see how it could take a while to unwind that python exception into an error in a temp file. I'd be okay with adding a little error handling to the parser. Would you be willing to do that?
,
May 10 2018
Re #7: Sure.
,
May 22 2018
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/cb7a6229bc447791b0e4ed60de6ffdd79667d9f7 commit cb7a6229bc447791b0e4ed60de6ffdd79667d9f7 Author: Kevin McNee <mcnee@chromium.org> Date: Tue May 22 14:51:00 2018 Notify of unreadable files in the FlagExpectations directory If a file that cannot be decoded as utf8 is inadvertently created in the FlagExpectations directory (e.g. a swap file from a text editor), identify the file for the user before termination. Bug: 841952 Change-Id: Id0c18d795c999ec91c759c67a78d2be3d3a01308 Reviewed-on: https://chromium-review.googlesource.com/1055474 Reviewed-by: Philip Rogers <pdr@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Reviewed-by: Quinten Yearsley <qyearsley@chromium.org> Commit-Queue: Kevin McNee <mcnee@chromium.org> Cr-Commit-Position: refs/heads/master@{#560581} [modify] https://crrev.com/cb7a6229bc447791b0e4ed60de6ffdd79667d9f7/third_party/blink/tools/blinkpy/web_tests/port/base.py [modify] https://crrev.com/cb7a6229bc447791b0e4ed60de6ffdd79667d9f7/third_party/blink/tools/blinkpy/web_tests/port/base_unittest.py
,
May 22 2018
We now identify the unreadable file. |
|||
►
Sign in to add a comment |
|||
Comment 1 by mcnee@chromium.org
, May 10 2018