Non-ascii file names in a ZIP file packed by some external archivers appear garbled |
||||||||||||||||||
Issue descriptionFollow-up of Issue 834544 . Chrome Version: 68.0.3439.0 Steps To Reproduce: (1) Zip a file or folder with non-ascii file name by the default archiver of Windows 10 (confirmed on version 1803/Build 17134.1/Japanese). (2) Copy the file to Chrome OS device, by either uploading the file to Drive or using removable media. (3) Open the file in the Chrome OS Files app. Expected Result: The file name appear as it looked in Windows Actual Result: The file name sometimes look garbled. For example: "ÉVé╡éóâeâLâXâg âhâLâàâüâôâg.txt" instead of "新規テキスト ドキュメント.txt" How frequently does this problem reproduce? (Always, sometimes, hard to reproduce?) This happens deterministically, depending on the file name and the archiver which created the ZIP file. What is the impact to the user, and is there a workaround? If so, what is it? Impact: The file contents can still be read. Workaround 1: <deleted, turned out to be invalid> Workaround 2: User may use third-party archivers on Windows which encodes file names in UTF-8. Workaround 3: If the file is on Drive, see file name in drive.google.com. I have only confirmed this with Japanese file names. I also tried with some accented Latin letters like "ç" in file name, but Windows Compressed Folder rejected zipping those files at step 1, thus could not be affected by this issue. However some third-party archivers may hit this issue.
,
May 24 2018
,
May 24 2018
,
May 25 2018
,
Jun 16 2018
> I have only confirmed this with Japanese file names. I also tried with some accented Latin letters like "ç" in file name, but Windows Compressed Folder rejected zipping those files at step 1, thus could not be affected by this issue. However some third-party archivers may hit this issue. Try to compress those files after changing the default encoding for non-Unicode apps to windows-1252. BTW, it's surprising that Windows-built-in zip compressor does not support a new spec for zip that allows UTF-8 filenames. Anyway, on Chrome OS, the encoding detector (compact encoding detector) can be used to detect the encoding of file names. Perhaps, it's better to detect the encoding after concatenating all the file names in a given zip archive'.
,
Jul 31
Employees working with my company's Chromebook are having trouble with this problem. Japanese is included in the compressed file sent from the bank. It is impossible for us to send the file without including Japanese to all the contractors, so I want you to solve it at an early date
,
Jul 31
In my company's chromebook there is a problem that contents are not displayed in the file application including Japanese
,
Jul 31
Thanks for the report. I think you are using version 67 or before. It is known issue and is partially resolved on M68. ( Issue 834544 ) We are fixing this issue in two steps. On M68, it will be able to read contents of the files, although the file names will appear garbled (文字化け in Japanese) with a certain type of ZIP file. Another fix will be made for reconstructing such file names. This is currently targeted by M69, but not assured right now as it depends on other priority tasks.
,
Aug 2
,
Aug 8
,
Aug 15
,
Aug 15
Added another workaround in the description as this was not mentioned so far. > Set "ZIP archiver - Unpacker" flag to "Disabled".
,
Aug 15
The new ZIP extension was enabled for reading ZIP file by default at version 65. Therefore all the versions since that are affected. (and Workaround 1 is effective since then)
,
Aug 17
I found the workaround #1 may still not work, as I found it shows no files with some example. I am investigating. Was it showing Shift-JIS filenames correctly with version 64 or even before?
,
Aug 17
According to the report in Issue 834544 , Workaround #1 will not work as I expected. It is because version 64 and before had same issue as 65--67. > - The customer tried to uncompress the same ZIP file titled in Japanese character from any managed Chromebook device with older version of Chrome 64, but the result the same. (I also tested several different OS versions, but could not resolved the issue. )
,
Aug 17
,
Sep 6
Friendly ping. Is there any updates on this issue? Can we expect to resolve the issue on M70?
,
Sep 6
,
Sep 6
We haven't came up with a single solution so far. This is not considered a release blocker for M70, but I will personally try to dig into one of the approaches in my mind tomorrow. We didn't plan to fix this at M70.
,
Sep 6
,
Sep 7
I'll finish a working patch early next week. However I expect it's going to affect other languages than Japanese in a bad manner in an unexpected way, unless tested and brushed up enough. So I basically think it should be done on M71, but we'll make the final decision next week.
,
Sep 13
,
Sep 28
http://www.mext.go.jp/a_menu/koutou/shinkou/07021403/002/002/001/003/001.zip Will be a good test case because it contains subdirectory with Japanese.
,
Oct 18
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/f3b6430b37261dd5180011eba64f4d5cb494fc9d commit f3b6430b37261dd5180011eba64f4d5cb494fc9d Author: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Date: Thu Oct 18 08:42:10 2018 Zip Archiver: Auto-detect and decode filenames when it's ShiftJIS. Some third-party archivers generate ZIP files which claim file names are coded in CP437 but actually put file names in other coding system. This change attempts to guess which encoding system they are, and decodes the file and directory names. Not all detected encodings are supported by the TextDecoder. There is also coding name difference between CED and TextDecoder. By this reason the set of encodings to be handled is whitelisted and translated by a mapping. We can simply add mappings to support other encodings. We also assume that all the files are coded in the uniform coding system, so as to stabilize the automated coding system detection. Bug: 846195 Test: browser_tests --gtest_filter='*ZipFiles*FilesAppBrowserTest*' --gtest_also_run_disabled_tests Cq-Include-Trybots: luci.chromium.try:closure_compilation Change-Id: I284c189431d4fbac5554fb62507f1646517d279d Reviewed-on: https://chromium-review.googlesource.com/c/1251684 Commit-Queue: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Reviewed-by: Anand Mistry <amistry@chromium.org> Reviewed-by: Ben Wells <benwells@chromium.org> Cr-Commit-Position: refs/heads/master@{#600696} [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/browser/chromeos/extensions/file_manager/private_api_misc.cc [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/browser/chromeos/extensions/file_manager/private_api_misc.h [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/browser/chromeos/file_manager/file_manager_browsertest.cc [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/browser/resources/chromeos/zip_archiver/cpp/volume.cc [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/browser/resources/chromeos/zip_archiver/js/volume.js [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/common/extensions/api/file_manager_private.idl [add] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/chrome/test/data/chromeos/file_manager/archive_sjis.zip [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/extensions/browser/extension_function_histogram_value.h [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/third_party/closure_compiler/externs/file_manager_private.js [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/tools/metrics/histograms/enums.xml [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/ui/file_manager/integration_tests/file_manager/zip_files.js [modify] https://crrev.com/f3b6430b37261dd5180011eba64f4d5cb494fc9d/ui/file_manager/integration_tests/test_util.js
,
Oct 19
,
Oct 20
Your change meets the bar and is auto-approved for M71. Please go ahead and merge the CL to branch 3578 manually. Please contact milestone owner if you have questions. Owners: benmason@(Android), kariahda@(iOS), kbleicher@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Oct 22
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5 commit 31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5 Author: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Date: Mon Oct 22 02:22:54 2018 Zip Archiver: Auto-detect and decode filenames when it's ShiftJIS. Some third-party archivers generate ZIP files which claim file names are coded in CP437 but actually put file names in other coding system. This change attempts to guess which encoding system they are, and decodes the file and directory names. Not all detected encodings are supported by the TextDecoder. There is also coding name difference between CED and TextDecoder. By this reason the set of encodings to be handled is whitelisted and translated by a mapping. We can simply add mappings to support other encodings. We also assume that all the files are coded in the uniform coding system, so as to stabilize the automated coding system detection. TBR=yamaguchi@chromium.org (cherry picked from commit f3b6430b37261dd5180011eba64f4d5cb494fc9d) Bug: 846195 Test: browser_tests --gtest_filter='*ZipFiles*FilesAppBrowserTest*' --gtest_also_run_disabled_tests Cq-Include-Trybots: luci.chromium.try:closure_compilation Change-Id: I284c189431d4fbac5554fb62507f1646517d279d Reviewed-on: https://chromium-review.googlesource.com/c/1251684 Commit-Queue: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Reviewed-by: Anand Mistry <amistry@chromium.org> Reviewed-by: Ben Wells <benwells@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#600696} Reviewed-on: https://chromium-review.googlesource.com/c/1293092 Reviewed-by: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Cr-Commit-Position: refs/branch-heads/3578@{#195} Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034} [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/browser/chromeos/extensions/file_manager/private_api_misc.cc [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/browser/chromeos/extensions/file_manager/private_api_misc.h [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/browser/chromeos/file_manager/file_manager_browsertest.cc [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/browser/resources/chromeos/zip_archiver/cpp/volume.cc [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/browser/resources/chromeos/zip_archiver/js/volume.js [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/common/extensions/api/file_manager_private.idl [add] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/chrome/test/data/chromeos/file_manager/archive_sjis.zip [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/extensions/browser/extension_function_histogram_value.h [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/third_party/closure_compiler/externs/file_manager_private.js [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/tools/metrics/histograms/enums.xml [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/ui/file_manager/integration_tests/file_manager/zip_files.js [modify] https://crrev.com/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5/ui/file_manager/integration_tests/test_util.js
,
Oct 22
To be confirmed on M71 and M72. This has only been fixed for ShiftJIS (Japanese). Feel free to file separate issues if there's more encodings (or languages) that needs to be supported.
,
Oct 23
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5 Commit: 31c8e86fa499c6ed9f5037ab5b93bf8ac3bfbbd5 Author: yamaguchi@chromium.org Commiter: yamaguchi@chromium.org Date: 2018-10-22 02:22:54 +0000 UTC Zip Archiver: Auto-detect and decode filenames when it's ShiftJIS. Some third-party archivers generate ZIP files which claim file names are coded in CP437 but actually put file names in other coding system. This change attempts to guess which encoding system they are, and decodes the file and directory names. Not all detected encodings are supported by the TextDecoder. There is also coding name difference between CED and TextDecoder. By this reason the set of encodings to be handled is whitelisted and translated by a mapping. We can simply add mappings to support other encodings. We also assume that all the files are coded in the uniform coding system, so as to stabilize the automated coding system detection. TBR=yamaguchi@chromium.org (cherry picked from commit f3b6430b37261dd5180011eba64f4d5cb494fc9d) Bug: 846195 Test: browser_tests --gtest_filter='*ZipFiles*FilesAppBrowserTest*' --gtest_also_run_disabled_tests Cq-Include-Trybots: luci.chromium.try:closure_compilation Change-Id: I284c189431d4fbac5554fb62507f1646517d279d Reviewed-on: https://chromium-review.googlesource.com/c/1251684 Commit-Queue: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Reviewed-by: Anand Mistry <amistry@chromium.org> Reviewed-by: Ben Wells <benwells@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#600696} Reviewed-on: https://chromium-review.googlesource.com/c/1293092 Reviewed-by: Tatsuhisa Yamaguchi <yamaguchi@chromium.org> Cr-Commit-Position: refs/branch-heads/3578@{#195} Cr-Branched-From: 4226ddf99103e493d7afb23a4c7902ee496108b6-refs/heads/master@{#599034} |
||||||||||||||||||
►
Sign in to add a comment |
||||||||||||||||||
Comment 1 by yamaguchi@chromium.org
, May 24 2018