Issue 860023 link

Starred by 1 user

Issue metadata

Status:	Assigned
Owner:	jbudorick@chromium.org
Cc:	martiniss@chromium.org
Components:	Infra>Client>Chrome
EstimatedDays:	----
NextAction:	----
OS:	Android
Pri:	3
Type:	Bug

Unicode output is garbled in CQ output on Android bots

Project Member Reported by wychen@chromium.org, Jul 3

Issue description

In the following job (https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-kitkat-arm-rel/27440), the Unicode output is good at the raw log (https://chromium-swarm.appspot.com/task?id=3e7a42409210be10&refresh=10&show_raw=1&wide_logs=true), but become garbled in the parsed log (https://logs.chromium.org/v/?s=chromium%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941998404952438896%2F%2B%2Fsteps%2Fbase_unittests_on_Android_device_Nexus_5__with_patch_%2F0%2Flogs%2FBreakIteratorTest.BreakWordJapanese%2F0).

Snippet from the raw log:


I  204.863s run_tests_on_device(097f1f50010ab93d)  [ RUN      ] BreakIteratorTest.BreakWordChineseEnglish
I  204.863s run_tests_on_device(097f1f50010ab93d)  ../../base/i18n/break_iterator_unittest.cc:197: Failure
I  204.863s run_tests_on_device(097f1f50010ab93d)  Expected equality of these values:
I  204.863s run_tests_on_device(097f1f50010ab93d)    UTF8ToUTF16(token1)
I  204.863s run_tests_on_device(097f1f50010ab93d)      Which is: 下载
I  204.863s run_tests_on_device(097f1f50010ab93d)    iter.GetString()
I  204.863s run_tests_on_device(097f1f50010ab93d)      Which is: 下
I  204.863s run_tests_on_device(097f1f50010ab93d)  Stack trace:
I  204.863s run_tests_on_device(097f1f50010ab93d)  #00 0x75cff8f3 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x005dd8f3
I  204.863s run_tests_on_device(097f1f50010ab93d)  #01 0x75972583 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x00250583
I  204.863s run_tests_on_device(097f1f50010ab93d)  #02 0x75d02053 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x005e0053
I  204.863s run_tests_on_device(097f1f50010ab93d)  
I  204.863s run_tests_on_device(097f1f50010ab93d)  ../../base/i18n/break_iterator_unittest.cc:200: Failure
I  204.863s run_tests_on_device(097f1f50010ab93d)  Value of: iter.IsWord()
I  204.863s run_tests_on_device(097f1f50010ab93d)    Actual: true
I  204.863s run_tests_on_device(097f1f50010ab93d)  Expected: false
I  204.863s run_tests_on_device(097f1f50010ab93d)  Stack trace:
I  204.863s run_tests_on_device(097f1f50010ab93d)  #00 0x75cff8f3 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x005dd8f3
I  204.863s run_tests_on_device(097f1f50010ab93d)  #01 0x75972659 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x00250659
I  204.863s run_tests_on_device(097f1f50010ab93d)  #02 0x75d02053 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x005e0053
I  204.863s run_tests_on_device(097f1f50010ab93d)  
I  204.863s run_tests_on_device(097f1f50010ab93d)  ../../base/i18n/break_iterator_unittest.cc:201: Failure
I  204.863s run_tests_on_device(097f1f50010ab93d)  Expected equality of these values:
I  204.863s run_tests_on_device(097f1f50010ab93d)    space
I  204.863s run_tests_on_device(097f1f50010ab93d)      Which is:  
I  204.863s run_tests_on_device(097f1f50010ab93d)    iter.GetString()
I  204.863s run_tests_on_device(097f1f50010ab93d)      Which is: 载
I  204.863s run_tests_on_device(097f1f50010ab93d)  Stack trace:
I  204.864s run_tests_on_device(097f1f50010ab93d)  #00 0x75cff8f3 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x005dd8f3
I  204.864s run_tests_on_device(097f1f50010ab93d)  #01 0x75972761 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x00250761
I  204.864s run_tests_on_device(097f1f50010ab93d)  #02 0x75d02053 /data/app-lib/org.chromium.native_test-1/lib_base_unittests__library.so+0x005e0053


Snippet from the parsed log:


[ RUN      ] BreakIteratorTest.BreakWordChineseEnglish
../../base/i18n/break_iterator_unittest.cc:197: Failure
Expected equality of these values:
  UTF8ToUTF16(token1)
    Which is: ??????
  iter.GetString()
    Which is: ???
Stack trace:
../../base/i18n/break_iterator_unittest.cc:200: Failure
Value of: iter.IsWord()
  Actual: true
Expected: false
Stack trace:
../../base/i18n/break_iterator_unittest.cc:201: Failure
Expected equality of these values:
  space
    Which is:
  iter.GetString()
    Which is: ???
Stack trace:


However, the parsed log works fine on iPhone bots:

https://logs.chromium.org/v/?s=chromium%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941999084297440000%2F%2B%2Fsteps%2Fbase_unittests__iPhone_6s_Plus_iOS_11.2_%2F0%2Flogs%2FBreakIteratorTest.BreakWordJapanese%2F0


Snippets:


BreakIteratorTest.BreakWordChineseEnglish:
../../base/i18n/break_iterator_unittest.cc:197: Failure
Expected equality of these values:
  UTF8ToUTF16(token1)
    Which is: 下载
  iter.GetString()
    Which is: 下
Stack trace:
0   base_unittests                      0x000000010e19e5dc base::debug::StackTrace::StackTrace() + 28
1   base_unittests                      0x000000010e0fa030 StackTraceGetter::CurrentStackTrace(int, int) + 64
2   base_unittests                      0x000000010e1096d7 testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 71
3   base_unittests                      0x000000010e108b1d testing::internal::AssertHelper::operator=(testing::Message const&) const + 125
4   base_unittests                      0x000000010d304dc2 base::i18n::BreakIteratorTest_BreakWordChineseEnglish_Test::TestBody() + 2322
../../base/i18n/break_iterator_unittest.cc:200: Failure
Value of: iter.IsWord()
  Actual: true
Expected: false
Stack trace:
0   base_unittests                      0x000000010e19e5dc base::debug::StackTrace::StackTrace() + 28
1   base_unittests                      0x000000010e0fa030 StackTraceGetter::CurrentStackTrace(int, int) + 64
2   base_unittests                      0x000000010e1096d7 testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 71
3   base_unittests                      0x000000010e108b1d testing::internal::AssertHelper::operator=(testing::Message const&) const + 125
4   base_unittests                      0x000000010d3051b8 base::i18n::BreakIteratorTest_BreakWordChineseEnglish_Test::TestBody() + 3336
../../base/i18n/break_iterator_unittest.cc:201: Failure
Expected equality of these values:
  space
    Which is:
  iter.GetString()
    Which is: 载
Stack trace:
0   base_unittests                      0x000000010e19e5dc base::debug::StackTrace::StackTrace() + 28
1   base_unittests                      0x000000010e0fa030 StackTraceGetter::CurrentStackTrace(int, int) + 64
2   base_unittests                      0x000000010e1096d7 testing::internal::UnitTestImpl::CurrentOsStackTraceExceptTop(int) + 71
3   base_unittests                      0x000000010e108b1d testing::internal::AssertHelper::operator=(testing::Message const&) const + 125
4   base_unittests                      0x000000010d305298 base::i18n::BreakIteratorTest_BreakWordChineseEnglish_Test::TestBody() + 3560

Comment 1 by martiniss@chromium.org, Jul 3

Components: -Infra>Client>Chrome Infra>Platform>Recipes

This is probably a recipes bug. I know in the past we've had issues with recipes handling unicode. Don't remember exactly what

Comment 2 by s...@google.com, Jul 10

Components: -Infra>Platform>Recipes Infra>Client>Android

I'd assume it isn't a recipe engine bug if it works correctly on iOS. I would assume that somewhere a user-written recipe is doing something wrong.

Comment 3 by bpastene@chromium.org, Jul 11

Components: -Infra>Client>Android Infra>Client>Chrome

Comment 4 by martiniss@chromium.org, Jul 12

Owner: martiniss@chromium.org
Status: Assigned (was: Untriaged)

I'm looking at this more, it's weird that it works in iOS but not regular. Trying to figure out why

Comment 5 by martiniss@chromium.org, Jul 12

Actually, I looked at this more. Compared https://ci.chromium.org/p/chromium/builders/luci.chromium.try/android-kitkat-arm-rel/27440 and https://ci.chromium.org/p/chromium/builders/luci.chromium.try/ios-simulator/42061

In the stdout of the swarming collect step, they both successfully print out unicode characters, not just "???". So something weird is happening in the step logs for chromium, but not ios.

They don't use the same code path for collecting tasks, so it looks like I have some code forensics to do.

Comment 6 by martiniss@chromium.org, Jul 12

https://chromium-review.googlesource.com/c/chromium/src/+/1134512 is a CL which breaks tests, you can see this very clearly in the tryjobs.

Comment 7 by martiniss@chromium.org, Jul 12

Owner: jbudorick@chromium.org

I found the culprit!

https://cs.chromium.org/chromium/build/scripts/slave/recipe_modules/test_utils/util.py?q=test_utils/util&sq=package:chromium&g=0&l=227

jbudorick@, you added this in https://chromium.googlesource.com/chromium/tools/build/+/90711a4918e74aedb10d05fe9a4400d8007ba0d2. Is there a reason that you chose to only do ascii?

Comment 8 by martiniss@chromium.org, Jul 12

Cc: martiniss@chromium.org

►

Issue 860023 link

Issue metadata

Unicode output is garbled in CQ output on Android bots

Issue description

Comment 1 by martiniss@chromium.org, Jul 3

Comment 1 Deleted

Comment 2 by s...@google.com, Jul 10

Comment 2 Deleted

Comment 3 by bpastene@chromium.org, Jul 11

Comment 3 Deleted

Comment 4 by martiniss@chromium.org, Jul 12

Comment 4 Deleted

Comment 5 by martiniss@chromium.org, Jul 12

Comment 5 Deleted

Comment 6 by martiniss@chromium.org, Jul 12

Comment 6 Deleted

Comment 7 by martiniss@chromium.org, Jul 12

Comment 7 Deleted

Comment 8 by martiniss@chromium.org, Jul 12

Comment 8 Deleted

Sign in to add a comment