Parrot shouldn't be running this test normally. Looks like a consequence of  issue 616380 .

@Pawel - We currently run video_WebRtcPerf as part of the go/crosbolt test suite which runs across all devices. Why shouldn't it be running against Parrot? Thanks!

 Issue 616380  might be a different reason as that was only happening on ARM devices.

Parrot crash seem to be happening from a very long time.

Ah sorry, I think I looked at the wrong log and misread it to be a HW acceleration error. Parrot doesn't have HW acceleration for WebRtc. But this test also tests SW. These fail with:

UnhandledTestFail: Unhandled DevtoolsTargetCrashException: Web content with index 0 may have crashed.

Rohit. Do you know who works on Devtools?

That is a standard telemetry crash stacktrace whenever the tab or the browser instance crashes during the test run.

This is a flaky failure. It failed once in the last 25 runs.

https://dashboards.corp.google.com/#/google::_24290669_64f2_42e0_879d_a8bad4e190e?f=milestone:in:54&f=board:in:parrot&f=status:in:ABORT,ERROR,FAIL,WARN,NO_RESULTS,GOOD&f=test:in:video_WebRtcPerf

Traceback (most recent call last):
  File "/usr/local/autotest/common_lib/test.py", line 804, in _call_test_function
    return func(*args, **dargs)
  File "/usr/local/autotest/common_lib/test.py", line 461, in execute
    dargs)
  File "/usr/local/autotest/common_lib/test.py", line 347, in _call_run_once_with_retry
    postprocess_profiled_run, args, dargs)
  File "/usr/local/autotest/common_lib/test.py", line 376, in _call_run_once
    self.run_once(*args, **dargs)
  File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 145, in run_once
    keyvals = self.test_decode_time(local_path)
  File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 335, in test_decode_time
    return self.test_webrtc(local_path, get_decode_time)
  File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 189, in test_webrtc
    self.start_loopback(cr)
  File "/usr/local/autotest/tests/video_WebRtcPerf/video_WebRtcPerf.py", line 106, in start_loopback
    tab = cr.browser.tabs[0]
  File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/browser/tab_list.py", line 18, in __getitem__
    return self._tab_list_backend.__getitem__(index)
  File "/usr/local/telemetry/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend_list.py", line 62, in __getitem__
    index, repr(self._filtered_context_ids)))
DevtoolsTargetCrashException: Web content with index 0 may have crashed. filtered_context_ids = []

R54-8540.0.0


2016-07-02T21:56:07.647497+00:00 WARNING crash_reporter[14628]: [user] Received crash notification for chrome[14394] sig 6, user 1000 (developer build - not testing - always dumping)
2016-07-02T21:56:07.649279+00:00 INFO crash_reporter[14628]: State of crashed process [14394]: D (disk sleep)
2016-07-02T21:56:07.821084+00:00 INFO kernel: [ 4932.677907] tpm_tis tpm_tis: command 0x41 (size 618) returned code 0x0
2016-07-02T21:56:08.850362+00:00 INFO crash_reporter[14628]: Stored minidump to /var/spool/crash/chrome.20160702.145607.14394.dmp
2016-07-02T21:56:08.850396+00:00 INFO crash_reporter[14628]: Leaving core file at /var/spool/crash/chrome.20160702.145607.14394.core due to developer image


2016-07-02T21:56:16.162468+00:00 WARNING crash_reporter[14814]: [user] Received crash notification for chrome[14647] sig 6, user 1000 (developer build - not testing - always dumping)
2016-07-02T21:56:16.164200+00:00 INFO crash_reporter[14814]: State of crashed process [14647]: D (disk sleep)
2016-07-02T21:56:17.086818+00:00 INFO crash_reporter[14814]: Stored minidump to /var/spool/crash/chrome.20160702.145616.14647.dmp
2016-07-02T21:56:17.086852+00:00 INFO crash_reporter[14814]: Leaving core file at /var/spool/crash/chrome.20160702.145616.14647.core due to developer image

I'm trying to see the stack in the core files.

The stack of chrome.20160702.145607.14394.core and chrome.20160702.145616.14647.core are the same.
(gdb) bt
#0  0x00007fbba6d63b82 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007fbba6d658a0 in __GI_abort () at abort.c:89
#2  0x00007fbba9d5f7f5 in base::debug::BreakDebugger () at ../../../../../../../home/chrome-bot/chrome_root/src/base/debug/debugger_posix.cc:249
#3  0x00007fbbaa090605 in net::NSSCertDatabase::NSSCertDatabase (this=0x16272f7c2660, public_slot=..., private_slot=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/net/cert/nss_cert_database.cc:93
#4  0x00007fbba9fcaf97 in net::NSSCertDatabaseChromeOS::NSSCertDatabaseChromeOS (this=0x16272f7c2660, public_slot=..., private_slot=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/net/cert/nss_cert_database_chromeos.cc:26
#5  0x00007fbba9a6f987 in (anonymous namespace)::NSSCertDatabaseChromeOSManager::DidGetPrivateSlot (this=0x16272fbc4640, private_slot=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/chrome/browser/net/nss_context_chromeos.cc:58
#6  0x00007fbba9a6fb40 in Invoke<base::WeakPtr<(anonymous namespace)::NSSCertDatabaseChromeOSManager>, std::unique_ptr<PK11SlotInfoStr, crypto::NSSDestroyer<PK11SlotInfoStr, PK11_FreeSlot> > > (receiver_ptr=<unknown type in /usr/local/google/home/wuchengli/Downloads/parrot-8540.0.0/debug/opt/google/chrome/chrome.debug, CU 0x1bb7d0f, DIE 0x1be44be>, method=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:137
#7  MakeItSo<void ((anonymous namespace)::NSSCertDatabaseChromeOSManager::* const&)(std::unique_ptr<PK11SlotInfoStr, crypto::NSSDestroyer<PK11SlotInfoStr, PK11_FreeSlot> >), base::WeakPtr<(anonymous namespace)::NSSCertDatabaseChromeOSManager>, std::unique_ptr<PK11SlotInfoStr, crypto::NSSDestroyer<PK11SlotInfoStr, PK11_FreeSlot> > > (weak_ptr=..., functor=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:224
#8  RunImpl<void ((anonymous namespace)::NSSCertDatabaseChromeOSManager::* const&)(std::unique_ptr<PK11SlotInfoStr, crypto::NSSDestroyer<PK11SlotInfoStr, PK11_FreeSlot> >), std::tuple<base::WeakPtr<(anonymous namespace)::NSSCertDatabaseChromeOSManager> > const&, 0ul> (
    unbound_args#0=<unknown type in /usr/local/google/home/wuchengli/Downloads/parrot-8540.0.0/debug/opt/google/chrome/chrome.debug, CU 0x1bb7d0f, DIE 0x1be18bd>, bound=..., 
    functor=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:270
#9  base::internal::Invoker<base::internal::BindState<void ((anonymous namespace)::NSSCertDatabaseChromeOSManager::*)(std::unique_ptr<PK11SlotInfoStr, crypto::NSSDestroyer<PK11SlotInfoStr, PK11_FreeSlot> >), base::WeakPtr<(anonymous namespace)::NSSCertDatabaseChromeOSManager> >, void(std::unique_ptr<PK11SlotInfoStr, crypto::NSSDestroyer<PK11SlotInfoStr, PK11_FreeSlot> >)>::Run(base::internal::BindStateBase *, <unknown type in /usr/local/google/home/wuchengli/Downloads/parrot-8540.0.0/debug/opt/google/chrome/chrome.debug, CU 0x1bb7d0f, DIE 0x1be4913>) (
    base=<optimized out>, unbound_args#0=<unknown type in /usr/local/google/home/wuchengli/Downloads/parrot-8540.0.0/debug/opt/google/chrome/chrome.debug, CU 0x1bb7d0f, DIE 0x1be4913>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:248
#10 0x00007fbba9dff917 in Run (args#0=..., this=0x16272f914750) at ../../../../../../../home/chrome-bot/chrome_root/src/base/callback.h:389
#11 crypto::(anonymous namespace)::ChromeOSUserData::SetPrivateSlot (this=0x16272fb287e0, private_slot=...) at ../../../../../../../home/chrome-bot/chrome_root/src/crypto/nss_util.cc:243
#12 0x00007fbba9e008ff in crypto::(anonymous namespace)::NSSInitSingleton::OnInitializedTPMForChromeOSUser (this=<optimized out>, username_hash=..., tpm_args=...)
    at ../../../../../../../home/chrome-bot/chrome_root/src/crypto/nss_util.cc:551
#13 0x00007fbba9dffda9 in Invoke<crypto::(anonymous namespace)::NSSInitSingleton*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot, std::default_delete<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot> > > (receiver_ptr=<optimized out>, 
    method=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:137
#14 MakeItSo<void (crypto::(anonymous namespace)::NSSInitSingleton::* const&)(std::basic_string<char> const&, std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot>), crypto::(anonymous namespace)::NSSInitSingleton*, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot, std::default_delete<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot> > > (functor=<optimized out>)
    at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:205
#15 RunImpl<void (crypto::(anonymous namespace)::NSSInitSingleton::* const&)(std::basic_string<char> const&, std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot>), std::tuple<base::internal::UnretainedWrapper<crypto::(anonymous namespace)::NSSInitSingleton>, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, base::internal::PassedWrapper<std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot, std::default_delete<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot> > > > const&, 0ul, 1ul, 2ul> (bound=..., functor=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:270
#16 base::internal::Invoker<base::internal::BindState<void (crypto::(anonymous namespace)::NSSInitSingleton::*)(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot, std::default_delete<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot> >), base::internal::UnretainedWrapper<crypto::(anonymous namespace)::NSSInitSingleton>, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, base::internal::PassedWrapper<std::unique_ptr<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot, std::default_delete<crypto::(anonymous namespace)::NSSInitSingleton::TPMModuleAndSlot> > > >, void()>::Run(base::internal::BindStateBase *) (base=<optimized out>) at ../../../../../../../home/chrome-bot/chrome_root/src/base/bind_internal.h:248
#17 0x00007fbba9d9a9dc in Run (this=0x16272fca7cf8) at ../../../../../../../home/chrome-bot/chrome_root/src/base/callback.h:389

The stack is the same as http://crbug.com/439192. xiyuan@ Please help triage. Thanks.

Is there a chrome user log for the crash in #12/#13 ?

I would like see if there is a

[773:1070:0111/231850:ERROR:nss_util.cc(901)] Error opening persistent database...

line there. It could indicate the crash is because of nss db corruption.

The chrome user log for #12/#13 is here. I don't see nss error in chrome and chrome_20160702-145513.

https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/68313888-haddowk/chromeos2-row3-rack1-host14/sysinfo/var/log/chrome/

Deleted: chrome.txt 12.6 KB

chrome.txt 12.6 KB View Download

Deleted: chrome_20160702-145513.txt 12.6 KB

chrome_20160702-145513.txt 12.6 KB View Download

Thanks for the link. The nss error is in chrome user log (under /home/chronos) not the system log (under /var/log). And it does have the same cause of issue 439192, where nssdb is corrupted.

https://pantheon.corp.google.com/storage/browser/chromeos-autotest-results/68313888-haddowk/chromeos2-row3-rack1-host14/debug/

[14394:14416:0702/145606:ERROR:nss_util.cc(851)] Error opening persistent database (configDir='sql:/home/chronos/u-b3edbf69d7a8d15949cdeb65073132d2277c7986/.pki/nssdb' tokenDescription='UserNSSDB b3edbf69d7a8d15949cdeb65073132d2277c7986'): NSS error code: -8018
[14394:14410:0702/145606:ERROR:owner_key_util_impl.cc(30)] Could not get size of /var/lib/whitelist/owner.key

Deleted: chrome.20160702.145607.14394.txt 10.5 KB

chrome.20160702.145607.14394.txt 10.5 KB View Download