New issue
Advanced search Search tips

Issue 641366 link

Starred by 3 users

Issue metadata

Status: Verified
Owner:
Closed: Sep 2016
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug

Blocked on:
issue 641369
issue 641372

Blocking:
issue 616854
issue 618392



Sign in to add a comment

Fix how cryptohomed works with mounts internally

Project Member Reported by apronin@chromium.org, Aug 26 2016

Issue description

There are several issues that lead to problems when cryptohome is performing some lengthy or delayed operation while the session is closed and umounts are happening. Here is the current state of things:

1) All known mounts are stored in a map (username -> mount) in mounts_ field of Service.

2) mounts_ is accessed from Service methods called from at least 3 threads: main (dbus) thread, mount_thread_, tpm_init_->work_thread_.

3) Occasionally, access to mounts_ is protected by mounts_lock_. But reads are typically unprotected.

4) InitializeTpmComplete called from tpm_init_->work_thread_ waits for mount_thread_'s tasks while holding mounts_lock_.

As a result:
(2) + (3) => Methods that read the mounts_ can iterate through the map, while some other thread deletes elements (during unmount) - unsafe.
(2) + (3) + (5) => Potential deadlock: It is possible that a task placed earlier on mount_thread_ is waiting on mounts_lock_ preventing other tasks from running, while InitializeTpmComplete waits on a later task while holding mounts_lock_.

On a plus side, it looks like all the tasks placed on mount_thread_ that have a pointer to mount as a parameter, have it AddRef'd, but that needs to be double-checked.

Examples of how these issues manifest themselves can be found in issue 618392,  issue 616854 , possibly issue 631640, etc.
 
Blockedon: 641369
Blockedon: 641372
Summary: Fix how cryptohomed works with mounts internally (was: Fix how cryptohome works with mounts internally)
Blocking: 616854
Blocking: 618392
Project Member

Comment 6 by bugdroid1@chromium.org, Sep 2 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform2/+/afefdcc425b89c577d57a6f16b26eaccd1664641

commit afefdcc425b89c577d57a6f16b26eaccd1664641
Author: Andrey Pronin <apronin@google.com>
Date: Fri Aug 26 23:43:33 2016

cryptohome: stop Service tasks in derived destructors

Service threads processing tasks must be stopped in derived
ServiceDistributed and ServiceMonolithic destructors. Otherwise, after
the derived destructor, all pure virtual functions from Service
overloaded in derived classes and all members defined in derived classes
will be destroyed, while mount_thread_ will continue running tasks
until stopped in Service destructor.

BUG= chromium:641366 
TEST=unittest ServiceTest.NoDeadlocksInInitializeTpmComplete

Change-Id: I1cf0dab532c7e3e1cb8ac7bed168266d1b23cad3
Reviewed-on: https://chromium-review.googlesource.com/376825
Commit-Ready: Andrey Pronin <apronin@chromium.org>
Tested-by: Andrey Pronin <apronin@chromium.org>
Reviewed-by: Darren Krahn <dkrahn@chromium.org>

[modify] https://crrev.com/afefdcc425b89c577d57a6f16b26eaccd1664641/cryptohome/service.h
[modify] https://crrev.com/afefdcc425b89c577d57a6f16b26eaccd1664641/cryptohome/service_distributed.cc
[modify] https://crrev.com/afefdcc425b89c577d57a6f16b26eaccd1664641/cryptohome/service.cc
[modify] https://crrev.com/afefdcc425b89c577d57a6f16b26eaccd1664641/cryptohome/service_monolithic.cc

Status: Started (was: Assigned)
Status: Fixed (was: Started)

Comment 9 by dchan@chromium.org, Oct 7 2016

Labels: VerifyIn-55
Labels: Needs-Feedback
@apronin Trying to verify thid bug. Please provide repro steps
Repro: run unit test from #6

Comment 12 by dchan@google.com, Nov 19 2016

Labels: VerifyIn-56

Comment 13 by dchan@google.com, Jan 21 2017

Labels: VerifyIn-57

Comment 14 by dchan@google.com, Mar 4 2017

Labels: VerifyIn-58
Status: Verified (was: Fixed)

Sign in to add a comment