New issue
Advanced search Search tips

Issue 780201 link

Starred by 1 user

Issue metadata

Status: Archived
Owner:
Closed: Dec 2017
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 3
Type: Bug

Blocking:
issue 769970
issue 780560



Sign in to add a comment

Address potential race condition in component updater

Project Member Reported by xiaochu@chromium.org, Oct 31 2017

Issue description

###Background###

IIUC, there is a race here. The following interleaving is possible:

[A component update begins]
RemoveComponent is called
ImageLoader removes component
[component update completes, ImageLoader installs new version of the component]
UnregisterComponent is called

In this case, we're left with an extra copy of the component owned by ImageLoader. Two ideas around this are:
(1) We could introduce a callback in UnregisterComponent and you could UnregisterComponent then call loader->RemoveComponent. In this case, we might have an race with shutdown situations, though.
(2) We could introduce a flag in the installer's OnCustomInstall that doesn't give ImageLoader a copy if RemoveComponent has been called. (Figuring out the threading requirements of the flag may be important.)


Let's say we use (1), I'm not sure what the race you mean by race with shutdown situations. But doing it in this way does not solve another existing race (happens rarely though):

[A component update begins]
[component update completes]
RemoveComponent is called
UnregisterComponent is called
ImageLoader removes component
[ImageLoader installs new version of the component]

The root cause for this is that we are not executing sequentially and transactionally here. Two proposals: (1.1) use SequencedTaskRunner to schedule ImageLoader operations. Then in ImageLoaderClient, make sure that operations are atomic and execution sequence is consistent with calling sequence. (1.2) on top of (1.1), introduce a mechanism to disallow UpdateComponent&RemoveComponent from interleaving each other. Doing it correctly should guarantee consistency. 

Regarding (2), it is a handy fix and should solve this properly. I'm a bit hesitant about adding the flag indicating 'Remove' is in progress since update_client already has an 'update in progress' state.

###Plan###

1. Currently OnCustomInstall which sends imageloader dbus operations is invoked in background thread: 

constexpr base::TaskTraits kTaskTraits = {
    base::MayBlock(), base::TaskPriority::BACKGROUND,
    base::TaskShutdownBehavior::SKIP_ON_SHUTDOWN};

Inside CustomInstall, we call imageloader dbus API on UI thread which guarantees sequence and thread safety.

2. add Uninstalling state in component updater which prevent component updater from starting other operations (update, install, uninstall, etc.).

3. cancel on-demand install when either uninstall or update is in progress.

4. during uninstall, remove (unmount) imageloader copy first before remove copy (main copy) in chrome. two benefits: (1) cancel uninstall if imageloader fails to remove (due to in use, fs failure, etc.), avoid dangling copy in imageloader. (2) only certain components are removable. if we remove main copy first, then we need to check twice in both chrome and imageloader.
 
Blocking: 769970
Blocking: 780560
Description: Show this description
Description: Show this description
Project Member

Comment 5 by bugdroid1@chromium.org, Nov 9 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/b8a2079913a44acf24cb112331cd72c57b4e2d02

commit b8a2079913a44acf24cb112331cd72c57b4e2d02
Author: Xiaochu Liu <xiaochu@chromium.org>
Date: Thu Nov 09 21:24:14 2017

Move OnCustomInstall body to SequencedTaskRunner

Currently, OnCustomInstall is invoked in a background thread but it
invokes imageloader on UI thread which does not bring any known benefits.

We move imageloader operation from UI thread over a SequnecedTaskRunner.

BUG= chromium:780201 
TEST=install is successful on DUT

Change-Id: I8deb2ebae9ff22fcf992f4c34d6198ddbcad0c70
Reviewed-on: https://chromium-review.googlesource.com/752622
Reviewed-by: Joshua Pawlicki <waffles@chromium.org>
Reviewed-by: Greg Kerr <kerrnel@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Commit-Queue: Xiaochu Liu <xiaochu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#515291}
[modify] https://crrev.com/b8a2079913a44acf24cb112331cd72c57b4e2d02/chrome/browser/component_updater/cros_component_installer.cc
[modify] https://crrev.com/b8a2079913a44acf24cb112331cd72c57b4e2d02/chrome/browser/component_updater/cros_component_installer.h

Project Member

Comment 6 by bugdroid1@chromium.org, Nov 17 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3f6ca074716a6df4bc82a1c002caffd73eed77fc

commit 3f6ca074716a6df4bc82a1c002caffd73eed77fc
Author: Xiaochu Liu <xiaochu@chromium.org>
Date: Fri Nov 17 02:45:02 2017

Revert "Move OnCustomInstall body to SequencedTaskRunner"

This reverts commit b8a2079913a44acf24cb112331cd72c57b4e2d02.

Reason for revert: it turns out that dbus method call in chrome requires UI thread (a DCHECK is broken). my initial plan to do blocking dbus call in OnCustomInstall is no longer viable. OnCustomInstall still needs to block its calling thread:  crbug.com/783495  and i will investigate change in component updater to achieve this. 

more discussions regarding dbus blocking method call are here: https://chromium-review.googlesource.com/c/chromium/src/+/762080

Original change's description:
> Move OnCustomInstall body to SequencedTaskRunner
> 
> Currently, OnCustomInstall is invoked in a background thread but it
> invokes imageloader on UI thread which does not bring any known benefits.
> 
> We move imageloader operation from UI thread over a SequnecedTaskRunner.
> 
> BUG= chromium:780201 
> TEST=install is successful on DUT
> 
> Change-Id: I8deb2ebae9ff22fcf992f4c34d6198ddbcad0c70
> Reviewed-on: https://chromium-review.googlesource.com/752622
> Reviewed-by: Joshua Pawlicki <waffles@chromium.org>
> Reviewed-by: Greg Kerr <kerrnel@chromium.org>
> Reviewed-by: Dan Erat <derat@chromium.org>
> Commit-Queue: Xiaochu Liu <xiaochu@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#515291}

TBR=derat@chromium.org,waffles@chromium.org,ejcaruso@chromium.org,kerrnel@chromium.org,xiaochu@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug:  chromium:780201 
Change-Id: Ieb0b29ee30dcbd6208bd1c2fcd176cbb10ad48d4
Reviewed-on: https://chromium-review.googlesource.com/775561
Commit-Queue: Xiaochu Liu <xiaochu@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Cr-Commit-Position: refs/heads/master@{#517257}
[modify] https://crrev.com/3f6ca074716a6df4bc82a1c002caffd73eed77fc/chrome/browser/component_updater/cros_component_installer.cc
[modify] https://crrev.com/3f6ca074716a6df4bc82a1c002caffd73eed77fc/chrome/browser/component_updater/cros_component_installer.h

Description: Show this description
Status: Fixed (was: Untriaged)
No longer needed since component updater no longer makes copies in OnCustomInstall.
Status: Archived (was: Fixed)

Sign in to add a comment