asvitkine@, could you please assign the right component?  I couldn't find anything variations / field-trials / Finch related...

(also -  issue 740701  probably should get the same component)

WIP CL @ https://chromium-review.googlesource.com/567034

The CL above hit an initialization-order-dependency problems.

Problem #1: blink::scheduler::RendererScheduler is unusable until WTF::Partitions::Initialize is called (between steps 1 and 3 below)

Problem #2: Consequently FieldTrialSyncer cannot be constructed until WTF::Partitions::Initialize is called (rather late in the initialization process / *after* ChildThreadImpl has already been constructed = while RenderThreadImpl is being constructed).  This conflicts with the CL above (which tried to move FieldTrialSyncer into ChildThreadImpl - between steps 1 and 2 below).  For a callstack associated with this problem see below (*).


Currently the renderer initialization happens in the following order:

step 1. Main thread TaskRunner is created:

    #1 blink::scheduler::TaskQueue::TaskQueue(...)
    #2 blink::scheduler::MainThreadTaskQueue::MainThreadTaskQueue(...)
    #3 scoped_refptr<blink::scheduler::MainThreadTaskQueue> blink::scheduler::TaskQueueManager::CreateTaskQueue<...>(...)
    #4 NewTaskQueue
    #5 blink::scheduler::MainThreadSchedulerHelper::MainThreadSchedulerHelper(...)
    #6 blink::scheduler::RendererSchedulerImpl::RendererSchedulerImpl(...)
    #7 blink::scheduler::RendererScheduler::Create()
    #8 content::RendererMain(...) content/renderer/renderer_main.cc:177


2. SetRuntimeFeatures is called:

    #1 content::RenderThreadImpl::InitializeWebKit(...) content/renderer/render_thread_impl.cc:1196
    #2 content::RenderThreadImpl::Init(...) content/renderer/render_thread_impl.cc:683
    #3 content::RenderThreadImpl::RenderThreadImpl(...)
    #4 content::RenderThreadImpl::Create(...)
    #5 content::RendererMain(...) content/renderer/renderer_main.cc:207



3. WTF::Partitions::Initialize is called:

    #1 WTF::Partitions::Initialize(...)
    #2 content::BlinkPlatformImpl::BlinkPlatformImpl(...)
    #3 content::RendererBlinkPlatformImpl::RendererBlinkPlatformImpl(...)
    #4 content::RenderThreadImpl::InitializeWebKit(...) content/renderer/render_thread_impl.cc:1201
    #5 content::RenderThreadImpl::Init(...) content/renderer/render_thread_impl.cc:683
    #6 content::RenderThreadImpl::RenderThreadImpl(...)
    #7 content::RenderThreadImpl::Create(...)
    #8 content::RendererMain(...) content/renderer/renderer_main.cc:207:5


4. field_trial_syncer_.InitFieldTrialObserving is called by RenderThreadImpl::RenderThreadImpl(...) content/renderer/render_thread_impl.cc:776


(*) Callstack showing a failed PostTask:

[1:1:0712/214527.880081:FATAL:Partitions.h(...)] Check failed: initialized_.
    ...
    #3 BufferPartition third_party/WebKit/Source/platform/wtf/allocator/Partitions.h:59:5
    #4 unsigned long WTF::PartitionAllocator::QuantizedSize<blink::scheduler::internal::TaskQueueImpl::Task>(...) third_party/WebKit/Source/platform/wtf/allocator/PartitionAllocator.h:38:0
    #5 AllocationSize third_party/WebKit/Source/platform/wtf/Vector.h:394:12
    #6 ExpandBuffer third_party/WebKit/Source/platform/wtf/Vector.h:477:0
    ...
    #9 void WTF::Deque<...>::emplace_back<...>(...) third_party/WebKit/Source/platform/wtf/Deque.h:516:0
    #10 blink::scheduler::internal::TaskQueueImpl::PushOntoImmediateIncomingQueueLocked(...) third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:318:32
    ...
    #12 blink::scheduler::internal::TaskQueueImpl::PostDelayedTask(...) third_party/WebKit/Source/platform/scheduler/base/task_queue_impl.cc:170:12
    #13 blink::scheduler::TaskQueue::PostDelayedTask(...) third_party/WebKit/Source/platform/scheduler/base/task_queue.cc:28:17
    #14 base::TaskRunner::PostTask(...) base/task_runner.cc:47:10
    #15 void base::ObserverListThreadSafe<base::FieldTrialList::Observer>::Notify<void (...) base/observer_list_threadsafe.h:143:24
    #16 base::FieldTrialList::NotifyFieldTrialGroupSelection(...) base/metrics/field_trial.cc:974:28
    #17 group base/metrics/field_trial.cc:356:5
    #18 base::FieldTrial::group_name() /base/metrics/field_trial.cc:362:0
    #19 base::FieldTrialList::FindFullName(...) base/metrics/field_trial.cc:624:25
    #20 content::SetRuntimeFeaturesDefaultsAndUpdateFromArgs(...) content/child/runtime_features.cc:229:7
    #21 content::RenderThreadImpl::InitializeWebKit(...) content/renderer/render_thread_impl.cc:1191:3
    #22 content::RenderThreadImpl::Init(...) content/renderer/render_thread_impl.cc:680:3 <- just calls InitializeWebKit
    #23 content::RenderThreadImpl::RenderThreadImpl(...) content/renderer/render_thread_impl.cc:640:3
    #24 content::RenderThreadImpl::Create(...) content/renderer/render_thread_impl.cc:564:14
    #25 content::RendererMain(...) content/renderer/renderer_main.cc:207:5


Any suggestions for what to do about the above?

From my side (i.e. things I can do to unblock https://chromium-review.googlesource.com/c/567034) I see the following options:

1. I could expose ChildThreadImpl::InitFieldTrialSyncer and call it manually (late!) from each derived class, but this would A) break encapsulation and B) would mean trial activations are not reported for a longer period of time.

2. I could swap steps 2 and 3 with a small change:

    diff --git a/content/renderer/render_thread_impl.cc b/content/renderer/render_thread_impl.cc
    index 7ad84ae09146..6030d0baceb0 100644
    --- a/content/renderer/render_thread_impl.cc
    +++ b/content/renderer/render_thread_impl.cc
    @@ -1187,13 +1187,12 @@ void RenderThreadImpl::InitializeWebKit(
         gin::Debug::SetJitCodeEventHandler(vTune::GetVtuneCodeEventHandler());
     #endif
     
    +  blink_platform_impl_.reset(new RendererBlinkPlatformImpl(
    +      renderer_scheduler_.get(), GetConnector()->GetWeakPtr()));
       SetRuntimeFeaturesDefaultsAndUpdateFromArgs(command_line);
       GetContentClient()
           ->renderer()
           ->SetRuntimeFeaturesDefaultsBeforeBlinkInitialization();
    -
    -  blink_platform_impl_.reset(new RendererBlinkPlatformImpl(
    -      renderer_scheduler_.get(), GetConnector()->GetWeakPtr()));
       blink::Initialize(blink_platform_impl_.get());

+skyostil@, the author of r411074 (which constructs a temporarily "defunct" TaskRunner)

+rmcilroy@, hlopko@, sammc@, ben@, tfarina@ - "blamed" for lines responsible for the current ordering of 1) WTF::Partitions::Initialize, 2) SetRuntimeFeaturesDefaults*.  Hopefully they can shout at me if my 2nd idea at the end of #c6 is wrong:

    rmcilroy@:  SetRuntimeFeaturesDefaultsAndUpdateFromArgs(command_line);
    hlopko@:    GetContentClient()
                    ->renderer()
                    ->SetRuntimeFeaturesDefaultsBeforeBlinkInitialization();
    sammc@:     blink_platform_impl_.reset(new RendererBlinkPlatformImpl(
    ben@:           renderer_scheduler_.get(), GetConnector()->GetWeakPtr()));
    tfarina@:   blink::Initialize(blink_platform_impl_.get());

FWIW, I am trying to see what trybots will say to the 2nd idea from #c6 - see the latest patchset of https://chromium-review.googlesource.com/c/567034

+palmer@ [the current owner of PartitionAlloc?]

If blink::scheduler::RendererScheduler depends on PartitionAlloc, then maybe RendererBlinkPlatformImpl (which initializes PartitionAlloc - see step3 in #c6) should be constructed before constructing blink::scheduler::RendererScheduler?
Note that this might require passing std::unique_ptr<RendererBlinkPlatformImpl> as an argument for RenderThreadImpl (and storing it in blink_platform_impl_).

WDYT?

PS. FWIW, my reordering change passed trybots.  Let's land it?  :->  https://chromium-review.googlesource.com/c/567034

I moved SetRuntimeFeaturesDefaultsAndUpdateFromArgs before Blink initialization since there are some WebRuntimeFeatures flags which need to be initialized correctly during Blink initialization. 

The one I know about is WebRuntimeFeatures::enableV8IdleTasks, which needs to be set to true (or false if the disable commandline flag is set) before V8Initializer::InitializeMainThread is called, which means before blink::Initialize().

It looks like your reordering CL moves the blink's platform creation before SetRuntimeFeaturesDefaultsAndUpdateFromArgs, but not blink::Initialize(), so this should be fine as far as I know, but other people on CC might know of different issues.

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5a02412fd428d67b7e7e511c8d7e308fd44d7419

commit 5a02412fd428d67b7e7e511c8d7e308fd44d7419
Author: Lukasz Anforowicz <lukasza@chromium.org>
Date: Wed Jul 19 21:31:57 2017

Move |field_trial_syncer_| into ChildThreadImpl.

There are 4 subclasses of ChildThreadImpl:
- RenderThreadImpl
- PpapiThread
- GpuChildThread
- UtilityThreadImpl

Before this CL, RenderThreadImpl and PpapiThread would have a
|field_trial_syncer_| field.  GpuChildThread would construct a
ChildProcessFieldTrialSyncer via ChromeContentGpuClient::Initialize, but
only if the GPU didn't share the browser process.  UtilityThreadImpl did
not have a |field_trial_syncer_| field.

Before this CL, RenderThreadImpl and GpuChildThread would notify the
browser process about trial activations by calling FieldTrialActivated
method of mojom::FieldTrialRecorder.  PpapiThread would notify via
PpapiHostMsg_FieldTrialActivated legacy IPC.  UtilityThreadImpl wouldn't
do anything.

After this CL, ChildProcessFieldTrialSyncer and
base::FieldTrialList::Observer are encapsulated and hidden by
ChildThreadImpl.

Also - after this CL, the //content -> //components/variations
dependency is sighly more restricted.

Bug:  740726 
Change-Id: I2fdf4e16f52a7339476e337f695b44e621d0b295
Reviewed-on: https://chromium-review.googlesource.com/567034
Commit-Queue: Lukasz Anforowicz <lukasza@chromium.org>
Reviewed-by: Nick Carter <nick@chromium.org>
Reviewed-by: Antoine Labour <piman@chromium.org>
Reviewed-by: Daniel Cheng <dcheng@chromium.org>
Reviewed-by: Alexei Svitkine (slow) <asvitkine@chromium.org>
Cr-Commit-Position: refs/heads/master@{#487993}
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/base/metrics/field_trial.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/chrome/gpu/chrome_content_gpu_client.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/chrome/gpu/chrome_content_gpu_client.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/components/variations/child_process_field_trial_syncer.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/components/variations/child_process_field_trial_syncer.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/components/variations/child_process_field_trial_syncer_unittest.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/browser/ppapi_plugin_process_host.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/browser/ppapi_plugin_process_host.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/child/BUILD.gn
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/child/DEPS
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/child/child_thread_impl.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/child/child_thread_impl.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/gpu/gpu_child_thread.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/gpu/gpu_child_thread.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/ppapi_plugin/BUILD.gn
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/ppapi_plugin/DEPS
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/ppapi_plugin/ppapi_thread.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/ppapi_plugin/ppapi_thread.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/public/child/child_thread.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/public/gpu/content_gpu_client.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/public/renderer/render_thread.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/renderer/BUILD.gn
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/renderer/DEPS
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/renderer/render_thread_impl.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/renderer/render_thread_impl.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/shell/gpu/shell_content_gpu_client.cc
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/content/shell/gpu/shell_content_gpu_client.h
[modify] https://crrev.com/5a02412fd428d67b7e7e511c8d7e308fd44d7419/ppapi/proxy/ppapi_messages.h

Things seem to be working fine in 61.0.3162.0 (Official Build) canary (64-bit) on Win10, so let me mark the bug as fixed.

Sweet!