New issue
Advanced search Search tips

Issue 757336 link

Starred by 6 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression



Sign in to add a comment

Windows ASan and Canary builds aren't working (in some configs)

Project Member Reported by och...@chromium.org, Aug 21 2017

Issue description

Since Friday we've been seeing the following error on ClusterFuzz (from the latest builds on http://commondatastorage.googleapis.com/chromium-browser-asan/index.html?prefix=win32-release/):

[0820/134717.761:ERROR:main_dll_loader_win.cc(134)] Failed to load Chrome DLL from c:\clusterfuzz\slave-bot\builds\chrome-test-builds_media_win32-release_e999b74784b15cd4b248500a1cd5cd3743d01362\revisions\asan-win32-release-495845\chrome.dll: The specified module could not be found. (0x7E)

[0820/134717.761:ERROR:main_dll_loader_win.cc(134)] Failed to load Chrome DLL from c:\clusterfuzz\slave-bot\builds\chrome-test-builds_media_win32-release_e999b74784b15cd4b248500a1cd5cd3743d01362\revisions\asan-win32-release-495845\chrome.dll: The specified module could not be found. (0x7E)

These are built by https://build.chromium.org/p/chromium.lkgr/builders/Win%20ASan%20Release/builds/6318 

rnk, any idea what's going wrong here?
 

Comment 1 by thakis@chromium.org, Aug 21 2017

See also "[chromium-dev] Today's Windows Canary crashing on startup"

Comment 2 by aarya@google.com, Aug 21 2017

Cc: brucedaw...@chromium.org scottmg@chromium.org jmad...@chromium.org
Adding folks from thread, also some windows experts.
I am seeing this with Canary on my local machine as well. Just filed  issue 757483  which may be a duplicate.
Cc: siggi@chromium.org
I can't repro on Win10 or a Win7 VM unfortunately.

Can we get a narrower bisect range via Clusterfuzz?

There was some chrome_elf changes recently that might change load order and confuse things (?)
 Issue 757483  has been merged into this issue.
Summary: Windows ASan and Canary builds aren't working (in some configs) (was: Windows ASan builds aren't working)
Unfortunately (or not) the Chromium build archive from https://commondatastorage.googleapis.com/chromium-browser-snapshots doesn't seem to repro the failure, at least the one I tried. (495953). Is there a repo of official builds for Canary?

Comment 8 by aarya@google.com, Aug 21 2017

From CF job type 1
495646 Bad
495601 Bad
495551 Good
495536 Good
495526 Good

From CF job type 2
495628 Bad
495573 Bad
495542 Good
495528 Good

So, regression range probably 495552:495573 ?

Comment 10 by siggi@chromium.org, Aug 21 2017

I reproed on Jamie's machine. It's not finding MF.dll to bind imports against, whatever that is.
Siggy was investigating on my machine locally, and turned up that MF.dll seems to be a problem:

12d0:2a60 @ 07203171 - LdrpProcessWork - ERROR: Unable to load DLL: "MF.dll", Parent Module: "C:\Users\jmadill\AppData\Local\Google\Chrome SxS\Application\62.0.3192.0\chrome.dll", Status: 0xc0000135

Looking at the imports:

    MF.dll
              11B3276C Import Address Table
              120CE708 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                   4E MFEnumDeviceSources
                   1F MFCreateDeviceSource

    MFPlat.DLL
              11B32778 Import Address Table
              120CE714 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                   C3 MFStartup
                   27 MFCreateAttributes

    MFReadWrite.dll
              11B32784 Import Address Table
              120CE720 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                    5 MFCreateSourceReaderFromMediaSource

I believe this is something to do with Media Foundation:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms694197(v=vs.85).aspx

eg this API references MF.dll:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa367379(v=vs.85).aspx
Looks like the culprit might be indeed:
https://chromium.googlesource.com/chromium/src/+/77be7aef526705f9e752c9cf1abf6a8d99e26803

According to Siggi it's likely this change to media/BUILD.gn:

  if (is_win) {
    ldflags = [
      "/DELAYLOAD:mf.dll",
      "/DELAYLOAD:mfplat.dll",
      "/DELAYLOAD:mfreadwrite.dll",
    ]
  }

Comment 13 by siggi@chromium.org, Aug 21 2017

Yups, these DLLs used to be delay loaded, and they're not anymore in Jamie's local Canary. I don't know what these DLLs provide, but apparently some people don't have them available.
This is likely because I have Windows N.. the one that ships without Windows Media Player and associated junk.

Comment 15 by r...@chromium.org, Aug 21 2017

Cc: r...@chromium.org
Owner: dalecur...@chromium.org
Hmm, I deleted these configs because I thought they were already being delay loaded from media/base/win. Will restore them to the main media BUILD file.
https://chromium-review.googlesource.com/624406 should fix this, working on verification now.
Confirmed that fixes it:

$ dumpbin /imports out\Release\chrome.dll

...

  Section contains the following delay load imports:
...

    MF.dll
              00000001 Characteristics
      000000018327BCA8 Address of HMODULE
      00000001830E2E20 Import Address Table
      00000001830B8FF8 Import Name Table
      00000001830BA698 Bound Import Name Table
      0000000000000000 Unload Import Name Table
                     0 time date stamp
                                    000000018224B80B     0 MFCreateDeviceSource
                                    000000018224B780     0 MFEnumDeviceSources
    MFPlat.DLL
              00000001 Characteristics
      000000018327BCB0 Address of HMODULE
      00000001830E2E38 Import Address Table
      00000001830B9010 Import Name Table
      00000001830BA6B0 Bound Import Name Table
      0000000000000000 Unload Import Name Table
                     0 time date stamp
                                    000000018224B81D     0 MFCreateAttributes
                                    00000001827F3866     0 MFStartup
    MFReadWrite.dll
              00000001 Characteristics
      000000018327BCB8 Address of HMODULE
      00000001830E2E50 Import Address Table
      00000001830B9028 Import Name Table
      00000001830BA6C0 Bound Import Name Table
      0000000000000000 Unload Import Name Table
                     0 time date stamp
Project Member

Comment 20 by bugdroid1@chromium.org, Aug 21 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/ebd7018dd8458d3a7e4bc80358964063e444620c

commit ebd7018dd8458d3a7e4bc80358964063e444620c
Author: Dale Curtis <dalecurtis@chromium.org>
Date: Mon Aug 21 23:32:22 2017

Ensure delay load of MediaFoundation propogates up build chain.

//media/base/win specified delay-load for MediaFoundation, but did
not ensure that was propogated to dependent targets as was done
before the //media refactoring.

The fix is to add a new "delay_load_mf" config which can be passed
through all_dependent_configs up the chain.

BUG= 757336 
TEST=inspect dll, observe MF is delay loaded.

Change-Id: I2a477bba53c8a63bf77adb14571b3923dbdca6e9
Reviewed-on: https://chromium-review.googlesource.com/624406
Reviewed-by: Thomas Guilbert <tguilbert@chromium.org>
Commit-Queue: Dale Curtis <dalecurtis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496117}
[modify] https://crrev.com/ebd7018dd8458d3a7e4bc80358964063e444620c/media/BUILD.gn
[modify] https://crrev.com/ebd7018dd8458d3a7e4bc80358964063e444620c/media/base/win/BUILD.gn

Status: Fixed (was: Assigned)

Comment 22 by grt@chromium.org, Aug 25 2017

Cc: grt@chromium.org ligim...@chromium.org pbomm...@chromium.org gov...@chromium.org
 Issue 758206  has been merged into this issue.
Labels: -Type-Bug ReleaseBlock-Dev M-62 Type-Bug-Regression

Sign in to add a comment