New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 696529 link

Starred by 9 users

Issue metadata

Status: Fixed
Owner:
Closed: Aug 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Mac
Pri: 1
Type: Bug



Sign in to add a comment

Waterfall bots are failing on "compile (no-op)" step due to safe_browsing_dmg having underspecified dependencies

Project Member Reported by serg...@chromium.org, Feb 27 2017

Issue description

Comment 1 Deleted

Labels: -Sheriff-Chromium Infra-Troopers
Past few builds have been green.

It fails because it thinks it needs to regenerate some test data. Just clobbered. Hopefully that will help.
Cc: dpranke@chromium.org tikuta@chromium.org thakis@chromium.org
Labels: OS-Mac
Still happens every now and then, and it's always for the safe_browsing_dmg test data.

+ tikuta@ who added the check in rf305cbc5994c4d16004944dd3f10b335e8ef6ea9

What should be doing about this failure?

Comment 4 by tikuta@chromium.org, Feb 28 2017

Cc: rsesek@chromium.org
The step is used to detect incomplete build dependency.

The confirm no-op step is flaky due to timing issue.
But if the step failures with ninja's '... is dirty' message, there should be wrong dependency in BUILD.gn.

Below log might explain what is the root cause of this issue.
https://build.chromium.org/p/chromium/builders/Mac/builds/24183/steps/compile%20confirm%20no-op/logs/stdio
ninja explain: output test_data/chrome/safe_browsing_dmg/hfsx_case_sensitive.img doesn't exist

Following BUILD.gn may be related?
https://cs.chromium.org/chromium/src/chrome/test/BUILD.gn?q=hfsx_case_sensitive.img+package:%5Echromium$&l=4984&dr=C
Components: Build
Labels: -Restrict-View-Google -Infra-Troopers
Status: Available (was: Untriaged)
This isn't an infra issue, it's a chromium / sheriff issue.

Also, there's no need for this to be R-V-G.
Summary: Waterfall bots are failing on "compile (no-op)" step due to safe_browsing_dmg having underspecified dependencies (was: Waterfall bots are failing on "compile (no-op)" step)
https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium%2FMac%2F24228%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout shows that 

[7523/50127] ACTION //chrome/test:mac_safe_browsing_test_data(//build/toolchain/mac:clang_x64)
[7524/50127] STAMP obj/third_party/WebKit/Source/devtools/frontend_protocol_sources.inputdeps.stamp
[7525/50127] STAMP obj/chrome/test/mac_safe_browsing_test_data.stamp

runs.  https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium%2FMac%2F24228%2F%2B%2Frecipes%2Fsteps%2Fcompile_confirm_no-op%2F0%2Fstdout says

inja explain: output test_data/chrome/safe_browsing_dmg/hfs_plus.img doesn't exist
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_GPTSPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_NONE.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_SPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDCO_GPTSPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDCO_NONE.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDCO_SPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDRO_GPTSPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDRO_NONE.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDRO_SPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDRW_GPTSPUD.dmg is dirty
ninja explain: test_data/chrome/safe_browsing_dmg/dmg_UDRW_NONE.dmg is dirty

(and so on)


https://cs.chromium.org/chromium/src/chrome/test/data/safe_browsing/dmg/generate_test_data.sh?q=generate_test_data+package:%5Echromium$&dr=C&l=24 doesn't do `set -eu` at the top, so maybe its call to https://cs.chromium.org/chromium/src/chrome/test/data/safe_browsing/dmg/make_hfs.sh?dr=C just fails every now and then and we don't notice?
Cc: katthomas@chromium.org erikc...@chromium.org
 Issue 602169  has been merged into this issue.
Owner: rsesek@chromium.org
Status: Assigned (was: Available)
@rsesek - any ideas?
Cc: chrome-trooper-bugs@chromium.org
 Issue 698355  has been merged into this issue.
 Issue 699605  has been merged into this issue.

Comment 11 by d...@chromium.org, Mar 9 2017

Cc: -chrome-trooper-bugs@chromium.org
Project Member

Comment 12 by bugdroid1@chromium.org, Mar 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5fe29f761d6791d746bd7e058d6f3d79227cc117

commit 5fe29f761d6791d746bd7e058d6f3d79227cc117
Author: rsesek <rsesek@chromium.org>
Date: Fri Mar 10 19:59:17 2017

Fix chrome/test/data/safe_browsing/dmg/generate_test_data.sh to not silently fail.

generate_test_data.sh uses `hdiutil create` and `hdiutil convert`. The former
can overwrite an existing output file, but the latter cannot. This led to the
following error:

  hdiutil: convert failed - File exists

But this message was getting swallowed because the script 1) did not do `set -e`
to stop on errors, and 2) redirect all output to /dev/null. This also fixes both
of those issues.

BUG= 696529 
R=thakis@chromium.org

Review-Url: https://codereview.chromium.org/2748453002
Cr-Commit-Position: refs/heads/master@{#456141}

[modify] https://crrev.com/5fe29f761d6791d746bd7e058d6f3d79227cc117/chrome/test/data/safe_browsing/dmg/generate_test_data.sh
[modify] https://crrev.com/5fe29f761d6791d746bd7e058d6f3d79227cc117/chrome/test/data/safe_browsing/dmg/make_hfs.sh

The above CL should fix some bugs in the script and allow us to see stderr now. But there's still the underlying bug in macOS that will cause this to fail. And AFAIK, there's nothing to be done about that.

Comment 14 by noel@chromium.org, Mar 11 2017

I received mail from the main builders that my patch broke the Mac compile and my patch had to do with safe_browsing_DMG 

https://codereview.chromium.org/2737763002

Compile error log for
  https://build.chromium.org/p/chromium/builders/Mac/builds/24658

[1912/49531] ACTION //chrome/test:mac_safe_browsing_test_data(//build/toolchain/mac:clang_x64)
FAILED: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_SPUD.dmg
[snip ...]
test_data/chrome/safe_browsing_dmg/hfs_plus.img test_data/chrome/safe_browsing_dmg/hfsx_case_sensitive.img test_data/chrome/safe_browsing_dmg/mach_o_in_dmg.dmg 
python ../../build/gn_run_binary.py ../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh test_data/chrome/safe_browsing_dmg
./../../chrome/test/data/safe_browsing/dmg/make_hfs.sh: line 37: pushd: /Volumes/SafeBrowsingDMG: No such file or directory

I saw examples of the above error, both before and after my change landed, and relaxed and searched crbug.com, and I found this  bug 696529 

I saw another variant FYI: failure to unmount /Volumes/SafeBrowsingDMG

Compile error log for
  https://build.chromium.org/p/chromium/builders/Mac/builds/24654

[4154/49531] ACTION //chrome/test:mac_safe_browsing_test_data(//build/toolchain/mac:clang_x64)
FAILED: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_SPUD.dmg
[snip ...]
test_data/chrome/safe_browsing_dmg/hfs_plus.img test_data/chrome/safe_browsing_dmg/hfsx_case_sensitive.img test_data/chrome/safe_browsing_dmg/mach_o_in_dmg.dmg 
python ../../build/gn_run_binary.py ../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh test_data/chrome/safe_browsing_dmg
Volume SafeBrowsingDMG on disk1 failed to unmount: dissented by PID=261 

Issue 700801 has been merged into this issue.
"Volume SafeBrowsingDMG on disk1 failed to unmount: dissented by PID=261" seems like a big problem. Something on the system is preventing the DMG from being unmounted, which means there's going to be other problems later down the line...
In comment #13, rsesek@ wrote:
> there's still the underlying bug in macOS that will cause this to fail.

What bug is that?
See #16 for one.
#16 doesn't actually tell me that that's a MacOS bug that we can't fix, it just says that *something* is holding a reference open ...
could we make the dmg building script print the output of `ps aux` on failure to see which process is to blame?
In issue 700801, it prints out that info:

Volume SafeBrowsingDMG on disk1 failed to unmount: dissented by PID=289 (/usr/libexec/lsd)


We could try looping over the unmount command, I guess, but for how long, etc?
Maybe start with `sleep 1, retry, sleep 2, retry, sleep 4, return sleep 8, retry, give up` and see if that helps?
And maybe file a rdar :-/
Labels: -Pri-2 Pri-0
This has been closing build tree so often.
Project Member

Comment 25 by bugdroid1@chromium.org, Mar 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fb87dbf4e2eae4532668bebc585b3017970ef3b9

commit fb87dbf4e2eae4532668bebc585b3017970ef3b9
Author: rsesek <rsesek@chromium.org>
Date: Tue Mar 14 12:58:15 2017

In make_hfs.sh, use `diskutil unmount force` instead of just unmount.

BUG= 696529 
TBR=dpranke@chromium.org

Review-Url: https://codereview.chromium.org/2747083002
Cr-Commit-Position: refs/heads/master@{#456680}

[modify] https://crrev.com/fb87dbf4e2eae4532668bebc585b3017970ef3b9/chrome/test/data/safe_browsing/dmg/make_hfs.sh

The error now seems to be consistently this:

[677/49600] ACTION //chrome/test:mac_safe_browsing_test_data(//build/toolchain/mac:clang_x64)
FAILED: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDSP_GPTSPUD.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDSP_NONE.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDSP_SPUD.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDTO_GPTSPUD.cdr test_data/chrome/safe_browsing_dmg/dmg_UDTO_NONE.cdr test_data/chrome/safe_browsing_dmg/dmg_UDTO_SPUD.cdr test_data/chrome/safe_browsing_dmg/dmg_UDZO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDZO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDZO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_SPUD.dmg test_data/chrome/safe_browsing_dmg/hfs_plus.img test_data/chrome/safe_browsing_dmg/hfsx_case_sensitive.img test_data/chrome/safe_browsing_dmg/mach_o_in_dmg.dmg 
python ../../build/gn_run_binary.py ../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh test_data/chrome/safe_browsing_dmg
./../../chrome/test/data/safe_browsing/dmg/make_hfs.sh: line 37: pushd: /Volumes/SafeBrowsingDMG: No such file or directory


So maybe the implicit mount from erasedisk isn't finishing. I'll try forcing that...
Project Member

Comment 27 by bugdroid1@chromium.org, Mar 14 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/575512843787644524196d04076a59ec9415cb89

commit 575512843787644524196d04076a59ec9415cb89
Author: rsesek <rsesek@chromium.org>
Date: Tue Mar 14 18:51:13 2017

In make_hfs.sh, explicitly mount the new volume.

This should happen automatically after erasedisk, but perhaps it's not
completing synchronously.

BUG= 696529 
R=thakis@chromium.org

Review-Url: https://codereview.chromium.org/2749033002
Cr-Commit-Position: refs/heads/master@{#456775}

[modify] https://crrev.com/575512843787644524196d04076a59ec9415cb89/chrome/test/data/safe_browsing/dmg/make_hfs.sh

FAILED: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDSP_GPTSPUD.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDSP_NONE.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDSP_SPUD.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDTO_GPTSPUD.cdr test_data/chrome/safe_browsing_dmg/dmg_UDTO_NONE.cdr test_data/chrome/safe_browsing_dmg/dmg_UDTO_SPUD.cdr test_data/chrome/safe_browsing_dmg/dmg_UDZO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDZO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDZO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_SPUD.dmg test_data/chrome/safe_browsing_dmg/hfs_plus.img test_data/chrome/safe_browsing_dmg/hfsx_case_sensitive.img test_data/chrome/safe_browsing_dmg/mach_o_in_dmg.dmg 
python ../../build/gn_run_binary.py ../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh test_data/chrome/safe_browsing_dmg
Volume on disk1 failed to mount; if it has a partitioning scheme, use "diskutil mountDisk"
If the volume is damaged, try the "readOnly" option


I literally have no idea anymore. Things work fine locally, repeatedly for me.
We've been rolling out 10.12.2. Both of the bots that I checked were 10.12.2, so maybe related? This error does not occur deterministically on either of those machines. I also tried sshing into build9-m1 and running the ninja command manually - that also worked.

Given that we're getting a mounting issue, perhaps we should use pmap with "-diagnostic" to see if the image seems sane/reasonable.

Comment 30 by noel@chromium.org, Mar 15 2017

#28 I literally have no idea anymore. Things work fine locally, repeatedly for me.

Idea: the script seems to auto-create the .dmg .img files.  Maybe run the script locally and produce the files, then wrap them all up in a .zip file (dunno how big the .zip would be or if that would even matter), and land a patch putting the .zip file in chrome/test/data/safe_browsing/dmg.

During the build step, the controlling script would be changed to unzip the files
into chrome/test/data/safe_browsing/dmg, and the tests that use this data would changed to read the .dmg .img files from there?

Basic idea is: chrome/test/data/safe_browsing/dmg always exists, so we should be able to read / write files therein for testing purposes I think.
Project Member

Comment 31 by bugdroid1@chromium.org, Mar 15 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3824c55fa682cafe038041adc75a0d478717dbca

commit 3824c55fa682cafe038041adc75a0d478717dbca
Author: erikchen <erikchen@chromium.org>
Date: Wed Mar 15 02:19:46 2017

Add temporary debugging to make_hfs.sh.

This will tell us about all available disk images, as well as specifics about
the RAMDISK being mounted.

BUG= 696529 

Review-Url: https://codereview.chromium.org/2750913003
Cr-Commit-Position: refs/heads/master@{#456956}

[modify] https://crrev.com/3824c55fa682cafe038041adc75a0d478717dbca/chrome/test/data/safe_browsing/dmg/make_hfs.sh

Erik's debugging logic landed at build #24791, and since then this hasn't flaked up to #24820. https://build.chromium.org/p/chromium/builders/Mac?numbuilds=200

Re: #30: Yeah the easy answer is to just check-in the data, but that would add ~48MB to the repo, which is why I wrote a script to generate it at build-time rather than doing that. Since the issue seems to be with creating the .img files (not the .dmg), we could *just* check in those (the results of make_hfs.sh), and that's only 4.7 MB.

Comment 33 by noel@chromium.org, Mar 17 2017

#32 Good to hear Erik's change worked (/me happy koalas ʕᵔᴥᵔʔ about that.)  The detail I was missing: agree, 48 Megs is a bit on the large side.


Comment 34 Deleted

Project Member

Comment 35 by sheriffbot@chromium.org, Mar 21 2017

Pri-0 bugs are critical regressions or serious emergencies, and this bug has not been updated in three days. Could you please provide an update, or adjust the priority to a more appropriate level if applicable?

If a fix is in active development, please set the status to Started.

Thanks for your time! To disable nags, add the Disable-Nags label.

For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
Labels: -Pri-0 Pri-2
 Issue 708889  has been merged into this issue.
Labels: -Pri-2 Pri-1
This was a tree closure today as well, and I reported it separately in  issue 708889 . The logs there may reveal something interesting, but probably not. Increasing priority.
Cc: tnagel@chromium.org emaxx@chromium.org dcheng@chromium.org tkent@chromium.org
 Issue 713643  has been merged into this issue.
Cc: -tikuta@chromium.org
Tree just closed, error details: https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium%2FMac%2F26768%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout

I see the following in the logs:

FAILED: test_data/chrome/safe_browsing_dmg/dmg_UDBZ_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDBZ_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDCO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDRW_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDSP_GPTSPUD.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDSP_NONE.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDSP_SPUD.sparseimage test_data/chrome/safe_browsing_dmg/dmg_UDTO_GPTSPUD.cdr test_data/chrome/safe_browsing_dmg/dmg_UDTO_NONE.cdr test_data/chrome/safe_browsing_dmg/dmg_UDTO_SPUD.cdr test_data/chrome/safe_browsing_dmg/dmg_UDZO_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UDZO_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UDZO_SPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_GPTSPUD.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_NONE.dmg test_data/chrome/safe_browsing_dmg/dmg_UFBI_SPUD.dmg test_data/chrome/safe_browsing_dmg/hfs_plus.img test_data/chrome/safe_browsing_dmg/hfsx_case_sensitive.img test_data/chrome/safe_browsing_dmg/mach_o_in_dmg.dmg 
python ../../build/gn_run_binary.py ../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh test_data/chrome/safe_browsing_dmg
Temporary debugging for  crbug.com/696529 .
/dev/disk0 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *214.7 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS vm682-m1                213.9 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3

/dev/disk1 (disk image):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   +524.3 KB   disk1

DIIsInitialized: returning YES
DIIsInitialized: returning YES
DIBackingStoreNewWithCFURL: entry with
file:///dev/disk1
writeable: false
DIBackingStoreInstantiatorProbe: entry
file:///dev/disk1
writeable: false
DIBackingStoreInstantiatorProbe: probing interface 0 CBSDBackingStore
CBSDBackingStore::newProbe score 100 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  0, score      100, CBSDBackingStore
DIBackingStoreInstantiatorProbe: probing interface 1 CBundleBackingStore
CBundleBackingStore::newProbe score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  1, score    -1000, CBundleBackingStore
DIBackingStoreInstantiatorProbe: probing interface 2 CRAMBackingStore
CRAMBackingStore::probe: scheme "file": not ram: or ramdisk: scheme.
CRAMBackingStore::probe: score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  2, score    -1000, CRAMBackingStore
DIBackingStoreInstantiatorProbe: probing interface 3 CCarbonBackingStore
CCarbonBackingStore::newProbe: setting initial rval to +100
CCarbonBackingStore::newProbe: [-1000] cannot get FSRef: FSPathMakeRef() failed -36.
CCarbonBackingStore::newProbe score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  3, score    -1000, CCarbonBackingStore
DIBackingStoreInstantiatorProbe: probing interface 4 CDevBackingStore
CDevBackingStore::newProbe score 900 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  4, score      900, CDevBackingStore
DIBackingStoreInstantiatorProbe: probing interface 5 CCURLBackingStore
CCURLBackingStore::probe: scheme is: file
CCURLBackingStore::probe: not recognized URL scheme.
CCURLBackingStore::probe: score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  5, score    -1000, CCURLBackingStore
DIBackingStoreInstantiatorProbe: probing interface 6 CVectoredBackingStore
CVectoredBackingStore::newProbe not "vectored" scheme.
CVectoredBackingStore::newProbe score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  6, score    -1000, CVectoredBackingStore
DIBackingStoreInstantiatorProbe: selecting CDevBackingStore
DIBackingStoreNewWithCFURL: CDevBackingStore
CDevBackingStore::setURL_Permissions_and_OpenFile: url, path, extension, device inode are set
CDevBackingStore::setPermission: entry with request 1
CBSDBackingStore::setPermission: opening /dev/rdisk1
CBSDBackingStore::OpenLockFriendly: mapping flags 0x00000000 -> 0x00000014 (locks are MANDATORY)
CBSDBackingStore::OpenLockFriendly: setting F_SETBACKINGSTORE
CBSDBackingStore::OpenLockFriendly: F_SETBACKINGSTORE returned errno = 22
(RO lock acquired)
CBSDBackingStore::setPermission: kDI_PERM_READONLY granted
CDevBackingStore::setPermission: CBSDBackingStore granted kDI_PERM_READONLY.
CDevBackingStore::setURL_Permissions_and_OpenFile: permission granted kDI_PERM_READONLY.
CDevBackingStore::setURL_Permissions_and_OpenFile: dev is open
CDevBackingStore::_grabDeviceInfo: got block size
CDevBackingStore::_grabDeviceInfo: ioctl(DKIOCGETMAXBLOCKCOUNTREAD) didn't work - using defaultCDevBackingStore::_grabDeviceInfo: ioctl(DKIOCGETMAXBLOCKCOUNTWRITE) didn't work - using defaultCDevBackingStore::_grabDeviceInfo: got block count
CDevBackingStore device size is 0x0000000000080000 (524288) bytes
CDevBackingStore block size is 0x00000200 (512) bytes
CDevBackingStore device size is 0x00000400 (1024) blocks
CDevBackingStore max read transfer size is 0x00000100 (256) blocks
CDevBackingStore max write transfer size is 0x00000100 (256) blocks
DIBackingStoreNewWithCFURL: instantiator returned 0
DIBackingStoreNewWithCFURL: returning 0
DIResolveURLToBackingStore: processing level 1 encodings.
DIFileEncodingNewWithBackingStore: entry for encoding level 1
DIFileEncodingInstantiatorProbe: entry for level 1
writeable: false
DIFileEncodingInstantiatorProbe: probing level 1 interface 0 CMacBinaryEncoding
00000000: 0000 0000 0000 0000 0000 0000 0000 0000   | ................ |
(repeated 7 times)
hdiutil: fileNameLength          $00000000
hdiutil: resourceForkLength      $00000000
hdiutil: dataForkLength          $00000000
hdiutil: commentLength           $00000000
hdiutil: MacBinary III signature        (0x00000000)
hdiutil: header CRC              $00000000
hdiutil: minimum decoder version $00000000
hdiutil: encoder version         $00000000
no MacBinary III signature - checking for MacBinary I or II
calculated CRC          $00000000
MacBinary II found
final MacBinary sanity check
ERROR kFileNameLength <= 0
failed final MacBinary sanity check...
DIFileEncodingInstantiatorProbe: interface  0, score    -1000, CMacBinaryEncoding
DIFileEncodingInstantiatorProbe: probing level 1 interface 1 CAppleSingleEncoding
00000000: 0000 0000 0000 0000 0000 0000 0000 0000   | ................ |
(repeated 1 times)
00000020: 0000 0000 0000 .... .... .... .... ....   | ................ |
CAppleSingleEncoding::isAppleSingleFile loadAppleSingleHeader failed with error 22
DIFileEncodingInstantiatorProbe: interface  1, score    -1000, CAppleSingleEncoding
DIFileEncodingInstantiatorProbe: probing level 1 interface 2 CEncryptedEncoding
CEncryptedEncoding::copyHeaderInformation: inBackingStore->openDataFork returned 0
CEncryptedEncoding::copyHeaderInformation: inBackingStore->getDataForkLength (stub header) returned 0
CEncryptedEncoding::copyHeaderInformation: backingStore data fork length is 524288
CEncryptedEncoding::copyHeaderInformation: reading V1 header from offset 523012
CEncryptedEncoding::copyHeaderInformation: inBackingStore->readDataFork (stub header) returned 0
CEncryptedEncoding::copyHeaderInformation: not recognized as v1 header
CEncryptedEncoding::copyHeaderInformation: reading V2 header from offset 0
CEncryptedEncoding::copyHeaderInformation: inBackingStore->readDataFork (stub header) returned 0
CEncryptedEncoding::copyHeaderInformation: not recognized as v2 header
(null)
DIFileEncodingInstantiatorProbe: interface  2, score    -1000, CEncryptedEncoding
DIFileEncodingInstantiatorProbe: nothing to select.
DIFileEncodingNewWithBackingStore: probe fails to find appropriate CFileEncoding class.
DIFileEncodingNewWithBackingStore: returning 110
DIResolveURLToBackingStore: level 1 encoding match failed. 110. 
(continuing)
DIResolveURLToBackingStore: processing level 2 encodings.
DIFileEncodingNewWithBackingStore: entry for encoding level 2
DIFileEncodingInstantiatorProbe: entry for level 2
writeable: false
DIFileEncodingInstantiatorProbe: probing level 2 interface 0 CUDIFEncoding
CUDIFEncoding::newProbe: failing because backing store is not UDIF file
DIFileEncodingInstantiatorProbe: interface  0, score    -1000, CUDIFEncoding
DIFileEncodingInstantiatorProbe: nothing to select.
DIFileEncodingNewWithBackingStore: probe fails to find appropriate CFileEncoding class.
DIFileEncodingNewWithBackingStore: returning 110
DIResolveURLToBackingStore: level 2 encoding match failed. 110. 
(continuing)
DIResolveURLToBackingStore: processing level 3 encodings.
DIFileEncodingNewWithBackingStore: entry for encoding level 3
DIFileEncodingInstantiatorProbe: entry for level 3
writeable: false
DIFileEncodingInstantiatorProbe: probing level 3 interface 0 CSegmentedNDIFEncoding
checkTypeCreator:     /    
checkTypeCreator: returning     , score 0
CSegmentedNDIFEncoding::isSegmentedNDIFFile checkTypeCreator failed
DIFileEncodingInstantiatorProbe: interface  0, score    -1000, CSegmentedNDIFEncoding
DIFileEncodingInstantiatorProbe: probing level 3 interface 1 CSegmentedUDIFEncoding
CSegmentedUDIFEncoding::isSegmentedUDIFFile backing store is not of type CUDIFEncoding
DIFileEncodingInstantiatorProbe: interface  1, score    -1000, CSegmentedUDIFEncoding
DIFileEncodingInstantiatorProbe: probing level 3 interface 2 CSegmentedUDIFRawEncoding
DIFileEncodingInstantiatorProbe: interface  2, score    -1000, CSegmentedUDIFRawEncoding
DIFileEncodingInstantiatorProbe: nothing to select.
DIFileEncodingNewWithBackingStore: probe fails to find appropriate CFileEncoding class.
DIFileEncodingNewWithBackingStore: returning 110
DIResolveURLToBackingStore: level 3 encoding match failed. 110. 
(continuing)
DIIsInitialized: returning YES
DIDiskImageNewWithBackingStore: entry with
writeable: false
DIDiskImageInstantiatorProbe: entry
writeable: false
DIDiskImageInstantiatorProbe: probing interface 0 CUDIFDiskImage
CUDIFDiskImage::checkBackingStoreType: backing store is wrong type
DIDiskImageInstantiatorProbe: interface  0, score    -1000, CUDIFDiskImage
DIDiskImageInstantiatorProbe: probing interface 1 CSparseBundleDiskImage
CSparseBundleDiskImage::newProbe: returning 0
DIDiskImageInstantiatorProbe: interface  1, score        0, CSparseBundleDiskImage
DIDiskImageInstantiatorProbe: probing interface 2 CSparseDiskImage
DIDiskImageInstantiatorProbe: interface  2, score        0, CSparseDiskImage
DIDiskImageInstantiatorProbe: probing interface 3 CRawDiskImage
CRawDiskImage: checkBackingStoreType score: 100
DIDiskImageInstantiatorProbe: interface  3, score      100, CRawDiskImage
DIDiskImageInstantiatorProbe: probing interface 4 CDARTDiskImage
DIDiskImageInstantiatorProbe: interface  4, score        0, CDARTDiskImage
DIDiskImageInstantiatorProbe: probing interface 5 CDiskCopy42DiskImage
DIDiskImageInstantiatorProbe: interface  5, score        0, CDiskCopy42DiskImage
DIDiskImageInstantiatorProbe: probing interface 6 CNDIFDiskImage
checkTypeCreator:     /    
checkTypeCreator: returning     , score 0
DIDiskImageInstantiatorProbe: interface  6, score        0, CNDIFDiskImage
DIDiskImageInstantiatorProbe: probing interface 8 CShadowedDiskImage
DIDiskImageInstantiatorProbe: interface  8, score     -100, CShadowedDiskImage
DIDiskImageInstantiatorProbe: probing interface 9 CCFPlugInDiskImage
DIDiskImageInstantiatorProbe: interface  9, score    -1000, CCFPlugInDiskImage
DIDiskImageInstantiatorProbe: probing interface 10 CWrappedDiskImage
DIDiskImageInstantiatorProbe: interface 10, score     -100, CWrappedDiskImage
DIDiskImageInstantiatorProbe: selecting CRawDiskImage
DIDiskImageNewWithBackingStore: CRawDiskImage
checkBackingStoreType returned score 100 type *WAR
CDevBackingStore::setPermission: entry with request 1
CDevBackingStore::setPermission: CBSDBackingStore granted kDI_PERM_READONLY.
DIDiskImageNewWithBackingStore: instantiator returned 0
DIDiskImageNewWithBackingStore: returning 0
process_pmap: creating MKMediaRef
process_pmap: scanning media
process_pmap: MKCFReadMedia() returned -5324: Inappropriate ioctl for device.
===== iso scan =====
(null)
(null)
===============================
none
0:
	partition-name: whole disk
	partition-start: 0
	partition-synthesized: true
	partition-length: 1024
	partition-hint: Apple_HFS
	partition-filesystems:
		HFS+: 
block-size: 512
burnable: true
appendable: true
scheme:     none
block size: 512
_ ## Type_________________ Name_________________ Start___ Size____
+    Apple_HFS             whole disk                   0     1024 

+ synthesized
RAMDISK_VOLUME:  + /dev/disk1
Volume on disk1 failed to mount; if it has a partitioning scheme, use "diskutil mountDisk"
If the volume is damaged, try the "readOnly" option
../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh failed with exit code 1

Cc: -tnagel@chromium.org
Issue 719665 has been merged into this issue.
Issue 720629 has been merged into this issue.

Comment 49 by tkent@chromium.org, May 10 2017

Cc: -tkent@chromium.org
I got a new failure, see a log point to this issue.

ython ../../build/gn_run_binary.py ../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh test_data/chrome/safe_browsing_dmg
Temporary debugging for  crbug.com/696529 .
/dev/disk0 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *214.7 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS vm682-m1                213.9 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1 (disk image):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                            SafeBrowsingDMG        +524.3 KB   disk1
DIIsInitialized: returning YES
DIIsInitialized: returning YES
DIBackingStoreNewWithCFURL: entry with
file:///dev/disk1
writeable: false
DIBackingStoreInstantiatorProbe: entry
file:///dev/disk1
writeable: false
DIBackingStoreInstantiatorProbe: probing interface 0 CBSDBackingStore
CBSDBackingStore::newProbe score 100 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  0, score      100, CBSDBackingStore
DIBackingStoreInstantiatorProbe: probing interface 1 CBundleBackingStore
CBundleBackingStore::newProbe score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  1, score    -1000, CBundleBackingStore
DIBackingStoreInstantiatorProbe: probing interface 2 CRAMBackingStore
CRAMBackingStore::probe: scheme "file": not ram: or ramdisk: scheme.
CRAMBackingStore::probe: score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  2, score    -1000, CRAMBackingStore
DIBackingStoreInstantiatorProbe: probing interface 3 CCarbonBackingStore
CCarbonBackingStore::newProbe: setting initial rval to +100
CCarbonBackingStore::newProbe: [-1000] cannot get FSRef: FSPathMakeRef() failed -36.
CCarbonBackingStore::newProbe score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  3, score    -1000, CCarbonBackingStore
DIBackingStoreInstantiatorProbe: probing interface 4 CDevBackingStore
CDevBackingStore::newProbe score 900 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  4, score      900, CDevBackingStore
DIBackingStoreInstantiatorProbe: probing interface 5 CCURLBackingStore
CCURLBackingStore::probe: scheme is: file
CCURLBackingStore::probe: not recognized URL scheme.
CCURLBackingStore::probe: score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  5, score    -1000, CCURLBackingStore
DIBackingStoreInstantiatorProbe: probing interface 6 CVectoredBackingStore
CVectoredBackingStore::newProbe not "vectored" scheme.
CVectoredBackingStore::newProbe score -1000 for file:///dev/disk1
DIBackingStoreInstantiatorProbe: interface  6, score    -1000, CVectoredBackingStore
DIBackingStoreInstantiatorProbe: selecting CDevBackingStore
DIBackingStoreNewWithCFURL: CDevBackingStore
CDevBackingStore::setURL_Permissions_and_OpenFile: url, path, extension, device inode are set
CDevBackingStore::setPermission: entry with request 1
CBSDBackingStore::setPermission: opening /dev/rdisk1
CBSDBackingStore::OpenLockFriendly: mapping flags 0x00000000 -> 0x00000014 (locks are MANDATORY)
CBSDBackingStore::OpenLockFriendly: setting F_SETBACKINGSTORE
CBSDBackingStore::OpenLockFriendly: F_SETBACKINGSTORE returned errno = 22
(RO lock acquired)
CBSDBackingStore::setPermission: kDI_PERM_READONLY granted
CDevBackingStore::setPermission: CBSDBackingStore granted kDI_PERM_READONLY.
CDevBackingStore::setURL_Permissions_and_OpenFile: permission granted kDI_PERM_READONLY.
CDevBackingStore::setURL_Permissions_and_OpenFile: dev is open
CDevBackingStore::_grabDeviceInfo: got block size
CDevBackingStore::_grabDeviceInfo: ioctl(DKIOCGETMAXBLOCKCOUNTREAD) didn't work - using defaultCDevBackingStore::_grabDeviceInfo: ioctl(DKIOCGETMAXBLOCKCOUNTWRITE) didn't work - using defaultCDevBackingStore::_grabDeviceInfo: got block count
CDevBackingStore device size is 0x0000000000080000 (524288) bytes
CDevBackingStore block size is 0x00000200 (512) bytes
CDevBackingStore device size is 0x00000400 (1024) blocks
CDevBackingStore max read transfer size is 0x00000100 (256) blocks
CDevBackingStore max write transfer size is 0x00000100 (256) blocks
DIBackingStoreNewWithCFURL: instantiator returned 0
DIBackingStoreNewWithCFURL: returning 0
DIResolveURLToBackingStore: processing level 1 encodings.
DIFileEncodingNewWithBackingStore: entry for encoding level 1
DIFileEncodingInstantiatorProbe: entry for level 1
writeable: false
DIFileEncodingInstantiatorProbe: probing level 1 interface 0 CMacBinaryEncoding
00000000: 0000 0000 0000 0000 0000 0000 0000 0000   | ................ |
(repeated 7 times)
hdiutil: fileNameLength          $00000000
hdiutil: resourceForkLength      $00000000
hdiutil: dataForkLength          $00000000
hdiutil: commentLength           $00000000
hdiutil: MacBinary III signature        (0x00000000)
hdiutil: header CRC              $00000000
hdiutil: minimum decoder version $00000000
hdiutil: encoder version         $00000000
no MacBinary III signature - checking for MacBinary I or II
calculated CRC          $00000000
MacBinary II found
final MacBinary sanity check
ERROR kFileNameLength <= 0
failed final MacBinary sanity check...
DIFileEncodingInstantiatorProbe: interface  0, score    -1000, CMacBinaryEncoding
DIFileEncodingInstantiatorProbe: probing level 1 interface 1 CAppleSingleEncoding
00000000: 0000 0000 0000 0000 0000 0000 0000 0000   | ................ |
(repeated 1 times)
00000020: 0000 0000 0000 .... .... .... .... ....   | ................ |
CAppleSingleEncoding::isAppleSingleFile loadAppleSingleHeader failed with error 22
DIFileEncodingInstantiatorProbe: interface  1, score    -1000, CAppleSingleEncoding
DIFileEncodingInstantiatorProbe: probing level 1 interface 2 CEncryptedEncoding
CEncryptedEncoding::copyHeaderInformation: inBackingStore->openDataFork returned 0
CEncryptedEncoding::copyHeaderInformation: inBackingStore->getDataForkLength (stub header) returned 0
CEncryptedEncoding::copyHeaderInformation: backingStore data fork length is 524288
CEncryptedEncoding::copyHeaderInformation: reading V1 header from offset 523012
CEncryptedEncoding::copyHeaderInformation: inBackingStore->readDataFork (stub header) returned 0
CEncryptedEncoding::copyHeaderInformation: not recognized as v1 header
CEncryptedEncoding::copyHeaderInformation: reading V2 header from offset 0
CEncryptedEncoding::copyHeaderInformation: inBackingStore->readDataFork (stub header) returned 0
CEncryptedEncoding::copyHeaderInformation: not recognized as v2 header
(null)
DIFileEncodingInstantiatorProbe: interface  2, score    -1000, CEncryptedEncoding
DIFileEncodingInstantiatorProbe: nothing to select.
DIFileEncodingNewWithBackingStore: probe fails to find appropriate CFileEncoding class.
DIFileEncodingNewWithBackingStore: returning 110
DIResolveURLToBackingStore: level 1 encoding match failed. 110. 
(continuing)
DIResolveURLToBackingStore: processing level 2 encodings.
DIFileEncodingNewWithBackingStore: entry for encoding level 2
DIFileEncodingInstantiatorProbe: entry for level 2
writeable: false
DIFileEncodingInstantiatorProbe: probing level 2 interface 0 CUDIFEncoding
CUDIFEncoding::newProbe: failing because backing store is not UDIF file
DIFileEncodingInstantiatorProbe: interface  0, score    -1000, CUDIFEncoding
DIFileEncodingInstantiatorProbe: nothing to select.
DIFileEncodingNewWithBackingStore: probe fails to find appropriate CFileEncoding class.
DIFileEncodingNewWithBackingStore: returning 110
DIResolveURLToBackingStore: level 2 encoding match failed. 110. 
(continuing)
DIResolveURLToBackingStore: processing level 3 encodings.
DIFileEncodingNewWithBackingStore: entry for encoding level 3
DIFileEncodingInstantiatorProbe: entry for level 3
writeable: false
DIFileEncodingInstantiatorProbe: probing level 3 interface 0 CSegmentedNDIFEncoding
checkTypeCreator:     /    
checkTypeCreator: returning     , score 0
CSegmentedNDIFEncoding::isSegmentedNDIFFile checkTypeCreator failed
DIFileEncodingInstantiatorProbe: interface  0, score    -1000, CSegmentedNDIFEncoding
DIFileEncodingInstantiatorProbe: probing level 3 interface 1 CSegmentedUDIFEncoding
CSegmentedUDIFEncoding::isSegmentedUDIFFile backing store is not of type CUDIFEncoding
DIFileEncodingInstantiatorProbe: interface  1, score    -1000, CSegmentedUDIFEncoding
DIFileEncodingInstantiatorProbe: probing level 3 interface 2 CSegmentedUDIFRawEncoding
DIFileEncodingInstantiatorProbe: interface  2, score    -1000, CSegmentedUDIFRawEncoding
DIFileEncodingInstantiatorProbe: nothing to select.
DIFileEncodingNewWithBackingStore: probe fails to find appropriate CFileEncoding class.
DIFileEncodingNewWithBackingStore: returning 110
DIResolveURLToBackingStore: level 3 encoding match failed. 110. 
(continuing)
DIIsInitialized: returning YES
DIDiskImageNewWithBackingStore: entry with
writeable: false
DIDiskImageInstantiatorProbe: entry
writeable: false
DIDiskImageInstantiatorProbe: probing interface 0 CUDIFDiskImage
CUDIFDiskImage::checkBackingStoreType: backing store is wrong type
DIDiskImageInstantiatorProbe: interface  0, score    -1000, CUDIFDiskImage
DIDiskImageInstantiatorProbe: probing interface 1 CSparseBundleDiskImage
CSparseBundleDiskImage::newProbe: returning 0
DIDiskImageInstantiatorProbe: interface  1, score        0, CSparseBundleDiskImage
DIDiskImageInstantiatorProbe: probing interface 2 CSparseDiskImage
DIDiskImageInstantiatorProbe: interface  2, score        0, CSparseDiskImage
DIDiskImageInstantiatorProbe: probing interface 3 CRawDiskImage
CRawDiskImage: checkBackingStoreType score: 100
DIDiskImageInstantiatorProbe: interface  3, score      100, CRawDiskImage
DIDiskImageInstantiatorProbe: probing interface 4 CDARTDiskImage
DIDiskImageInstantiatorProbe: interface  4, score        0, CDARTDiskImage
DIDiskImageInstantiatorProbe: probing interface 5 CDiskCopy42DiskImage
DIDiskImageInstantiatorProbe: interface  5, score        0, CDiskCopy42DiskImage
DIDiskImageInstantiatorProbe: probing interface 6 CNDIFDiskImage
checkTypeCreator:     /    
checkTypeCreator: returning     , score 0
DIDiskImageInstantiatorProbe: interface  6, score        0, CNDIFDiskImage
DIDiskImageInstantiatorProbe: probing interface 8 CShadowedDiskImage
DIDiskImageInstantiatorProbe: interface  8, score     -100, CShadowedDiskImage
DIDiskImageInstantiatorProbe: probing interface 9 CCFPlugInDiskImage
DIDiskImageInstantiatorProbe: interface  9, score    -1000, CCFPlugInDiskImage
DIDiskImageInstantiatorProbe: probing interface 10 CWrappedDiskImage
DIDiskImageInstantiatorProbe: interface 10, score     -100, CWrappedDiskImage
DIDiskImageInstantiatorProbe: selecting CRawDiskImage
DIDiskImageNewWithBackingStore: CRawDiskImage
checkBackingStoreType returned score 100 type *WAR
CDevBackingStore::setPermission: entry with request 1
CDevBackingStore::setPermission: CBSDBackingStore granted kDI_PERM_READONLY.
DIDiskImageNewWithBackingStore: instantiator returned 0
DIDiskImageNewWithBackingStore: returning 0
process_pmap: creating MKMediaRef
process_pmap: scanning media
process_pmap: MKCFReadMedia() returned -5324: Inappropriate ioctl for device.
===== iso scan =====
(null)
(null)
===============================
none
0:
	partition-name: whole disk
	partition-start: 0
	partition-synthesized: true
	partition-length: 1024
	partition-hint: Apple_HFS
	partition-filesystems:
		HFS+: 
block-size: 512
burnable: true
appendable: true
scheme:     none
block size: 512
_ ## Type_________________ Name_________________ Start___ Size____
+    Apple_HFS             whole disk                   0     1024 
+ synthesized
RAMDISK_VOLUME:  + /dev/disk1
Volume on disk1 failed to mount; if it has a partitioning scheme, use "diskutil mountDisk"
If the volume is damaged, try the "readOnly" option
../../chrome/test/data/safe_browsing/dmg/generate_test_data.sh failed with exit code 1

https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium%2FMac%2F27334%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout
Labels: Type-Bug
erik/robert - what's the advice to build sheriffs for this? (Just reopen?)

another - https://luci-logdog.appspot.com/v/?s=chromium%2Fbb%2Fchromium%2FMac%2F27364%2F%2B%2Frecipes%2Fsteps%2Fcompile%2F0%2Fstdout

Comment 52 by groby@chromium.org, May 17 2017

Also: I just had a local build of unit tests fail. I'm all for instrumentation, but can we constrain it so it only affects the bots? :)

Either way - a new build succeeded. The failed build had one run of generate_test_data.sh, the succeeded build had two (successful) runs of generate_test_data.sh

All of them attached. The main difference between failed and first successful run is that the failed one used /dev/disk2, the successful one used /dev/disk3
log-bad
11.9 KB View Download
log-good
11.8 KB View Download
log-good-2
11.9 KB View Download
This just failed for me in my local build of unit_tests at 5c29de4fcc53.  It fails repeatedly at that step.  Is there a known workaround?

Logs attached.

log-bad.txt
13.9 KB View Download

Comment 54 by vabr@chromium.org, Jun 22 2017

Another case seen in https://build.chromium.org/p/chromium/builders/Mac/builds/29190
compile.txt
277 KB View Download
I don't see that as the same failure whatsoever - it's not in the compile (no-op) step.  I think some time in April a separate issue got glommed together which is occasional failure of safe_browsing/dmg/generate_test_data.sh

I get that the changes to fix the first one may have lead to the second, but they have different contexts now.  One needs GN/ninja/build experts and the other needs Apple/MacOS/Ramdisk experts.

rsesek: I suggest you split the two issues.
Cc: mortonm@google.com
 Issue 751144  has been merged into this issue.
Note that I can reproduce this locally 100% of the time (i.e., I can't build unit_tests on Mac unless I remove this target).
This happened again today: https://luci-milo.appspot.com/buildbot/chromium/Mac/31328

rsesek: what's the plan here?
Flaky tests should generally be disabled if they aren't fixed (Or while they're being fixed).  A flake build rule seems an order of magnitude worth than that since it prevents all tests from running and closes the tree.  Should we just remove the target and disable the tests, so it can be fixed offline?
There's been a fix in the cq for 10h or so (https://chromium-review.googlesource.com/#/c/609143/)
That's great!
Project Member

Comment 69 by bugdroid1@chromium.org, Aug 10 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/c680ef2f77f42cb21b8b99976533468d2532d267

commit c680ef2f77f42cb21b8b99976533468d2532d267
Author: Robert Sesek <rsesek@chromium.org>
Date: Thu Aug 10 01:12:05 2017

Check in Safe Browsing DMG HFS test data.

The make_hfs.sh step has a tendency to fail for mysterious reasons (it appears
a macOS daemon gets wedged). Rather than generating these HFS data at
build-time, it is now generated independently and checked in as a compressed
archive hfs_raw_images.tar.bz2. The archive is then extracted as part of the
build.

Bug:  696529 
Change-Id: I0a8cd286973eb796d0a132d0b24588179ec7008d
Reviewed-on: https://chromium-review.googlesource.com/609143
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#493207}
[modify] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/BUILD.gn
[add] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/data/safe_browsing/dmg/.gitignore
[add] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/data/safe_browsing/dmg/Makefile
[add] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/data/safe_browsing/dmg/README.md
[modify] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/data/safe_browsing/dmg/generate_test_data.sh
[add] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/data/safe_browsing/dmg/hfs_raw_images.tar.bz2
[modify] https://crrev.com/c680ef2f77f42cb21b8b99976533468d2532d267/chrome/test/data/safe_browsing/dmg/make_hfs.sh

Status: Fixed (was: Assigned)
This hasn't happened in the past two weeks, so calling this fixed.

Sign in to add a comment