I'd recommend 1.

yasm is much closer to masm than LLVM. Third-party projects that Chromium doesn't control are much more likely to accept patches to make their assembly yasm-compatible. Sending a patch to rewrite their .asm file to use GNU-as-compatible syntax (even with .intel_syntax) might not go over as well.

I think yasm is basically a rewrite of nasm, so this should describe what needs to be done:
http://www.nasm.us/doc/nasmdoc2.html#section-2.2

SkJumper_generated_win.S differs from SkJumper_generated.S in two ways today:
  A) it's in ml.exe syntax
  B) it translates from MS fastcall to the standard Sys-V ABI inside start_pipeline, which is then the entry point to all other code in the file.

Since it's all generated, we could pretty easily generate a third .S file that does B) but not A), i.e. a Clang compatible assembly file that still does the important ABI switch.  I think this is similar to your idea 3. without breaking compatibility of other clients without Clang.

I guess I should say, we'd be happy to generate whatever dialect of assembly you like.

I think I'd probably lean towards #3 a bit for Crashpad due to the out-of-repo build.

Crashpad doesn't currently have .gn files (other than in Chromium src.git). The current build is gyp, and those expect gyp's magic handling of asm.

If Crashpad were to write .gn files for real that live in the Crashpad repo (as is needed for the Fuchsia-the-OS-side use of Crashpad, and would just generally be better) it would be pretty heavy to have to pull yasm/nasm/whatever and make a GN build for that just to assemble the one .asm file.

(It's actually two .asm files in crashpad, win/safe_terminate_process.asm too.)

The following revision refers to this bug:
  https://skia.googlesource.com/skia/+/6492afa7971cf295a3c3cb92a85218917c02bb4a

commit 6492afa7971cf295a3c3cb92a85218917c02bb4a
Author: Nico Weber <thakis@chromium.org>
Date: Sat Sep 16 00:28:48 2017

Disable SkJumper assembly in cross builds for now.

Bug: chromium:762167
Change-Id: Ia23f6dbfc0466aef4ca9d1a5b9ff343d79dc83bb
Reviewed-on: https://skia-review.googlesource.com/47460
Reviewed-by: Mike Klein <mtklein@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>

[modify] https://crrev.com/6492afa7971cf295a3c3cb92a85218917c02bb4a/src/jumper/SkJumper.cpp
[modify] https://crrev.com/6492afa7971cf295a3c3cb92a85218917c02bb4a/gn/core.gni

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/78abf62baeea20940b0fcfb5e8510a88703be81b

commit 78abf62baeea20940b0fcfb5e8510a88703be81b
Author: Nico Weber <thakis@chromium.org>
Date: Sat Sep 16 15:28:47 2017

skia: Hook up new skia_core_defines in chrome build.

It was added in https://skia-review.googlesource.com/c/skia/+/47460

Bug: 762167
Change-Id: I41e047550d7695a5c4930dd7bf1691d1fc076480
Reviewed-on: https://chromium-review.googlesource.com/669585
Commit-Queue: Nico Weber <thakis@chromium.org>
Commit-Queue: Mike Klein <mtklein@chromium.org>
Reviewed-by: Mike Klein <mtklein@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502503}
[modify] https://crrev.com/78abf62baeea20940b0fcfb5e8510a88703be81b/skia/BUILD.gn

FYI, just filed issue 766721 about switching to nasm if that changes anything here. 

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/912c9907d5636ec45d59f95cd0902f4215e027d9

commit 912c9907d5636ec45d59f95cd0902f4215e027d9
Author: Nico Weber <thakis@chromium.org>
Date: Mon Oct 23 16:50:42 2017

Use inline asm instead of ml.exe for SafeTerminateProcess().

It's a bit less code, and it removes one of two uses of ml.exe in the chrome
build.  No intended behavior change.

clang-cl's output for the new code:
	mov	eax, dword ptr [__imp__TerminateProcess@8]

	push	ebp
	mov	ebp, esp
	push	dword ptr [ebp + 12]
	push	dword ptr [ebp + 8]
	call	dword ptr [eax]
	test	eax, eax
	setne	al
	mov	esp, ebp
	pop	ebp
	ret


cl.exe's output for the new code:
	push	ebp
	mov	ebp, esp
	push	DWORD PTR [ebp+12]
	push	DWORD PTR [ebp+8]
	call	DWORD PTR __imp__TerminateProcess@8
	test	eax, eax
	setne	al
	mov	esp, ebp
	pop	ebp
	ret	0

Bug: 762167
Change-Id: I54266504a09c150ef6ae8fb8defa42af65ac7e6c
Reviewed-on: https://chromium-review.googlesource.com/732030
Reviewed-by: Mark Mentovai <mark@chromium.org>
Cr-Commit-Position: refs/heads/master@{#510822}
[modify] https://crrev.com/912c9907d5636ec45d59f95cd0902f4215e027d9/build/secondary/third_party/crashpad/crashpad/util/BUILD.gn
[delete] https://crrev.com/c171b524e0ac3b2de57d7d0b230c13adc2570178/third_party/crashpad/crashpad/util/win/safe_terminate_process.asm
[add] https://crrev.com/912c9907d5636ec45d59f95cd0902f4215e027d9/third_party/crashpad/crashpad/util/win/safe_terminate_process.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fc1ac734b09f7f8017c01d66947b3d17b1ba1718

commit fc1ac734b09f7f8017c01d66947b3d17b1ba1718
Author: Nico Weber <thakis@chromium.org>
Date: Mon Oct 23 18:22:42 2017

crashpad: In win cross builds, provide dummy CaptureContext().

This is not fit for production, but it unblocks building 'chrome',
'browser_tests', and almost all other targets on Linux hosts.

Bug: 762167
Change-Id: Ib0b7de5cbc71c87ca3630254d6ddda0ad38f1769
Reviewed-on: https://chromium-review.googlesource.com/733601
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Mark Mentovai <mark@chromium.org>
Cr-Commit-Position: refs/heads/master@{#510853}
[modify] https://crrev.com/fc1ac734b09f7f8017c01d66947b3d17b1ba1718/build/secondary/third_party/crashpad/crashpad/util/BUILD.gn
[add] https://crrev.com/fc1ac734b09f7f8017c01d66947b3d17b1ba1718/third_party/crashpad/crashpad/util/win/capture_context_broken.cc

I'm slowly coming around that we should teach clang's assembler to also accept masm syntax. The only way to currently use assembly without additional deps is to do .S files for non-Win and .asm for Win, and clang should Just Work in that world.

(I did try using yasm for some .asm file in cross-builds only given rnk's suggestion on the bug, but http://yasm.tortall.net/HowToContribute.html lists "masm syntax support" as an open project, and it didn't work in practice, so that's out.)

However, getting masm syntax support will be a bit of work. For now, I've been able to remove all uses of .asm files in cross builds (with some fallout; for example skia code is less optimized.)

So cross builds are unblocked for now, but we'll need to do something better. Keeping this open for that.

(Maybe nasm instead? It's supposed to support MASM-ish according to https://en.wikipedia.org/wiki/Netwide_Assembler. I don't know if that's more hassle than it's worth though.)

If it's getting down to Skia, don't forget that our assembly is all generated.  We can certainly generate separate assembly files for clang, clang-cl.exe, and ml[64].exe.

We're also happy to work with a _very_ restricted subset of masm syntax.  We do just about everything with DB, ALIGN, a couple IF/ENDIF, and of course comments.

Yes, skia isn't much of an issue.

There's a single .asm file in crashpad for capturing stack traces. And then there's a whole bunch of .asm files in enable_nacl=true builds (enable_nacl defaults to false for cross builds).

More interestingly, if someone were to add a .asm file for something in the future, everything except "have clang accept masm syntax" will cause annoyances.

So I think adding masm support is probably the lowest-annoyance thing going forward, but for now we managed to punt on this and still got the cross build working.

The following revision refers to this bug:
  https://chromium.googlesource.com/crashpad/crashpad.git/+/55133d332b6c32777107cddbce60eacf0ad35497

commit 55133d332b6c32777107cddbce60eacf0ad35497
Author: Mark Mentovai <mark@chromium.org>
Date: Mon Oct 23 19:14:05 2017

win: Use inline asm instead of ml.exe for SafeTerminateProcess()

This upstreams
https://chromium.googlesource.com/chromium/src/+/912c9907d5636ec45d59f95cd0902f4215e027d9
(slightly modified).

Bug: chromium:762167
Change-Id: I69c605f693da8691d32222b5617f62637c1c2dcd
Reviewed-on: https://chromium-review.googlesource.com/734100
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>

[modify] https://crrev.com/55133d332b6c32777107cddbce60eacf0ad35497/util/util.gyp
[delete] https://crrev.com/ce084d37c83dbe7bec296a8ce0420228f75de9f9/util/win/safe_terminate_process.asm
[add] https://crrev.com/55133d332b6c32777107cddbce60eacf0ad35497/util/win/safe_terminate_process.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/crashpad/crashpad.git/+/2f4516f938386a242ff1204755dc551f4ae98079

commit 2f4516f938386a242ff1204755dc551f4ae98079
Author: Mark Mentovai <mark@chromium.org>
Date: Mon Oct 23 19:56:35 2017

win: Provide broken CHECK()ing dummy CaptureContext() for cross builds

This upstreams
https://chromium.googlesource.com/chromium/src/+/fc1ac734b09f7f8017c01d66947b3d17b1ba1718
(slightly modified).

This dummy implementation is not used by Crashpad anywhere at all, and
is only used by Crashpad in Chromium during a cross build targeting
Windows without the benefit of Microsoft’s ml.exe/ml64.exe assembler.
Strictly speaking, this file does not need to be checked in to the
Crashpad repository, but since Chromium needs it to unblock its
not-production-ready cross build for Windows, it’s being landed here to
avoid Chromium’s copy of Crashpad appearing as modified or “dirty”
relative to this upstream copy. (Even though this file is really dirty.)

Bug: chromium:762167
Change-Id: Ibfdc316c1f5fe81d4b3a1d86f4032adccac467e5
Reviewed-on: https://chromium-review.googlesource.com/734102
Reviewed-by: Mark Mentovai <mark@chromium.org>
Commit-Queue: Mark Mentovai <mark@chromium.org>

[add] https://crrev.com/2f4516f938386a242ff1204755dc551f4ae98079/util/win/capture_context_broken.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/af5f31ed615bdbf0ad246cff643a2802a1e0d702

commit af5f31ed615bdbf0ad246cff643a2802a1e0d702
Author: Nico Weber <thakis@chromium.org>
Date: Tue Oct 24 13:15:02 2017

crashpad: Revert inlin-asm-for-SafeTerminateProcess(); call TerminateProcess() in cross builds instead.

The inline asm doesn't work in clang builds for reasons I don't yet understand.
Since we just switched to clang by default, lets get back to a good state first.

TBR=mark@chromium.org

Bug:  777579 , 777741 ,762167
Change-Id: I928ebaeaca033da69eaf1d832f477fb3fdca0283
Reviewed-on: https://chromium-review.googlesource.com/735619
Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Hans Wennborg <hans@chromium.org>
Cr-Commit-Position: refs/heads/master@{#511127}
[modify] https://crrev.com/af5f31ed615bdbf0ad246cff643a2802a1e0d702/build/secondary/third_party/crashpad/crashpad/util/BUILD.gn
[add] https://crrev.com/af5f31ed615bdbf0ad246cff643a2802a1e0d702/third_party/crashpad/crashpad/util/win/safe_terminate_process.asm
[delete] https://crrev.com/5ebcc615a08792ecd11ef69119a6a51b3ad615c3/third_party/crashpad/crashpad/util/win/safe_terminate_process.cc
[add] https://crrev.com/af5f31ed615bdbf0ad246cff643a2802a1e0d702/third_party/crashpad/crashpad/util/win/safe_terminate_process_broken.cc

The following revision refers to this bug:
  https://chromium.googlesource.com/crashpad/crashpad.git/+/6d5bd1d04d5f50d9f4a58d528ccfb38c46423f23

commit 6d5bd1d04d5f50d9f4a58d528ccfb38c46423f23
Author: Mark Mentovai <mark@chromium.org>
Date: Tue Oct 24 19:07:38 2017

win: Go back to using ml.exe for SafeTerminateProcess()

This reverts 55133d332b6c and adds a broken dummy SafeTerminateProcess()
for cross builds instead. It’s similar to 2f4516f93838, which was for
CaptureContext().

This upstreams
https://chromium.googlesource.com/chromium/src/+/af5f31ed615bdbf0ad246cff643a2802a1e0d702
(slightly modified).

The dummy implementation in the “broken” file affords no protection
against third-party code patching TerminateProcess() badly. The “broken”
file is not used by Crashpad anywhere at all, and is only used by
Crashpad in Chromium during a cross build targeting Windows without the
benefit of Microsoft’s ml.exe assembler. Strictly speaking, this file
does not need to be checked in to the Crashpad repository, but since
Chromium needs it to unblock its not-production-ready cross build for
Windows, it’s being landed here to avoid Chromium’s copy of Crashpad
appearing as modified or “dirty” relative to this upstream copy.

Bug: chromium:762167,  chromium:777924 
Change-Id: Iba68c0cab142fbe9541ea254a9a856b8263e4c70
Reviewed-on: https://chromium-review.googlesource.com/735078
Reviewed-by: Mark Mentovai <mark@chromium.org>

[modify] https://crrev.com/6d5bd1d04d5f50d9f4a58d528ccfb38c46423f23/util/util.gyp
[add] https://crrev.com/6d5bd1d04d5f50d9f4a58d528ccfb38c46423f23/util/win/safe_terminate_process.asm
[delete] https://crrev.com/025455e77af4eb0cffad42dda61aff0151ba1bfa/util/win/safe_terminate_process.cc
[add] https://crrev.com/6d5bd1d04d5f50d9f4a58d528ccfb38c46423f23/util/win/safe_terminate_process_broken.cc

(note to self: the revert was due to https://bugs.llvm.org/show_bug.cgi?id=35056, see other bug linked in that cl description)

The skia issue resolved itself in https://skia-review.googlesource.com/c/skia/+/110641

Yep, hoping Skia stays back in the wonderful boring all C++ world from now on.

Cool :)

We should track this, though:
> I have had to drop Windows down to __vectorcall and our narrower
> stage calling convention that keeps the d-registers on the stack.
> I tried forcing sysv_abi, but that crashed Clang.  :/

I wasn't able to reproduce this, but I did notice that sysv_abi doesn't work at all on Windows. We effectively have the reverse of https://llvm.org/pr31362, we apply the Windows CC rules in the frontend instead of the SysV ones.

Yeah, great minds think alike: Nico just mentioned the same on https://skia-review.googlesource.com/c/skia/+/110641.  I'm in the middle of declawing Bit9 from my freshly re-imaged Windows workstation, and while waiting following along https://chromium.googlesource.com/chromium/src/+/master/docs/win_cross.md to see what I can do from a cross compile.

You should be able to see the failure on some of the red bots on PS 11.  They're all roughly this:

FAILED: obj/src/opts/sse41.SkOpts_sse41.obj 
"c:\b\s\w\ir\clang_win/bin/clang-cl.exe" /nologo /showIncludes /FC @obj/src/opts/sse41.SkOpts_sse41.obj.rsp /c ../../../src/opts/SkOpts_sse41.cpp /Foobj/src/opts/sse41.SkOpts_sse41.obj /Fd"obj/sse41_c.pdb"
fatal error: error in backend: stack allocation size is not a multiple of 8


My attempt to try sysv_abi again was https://skia-review.googlesource.com/c/skia/+/113705, but it looked too green to be true.  I suspect I've mixed up #defines and am not really falling into the sysv_abi case.

There's an existing bug for some of the perf knock on effects of that CL (https://bugs.chromium.org/p/chromium/issues/detail?id=821012) but we might want a different standalone "sysv_abi on Windows" bug.

I filed https://bugs.llvm.org/show_bug.cgi?id=36806 as a standalone "make sysv_abi work on Windows" bug, but it's possible it can't be made to work. We'll see.

You might also consider trying Intel's __regcall calling convention. 

Hey, major props to Windows cross compiling effort.  I've got it working, and with a small patch to Skia was able to get ideal sysv_abi code, at least in the couple functions I always use as my smoke test:

0000000000000000 <?premul@hsw@@YAX_KPEAPEAX00T__m256@@2222222@Z>:
   0:   c5 fc 59 c3             vmulps %ymm3,%ymm0,%ymm0
   4:   c5 f4 59 cb             vmulps %ymm3,%ymm1,%ymm1
   8:   c5 ec 59 d3             vmulps %ymm3,%ymm2,%ymm2
   c:   48 ad                   lods   %ds:(%rsi),%rax
   e:   48 ff e0                rex.W jmpq *%rax

0000000000000000 <?premul_dst@hsw@@YAX_KPEAPEAX00T__m256@@2222222@Z>:
   0:   c5 dc 59 e7             vmulps %ymm7,%ymm4,%ymm4
   4:   c5 d4 59 ef             vmulps %ymm7,%ymm5,%ymm5
   8:   c5 cc 59 f7             vmulps %ymm7,%ymm6,%ymm6
   c:   48 ad                   lods   %ds:(%rsi),%rax
   e:   48 ff e0                rex.W jmpq *%rax

Compare that with vectorcall:

0000000000000000 <?premul@hsw@@YQXPEAUParams@1@PEAPEAXT__m256@@222@Z>:
   0:   c5 ec 59 d5                     vmulps %ymm5,%ymm2,%ymm2
   4:   c5 e4 59 dd                     vmulps %ymm5,%ymm3,%ymm3
   8:   c5 dc 59 e5                     vmulps %ymm5,%ymm4,%ymm4
   c:   48 8b 02                        mov    (%rdx),%rax
   f:   48 83 c2 08                     add    $0x8,%rdx
  13:   48 ff e0                        rex.W jmpq *%rax

0000000000000000 <?premul_dst@hsw@@YQXPEAUParams@1@PEAPEAXT__m256@@222@Z>:
   0:   c5 fc 28 81 80 00 00 00         vmovaps 0x80(%rcx),%ymm0
   8:   c5 fc 59 49 20                  vmulps 0x20(%rcx),%ymm0,%ymm1
   d:   c5 fc 29 49 20                  vmovaps %ymm1,0x20(%rcx)
  12:   c5 fc 59 49 40                  vmulps 0x40(%rcx),%ymm0,%ymm1
  17:   c5 fc 29 49 40                  vmovaps %ymm1,0x40(%rcx)
  1c:   c5 fc 59 41 60                  vmulps 0x60(%rcx),%ymm0,%ymm0
  21:   c5 fc 29 41 60                  vmovaps %ymm0,0x60(%rcx)
  26:   48 8b 02                        mov    (%rdx),%rax
  29:   48 83 c2 08                     add    $0x8,%rdx
  2d:   48 ff e0                        rex.W jmpq *%rax


It's possible that we were using an older clang-cl back when I failed the first time?  Unclear... things seem to build fine now.

__regcall is a really good idea.  I'd ruled it out earlier back in the offline-compilation-and-disassemble days because it mangled names in a somewhat tricky way, I think?  But now I really don't care about that anymore.  We've actually been wanting to pass a 9th vector register (containing coverage) through our pipeline, and this might be a way forward there.

(I have not yet run this code, nor even linked.)

Got my Windows machine back up... building with clang-cl there is still hitting the fatal error:

"D:\src\clang_win/bin/clang-cl.exe" /nologo /showIncludes /FC @obj/src/opts/sse41.SkOpts_sse41.obj.rsp /c ../src/opts/SkOpts_sse41.cpp /Foobj/src/opts/sse41.SkOpts_sse41.obj /Fd"obj/sse41_c.pdb"
fatal error: error in backend: stack allocation size is not a multiple of 8

"D:\src\clang_win/bin/clang-cl.exe" -- where did you get that binary from? Is that the chromium one? What's `D:\src\clang_win/bin/clang-cl.exe --version`?

Yep, it's the Chromium one at 327688.  Same deal happens inside the Chromium tree.

We fetched it like this:

  CLANG_REVISION = '327688'
  CLANG_SUB_REVISION = '1'
  CLANG_PKG_VERSION = '%s-%s' % (CLANG_REVISION, CLANG_SUB_REVISION)
  GS_URL = ('https://commondatastorage.googleapis.com/chromium-browser-clang'
            '/Win/clang-%s.tgz' % CLANG_PKG_VERSION)

This bug has an owner, thus, it's been triaged. Changing status to "assigned".

ml64.exe always writes the coff timestamp header as far as I can tell (it doesn't print a "unknown flag" if you pass /Brepro like it does for other flags, but that flag doesn't have the effect of putting something fixed in the header like it does for cl). So that's a (weak) argument for rolling our own assembler eventually.

What's worse, ml64.exe seems to write a .debug$S section containing an absolute path to the obj name, making deterministic builds difficult (cf https://chromium-review.googlesource.com/c/chromium/src/+/1273479 which works for everything except the two asm files in crashpad)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/de2f0bfe2c61fe2a823640d2f5ffcada755cac59

commit de2f0bfe2c61fe2a823640d2f5ffcada755cac59
Author: Nico Weber <thakis@chromium.org>
Date: Tue Oct 30 10:21:53 2018

win: Add hacky ml.exe wrapper that makes ml's output deterministic after the fact.

Without this, chrome.exe.pdb contains two absolute paths (to the two obj files
built by ml64 for crashpad).

Bug: 762167,330260,899438
Change-Id: If4e0195b51f7d9ff70d890fe3a5302f5961c94d6
Reviewed-on: https://chromium-review.googlesource.com/c/1303887
Reviewed-by: Reid Kleckner <rnk@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Commit-Position: refs/heads/master@{#603852}
[modify] https://crrev.com/de2f0bfe2c61fe2a823640d2f5ffcada755cac59/build/toolchain/win/BUILD.gn
[add] https://crrev.com/de2f0bfe2c61fe2a823640d2f5ffcada755cac59/build/toolchain/win/ml.py