New issue
Advanced search Search tips

Issue 623342 link

Starred by 3 users

Issue metadata

Status: Untriaged
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 3
Type: Bug



Sign in to add a comment

Windows: Cannot set custom toolchain args with GN

Project Member Reported by marshall@chromium.org, Jun 25 2016

Issue description

Version: Chromium revision 318e6f54 (#400326)
OS: Windows 10 64-bit

What steps will reproduce the problem?
(1) Try to use a custom toolchain for `gn gen` on  Windows by specifying some combination of windows_sdk_path, wdk_path, visual_studio_path, visual_studio_version, visual_studio_runtime_dirs.

What is the expected output?
Some combination of these values should succeed.

What do you see instead?
Various errors. See below.

Please use labels and text to provide additional information.

I believe only the windows_sdk_path, visual_studio_path, visual_studio_version and visual_studio_runtime_dirs values should be required and set by the user. For example:

Directory structure:

C:\cache\vs-professional-2015\
  VC\
    atlmfc\
    bin\
    crt\
    include\
    lib\
    redist\
  win10sdk\
    bin\
    Include\
    Lib\
  SystemCRT\
    api-ms-win-core-file-l1-2-0.dll
    api-ms-win-core-file-l2-1-0.dll
    ...

Associated GN arguments:
windows_sdk_path="C:\cache\vs-professional-2015\extracted\win10sdk"
visual_studio_path="C:\cache\vs-professional-2015\extracted"
visual_studio_version="2015"
visual_studio_runtime_dirs="C:\cache\vs-professional-2015\extracted\SystemCRT"


The problems are as follows:

1. wdk_path is required by build/config/win/visual_studio_version.gni but does not appear to be used anywhere in the Chromium GN config. The assert for this value should probably be removed.

2. Cannot override visual_studio_runtime_dirs and cannot use the default value. The default value should probably be removed and visual_studio_runtime_dirs moved to a declare_args() block.

2a. Error when no visual_studio_runtime_dirs value is specified:

ERROR at //build/toolchain/win/BUILD.gn:300:38: This is not a string.
                                     visual_studio_runtime_dirs,
                                     ^-------------------------
Instead I see a list = []

2b. Error when visual_studio_runtime_dirs value is specified:

ERROR at //build/toolchain/win/BUILD.gn:6:1: Value collision.
import("//build/config/win/visual_studio_version.gni")
^----------------------------------------------------
This import contains "visual_studio_runtime_dirs"
See //build/config/win/visual_studio_version.gni:43:32: defined here.
  visual_studio_runtime_dirs = []
                               ^
Which would clobber the one in your current scope

 
Components: Build
Oops, minor typo. The "extracted\" path component should be removed from the above example.
Currently, in order to correctly generate environment files (environment.x64, environment.x86) you must also:

- set DEPOT_TOOLS_WIN_TOOLCHAIN=0
- set GYP_MSVS_OVERRIDE_PATH to the VS root directory.

In build/toolchain/win/setup_toolchain.py it loads the INCLUDE, PATH and LIB variables by parsing %GYP_MSVS_OVERRIDE_PATH%\VC\vcvarsall.bat. This doesn't work if you've packaged up Visual Studio into a zip and then extracted it to some temp location on the build machine. It would be better if the setup_toolchain.py script supported explicit definition of these values (e.g. read them from INCLUDE/PATH/LIB environment variables).

There's also a toolchain.ninja file that appears to be new with the GN build. I'm not sure where the VS paths in this file are coming from.
Below is a patch that I believe addresses the GN variable behaviors from the original comment. It does not address the setup_toolchain.py issues from comment #3.

diff --git a/build/config/win/visual_studio_version.gni b/build/config/win/visual_studio_version.gni
index 05b369f..8be3a47 100644
--- a/build/config/win/visual_studio_version.gni
+++ b/build/config/win/visual_studio_version.gni
@@ -12,9 +12,8 @@ declare_args() {
   # Use "2013" for Visual Studio 2013, or "2013e" for the Express version.
   visual_studio_version = ""

-  # Directory of the Windows driver kit. If visual_studio_path is empty, this
-  # will be auto-filled.
-  wdk_path = ""
+  # Path to Visual Studio runtime libraries.
+  visual_studio_runtime_dirs = ""

   # Full path to the Windows SDK, not including a backslash at the end.
   # This value is the default location, override if you have a different
@@ -33,12 +32,11 @@ if (visual_studio_path == "") {
   visual_studio_path = toolchain_data.vs_path
   windows_sdk_path = toolchain_data.sdk_path
   visual_studio_version = toolchain_data.vs_version
-  wdk_path = toolchain_data.wdk_dir
   visual_studio_runtime_dirs = toolchain_data.runtime_dirs
 } else {
   assert(visual_studio_version != "",
          "You must set the visual_studio_version if you set the path")
-  assert(wdk_path != "",
-         "You must set the wdk_path if you set the visual studio path")
-  visual_studio_runtime_dirs = []
+  assert(visual_studio_runtime_dirs != "",
+         "You must set the visual_studio_runtime_dirs if you set the visual " +
+         "studio path")
 }
Below is a patch to setup_toolchain.py that reads variables from the environment when %GYP_MSVS_OVERRIDE_PATH%\VC\vcvarsall.bat doesn't exist. This correctly populates the environment files (environment.x64, environment.x86) and toolchain.ninja based on what you set via INCLUDE, PATH and LIB.

diff --git a/build/toolchain/win/setup_toolchain.py b/build/toolchain/win/setup_toolchain.py
index d58cb85..fd608ba 100644
--- a/build/toolchain/win/setup_toolchain.py
+++ b/build/toolchain/win/setup_toolchain.py
@@ -124,11 +124,14 @@ def _LoadToolchainEnv(cpu, sdk_dir):
     script_path = os.path.normpath(os.path.join(
                                        os.environ['GYP_MSVS_OVERRIDE_PATH'],
                                        'VC/vcvarsall.bat'))
-    if not os.path.exists(script_path):
-      raise Exception('%s is missing - make sure VC++ tools are installed.' %
-                      script_path)
-    args = [script_path, 'amd64_x86' if cpu == 'x86' else 'amd64']
-    variables = _LoadEnvFromBat(args)
+    if os.path.exists(script_path):
+      args = [script_path, 'amd64_x86' if cpu == 'x86' else 'amd64']
+      variables = _LoadEnvFromBat(args)
+    else:
+      variables = []
+      for k in sorted(os.environ.keys()):
+        variables.append('%s=%s' % (str(k), str(os.environ[k])))
+      variables = '\n'.join(variables)
   return _ExtractImportantEnvironment(variables)

Here's a CL with the above changes: https://codereview.chromium.org/2128993002

I won't take ownership of this issue since I'm not sure what other changes might be required to support build environments different from my own.
Closing the above CL for now. The use of setup_toolchain.py, vs_toolchain.py, environment.* files, DEPOT_TOOLS_WIN_TOOLCHAIN, GYP_MSVS_OVERRIDE_PATH, various GN variables, and VS detection in general feels overly complicated. Maybe it would be better to agree on a single source of truth (JSON file or GN variables), and then just reference that from everywhere.
When DEPOT_TOOLS_WIN_TOOLCHAIN=0 the SetEnvironmentAndGetRuntimeDllDirs() function in build/vs_toolchain.py tries to copy toolchain binaries from system directories to build directories [1]. This fails when the toolchain is not installed locally. It should instead look for these binaries under visual_studio_runtime_dirs and visual_studio_path, or just assume that they're already available under PATH.

[1] https://cs.chromium.org/chromium/src/build/vs_toolchain.py?dr&q=vs_toolchain.py+Sysnative&sq=package:chromium&l=88

Sign in to add a comment