New issue
Advanced search Search tips

Issue 871574 link

Starred by 2 users

Issue metadata

Status: Assigned
Owner:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux
Pri: 2
Type: Bug

Blocked on:
issue 899437
issue 911183

Blocking:
issue 869348
issue 899438
issue 330263



Sign in to add a comment

Generated build files should be deterministic with respect to build path

Project Member Reported by thomasanderson@chromium.org, Aug 6

Issue description

I care about deterministic build files because I often mount my build directory at different locations (eg. in a shared directory inside a VM).

Right now, the generated build files on Linux with default configuration has just a few lines that depend on the name of the path to the source directory, or the name of the build directory relative to the source directory.  The diff is shown below.  We should fix this and perhaps add a step on a CQ bot that simply does "gn gen out/Default1; gn gen out/Default2; diff -ru out/Default1 out/Default2" and "grep `pwd` out/Defualt1 -r".

thomasanderson@thomasanderson:~$ diff -ru dev/chromium_official/src/out/Default1/ dev/chromium_1/src/out/Default2
diff -ru dev/chromium_official/src/out/Default1/toolchain.ninja dev/chromium_1/src/out/Default2/toolchain.ninja
--- dev/chromium_official/src/out/Default1/toolchain.ninja	2018-08-06 16:07:08.617735701 -0700
+++ dev/chromium_1/src/out/Default2/toolchain.ninja	2018-08-06 16:22:19.960980339 -0700
@@ -15079,7 +15079,7 @@
 subninja obj/components/url_formatter/unit_tests.ninja
 subninja obj/components/url_formatter/url_formatter.ninja
 rule __components_url_formatter_top_domains_generate_top_domains_test_trie___build_toolchain_linux_clang_x64__rule
-  command = python ../../build/gn_run_binary.py top_domain_generator /usr/local/google/home/thomasanderson/dev/chromium_official/src/components/url_formatter/top_domains/test_domains.skeletons /usr/local/google/home/thomasanderson/dev/chromium_official/src/components/url_formatter/top_domains/top_domains_trie.template gen/components/url_formatter/top_domains/test_domains-trie-inc.cc
+  command = python ../../build/gn_run_binary.py top_domain_generator /usr/local/google/home/thomasanderson/dev/chromium_1/src/components/url_formatter/top_domains/test_domains.skeletons /usr/local/google/home/thomasanderson/dev/chromium_1/src/components/url_formatter/top_domains/top_domains_trie.template gen/components/url_formatter/top_domains/test_domains-trie-inc.cc
   description = ACTION //components/url_formatter/top_domains:generate_top_domains_test_trie(//build/toolchain/linux:clang_x64)
   restat = 1
   pool = build_toolchain_action_pool
@@ -15088,7 +15088,7 @@
 
 build obj/components/url_formatter/top_domains/generate_top_domains_test_trie.stamp: stamp gen/components/url_formatter/top_domains/test_domains-trie-inc.cc
 rule __components_url_formatter_top_domains_generate_top_domains_trie___build_toolchain_linux_clang_x64__rule
-  command = python ../../build/gn_run_binary.py top_domain_generator /usr/local/google/home/thomasanderson/dev/chromium_official/src/components/url_formatter/top_domains/alexa_domains.skeletons /usr/local/google/home/thomasanderson/dev/chromium_official/src/components/url_formatter/top_domains/top_domains_trie.template gen/components/url_formatter/top_domains/alexa_domains-trie-inc.cc
+  command = python ../../build/gn_run_binary.py top_domain_generator /usr/local/google/home/thomasanderson/dev/chromium_1/src/components/url_formatter/top_domains/alexa_domains.skeletons /usr/local/google/home/thomasanderson/dev/chromium_1/src/components/url_formatter/top_domains/top_domains_trie.template gen/components/url_formatter/top_domains/alexa_domains-trie-inc.cc
   description = ACTION //components/url_formatter/top_domains:generate_top_domains_trie(//build/toolchain/linux:clang_x64)
   restat = 1
   pool = build_toolchain_action_pool
@@ -32207,7 +32207,7 @@
 subninja obj/testing/libfuzzer/fuzzers/javascript_parser_proto.ninja
 subninja obj/testing/libfuzzer/fuzzers/javascript_parser_proto_fuzzer.ninja
 rule __testing_libfuzzer_fuzzers_javascript_parser_proto_fuzzer.owners___build_toolchain_linux_clang_x64__rule
-  command = python ../../testing/libfuzzer/gen_fuzzer_owners.py --owners javascript_parser_proto_fuzzer.owners --sources out/Default1/gen/testing/libfuzzer/fuzzers/javascript_parser_proto_to_string.cc testing/libfuzzer/fuzzers/javascript_parser_proto_fuzzer.cc testing/libfuzzer/fuzzers/javascript_parser_proto_to_string.h
+  command = python ../../testing/libfuzzer/gen_fuzzer_owners.py --owners javascript_parser_proto_fuzzer.owners --sources out/Default2/gen/testing/libfuzzer/fuzzers/javascript_parser_proto_to_string.cc testing/libfuzzer/fuzzers/javascript_parser_proto_fuzzer.cc testing/libfuzzer/fuzzers/javascript_parser_proto_to_string.h
   description = ACTION //testing/libfuzzer/fuzzers:javascript_parser_proto_fuzzer.owners(//build/toolchain/linux:clang_x64)
   restat = 1
   pool = build_toolchain_action_pool

 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 14

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/24bc6198d5553fc0659da2a37580df516527400c

commit 24bc6198d5553fc0659da2a37580df516527400c
Author: Tom Anderson <thomasanderson@chromium.org>
Date: Tue Aug 14 00:10:28 2018

Relativize paths in build files

BUG=871574

Change-Id: I8eca9b8fabf1e90b0e9c054dd5283be21508bae4
Reviewed-on: https://chromium-review.googlesource.com/1164536
Reviewed-by: Johann Koenig <johannkoenig@google.com>
Reviewed-by: Scott Violet <sky@chromium.org>
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Cr-Commit-Position: refs/heads/master@{#582764}
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/android_webview/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/android_webview/glue/generate_resource_rewriter.gni
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/build/config/android/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/build/config/android/rules.gni
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/chrome/browser/chromeos/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/chrome/chrome_cleaner/test/resources/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/chromecast/build/tests/cast_test.gni
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/chromeos/ime/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/components/url_formatter/top_domains/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/components/url_formatter/top_domains/top_domain_generator.cc
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/remoting/credits/credits.gni
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/remoting/host/installer/win/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/testing/libfuzzer/fuzzer_test.gni
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/testing/libfuzzer/fuzzers/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/third_party/libvpx/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/third_party/mockito/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/third_party/robolectric/BUILD.gn
[modify] https://crrev.com/24bc6198d5553fc0659da2a37580df516527400c/tools/grit/grit_rule.gni

Blocking: 330263
We have https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Deterministic%20Linux on the main waterfall. It runs tools/determinism/compare_build_artifacts.py to compare all files referenced from isolated files (i.e. everything that's built). It could probably also compare generated ninja files. The only thing to watch out for is that we don't delete stale ninja files, and we probably wouldn't want to compare those.
Blocking: 869348
Blockedon: 899437
Blocking: 899438
Blockedon: 911183

Sign in to add a comment