This project is mirrored from https://github.com/bazelbuild/bazel.git.
Pull mirroring updated .
- Feb 06, 2024
-
-
Xùdōng Yáng authored
-
Tiago Quelhas authored
For now, it knows how to convert between the binary and JSON formats. Ability to convert from the new compact format will be added in a followup. Tested by running manually. I will add end-to-end test coverage once the converter has been fully implemented. PiperOrigin-RevId: 603684017 Change-Id: Idf0d851feb9ea22e7021a1b62f003f0978bcd378
-
- Feb 03, 2024
-
-
Tiago Quelhas authored
For now, it knows how to convert between the binary and JSON formats. Ability to convert from the new compact format will be added in a followup. Tested by running manually. I will add end-to-end test coverage once the converter has been fully implemented. PiperOrigin-RevId: 603684017 Change-Id: Idf0d851feb9ea22e7021a1b62f003f0978bcd378
-
Tiago Quelhas authored
PiperOrigin-RevId: 585632524 Change-Id: I559bc9f6abc859ab7578a8918efb863c5bfe9e9b
-
Tiago Quelhas authored
PiperOrigin-RevId: 603371194 Change-Id: Id16f33c85a412c262acb0468d431b64bbddb7932
-
Tiago Quelhas authored
PiperOrigin-RevId: 602985772 Change-Id: I8e05a462069ea43940bd6121987673a1f5b918a3
-
- Feb 02, 2024
-
-
Tiago Quelhas authored
This should have been a part of https://github.com/bazelbuild/bazel/commit/9fd9aa57c456f5324383528c4094995f19124889. PiperOrigin-RevId: 602999711 Change-Id: Ic817e5682d8fbbb477e03a0a7e217c61735cc90b
-
Tiago Quelhas authored
In addition, check that an output is of the expected type before emitting it, and otherwise deem it nonexistent. Strictly speaking, this is a breaking change for the old (expanded) log formats, but any such spawn will later cause the respective action to fail, so it doesn't materially affect the ability to analyze cache misses. PiperOrigin-RevId: 602724835 Change-Id: Id3ccb2db8362e19d7e26bf8e6bc5cb7fb9478d52
-
Tiago Quelhas authored
The digest function is the same for every digest computed by an invocation. To save space, it is stored once in the Invocation log entry. Also embed a Digest proto in File and Spawn messages, with the proviso that the hash_function_name field will be empty. The size is always required because the hash alone might not be sufficient to retrieve an entry from a remote cache. PiperOrigin-RevId: 602684865 Change-Id: I6f4882051aad9ea72bea0232f8ad4b2cd7b8e335
-
- Feb 01, 2024
-
-
bazel.build machine account authored
Similar to the fix for https://github.com/bazelbuild/bazel/issues/20721, we write recorded repo mapping entries into the marker file so that repo fetching is sensitive to any relevant repo mapping changes. Fixes #20722. Closes #21131. Commit https://github.com/bazelbuild/bazel/commit/9edaddd6273de776a3f60245f5c56f82f9622caf PiperOrigin-RevId: 603310262 Change-Id: I806f383e8579fed3533fac9b181efd8b75e76fcd Co-authored-by:
Xdng Yng <wyverald@gmail.com>
-
Tiago Quelhas authored
It's just a general interface for encoding a stream of protobuf messages. Some implementations, notably JsonOutputStreamWrapper, don't encode in length-delimited wire format. PiperOrigin-RevId: 602907082 Change-Id: Ib35eeb15d92b4eca9b827f44927b6a9370d3107c
-
Tiago Quelhas authored
[7.1.0] Document that the compact execution log isn't guaranteed to be serialized in increasing ID order. (#21165) PiperOrigin-RevId: 602664512 Change-Id: Ic23e7f9c78fc15e6bb6f5b6a86b0bce8c608e966
-
Tiago Quelhas authored
This is to allow StableSort (and other utilities to be added soon) to be shared between Bazel and a future log conversion tool, without bringing in unnecessary dependencies for the latter. PiperOrigin-RevId: 602385146 Change-Id: I29419c2715338a1845f79568d86119033f7ff5ff
-
Liam Miller-Cushon authored
Cherry-pick of f697132a https://github.com/bazelbuild/bazel/issues/21119
-
Tiago Quelhas authored
Cuts down the stableSort() run time by 60%, and the overall run time by 20%, when sorting a binary log obtained by building Blaze itself. PiperOrigin-RevId: 602356778 Change-Id: Id0cd577e83216553c590b0bc8e3cd4b70d8ee957
-
- Jan 31, 2024
-
-
Tiago Quelhas authored
[7.1.0] Make some minor adjustments to the compact execution log format and document it better. (#21146) * Remove the Output top-level payload type. It can only appear as a sub-message of a Spawn. * Add an Invocation top-level payload type. It is currently empty but reserved for anticipated needs (e.g. to report the digest function used). * Spell out some guarantees (and non-guarantees) about the format. Renumbering is fine, since the format hasn't made it into a release yet. PiperOrigin-RevId: 602352295 Change-Id: I6044c9a9a51360dfce5701071883b318b1e9a5c2
-
bazel.build machine account authored
Completion for both repository names and targets in (fetched) external repositories is provided based on `bazel mod dump_repo_mapping`. RELNOTES: Bazel's Bash completion can now complete external repository labels when using `--enable_bzlmod`. Closes #20693. Commit https://github.com/bazelbuild/bazel/commit/a4c3d899212bb216abe3789a13657e7413a3eefc PiperOrigin-RevId: 602923007 Change-Id: I7c7dfe3d90a2921f473c2d2fedbc054da67932ac Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
Tiago Quelhas authored
PiperOrigin-RevId: 601430837 Change-Id: Ie7739e2e4aead81a8a3ab7d9eedf47294ed9f855
-
bazel.build machine account authored
This is a resubmission of @stepancheg's #12782, which became stale, with this original description: ==== For certain calls like `type(x)` or `str(x)` or `l.append(x)` we can reuse `positional` array argument of `fastcall` to pass it to the `MethodDescriptor.call` and skip all runtime checks in `BuiltinFunction`. This optimization cannot be applied if * a function has extra positional or named arguments * has arguments guarded by flag * accepts extra `StarlarkThread` parameter * has unfilled arguments with default values So this optimation won't be used for calls `list(x)` or `bool()`. For `type(1)` called in loop, the result is 15% speedup: ``` A: n=30 mean=5.705 std=0.387 se=0.071 min=5.131 med=5.827 B: n=30 mean=4.860 std=0.345 se=0.064 min=4.396 med=4.946 B/A: 0.852 0.820..0.884 (95% conf) ``` For `bool()` (no argument) called in loop, it results in 1% slowdown: ``` A: n=29 mean=9.045 std=0.603 se=0.113 min=8.096 med=9.400 B: n=29 mean=9.134 std=0.520 se=0.098 min=8.248 med=9.448 B/A: 1.010 0.976..1.045 (95% conf) ``` ==== Beyond the original PR, this includes benchmarks, a test case addressing a review comment and small adaptions to the implementation to make it work with recent changes. Benchmark results with JDK 21 and `--seconds 5`: ``` before: File src/test/java/net/starlark/java/eval/testdata/bench_call.star: benchmark ops cpu/op wall/op steps/op alloc/op bench_call_bool 67108863 100ns 100ns 3 87B bench_call_dict_get 33554431 159ns 158ns 5 143B bench_call_dict_get_none 33554431 180ns 179ns 6 143B bench_call_list_append 33554431 170ns 169ns 5 207B bench_call_type 67108863 139ns 138ns 4 111B after: File src/test/java/net/starlark/java/eval/testdata/bench_call.star: benchmark ops cpu/op wall/op steps/op alloc/op bench_call_bool 67108863 100ns 100ns 3 87B bench_call_dict_get 33554431 155ns 154ns 5 143B bench_call_dict_get_none 33554431 174ns 174ns 6 143B bench_call_list_append 67108863 141ns 141ns 5 183B bench_call_type 67108863 115ns 114ns 4 87B ``` Closes #19708. Commit https://github.com/bazelbuild/bazel/commit/aded06b8bd74c39d916bfb10645cf83add42f749 PiperOrigin-RevId: 602821608 Change-Id: If40e2eb1e09ec43d56b36ba13987aa5312fd47ec Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
Tiago Quelhas authored
[7.1.0] Avoid using `InputStream.available()` to detect EOF while reading delimited protos. (#21143) It's not a reliable method to check for EOF: it returns how many bytes are guaranteed to be read without blocking, and the base implementation always returns 0. Instead, leverage the fact that parseDelimitedFrom() returns null if the stream is at EOF. PiperOrigin-RevId: 602257598 Change-Id: I61e51774611196fb44745dc0aa2dc836b41fcd68
-
- Jan 30, 2024
-
-
keertk authored
Commit https://github.com/bazelbuild/bazel/commit/275000c430de7436a1dd09f0874f6151a7ca9162 PiperOrigin-RevId: 602440856 Change-Id: I5c1a3c4ba3fa70d1ea928ad3d23f472da7af65d7
-
- Jan 29, 2024
-
-
bazel.build machine account authored
As discussed in #18997, compression causes performance degradation for small blobs. The flag defaults to zero (always compress) for backwards compatibility, but the discussion suggests that 100 might be an appropriate value to set it to. Fixes #18997. Commit https://github.com/bazelbuild/bazel/commit/69eb0ec181012d31123f4ee862effe5d51c1725e PiperOrigin-RevId: 602353295 Change-Id: Ie9ef8c10e221e9190d3c5cb5f267fcf35a63fd3b Co-authored-by:
Googler <tjgq@google.com>
-
Ian (Hee) Cha authored
Bazel forces use of `lld` and `ld.gold` if found, but performed feature detection on the default linker instead. Fixes #20834 Closes #20833. Commit https://github.com/bazelbuild/bazel/commit/2bfe045ff2d6550e443625128b0dfeb2941ebfbc PiperOrigin-RevId: 598565598 Change-Id: I4890f278c5cc33d4e6a6ebb10d796fb1c22f9ba6 --------- Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
bazel.build machine account authored
Unfortuantely, there isn't an easy way to add integration test for this specific edge case on Windows. Fixes #20741. Closes #20752. Commit https://github.com/bazelbuild/bazel/commit/958e0c485d5afc9f69f1cdaa85c2ea7c0afc4007 PiperOrigin-RevId: 596547046 Change-Id: I4f517b161c03793329d5a4e21ec8ac4a5b53abb0 Co-authored-by:
Chi Wawng <coeuvre@gmail.com>
-
bazel.build machine account authored
Apple has since changed visionOS to only support arm macs Closes #20335. Commit https://github.com/bazelbuild/bazel/commit/ce9fa8eff5d4705c9f6bf6f6642fa9ed45eb0247 PiperOrigin-RevId: 601022333 Change-Id: I6322c68102129eb230bbbb4546249cb22899fe94 Co-authored-by:
Keith Smiley <keithbsmiley@gmail.com>
-
- Jan 28, 2024
-
-
bazel.build machine account authored
When building without the bytes, outputs are only materialized when either (1) an action prefetches them, or (2) they've been explicitly requested. When both --remote_download_minimal and --noexperimental_check_output_files are set, this presents a problem for a build command followed by a run command for the same target: the build command won't download the outputs and the run command won't check for their presence, proceeding without them. A non-incremental run command works, because the outputs are considered top-level even in minimal mode. This is only a problem for a run command because it exists outside of action execution, and doesn't get a chance to prefetch inputs; it would have been better to implement the run command by injecting a specially crafted action into the build graph, but that will have to wait for another day. The present fix might theoretically slow things down if output checking is truly unnecessary, but I doubt that would cause a significant regression in practice. Fixes #20843. Closes #20853. Commit https://github.com/bazelbuild/bazel/commit/2f899ef8b39935d9ca131b71a24e67ef286bc378 PiperOrigin-RevId: 597909909 Change-Id: I66aedef4994fbda41fe5378c80940fa8ba637bfd Co-authored-by:
Tiago Quelhas <tjgq@google.com>
-
- Jan 27, 2024
-
-
Tiago Quelhas authored
[7.1.0] Fix up permissions error in getInputStream, like we already do for getOutputStream. (#21087) We're likely reporting permission errors as FileNotFoundException in some cases, which makes debugging more difficult and might have correctness implications. Better yet would be to migrate to the standard java.nio.file.FileSystemException types, but that's a big and scary change, and it will have to wait for another day. PiperOrigin-RevId: 601092545 Change-Id: I1b8777850c946c97e7faf2a47a60255f4e591a3a
-
Tiago Quelhas authored
Unlike on Unix, we can't fall back to an alternative implementation. Failure to emit an early error would cause later native method calls to produce a less informative error message. See https://github.com/bazelbuild/bazel/issues/20677 for a recent example where this would have been helpful. Also note some minor changes: * Since we attempt to load the JNI in two different ways, make sure to propagate both exceptions (one suppressed by the other). * The default switch case is unnecessary, as errorprone already enforces exhaustiveness. PiperOrigin-RevId: 595206761 Change-Id: I4b6258d59d9a4fcc9f83418375acc51a32d7f4a4
-
Tiago Quelhas authored
Although the semantics weren't explicitly specified by the FileSystem interface, the Unix implementation calls readdir(3), which always dereferences a symlink for the directory itself. Also deduplicate the common logic. PiperOrigin-RevId: 598543913 Change-Id: I9bcab70c57ef4e8c4da58f4871397c6f2362f43c
-
bazel.build machine account authored
This requires fixing an actual bug that causes the Windows Java launcher to not find the Java runtime with `--nolegacy_external_runfiles`. Work towards https://github.com/bazelbuild/bazel/issues/12821 Closes #20680. Commit https://github.com/bazelbuild/bazel/commit/3d4491c9ab0c063356d3c017a9308c50fa604294 PiperOrigin-RevId: 601826685 Change-Id: Ib45e60901d47efc06bf875c334edafcaeabc5f1e Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
Tiago Quelhas authored
[7.1.0] Clarify where to find the definition of the --experimental_remote_scrubbing_config configuration format. (#21089) PiperOrigin-RevId: 597764071 Change-Id: I9ea7000509e29c327034db5fd0087b35c0fcdfb9
-
Tiago Quelhas authored
Previously, they were both displayed as `remote-cache`; there's now a separate `disk-cache` form. If a combined cache is used, one or both forms will appear, depending on which caches were looked up. As a result, the progress status reporting is moved to the individual cache implementations. While this is kind of unfortunate from an architectural standpoint, it's likely the best we can do until we recast cache lookups as spawn strategies (see #19904). Closes #20935. PiperOrigin-RevId: 601748051 Change-Id: I03710219973c95d4fca999d931b3513f6d240d94
-
bazel.build machine account authored
These cause failures when relying on `depGraph` containing `ModuleKey`s for all modules, which is the case in production. Work towards #20997 Closes #21037. Commit https://github.com/bazelbuild/bazel/commit/27419e33dc8c48fb5ca6d97f5d6788f9aca2032f PiperOrigin-RevId: 601822357 Change-Id: Ifcad9d7b73835491c1f1fca975e05834057a6825 Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
Xùdōng Yáng authored
RELNOTES: The flag `--experimental_worker_for_repo_fetching` now defaults to `auto`, which uses virtual threads from JDK 21 if it's available. This eliminates restarts during repo fetching.
-
bazel.build machine account authored
`bazel mod dump_repo_mapping` with no arguments is explicitly made an error so that a new mode that dumps all repository mappings with a single Bazel invocation can be added later if needed, e.g. to support IntelliJ's "sync" workflow. RELNOTES: `bazel mod dump_repo_mapping <canonical repo name>...` returns the repository mappings of the given repositories in NDJSON. This information can be used by IDEs and Starlark language servers to resolve labels with `--enable_bzlmod`. Work towards #20631 Closes #20686. Commit https://github.com/bazelbuild/bazel/commit/59ac9ce297eb1bc0c47d4dc5de788908208735e0 PiperOrigin-RevId: 601332180 Change-Id: I828d7c88637bea175e11eccc52c6202f6da4c32c Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
Xùdōng Yáng authored
bzlmod: support git repos in source.json Closes #20912. RELNOTES: Added `init_submodules` attribute to `git_override`. Registries now support the `git_repository` type in `source.json`. Change-Id: Ia267131e6d081572a642888ba34e285805bbbe9f PiperOrigin-RevId: 601268823 --------- Co-authored-by:
Nachum Goldstein <nachum@google.com> Co-authored-by:
Keith Smiley <keithbsmiley@gmail.com>
-
- Jan 26, 2024
-
-
Xùdōng Yáng authored
-
bazel.build machine account authored
Fixes https://github.com/bazelbuild/bazel/issues/20919 Closes #20924. Commit https://github.com/bazelbuild/bazel/commit/3d7c5ae47e2a02ccd81eb8024f22d56ae7811c9b PiperOrigin-RevId: 601022242 Change-Id: I257023dcf9983cb2b3ce16079675f6b5f9409b86 Co-authored-by:
Keith Smiley <keithbsmiley@gmail.com>
-
- Jan 25, 2024
-
-
bazel.build machine account authored
Work towards #20942 On the referenced bug, this prints: ``` ERROR: Circular definition of repositories generated by module extensions and/or .bzl files: .-> repository mapping of @@ | WORKSPACE file | //python:pip.bzl | //python/pip_install:requirements.bzl | //python:defs.bzl | //python:py_import.bzl | //python:py_info.bzl | @@_main~internal_deps~rules_python_internal//:rules_python_config.bzl | @@_main~internal_deps~rules_python_internal | extension 'internal_deps' defined in //python/private/bzlmod:internal_deps.bzl `-- repository mapping of @@ ERROR: Error computing the main repository mapping: cycles detected during computation of main repo mapping ``` Closes #20958. Commit https://github.com/bazelbuild/bazel/commit/fab64142b3a2839059d19acadb6c10ebfebf4402 PiperOrigin-RevId: 601183514 Change-Id: If321005b1f6918117d0f5b5763ae2bda4d8e34f1 Co-authored-by:
Fabian Meumertzheim <fabian@meumertzhe.im>
-
Salma Samy authored
If a local registry is specified using a workspace placeholder, the lockfile stores location details of the files with resolved workspace information, including user-specific paths. To fix that: Updating the logic to use the workspace placeholder %workspace% during lockfile writing and reverting to the resolved paths when reading. PiperOrigin-RevId: 600876401 Change-Id: I6162d8e8f1fe5e29b7899fc5bb218d1b6be926d0
-