Baseline: 8e8ddaba

Cherry picks:

   + bbc51a06:
     Fix `cc_test` coverage broken by
     https://github.com/bazelbuild/bazel/commit/92cba040904c2ccbd0b5a7
     0263de739f72c592ae
   + 162cacd3:
     Automated rollback of commit
     915fb3e8.
   + bf6ebe9f:
     Make tree artifacts that are symlinks to absolute paths work
     correctly.

This release contains contributions from many people at Google, as well as .

Previously, the code assumed that they are always symlinks whose realpath is under the execroot.

RELNOTES: None.
PiperOrigin-RevId: 601536298
Change-Id: Ia6a2c7f2c786e4d69b0d9e0be9c7f65af9931418

*** Reason for rollback ***

Might cause build to hang forever.

b/320630578

*** Original change description ***

Optimize prefetchInputs.

Use a pre-allocated array to hold the intermediate transfers to avoid allocations. Replace some of RxJava code with Futures to avoid RxJava overheads.

This improves the perfromance of prefetchInputs on a large set of inputs from ~400ms to ~16ms.

Fixes #20555.

Closes #20557.

PiperOrigin-RevId: 599135847
Change-Id: Idae6a1c57e634d16091e31e097b16ca97a67e62d

Fix `cc_test` coverage broken by https://github.com/bazelbuild/bazel/commit/92cba040904c2ccbd0b5a70263de739f72c592ae

PiperOrigin-RevId: 597562752
Change-Id: I799eb0a1eab433a9c6ceeaf53027570d1cd49be1

* Fixes a soundness error with ImmutableMapCodec which would cause a crash if
  the declared type is ImmutableSortedMap by splitting into 2 codecs.
* This necessarily includes ImmutableBiMapCodec and
  ImmutableClassToInstanceMapCodec because those two previously had
  dependencies on ImmutableMapCodec.

PiperOrigin-RevId: 596078698
Change-Id: I23b928307a3f1b1722d31b2907366fe46352ee6f

PiperOrigin-RevId: 596062007
Change-Id: I97d95c3944e381caaa01ac995eee607ebfdb9f2e

#19000

PiperOrigin-RevId: 596060323
Change-Id: Ifbef6ea29918e4af7856a675b18f1ebfd1a14f37

This alters an error message that used some obsolete verbiage (it said "generated label" when really it could be any non-input-file target).

- Eliminate GeneratedLabelConflict subclass. It's unnecessary to distinguish the case of input file conflicts from other types of name conflicts.

- Clarify comment about nameConflictCheckingPolicy.

- Remove addInputFile() overload that did its own creation of the InputFile. This makes custody of the input file object more explicit and allows it to be passed to helper functions like nameConflict().

- In createInputFile(), reorder clauses for readability and reuse of InputFile object across two cases.

Work toward #19922.

PiperOrigin-RevId: 596059017
Change-Id: Ie7acc7ba7592875c8726271b01c81f4a56c42a2d

PiperOrigin-RevId: 596051669
Change-Id: Ibfa7b49e4a7dc3e2282a554a00d37586ab6ebb67

Update `PrecomputedValue.Key` and `PrecomputedValue.UnshareableKey` to use `@AutoCodec.Interner` instead of `@AutoCodec.Instantiator`.

PiperOrigin-RevId: 596048876
Change-Id: I7f23da6170e787d20e45a2cded0c279ca74da9b4

PiperOrigin-RevId: 596045519
Change-Id: I4523b762770f601e91ffd4ae0d3f4c122ae3b537

This makes the affected error message more symmetric with other related errors that do not include the package name. The package name should be added somewhere higher up in the handling logic anyway.

Work toward #19922. (Drive-by fix)

PiperOrigin-RevId: 596041531
Change-Id: Ibfc3731f941de3a5020d751d46ac33f77e7d2019

PiperOrigin-RevId: 596041334
Change-Id: I6e29b07a3c21452642169c52494383c5fac0b09c

This change is a no-op.

- The idiom of checking the `targets` map for a newly added target and failing if it finds one, is factored into a new helper checkNameConflict().

- Updated javadoc to checkForConflicts(). Ensured that all existing checks are mentioned, and deleted reference to two conditions that don't appear to actually be checked: "no rule with errors is inserted into the package" -- we seem to do the opposite and propagate error bits from rules to the package; and "the generating rule of every output must itself be in the package" -- perhaps this is enforced elsewhere, but it's not part of this method.

- Add TODO of me pondering whether we care if an output file prefix clashes with a non-output-file target.

- Merge duplicateOutputFile() into nameConflict(). (This doesn't change the error message because nameConflict() was never called with an OutputFile as its first arg.)

- Add TODO identifying that we don't actually verify that an output doesn't clash with its own generating rule, even though we check it against other previously introduced rules.

- Rename conflictingOutputFile() -> overlappingOutputFilePrefixes() to distinguish from the other kinds of conflicts we're talking about.

- Inline checkForInputOutputConflicts() and inputOutputNameConflict(), which each only had one call site and are more readable as part of checkForConflicts().

- Use String.format() more.

Work toward #19922.

PiperOrigin-RevId: 596031120
Change-Id: I02ed6bb4d0f4c1f5aa2bfa16c91835d92e4cef39

FlatDeserializationContext only accepts deserializeFully. It's clear that
interned types actually only work with deserializeFully because partially
deserialized values would not be correctly interned.

* Make InterningObjectCodec use FlatDeserializationContext instead of
  AsyncDeserializationContext.
* Splits out FLAT_OBJECT_ARRAY_PROCESSOR from OBJECT_ARRAY_PROCESSOR that uses
  the FlatDeserializationContext.

PiperOrigin-RevId: 596016993
Change-Id: Ic799000d8caa446e2291858737669c16ab7bafa7

*** Reason for rollback ***

Broke tests

*** Original change description ***

Update CompatDexBuilder to compile using the DexFilePerClassFileConsumer.

The builder is already compiling a single classfile to a single dex file (in DexIndexed mode). The use of the DexFilePerClassFileConsumer mode will not change that, but it informs D8 that the compilation is an "intermediate" compilation and more closely matches the usage.

PiperOrigin-RevId: 596000561
Change-Id: If44a3c63264cdbb482e0de5809ee2e6479b5c614

*** Reason for rollback ***

Rollforward after fixing the underlying problem in https://github.com/bazelbuild/bazel/commit/9b027c88051797ada7350098ef190191b953014a

*** Original change description ***

Disable PyTest.testSmoke on macOS

Due to https://github.com/bazelbuild/bazel/issues/20660

RELNOTES: None
PiperOrigin-RevId: 595998556
Change-Id: Idddf6149a918de4408a3fda016d605380a4ab6af

This change is primarily done to support CppLinkAction which already prints names like this.

Alternatively we could add new placeholders for short names, however I believe it's better we only keep one form consistently thoughout the rules.

RELNOTES[INC]: progress messages print short primary input and output without root base

PiperOrigin-RevId: 595968580
Change-Id: I0e612763102e207511f38bdb014862d6084cd500

The builder is already compiling a single classfile to a single dex file (in DexIndexed mode). The use of the DexFilePerClassFileConsumer mode will not change that, but it informs D8 that the compilation is an "intermediate" compilation and more closely matches the usage.

PiperOrigin-RevId: 595956205
Change-Id: I063e8e2e71fe1057c7d651f88e77ecd95a89c1d6

This method is in fact not available from Starlark.

RELNOTES: None.
PiperOrigin-RevId: 595949407
Change-Id: I1cc9d8583c2edca42e2a2eadf27bbd74b8f9604f

Fixes https://github.com/bazelbuild/bazel/issues/20743

PiperOrigin-RevId: 595935153
Change-Id: I0409552aa92f3886c5abf3bd3ce50d67594dab7e

Before `getRawLinkArgv` generated a joint/mixed command line, which was composed of 'linker executable' (first argument) and regular arguments. Reference to 'param file' was mixed into the regular arguments.

Linker command line is long, too long. To fix this, most of linkers are called like `linker @param.file`. The long list of arguments is hidden in `param.file`.

`splitCommandline` used `isLikelyParamFile` to filter out what that `param file` reference was. The result were 2 command lines. First one with 'linker executable' and reference to 'param file' and second command line contained regular arguments, that are written to param file.

Generate those 2 command lines directly. First one is returned by `getCommandLine` and second one (the regular arguments) by `getParamCommandLine`

Param file reference is formatted using `linker_param_file` feature, using `%{linker_param_file}` substitution. In most cases this is just `@%{linker_param_file}`, but there are some linkers that need different formatting.

This change is also needed for Starlarkification of CppLinkAction, because the Starlark interface requires to set how param file is formatted, directly. See https://bazel.build/rules/lib/builtins/Args#use_param_file

Fixes: https://github.com/bazelbuild/bazel/issues/18074
PiperOrigin-RevId: 595912823
Change-Id: I54b36113d87f975af63341b2dec17b2f861c0ffa

PiperOrigin-RevId: 595844312
Change-Id: I3f7af05ad57bb3b233a693665bac079388638184

* Renames ImmutableSetRuntimeCodec to ImmutableSetCodec for consistency.
* Changes Collections.emptySet to serialize using a reference constant.
* Adds a unit test.

PiperOrigin-RevId: 595820617
Change-Id: Ic2592216a1463e8e17168f4d308e4c49f83644c9

... that use repo mapping. This is a rather obscure case of the lockfile being stale; if the `Label()` constructor is called in an extension impl function, and that call uses repo mapping of any form (i.e. the argument looks like `@foo//bar`), then we need to be ready to rerun the extension if `@foo` were to suddenly map to something else.

I also did a minor refactoring in `SingleExtensionEvalFunction` around the logic to decide whether the locked extension is up-to-date. Right now we perform a "diff" between the locked extension and what we expect to be up-to-date, and if a "diff" is found *and* `--lockfile_mode=error`, we basically perform a diff again. We also don't short circuit; that is, if the transitive bzl digest has changed, there's no point in seeing whether any files have changed, but we do right now.

A follow-up will be sent to fix the analogous bug for repo rules.

Fixes #20721.

Closes #20742.

PiperOrigin-RevId: 595818144
Change-Id: Id660b7a659a7f2e4dde19c16784c2ab18a9ceb69

This is achieved by rewriting the user-specified mounts to mounts onto subdirectories of the hermetic sandbox tmp directory.

Fixes #20527

Closes #20583.

PiperOrigin-RevId: 595815029
Change-Id: Ibfe5f67fb8fb59131b6c82a826ed5200f2b10a94

The implementation of both methods is the same. Because the second one is already public, this reduces the number of public methods of LinkCommandLine.

The change is a no-op.

PiperOrigin-RevId: 595771439
Change-Id: Ia5a494746c1f15be7a16149ee28ae349c57c955d

platform.html does not exist (anymore)

it now points to: https://bazel.build/reference/be/platforms-and-toolchains#constraint_value

Closes #20694.

PiperOrigin-RevId: 595743748
Change-Id: I5ea30e4ce2b89af86d42456a724986e648c9140c

PiperOrigin-RevId: 595703930
Change-Id: Ic4e836b8384ba788ae6f7ecd968e4e90ca889771

Previously, it was possible in theory, but not in practice, to call both addRule() and addRuleUnchecked() on the same Package.Builder. This CL removes that possibility, and in doing so makes it easier to reason about whether the ruleLabels map will be null. It also makes the outputFilePrefixes map nullable along the same lines as ruleLabels. Both maps are only used for validation, and so are irrelevant to Builders that use addRuleUnchecked.

For completeness/correctness, replaceTarget is also updated to mention its interaction with this invariant.

Work toward #19922.

PiperOrigin-RevId: 595696336
Change-Id: I3965033784025e3dd6cfc85ec0b591a21c24de8c

outputFilePrefixes is a map used to help check whether one OutputFile's name clashes with a directory prefix of another. Since it's updated with info from every OutputFile of a rule passed to checkForConflicts(), there's no need to also update it in addRuleUnchecked().

This is a no-op assuming that no user of Package.Builder calls both addRule() and addRuleUnchecked() -- which is currently the case, and ought to be enforced, perhaps in a follow-up CL.

(Note that the alternative approach, of deleting the map update in checkForConflicts() while leaving it intact in addRuleUnchecked(), is not only less readable, but also incorrect since it doesn't catch conflicts between OutputFiles of the same generating rule.)

Work toward #19922.

PiperOrigin-RevId: 595683434
Change-Id: I8b660807edfa1eef5101fe362ddaed4352e3f684

The implementation was exactly the same. (Probably historically that wasn't the case).

PiperOrigin-RevId: 595674237
Change-Id: I608b6d5f06db0d81a8abf631505e2552ce45a7b3

Fix missing triple-backtick introduced in https://github.com/bazelbuild/bazel/commit/13ecdf583301a94484a1ae0eb27c56fcf3248dc5

PiperOrigin-RevId: 595654761
Change-Id: Ib02ad8c5f5a3e29bd022ffd006a71cb182ab7466

To mitigate https://github.com/bazelbuild/bazel/issues/20743

PiperOrigin-RevId: 595652084
Change-Id: I349024873d86f9b57dffca3c53429bf8ec3ed453

It's used only in tests, where it can be more accurtely determined if there is an .so artifact present.

PiperOrigin-RevId: 595644582
Change-Id: Iffc2b89fdbef322cc63d92f5acf4aa1261f1fca3

The previous implementation contained unsound casts. LinkedHashMultimap is
a declarable type that cannot be assigned from ImmutableSetMultimap which
would have led to a runtime crash. So the codecs are broken out and registered
individually.

Minor ArrayProcessor changes.
* Use a static import for array offset constants.
* Add a deserializeObjectArrayFully method.

PiperOrigin-RevId: 595569849
Change-Id: I8add48dacd13d4e08159c48e4145632055af2e15

Observed NPE in a debugger that was trying to show a string representation.

Also make the string representation of the digest consistent with `RemoteFileArtifactValue`.

PiperOrigin-RevId: 595554648
Change-Id: I705ed0ffabf4bde81bb3b1f653f30b7565e65b82

This CL adds aspect nodes as explicit targets in cquery deps by making
ConfiguredTargetQueryEnvironment generic over a new interface,
CqueryNode, rather than ConfiguredTarget.

CqueryNode is implemented by AspectKey as well as ConfiguredTarget so
that both can be traversed in a deps query.

PiperOrigin-RevId: 595509373
Change-Id: I8a637cd3ed640907d2b1501bcd2b4a4507d183e7

This is achieved by mounting tmpfs after regular mounts in the sandbox binary as well as creating the directories at which tmpfs are mounted under the sandbox tmp directory.

Closes #20658.

PiperOrigin-RevId: 595500822
Change-Id: Icf3d51bffdd004f668ba4fbbdbd5f833c20db3d9

The test was failing because the `b.py` file couldn't be imported because, starting with Python 3.11, the `PYTHONSAFEPATH` environment variable (set by the bootstrap startup) is respected. This setting inhibits the default Python behavior of adding the main script's directory to sys.path.

To fix, use `py_library.imports` to explicitly add the necessary directory to `sys.path`.

Fixes https://github.com/bazelbuild/bazel/issues/20660

Closes #20738.

PiperOrigin-RevId: 595498775
Change-Id: I0c9521b210fe9e2c40692727fc87d41995e0968a