Baseline: 19ae1975

Cherry picks:

   + 18bfed26:
     Properly preserve state when interrupted during a deletion. Also
     use best Bazel practices for reinitializing maps by reassigning,
     since just clearing can retain memory. When enqueuing deleted
     nodes, don't use more threads than there are nodes. Finally, fix
     a test accidentally changed in
     https://github.com/bazelbuild/bazel/commit/3ea75594bfdb5e67942501
     55995c45a5adc00c28.

Incompatible changes:

  - Flipped --incompatible_disallow_resource_jars (see
    https://github.com/bazelbuild/bazel/issues/13221).

Important changes:

  - Disable --all_incompatible_changes flag.

This release contains contributions from many people at Google, as well as Benjamin Peterson, Ben Lee, Grzegorz Lukasik, Ikko Ashimine.

Properly preserve state when interrupted during a deletion. Also use best Bazel practices for reinitializing maps by reassigning, since just clearing can retain memory. When enqueuing deleted nodes, don't use more threads than there are nodes. Finally, fix a test accidentally changed in https://github.com/bazelbuild/bazel/commit/3ea75594bfdb5e6794250155995c45a5adc00c28.

PiperOrigin-RevId: 392680387

We already collect information about the number of `FileStateValue` nodes included in the diff. Expand the event with number of `FileValue` nodes invalidated during evaluation.

PiperOrigin-RevId: 392537693

PiperOrigin-RevId: 392535347

These can be unboundedly huge, which besides producing more information than
most could reasonably consume, it can crash the client<->server grpc interaction
due to trying to push too much data at once.

PiperOrigin-RevId: 392483926

Step 1 of https://github.com/bazelbuild/bazel/issues/13892

RELNOTES:
Disable --all_incompatible_changes flag.
PiperOrigin-RevId: 392476629

Closes #13869.

PiperOrigin-RevId: 392474715

continous -> continuous

Closes #13881.

PiperOrigin-RevId: 392467201

I just figured that other people might run into the same problem as we did on Bazel CI and hopefully this helps them fix the issue quickly.

RELNOTES: None.
PiperOrigin-RevId: 392466872

Stop including synthetic exception with longest stack trace in bug report for slow interrupts: the exceptions are not being aggregated properly by the server.

PiperOrigin-RevId: 392448135

The resource_jars is not supported by java_common.compile function. In
order to rewrite java_library to Starlark the call needs to be extended or the
feature disabled. Because the migration is relatively simple I'm opting
to flip the flag and remove it.

RELNOTES[INC]: Flipped --incompatible_disallow_resource_jars (see https://github.com/bazelbuild/bazel/issues/13221).

Closes #13891.

PiperOrigin-RevId: 392427418

This reflects the change in recent Linux distributions that no longer ship a /usr/bin/python binary.

I'm not sure where this is actually used, now that if I understand correctly the shebang defaults to the Python binary found via auto-detection, but we should probably still fix this path (or remove this default fallback completely?).

Related to #11201. Closes #13851.

PiperOrigin-RevId: 392427248

This filters the command line output to only show the binary's output.

PiperOrigin-RevId: 392364638

PiperOrigin-RevId: 392098303

To Blaze Sheriff: Thsi is expected to break
//third_party/tulsi/src/TulsiGeneratorIntegrationTests (see
https://github.com/bazelbuild/bazel/commit/99a85e79a11cef98d0e08db21ab94833398381af).  Please ignore it.

PiperOrigin-RevId: 392091288

Plus some minor simplifications (builder.owned was unnecessary).

Intent is a nearby regression test for the one time we were attempting to submit
work cross ForkJoinQuiescingExecutors, but wound up submitting all of the work
to the wrong pool because of unsound usage of ForkJoinTask.inForkJoinPool().

PiperOrigin-RevId: 392045189

PiperOrigin-RevId: 391977860

Filegroup implementation breaks queries.

PiperOrigin-RevId: 391977316

PiperOrigin-RevId: 391976737

The warnings can't be generated from Starlark. Disabling this test to make the switch to Starlark java_library easier.

PiperOrigin-RevId: 391976273

Modifying the original builder causes the code to behave differently after the first export.

PiperOrigin-RevId: 391943813

PiperOrigin-RevId: 391893910

Fix terrible ForkJoinQuiescingExecutor latent bug exposed by new usage of ForkJoinPool: only "adapt" a runnable to run in an existing FJP if that existing FJP is the same as the executor's: don't enqueue them to run in whatever random FJP is trying to enqueue this runnable. Big props to michajlo@ for quickly diagnosing this and suggesting the fix.

PiperOrigin-RevId: 391881393

Make some performance optimizations by reusing a canonical `ReqiuiredConfigFragmentsProvider#EMPTY` instance and avoiding a copy if merging with `EMPTY`.

PiperOrigin-RevId: 391842481

Rule declarations pick up the [allowlist](https://github.com/bazelbuild/bazel/blob/897aa816186327f7a5af19cc4983b9778ff3e8b7/src/main/java/com/google/devtools/build/lib/analysis/starlark/StarlarkRuleClassFunctions.java#L260-L264) which [comes from the `StarlarkThread`](https://github.com/bazelbuild/bazel/blob/897aa816186327f7a5af19cc4983b9778ff3e8b7/src/main/java/com/google/devtools/build/lib/analysis/starlark/StarlarkRuleClassFunctions.java#L304-L311). Add the allowlist to `BazelStarlarkContext` used when declaring Starlark rules so that it adds an attribute with the allowlist in them.

Remove `$` and `allowlist` from the allowlist attribute base name since `Allowlist.getAttributeFromAllowlistName` already adds those.

PiperOrigin-RevId: 391837774

PiperOrigin-RevId: 391794612

built-ins.

Rules are necessary so that they can later be overriden by Starlark ones later.

PiperOrigin-RevId: 391776104

Baseline: ead44955

Cherry picks:

   + 99a85e79:
     Automated rollback of commit
     92f7b586.
   + 16523615:
     Fix a race condition in scheduling code in
     `FileSystemValueCheckerInferringAncestors`.
   + ed251187:
     Add incompatible flag to guard top-level aspects dependencies

Incompatible changes:

  - Removing java_common.javac_jar Starlark call.
  - native.existing_rule now returns select values in a form that is
    accepted by rule instantiation. This is a breaking API change
    because there is some code that relies on the precise type
    returned, including brittle workarounds for this bug specifically
    and insufficiently flexible workarounds for other issues with the
    intersection of select and native.existing_rule.
  - flipped incompatible_use_toolchain_resolution_for_java_rules, see
    #7849
  - Query output=xml/proto/location for source files will now show
    the location of line 1 of the source file (as the new default)
    instead of its location in the BUILD file.
  - Specifying a target pattern underneath a directory specified by
    .bazelignore will now emit a warning, not an error.
  - Query `--order_output=auto` will now sort lexicographically.
    However, when `somepath` is used as a top level function (e.g.
    `query 'somepath(a, b)'`), it will continue to output in
    dependency order. If you do not want the lexicographical output
    ordering, specify another `--order_output` value (`no`, `deps` or
    `full`) based on what ordering you require.
  - In the build event stream,
    BuildMetrics.TargetMetrics.targets_loaded is no longer populated.
    Its value was always mostly meaningless.
    BuildMetrics.TargetMetrics.targets_configured and
    BuildMetrics.ActionSummary.actions_created now include configured
    aspect data.
  - //visibility:legacy_public has been removed.
  - Flip and remove incompatible_dont_collect_so_artifacts
    (https://github.com/bazelbuild/bazel/issues/13043).
  - Remove flag --experimental_no_product_name_out_symlink: it is
    always true.
  - The Starlark method generate_dsym in objc fragment has
    been deleted.  Please use the equivalent apple_generate_dsym in
    cpp
    fragment instead.
  - Native libraries in data attribute are not collected. See
    https://github.com/bazelbuild/bazel/issues/13550 for details
  - Enforce the `--profile` path to be absolute.
  - Enforce the --memory_profile path to be absolute.
  - JavaToolchainInfo.jvm_opt returns Depset instead of a list.
  - --apple_sdk has been deleted.  It is a no-op.

Important changes:

  - Flag --incompatible_objc_compile_info_migration is removed.  See
    #10854.
  - Flag --incompatible_objc_compile_info_migration is removed.  See
    #10854.
  - Flag --incompatible_objc_compile_info_migration is removed.  See
    #10854.
  - none
    PAIR=cmita
  - The --incompatible_load_python_rules_from_bzl flag is now a no-op.
  - Filter all (instead of just C++) source files for coverage output
    according to --instrumentation_filter and
    --instrument_test_targets.
  - The `--incompatible_disable_native_apple_binary_rule` flag has
    been added which disables the native `apple_binary` rule. Users
    who need to use `apple_binary` directly (if they cannot use one
    of the more specific Apple rules) should load it from
    https://github.com/bazelbuild/rules_apple.
  - The Android rules' --use_singlejar_apkbuilder is now a no-op.
    SingleJar will always be used to build APKs.
  - dict.setdefault(key, ...) now fails if dict is frozen, even if it
    already contains key. This is an incompatible API change.
  - Flag --incompatible_objc_provider_remove_compile_info is removed.
     See #11359.
  - Starlark now permits def statements to be nested (closures).
  - native.existing_rule now returns select values in a form that is
    accepted by rule instantiation. This is a breaking API change,
    though the fallout is expected to be small.
  - Starlark now supports lambda (anonymous function) expressions.
  - The "test" and "coverage" commands no longer return 3 when a
    test action fails because of a system error. Instead, the exit
    code
    reflects the type of system error.
  - The undocumented ctx.expand feature no longer exists.
  - Make --legacy_dynamic_scheduler a no-op flag.
  - Multiplex persistent workers can now use the JSON protocol.
  - native.existing_rule now returns a mutable list, not a tuple, for
    a list-valued attributes. This is an incompatible API change.
  - Roll back change to have native.existing_rules use list instead
    of tuple.
  - BEP includes test suite expansions.
  - config_setting now honors `visibility` attribute (and defaults to
    `//visibility:public`)
  - Change the MultiArchSplitTransitionProvider to be based on
    platform type + CPU instead of fixed "ios_" + cpu.
  - enforce config_setting visibility. See
    https://github.com/bazelbuild/bazel/issues/12932 for details.
  - add a flag to build v4 signature file
  - Added _direct_source_jars output group to Java related targets.
    END_PUBLIC
  - pkg_deb is no longer part of @bazel_tools//build_defs/pkg:pkg.bzl.
    Use https://github.com/bazelbuild/rules_pkg/tree/main/pkg instead
  - Allowing the lipo operations to be conditional in the
    linkMultiArchBinary API for Apple binaries. Single architecture
    slices are now returned through AppleBinaryOutput and the
    Starlark API.
  - Release restriction for "-" in the package name for Python
    sources. Now `py_binary` and `py_test` targets can have main
    source file with "-" in the path.
  - Users consuming BEP may assume that a `named_set_of_files` event
    will
    appear before any event referencing that `named_set` by ID. This
    allows consumers
    to process the files for such events (eg. `TargetCompleted`)
    immediately.
  - BEP includes all files from successful actions in requested
    output groups.
    Previously, an output group's files were excluded if any file in
    the output group
    was not produced due to a failing action. Users can expect BEP
    output to be larger
    for failed builds.
  - In BEP, TargetComplete.output_group has a new field `incomplete`
    indicating that the file_sets field is missing one or more
    declared artifacts
    whose generating actions failed.
  - The flag `--toolchain_resolution_debug` now takes a regex
    argument, which is used to check which toolchain types should
    have debug info printed. You may use `.*` as an argument to keep
    the current behavior of debugging every toolchain type.
  - Add runfiles.merge_all() for merging a sequence of runfiles
    objects.
  - runfiles.merge() and merge_all() now respect
    --nested_set_depth_limit.
    If you hit the depth limit because you were calling merge() in a
    loop, use
    merge_all() on a sequence of runfiles objects instead.
  - Bazel will no longer create a bazel-out symlink if
    --symlink_prefix is specified: the directory pointed to via the
    bazel-out symlink is accessible via ${symlink_prefix}-out. If
    this causes problems for you, set
    --experimental_no_product_name_out_symlink=false in your builds
    and file an issue.
  - Updates worker protocol with cancellation fields, and adds
    experimental_worker_cancellation flag to control cancellation.
  - Simplify build failure output by always using `NNN arguments`.
  - trim_test_configuration now defaults to on
  - Mark genrule.srcs as a source attribute for coverage.
  - When using --allow_analysis_failures (for example, via
    bazel-skylib's
    analysistest with `expect_failure = True`), analysis-time
    failures in aspect
    implementation functions will now be propagated and saved in
    AnalysisFailureInfo, just like analysis-time failures in rules.
  - cquery --noimplicit_deps now correctly filters out resolved
    cc_toolchains
  - Sign apks deterministically.
  - Make gcov optional in cc_toolchain tools.
  - If --experimental_prefer_mutual_xcode is passed, Bazel will
    choose the local default (instead of the newest mutually
    available version) if it's available both locally and remotely.
  - Remove java_lite_proto_library.strict_deps attribute.
  - Generate proguard configurations deterministically.
  - Adds a new flag, `--incompatible_enable_cc_test_feature` which
    switches from the use of build variables to the feature of the
    same name.
  - Dropped fragile xz support from built in pkg_tar. Users requiring
    xz
    compression should switch to bazlebuild/rules_pkg.
  - If all strategies of one branch (the local or remote execution
    branch) of the `dynamic` strategy fail to even accept (via the
    response they give from `canExec`) the action, `dynamic` will now
    try to see if the other branch can accept it. (Trying to run it
    and it failing will still cause a failure if it was the first
    result, this is about strategies claiming they can't even try the
    action)
  - Add `disable_annotation_processing` option to
    `java_common.compile`, which disables any annotation processors
    passed to `plugins` or in `exported_plugins` of `deps`
  - Remove obsolete --incompatible_prohibit_aapt1
  - The minimum Android build tools version for the Android rules is
    now 30.0.0
  - Adds --experimental_reuse_sandbox_directories flag to reuse
    already-created non-worker sandboxes with cleanup.
  - --experimental_force_gc_after_build is deprecated and will be
    removed soon. Use --bep_publish_used_heap_size_post_build instead
  - Forward coverage-instrumented files from non-tool dependencies by
    default.
  - The used_heap_size_post_build field in BEP is populated when the
    --memory_profile flag is set
  - --run_validations defaults to true.
  - Consider label_keyed_string_dict attributes when gathering
    instrumented files for coverage.
  - Remove flag
    --experimental_forward_instrumented_files_info_by_default, now
    that this behavior is the default.
  - When using MemoryProfiler with multiple GCs via the
    --memory_profile_stable_heap_parameters flag, we do a more
    precise calculation of heap used at the end of the build. This
    will generally result in lower values.
  - --bep_publish_used_heap_size_post_build is deprecated. Use
    --memory_profile=/dev/null instead.

This release contains contributions from many people at Google, as well as Adam Liddell, Alex Eagle, Andrew Katson, Anthony Pratti, Austin Schuh, Benedek Thaler, Benjamin Lee, Benjamin Peterson, Benjamin Peterson, bromano, Cameron Mulhern, Christopher Peterson Sauer, Christopher Sauer, Cristian Hancila, Dan Bamikiya, Daniel McCarney, Daniel Wagner-Hall, Danny Wolf, Dave MacLachlan, Dave Nicponski, David Cummings, David, David Ostrovsky, Delwin9999, Denys Kurylenko, Dmitry Ivankov, dorranh, Ed Schouten, Eitan Adler, Elliotte Rusty Harold, erenon, Eric Cousineau, Ethan Steinberg, FaBrand, Felix Ehrenpfort, Finn Ball, Fredrik Medley, Garrett Holmstrom, Gautam Korlam, George Gensure, goodspark, Greg Estren, Grzegorz Lukasik, Jesse Chan, Joe Lencioni, Johannes Abt, John Laxson, Jonathan Schear, Justus Tumacder, Keith Smiley, Kevin Hogeland, Lauri Peltonen, Liu Liu, Lszl Csomor, m, Marc Zych, Mark Karpov, Masoud Koleini, Matt Mackay, Max Liu, Menny Even Danan, menny, Michael Chinen, Nathaniel Brough, Nick Korostelev, Niek Peeters, Nikolay Shelukhin, odisseus, Oleh Stolyar, Olek Wojnar, Olle Lundberg, Philipp Schrader, Pras Velagapudi, Qais Patankar, Rai, Ryan Beasley, samhowes, Samuel Giddins, Sebastian Olsson, Steve Siano, steve-the-bayesian, Stiopa Koltsov, susinmotion, tatiana, Thi Doan, ThomasCJY, Timothy Klim, Tobi, Torgil Svensson, Trustin Lee, Ulf Adams, Uri Baghin, Vaidas Pilkauskas, Vertexwahn, wisechengyi, Xavier Bonaventura, Yannic Bonenberger, Yannic Bonenberger, Yannic, Yury Evtikhov, Yuval Kaplan, Yuval, [zqzzq].

https://docs.google.com/document/d/12sC92mq7WasTvDWsm4U782oCegitfi1jbgOnMIGCe-A/edit

PiperOrigin-RevId: 391720461

JavaPluginInfo is a toplevel symbol and cannot be retrieved conditionally. This is needed for backward compatibility - Bazel versions that don't have this provider yet.

PiperOrigin-RevId: 391709899

PiperOrigin-RevId: 391651166

PiperOrigin-RevId: 391647723

Adding a link to specify the definition of package and subpackage for beginners who come directly to this page without looking at[]

PiperOrigin-RevId: 391627913

The TODO was fixed in a previous commit (`b07cd064`) but I must have messed up my merge and incorrectly didn't delete it.

PiperOrigin-RevId: 391623531

Enable Apple Silicon iOS simulator support via use of new CPU type. This will be removed in favor of platforms once moved onto the new toolchain resolution system.

PiperOrigin-RevId: 391596112

Remove the preconditions check that an action is not running when accessing its aggregated elapsed time.

It may have transitioned back to running if it was rewound. In this case, using the completed execution's aggregated elapsed time seems reasonable.

PiperOrigin-RevId: 391572822

Closes #13867.

PiperOrigin-RevId: 391561240

PiperOrigin-RevId: 391556458

PiperOrigin-RevId: 391551441

The value is not used anywhere anymore.

PiperOrigin-RevId: 391526098