We currently already do this in runtime_build_and_test, we can reuse the same technique in other workflows to speed them up.