Sourced from ray's releases.

Ray-2.55.1

• Fixes SSH connectivity issue in the ray-llm image (#62625 / #62718).
• Upgrade apt packages in slim base (#62666 / #62717).

Ray-2.55.0

Ray Data

🎉 New Features

• Add DataSourceV2 API with scanner/reader framework, file listing, and file partitioning (#61220, #61615, #61997)
• Support GPU shuffle with rapidsmpf 26.2 (#61371, #62062)
• Add Kafka datasink, migrate to confluent-kafka, support datetime offsets (#60307, #61284, #60909)
• Add Turbopuffer datasink (#58910)
• Add 2-phase commit checkpointing with trie recovery and load method (#61821, #60951)
• Queue-based autoscaling policy integrated with task consumers (#59548, #60851)
• Enable autoscaling for GPU stages (#61130)
• Expressions: add random(), uuid(), cast, and map namespace support (#59656, #60695, #59879)
• Add support for Arrow native fixed-shape tensor type (#56284)
• Support writing tensors to tfrecords (#60859)
• Add pathlib.Path support to read_* functions (#61126)
• Add cudf as a batch_format (#61329)
• Allow ActorPoolStrategy for read_datasource() via compute parameter (#59633)
• Introduce ExecutionCache for streamlined caching (#60996)
• Support strict=False mode for StreamingRepartition (#60295)
• Port changes from lance-ray into Ray Data (#60497)
• Enable PyArrow compute-to-expression conversion for predicate pushdown (#61617)
• Add vLLM metrics export and Data LLM Grafana dashboard (#60385)
• Include logical memory in resource manager scheduling decisions (#60774)
• Add monotonically increasing ID support (#59290)

💫 Enhancements

• Performance: cache _map_task args, heap-based actor ranking, actor pool map improvements (#61996, #62114, #61591)
• Optimize concat tables and PyArrow schema hashing (#61315, #62108)
• Reduce default DownstreamCapacityBackpressurePolicy threshold to 50% (#61890)
• Improve reproducibility for random APIs (#59662)
• Clamp batch size to fall within C++ 32-bit int range (#62242)
• Account for external consumer object store usage in resource manager budget (#62117)
• Make get_parquet_dataset configurable in number of fragments to scan (#61670)
• Consolidate schema inference and make all preprocessors implement SerializablePreprocessorBase (#61213, #61341)
• Disable hanging issue detection by default (#62405)
• Make execution callback dataflow explicit to prevent state leakage (#61405)
• Log DataContext in JSON format at execution start for traceability (#61150, #61428)
• Autoscaler: configurable traceback, Prometheus gauges, relaxed constraints (#62210, #62209, #61917, #61385)
• Add metrics for task scheduling time, output backpressure, and logical memory (#61192, #61007, #61436)
• Prevent operators from dominating entire shared object store budget (#61605)
• Eliminate generators to avoid intermediate state pinning (#60598)
• Default log encoding to UTF-8 on Windows (#61143)
• Remove legacy BlockList, locality_with_output, old callback API, PyArrow 9.0 checks (#60575, #61044, #62055, #61483)
• Upgrade to pyiceberg 0.11.0; cap pandas to <3 (#61062, #60406)
• Refactor logical operators to frozen dataclasses (#61059, #61308, #61348, #61349, #61351, #61364, #61481)

... (truncated)

• 237c245 cherrypick #62758 (#62759)
• a4c5c8c cherrypick #62625 (#62718)
• 9c12c02 cherrypick #62666 (#62717)
• cf6616e updating ray version 2.55.1 (#62672)
• 3bbcfa7 [docker] Update latest Docker dependencies for 2.55.0 release (#62626)
• 58af3fc Cherry picking to 2.55 (#62517)
• 2ecf83a cherrypick part of #62485 (#62520)
• 3c9dfdf [train] Cherrypick validation resumption deflake (#62448)
• 6fcc3f4 [core] Cherry-pick: Deflake test_dashboard_port_conflict (#62413)
• 5092068 windows base build fix: install together (#62415)
• Additional commits viewable in compare view

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)