Sourced from daft's releases.

v0.5.19

What's Changed 🚀

We have a pretty crazy release this time around. Some especially notable features include:

• Interactive DataFrames in Jupyter Notebooks, with special support for some multimodal types
• An async API for LLM text generation, particularly with OpenAI
• A new .into_batches DataFrame API, the modern alternative to .into_partitions
• Adding support for .offset / OFFSET operator across the engine. Thanks @​plotor for the great work!
• Various Flotilla performance and reliability improvements
• Various casting improvements

✨ Features

• feat: Async open ai llm generate @​colin-ho (#4879)
• feat: Add offset syntax support to SQL @​plotor (#4707)
• feat: adds support for SQL GROUP BY column position @​rchowell (#4955)
• feat: better dtype type inference @​universalmind303 (#4973)
• feat: Casting from Python into struct or list types @​srilman (#4957)
• feat: support creating partitioned tables in Iceberg via the Catalog interface. @​redpheonixx (#4951)
• feat: implement into_batches operator on flotilla distrubted engine @​ohbh (#4958)
• feat: literal variants for (pretty much) all types @​kevinzwang (#4947)
• feat: Add offset support to Flotilla Engine @​plotor (#4918)
• feat: implement into_batches on the swordfish native daft runner @​ohbh (#4935)
• feat: Flotilla broadcast join @​colin-ho (#4867)

🐛 Bug Fixes

• fix: Always just use actor for flotilla scheduler @​colin-ho (#4978)
• fix: Add handle for swordfish runtime stats manager @​colin-ho (#4970)
• fix: Dudep lance read required columns @​xloya (#4967)
• fix: Don't use wildcard for logical plan match in pushdown rules @​colin-ho (#4945)
• fix: Coerce arrow schema for parquet decoding @​colin-ho (#4948)
• fix: use associate type for swordfish into_batches operator state @​ohbh (#4956)
• fix: raise error on invalid cross join parameters @​rchowell (#4952)
• fix: interactive html fixes @​colin-ho (#4943)

♻️ Refactor

• refactor: Use associate type for swordfish operator state @​colin-ho (#4921)

📖 Documentation

• docs: update links in document processing example @​ccmao1130 (#4946)
• docs: improve daft.func documentation and type inference @​universalmind303 (#4942)
• docs: fix link for pandas @​universalmind303 (#4941)

👷 CI

• ci: add 3.11 to matrix and always include typing_extensions @​rchowell (#4950)

... (truncated)

• 23549cd fix: Always just use actor for flotilla scheduler (#4978)
• 4f14d6a fix: Add handle for swordfish runtime stats manager (#4970)
• 4533257 feat: Async open ai llm generate (#4879)
• 0129008 feat: Add offset syntax support to SQL (#4707)
• 5c7baae feat: adds support for SQL GROUP BY column position (#4955)
• acb838a feat: better dtype type inference (#4973)
• c555b0c fix: Dudep lance read required columns (#4967)
• 5a7f4b0 feat: Casting from Python into struct or list types (#4957)
• 9902072 feat: support creating partitioned tables in Iceberg via the Catalog interfac...
• 7fc5d47 feat: implement into_batches operator on flotilla distrubted engine (#4958)
• Additional commits viewable in compare view

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)