pola-rs/polars: Python Polars 0.20.17

🏆 Highlights

• Full plan CSE (#15264)

⚠️ Deprecations

• Rename parameter by to group_by in DataFrame.upsample/group_by_dynamic/rolling (#14840)
• Rename from_repr parameter from tbl to data (#15156)

🚀 Performance improvements

• load bits one word at a time for BitmapIter (#15333)
• Ipc exec multiple paths (#15040)
• add SIMD support for if-then-else kernels (#15131)

✨ Enhancements

• Remove 'FileCacher' optimization (#15357)
• check input type in entropy (#15351)
• Implements arr.n_unique (#15296)
• CSE don't scan share if predicate pushdown predicates don't match (#15328)
• Add read_database support for SurrealDB ("ws" and "http") (#15269)
• Only allow inputs of type Sequence in from_records (#15329)
• In hypothesis testing strategies, enable Decimal strategy by default (#15321)
• Remove cached nodes when finished (#15310)
• Full plan CSE (#15264)
• More robust handling of async database calls (#15202)
• Add name parameter to GroupBy.len method (#15235)
• Add IR for expressions. (#15168)
• Improve read_database when reading from Kùzu graph database (#15218)
• Warn if map_elements is called without return_dtype specified (#15188)
• Add support for async SQLAlchemy connections to read_database (#15162)
• Infer time_unit in pl.duration when nanoseconds is specified (#14987)
• Add strict parameter to from_dict/from_records (#15158)

🐞 Bug fixes

• Return correct dtype for s.clear() when dtype is Object (#15315)
• ensure first datapoint is always included in group_by_dynamic (#15312)
• Non-exhaustive patterns: arrow-schema::DataType in polars-arrow (#15250)
• use dynamic stacks for problematic recursive functions (#15355)
• Adding default ddof for Series.list.std and Series.list.var (#15267)
• Raise properly for slices not supported by LazyFrame (#15331)
• Propagate strictness in from_dicts (#15344)
• Raise error when schema_overrides contains nonexistent column name (#15290)
• Enforce integer dtype input for int_range and int_ranges (#15339)
• Preserve Decimal precision when constructing empty Series (#15320)
• Fix cache dot visualization (#15311)
• Handle special case correctly when slicing a LazyFrame (#15297)
• Properly propagate strict flag when constructing a Struct Series from any values (#15302)
• Consistent expansion of nested struct data during DataFrame init from dict (#15217)
• Raise when join projects name with suffix that doesn't exist (#15256)
• Ensure ooc_start is set (#15255)
• Fix bug where rolling operations were ignoring check_sorted in some cases (#15227)
• Fix lazy schema for rle expression (#15248)
• incorrect negative offset in multi-byte string slicing (#15140)
• do not clamp negative offsets to start of array prematurely (#15242)
• allow null index in list.get and array.get (#15239)
• Avoid loading all columns in read_parquet when columns parameter is specified (#15229)
• properly support nulls_last + descending (#15212)
• fix nested runtime panic (#15216)
• Block rounding/truncating to negative durations (#15175)
• Ensure the cs.temporal() selector uses wildcard time zone matching for Datetime (#13683)
• Consistently raise TypeError on constructor failure (#15178)
• Properly propagate strictness in some constructor cases (#15166)
• Fix constructing a Series from a list of Series with given dtype (#15144)

📖 Documentation

• Fix time unit in timestamp example (#15281)
• Fix link to renamed method (.list.lengths -> .list.len) (#15228)
• Update Excel and database pages in user guide (#14721)
• Add examples for Series.search_sorted (#14737)
• Correct the definition of an expression in the user guide (#14750)
• Add a note about the behaviour of lower/upper bounds for is_between, and add an example (#15197)

📦 Build system

• Update Cargo lock (#15370)

🛠️ Other improvements

• Memory usage test infrastructure, plus a test for #15098 (#15285)
• Update CODEOWNERS (#15352)
• remove try_apply_values (#15336)
• always use non-legacy float_sum for mean (#15343)
• remove legacy bitmap module (#15335)
• Fix test not writing to temporary directory (#15318)
• Reorganize tests for clear operation (#15304)
• Rename Cache[count] to Cache[cache_hits] (#15300)
• Cleanup file_caching optimization call (#15299)
• Minor refactor of PyDataFrame.from_dicts (#15274)
• remove 'fast-projection' node (#15253)
• cleanup column names in optimizer (#15252)
• remove left_most_input_name from expr ir (#15251)
• add AlignedBitmapSlice (#15171)
• Run non-benchmark tests in benchmark workflow (#15207)
• Add wrapping_abs to arithmetic kernel (#15210)
• remove raw buffers from BinViewArray (#15206)
• Enable RUST_BACKTRACE=1 in the CI test suite (#15204)
• Split read_database functionality into cleaner module structure (#15201)
• Clean up some of the AnyValue conversion logic (#15190)
• remove parts of legacy bit_util (#15169)
• remove legacy arrow compute (#15164)
• Split up dataframe module in PyO3 bindings (#15165)
• Remove unused private constructors (#15160)

Thank you to all our contributors for making this release possible! @MarcoGorelli, @alexander-beedie, @braaannigan, @c-peters, @cojmeister, @deanm0000, @dependabot, @dependabot[bot], @itamarst, @kszlim, @mbuhidar, @mcrumiller, @mickvangelderen, @orlp, @petrosbar, @reswqa, @ritchie46, @rob-sil, @sportfloh, @stinodego and @yutannihilation