v1.2.646-nightly

What's Changed

Exciting New Features ✨

• feat(query): Reduce serialisation memory usage when spilling to local disk by @forsaken628 in #16580
• feat(query): Support show dictionaries DQL. by @Winnie-Hong0927 in #16602

Thoughtful Bug Fix 🔧

• fix: subquery returns wrong results under cluster by @xudong963 in #16606
• fix(query): ntile frame should be unbounded by @TCeason in #16604

Code Refactor 🎉

• refactor: isolate session_id and session states of diff users. by @youngsofun in #16592

Others 📒

• chore: refine table sample tests by @xudong963 in #16600
• chore(query): add more logs on aggregation by @sundy-li in #16552

Full Changelog: v1.2.645-nightly...v1.2.646-nightly

v1.2.645-nightly

What's Changed

Exciting New Features ✨

• feat(query): add date_diff function by @chagelo in #16560

Thoughtful Bug Fix 🔧

• fix(query): fix left outer join wrong result by @Dousir9 in #16601

Build/Testing/CI Infra Changes 🔌

• ci: Add iceberg tpch sqllogictests by @Xuanwo in #16598

Others 📒

• chore(query): improve variable parser by @sundy-li in #16599

New Contributors

• @chagelo made their first contribution in #16560

Full Changelog: v1.2.644-nightly...v1.2.645-nightly

v1.2.644-nightly

What's Changed

Exciting New Features ✨

• feat(query): window partition by spill to disk by @forsaken628 in #16441
• feat(query): Simplified version Pattern S/L/D/G for number to_char by @forsaken628 in #16569
• feat(query): add json array lambda functions by @sundy-li in #16573
• feat(query): support describe user statement by @b41sh in #16579
• feat: copy option 'pattern' support variable. by @youngsofun in #16525
• feat(query): support json array functions by @b41sh in #16571

Thoughtful Bug Fix 🔧

• fix: direct io syscall not available on mac by @forsaken628 in #16574
• fix: window in scalar subquery returns wrong results by @xudong963 in #16567
• fix: avoid eliminating the left child of union in RuleEliminateUnion by @xudong963 in #16584
• fix(query): fix inverted index panic caused by null value by @b41sh in #16576
• fix(query): fix cannot obtain stack trace when segmentation fault by @zhang2014 in #16566
• fix: merge into panic by @zhyass in #16581
• fix(query): list_dictionaries should not ignore db_id by @TCeason in #16587
• fix(iceberg): Make sure iceberg table has been loaded by @Xuanwo in #16593
• fix: Disable prewhere for iceberg table for now by @Xuanwo in #16597

Code Refactor 🎉

• refactor: Automatically chunk large key sets in KVPbApi methods by @drmingdrmer in #16561
• refactor: simplify get_tables_history() by @drmingdrmer in #16572
• refactor(iceberg): Use iceberg file IO instead of our own operator by @Xuanwo in #16577
• refactor: add SchemaApi::get_table_in_db by @drmingdrmer in #16582
• refactor: SchemaApi::get_table_meta_history() by @drmingdrmer in #16586
• refactor: fix error message. by @youngsofun in #16578
• refactor(storage): inverted index use merge io to read data by @b41sh in #16589

Others 📒

• chore(query): SHOW/DESC is not DDL by @TCeason in #16583
• chore: check lock revision before watch delete by @zhyass in #16590
• chore(ci): flaky test by @zhyass in #16596

Full Changelog: v1.2.643-nightly...v1.2.644-nightly

v1.2.643-nightly

What's Changed

Exciting New Features ✨

• feat: unset table options by @dantengsky in #16544

Thoughtful Bug Fix 🔧

• fix: move TableInfo building out of SchemaApi by @drmingdrmer in #16548
• fix: identify failed vacuum db by id by @drmingdrmer in #16553
• fix(query): fix json object compare cause panic by @b41sh in #16555

Code Refactor 🎉

• refactor: get_history_tables_for_gc() should not return TableInfo, but just table name, id and values by @drmingdrmer in #16545
• refactor(storage): improve inverted index match phrase query by @b41sh in #16547

Others 📒

• chore(storage): refactor compact source by @zhyass in #16527

Full Changelog: v1.2.642-nightly...v1.2.643-nightly

v1.2.642-nightly

What's Changed

Exciting New Features ✨

• feat: short sql length setting by @arkzuse in #16502

Code Refactor 🎉

• refactor: SchemaAPI::do_get_table_history by @drmingdrmer in #16540
• refactor: DroppedId for listing db/tables for gc by @drmingdrmer in #16542

Others 📒

• chore: update repo for workflows by @everpcpc in #16546

New Contributors

• @arkzuse made their first contribution in #16502

Full Changelog: v1.2.641-nightly...v1.2.642-nightly

v1.2.641-nightly

What's Changed

Exciting New Features ✨

• feat: add fast path for cross join to optimize tpcds q23 (~50x) by @xudong963 in #16504
• feat(catalog): ignore error when listing databases by @BohuTANG in #16514
• feat: limit stage name to plain_identifier by @youngsofun in #16522
• feat(query): Initial implementation of postgres-compatible number to_char function by @forsaken628 in #16507
• feat(query): Procedure support drop if exists and create or replace by @TCeason in #16500
• feat(query): DATE_ADD Functions to support week as a unit by @TracyZYJ in #16530
• feat: http handler add X-DATABEND-VERSION in each response. by @youngsofun in #16518

Thoughtful Bug Fix 🔧

• chore: add extra bracket for and and or to make explain clear by @xudong963 in #16494
• fix(query): fix planner cache cause modify async function does not take effect by @b41sh in #16510
• fix: ignore_result doesn't have an effect on union by @xudong963 in #16515
• fix(query): fix copy into table with disable distributed copy into by @zhang2014 in #16529
• fix(query): fix column leaf_index by @Dousir9 in #16537
• fix: Only when all tables are included, the db can be dropped by @drmingdrmer in #16539

Code Refactor 🎉

• refactor(query): refactor license manager by @zhang2014 in #16492
• refactor: simplify DroppedId. Replace manual comparison with string comparison by @drmingdrmer in #16495
• refactor: update dependency datafusion-orc. by @youngsofun in #16491
• refactor: get_tableinfos_by_ids() by @drmingdrmer in #16503
• chore(query): refactor like function by @sundy-li in #16493
• chore: simplify SchemaApi::truncate_table() by @drmingdrmer in #16506
• refactor: remove get_pb_value() by @drmingdrmer in #16516
• refactor: Output whole error message with source while retry by @Xuanwo in #16519
• refactor: unify path of write segment by @SkyFan2002 in #16517
• refactor: Improve gc_dropped_table_by_id() method by @drmingdrmer in #16528
• refactor: dropped table listing and GC by @drmingdrmer in #16531
• refactor: gc_dropped_db_by_id() by @drmingdrmer in #16538

Others 📒

• chore: remove logs from block writer by @dantengsky in #16497
• chore: simplify get_gc_table_info() by @drmingdrmer in #16499
• chore(query): replace to datafuse-extras/dtparse by @TCeason in #16509
• chore(storage): hook recluster avoid scan all segments by @zhyass in #16498
• chore: update source repo for async-compression by @everpcpc in #16513
• chore(query): move recursion & backtrace crate to datafuse-extras by @zhang2014 in #16520
• chore(query): add more asserts by @sundy-li in #16536

Full Changelog: v1.2.640-nightly...v1.2.641-nightly

v1.2.640-nightly

What's Changed

Exciting New Features ✨

• feat(query): push rank limit into aggregate partial node by @sundy-li in #16466
• feat: add eliminate union optimizer rule by @xudong963 in #16478
• feat(query): parallel k way merge sort by @forsaken628 in #16340

Code Refactor 🎉

• refactor: add DatabaseMeta.gc_in_progress by @drmingdrmer in #16464
• refactor(query): use pipeline exchange to refactor window scatter by @zhang2014 in #16471
• refactor(query): simplify expression kernel by @sundy-li in #16482
• refactor: add KVPbCrudApi as abstraction layer for commonly used CRUD operations by @drmingdrmer in #16479
• refactor: add KVPbCrudApi::crud_try_insert() and crud_try_upsert() by @drmingdrmer in #16484
• refactor: SchemaApi::list_databases() by @drmingdrmer in #16485
• refactor: SchemaApi::get_database_history() by @drmingdrmer in #16487
• refactor: test SchemaApi::get_tenant_history_database() by @drmingdrmer in #16488
• refactor(query): refactor license manager by @zhang2014 in #16489

Others 📒

• chore(ci): remove hdfs release by @everpcpc in #16477
• chore(query): null first behavior by @sundy-li in #16475
• chore: update nom-rule by @andylokandy in #16483
• chore: bump proj4rs 0.1.4 by @b41sh in #16474

Full Changelog: v1.2.639-nightly...v1.2.640-nightly

v1.2.639-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix: typo for query & profile logging config by @everpcpc in #16476

Full Changelog: v1.2.638-nightly...v1.2.639-nightly

v1.2.638-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix: Multi-thread recreate view/table gives error by @zhyass in #16465
• fix: log query detail and profile with identical format by @everpcpc in #16469
• fix: revert log format breakage from #16249 by @andylokandy in #16470

Others 📒

• chore(executor): support for data exchange between different processors within the pipeline by @zhang2014 in #16467
• chore(executor): more edge detail for executing graph by @zhang2014 in #16468

Full Changelog: v1.2.637-nightly...v1.2.638-nightly

v1.2.637-nightly

What's Changed

Exciting New Features ✨

• feat(query): Procedure Part2 support arguments by @TCeason in #16453
• feat(query): refactor window spill by @Dousir9 in #16448

Thoughtful Bug Fix 🔧

• fix: wrong order by result by @xudong963 in #16422

Code Refactor 🎉

• refactor(query): sort kernel optimization by @sundy-li in #16458
• refactor(storage): improve inverted index read fst file first to reduce load index by @b41sh in #16385
• refactor: refactor vacuum by @drmingdrmer in #16454
• chore: simplify get_sequence() and get_sequence_next_value() by @drmingdrmer in #16463

Others 📒

• chore(ci): release with musl by @everpcpc in #16460

Full Changelog: v1.2.636-nightly...v1.2.637-nightly