feat!: combine LanceFragment.merge_columns and LanceDataset.add_columns in one implementation

[westonpace]

These two methods were very similar but took different arguments. This combines the implementation of the two methods.

In addition:

• The python implementation of Updater was removed as:
  • It is no longer used (since merge_columns now uses Fragment::add_columns)
  • It was internal and could potentially be a maintenance burden in the future
• The python method LanceFragment.add_columns was removed since it has been deprecated for six months and could be confusing.

In a few weeks it might be good to deprecate LanceFragment.merge_columns and add LanceFragment.add_columns back with a new signature that mirrors LanceDataset.add_columns.

BREAKING CHANGE: The rust method LanceDataset.add_columns now has an optional batch_size parameter.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 62.29508% with 23 lines in your changes missing coverage. Please review.

Project coverage is 78.15%. Comparing base (f9024ce) to head (7b5f836).

Files with missing lines	Patch %	Lines
rust/lance/src/dataset/fragment.rs	5.55%	17 Missing ⚠️
rust/lance/src/dataset/schema_evolution.rs	87.17%	3 Missing and 2 partials ⚠️
rust/lance/src/dataset/updater.rs	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3021      +/-   ##
==========================================
- Coverage   78.19%   78.15%   -0.05%     
==========================================
  Files         239      239              
  Lines       76782    76825      +43     
  Branches    76782    76825      +43     
==========================================
+ Hits        60043    60045       +2     
- Misses      13669    13693      +24     
- Partials     3070     3087      +17     

Flag	Coverage Δ
unittests	78.15% <62.29%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wjones127]

Nice work. This looks great!

[wjones127]

In a few weeks it might be good to deprecate LanceFragment.merge_columns and add LanceFragment.add_columns back with a new signature that mirrors LanceDataset.add_columns.

Agreed. The naming inconsistency makes me sad :(