From c578a03289e4cc885c73b4c3ba79d2ff94dc160e Mon Sep 17 00:00:00 2001 From: Per Fuchs Date: Thu, 20 Jul 2023 20:52:50 +0200 Subject: [PATCH] Arrow support is now stable and ready for production (#43) * Remove the experimental warning from the external formats page * Remove the experimental settings section from the hyper_process page as Arrow was the only setting mentioned there. * Add to the release notes * Remove broken link from release notes --- website/docs/hyper-api/hyper_process.md | 8 ++------ website/docs/releases.md | 6 +++--- website/docs/sql/external/formats.md | 7 ------- 3 files changed, 5 insertions(+), 16 deletions(-) diff --git a/website/docs/hyper-api/hyper_process.md b/website/docs/hyper-api/hyper_process.md index 2591a8a..b4d0ede 100644 --- a/website/docs/hyper-api/hyper_process.md +++ b/website/docs/hyper-api/hyper_process.md @@ -248,7 +248,7 @@ unsupported version 2 (max supported version: 1). To open this database, please update your product. (error code 0AS01)" ::: -### Experimental Settings {#experimentalsettings} + diff --git a/website/docs/releases.md b/website/docs/releases.md index f4e1a0c..8a97964 100644 --- a/website/docs/releases.md +++ b/website/docs/releases.md @@ -27,6 +27,7 @@ In case you are wondering why all our releases start with `0.0`, read [this FAQ * The functions `EXTRACT`, `date_part` and `date_trunc` have the following fiscal calendar fields: `fiscal_week`, `fiscal_month`, `fiscal_quarter`, and `fiscal_year`. * Fiscal calendar options for the newly-added fields were added. See [Fiscal Calendar Options](/docs/sql/scalar_func/datetime#fiscal-calendar-options). * Updated OpenSSL version from 1.1.1t to 1.1.1u. +* Support for reading and writing [Arrow](/docs/sql/external/formats#external-format-arrow) is now stable and ready for use in production ### v0.0.17360 [July 5, 2023] @@ -166,9 +167,8 @@ In case you are wondering why all our releases start with `0.0`, read [this FAQ Improved external file format support (CSV & Apache Parquet): Now you can use Hyper as a SQL query engine directly on top of open formats and data lakes. - * Hyper now has experimental support for reading external data directly from Amazon S3. You need to enable the - [experimental_external_s3](/docs/hyper-api/hyper_process#experimentalsettings) - setting to use this feature and be aware that it can **change or be removed at any time without prior notice**. + * Hyper now has experimental support for reading external data directly from Amazon S3. ~~You need to enable the experimental_external_s3 + setting to use this feature and be aware that it can **change or be removed at any time without prior notice**.~~ The feature is considered stable as of v0.0.15735. The experimental flag is not necessary anymore. Hyper's S3 capabilities are highly optimized (using techniques such as concurrent requests, request hedging and prefetching). For maximum performance, diff --git a/website/docs/sql/external/formats.md b/website/docs/sql/external/formats.md index 387b1d8..82c5bbf 100644 --- a/website/docs/sql/external/formats.md +++ b/website/docs/sql/external/formats.md @@ -371,13 +371,6 @@ production workloads yet. ## Apache Arrow Format {#external-format-arrow} -:::warning Experimental Feature -Hyper's support for the Arrow format is still in an early state and -should be considered experimental. We recommend to not use it in -production workloads yet. Set the `experimental_external_format_arrow` [setting](/docs/hyper-api/hyper_process#experimentalsettings) -to `true` to activate it. -::: - The `FORMAT => 'arrowfile'` or `FORMAT => 'arrowstream'` option enables reading the columnar binary format [Apache Arrow](https://arrow.apache.org/). As data is stored in a columnar format, Hyper does not need to read the whole file if only a subset of columns is selected. Hyper can read only