Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for reading Parquet files #576

Open
fb64 opened this issue Jan 28, 2024 · 4 comments · May be fixed by #577
Open

Add support for reading Parquet files #576

fb64 opened this issue Jan 28, 2024 · 4 comments · May be fixed by #577
Labels
enhancement New feature or request
Milestone
0.15.0

Comments

@fb64
Copy link
Contributor

fb64 commented Jan 28, 2024

No description provided.

fb64 added a commit to fb64/dataframe that referenced this issue Jan 28, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
158ed95
@zaleslaw
Copy link
Collaborator

Thanks for the issue, could you please provide a use-case for this task? Do you have data in this format to explore?

@fb64
Copy link
Contributor Author

fb64 commented Jan 29, 2024 •
edited
Loading

IMHO Parquet format is now a standard in term of data exchange format as it provides both good performance and good compression rate. Lot of data can be found in this format and it is supported by most data science tools.
In my PR you can find a parquet file example.
Some open data could also be found as parquet file like Newyork Yellow taxi trip for example : https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page

fb64 added a commit to fb64/dataframe that referenced this issue Jan 29, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
3c6e600
@Jolanrensen Jolanrensen linked a pull request Jan 29, 2024 that will close this issue
Open
@Jolanrensen
Copy link
Collaborator

Can confirm, Parquet support would be nice, it's used a lot with Spark as well.

@Jolanrensen Jolanrensen added the enhancement New feature or request label Jan 29, 2024
fb64 added a commit to fb64/dataframe that referenced this issue Jan 31, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
0dd7498
@kopczynski-9livesdata
Copy link

I second this. Parquet format is massively popular when working with Spark and having this in Kotlin Dataframe would be great.

@zaleslaw zaleslaw added this to the 0.14.0 milestone Apr 23, 2024
fb64 added a commit to fb64/dataframe that referenced this issue Apr 24, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
71b06f5
fb64 added a commit to fb64/dataframe that referenced this issue Apr 24, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
8b8f706
fb64 added a commit to fb64/dataframe that referenced this issue Aug 13, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
5ce70b9
@zaleslaw zaleslaw modified the milestones: 0.14.0, Backlog, 0.15.0 Sep 3, 2024
fb64 added a commit to fb64/dataframe that referenced this issue Sep 6, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
79fd37d
fb64 added a commit to fb64/dataframe that referenced this issue Oct 15, 2024
@fb64
Add support for reading parquet file thanks to arrow-dataset Kotlin#576
e66906a
@laurentperez laurentperez mentioned this issue Oct 17, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
0.15.0
Development

Successfully merging a pull request may close this issue.

Add support for reading parquet file thanks to arrow-dataset #576 fb64/dataframe
4 participants
@zaleslaw @fb64 @Jolanrensen @kopczynski-9livesdata