Skip to content

0.4.0
Compare
Choose a tag to compare
@chathurawidanage chathurawidanage released this 21 Apr 16:49
· 13 commits to 0.4.0 since this release
0.4.0
76d150e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Cylon 0.4.0 is a major release with the following features.

Major Features

Python

  • DataFrame API similar to Pandas supporting around 40 operators commonly used in Pandas.
  • Conda build and conda based binaries for Linux for installing.
  • Python binding to all the operators added on the C++ level.
  • Providing compute functions with both Arrow and Numpy for filtering, math operations and comparison operators.
  • Added operator benchmarks.
  • Added new options for CSV reading supporting all the options in PyArrow for reading CSV.

C++

  • Added distributed multi-column operations on tables for join, union, intersection, set difference and sort.
  • Added improved hash operations using Bytell Hash Maps. Improved performance by 2 times for union, intersection, set difference and unique.
  • Added new aggregate operations for GroupBy operation (Mean, Variance, Std Dev, Quantile, NUnique, Median).
  • Implemented GroupBy aggregators using CRTP (Curiously recurring template pattern).
  • Improved indexing at the core by Added more types, improved performance of indexed lookups.
  • Added unique distributed operator.
  • Added temporal data types like DateTime, Date32 (seconds resolution), Date64 (milliseconds resolution) and TImestamp (with time zone information).
  • Other performance improvements and bug fixes.

Build

  • Compiling using external Apache Arrow installation (local/ pip).

Applications and Benchmarks

  • Implementing a subset of TPC-XBB queries (Queries 6, 7, 9, 14, 22, 23) and the rest is ongoing.
  • Applications with connections to deep learning.

You can download source code from Github

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Assets 2
Loading