Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark 3.2 new functions #234

Open
4 of 16 tasks
eruizalo opened this issue Jun 24, 2022 · 0 comments
Open
4 of 16 tasks

Spark 3.2 new functions #234

eruizalo opened this issue Jun 24, 2022 · 0 comments
Labels
API enhancement New feature or request good first issue Good for newcomers spark_3.2 PR changes to spark 3.2

Comments

@eruizalo
Copy link
Collaborator

Look into the following functions if they already are in doric or we have to add them:

  • def typedlit[T : TypeTag](literal: T): Column
  • def count_distinct(expr: Column, exprs: Column*): Column
  • def product(e: Column): Column
  • def sum_distinct(e: Column): Column ✅ covered in Feat/#60 implemented aggregate functions #123
  • def lag(e: Column, offset: Int, defaultValue: Any, ignoreNulls: Boolean): Column
  • def lead(e: Column, offset: Int, defaultValue: Any, ignoreNulls: Boolean): Column
  • def bitwise_not(e: Column): Column ⁉️ WIP in Non-aggregate functions #68
  • def shiftleft(e: Column, numBits: Int): Column ✅ covered in feat: [+] #66 math functions #223
  • def shiftright(e: Column, numBits: Int): Column ✅ covered in feat: [+] #66 math functions #223
  • def shiftrightunsigned(e: Column, numBits: Int): Column ✅ covered in feat: [+] #66 math functions #223
  • def sentences(string: Column, language: Column, country: Column): Column
  • def sentences(string: Column): Column
  • def next_day(date: Column, dayOfWeek: Column): Column
  • def session_window(timeColumn: Column, gapDuration: String): Column
  • def session_window(timeColumn: Column, gapDuration: Column): Column
  • def call_udf(udfName: String, cols: Column*): Column
@eruizalo eruizalo added enhancement New feature or request good first issue Good for newcomers spark_3.2 PR changes to spark 3.2 API labels Jun 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API enhancement New feature or request good first issue Good for newcomers spark_3.2 PR changes to spark 3.2
Projects
Status: Todo
Development

No branches or pull requests

1 participant