Skip to content
This repository has been archived by the owner on Sep 4, 2024. It is now read-only.

Releases: astronomer/astro-provider-databricks

0.1.3

27 Apr 13:35
@tatiana tatiana
0.1.3
71a39cc
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.1.3

Enhancements

  • Associate a DatabricksNotebookOperator to a DatabricksWorkflowTaskGroup even if there are up to three levels TaskGroups in between (#29 by @jlaneve )
  • Support templating the field notebook_params of the DatabricksNotebookOperator (#33 by @tatiana )
  • Extend notebook_params of the DatabricksNotebookOperator with the values defined in DatabricksWorkflowTaskGroup (#33 by @tatiana )
  • Improve example DAGs (#29 by @jlaneve )

Contributors

tatiana and jlaneve
Assets 2
Loading

0.1.2

11 Apr 23:21
@jlaneve jlaneve
0.1.2
a3efcde
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.1.2

What's Changed

New Contributors

Full Changelog: 0.1.1...0.1.2

Contributors

jlaneve and josh-fell
Assets 2
Loading

0.1.1

13 Mar 18:06
@tatiana tatiana
0.1.1
392849f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
0.1.1
  • DatabricksWorkflowTaskGroup and DatabricksNotebookOperator log the Databricks Job URL (#20)
  • README improvement (#21)
Assets 4
Loading

0.1.0

13 Mar 10:40
@tatiana tatiana
0.1.0
684c603
Compare
Choose a tag to compare
0.1.0
  • DatabricksWorkflowTaskGroup: Airflow Task Group <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html#taskgroups>_ that allows users to create a Databricks Workflow <https://www.databricks.com/product/workflows>_.
  • DatabricksNotebookOperator: Airflow Operator <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/operators.html>_ which abstracts a pre-existing Databricks Notebook <https://docs.databricks.com/notebooks/>_. Can be used independently to run the Notebook, or within a Databricks Workflow Task Group.
  • AstroDatabricksPlugin: An Airflow Plugin <https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/plugins.html>_ which is installed by the default. It allows users, by using the UI, to view a Databricks job and retry running it in case of failure.

Known limitations

  • Cancelling the Airflow task does not stop its execution in Databricks (issue #1 <https://github.com/astronomer/astro-provider-databricks/issues/1>_).
  • Users should not click the buttons "repair all" or "repair single task" while the DAG/task is running (issue #2 <https://github.com/astronomer/astro-provider-databricks/issues/2>_).
Assets 4
Loading