Releases: PrefectHQ/prefect
Release 2.19.2
This release includes a few bug fixes, ensuring:
- 🛠️ runs created from the
deployments/{id}/create_flow_run
endpoint hydrates workspace variables - 🔢 proper integer value display on the Variables page of the Prefect UI — PrefectHQ/prefect-ui-library#2454
- ⚙️ "Run a deployment" automation action parameter display input configuration persists during editing — PrefectHQ/prefect-ui-library#2458
- 📌
requests
dependency pinned to<2.32.0
inrequirements-dev.txt
- #13538 - 📄 Jinja template example is renderable in automations documentation — #13421
2.19.1
Release 2.19.1
Enhancements to prefect-dbt
for running dbt-core
commands
Introducing prefect-dbt
summary artifacts! With summary artifacts, you get a view of all of the runs that succeeded, failed, or were skipped as well as where the failed models live in your dbt project and why they failed. This exposes information about each dbt node in a succinct format that teams can use for development, debugging, insights, and so much more.
These changes are available in prefect-dbt
version 0.5.0!
See the following pull requests for implementation details:
Enhancements
- Enable flow run infra overrides section in the Prefect UI - PrefectHQ/prefect-ui-library#2417
Experimental
Events and Automations
- Add ability to create and manage automations in the Prefect UI - #13342
Documentation
- Update documentation formatting for
prefect-ray
- #13385
Integrations
- Update all integrations libraries to pin
prefect<3.0.0
- #13408
Integration library releases
This release was accompanied by the following integration library releases:
prefect-aws
- 0.4.17prefect-azure
- 0.3.10prefect-bitbucket
- 0.2.5prefect-dask
- 0.2.9prefect-databricks
- 0.2.7prefect-dbt
- 0.5.0prefect-docker
- 0.5.2prefect-email
- 0.3.5prefect-gcp
- 0.5.12prefect-github
- 0.2.5prefect-gitlab
- 0.2.5prefect-kubernetes
- 0.3.10prefect-ray
- 0.3.6prefect-shell
- 0.2.5prefect-slack
- 0.2.6prefect-snowflake
- 0.27.6prefect-sqlalchemy
- 0.4.3
All changes: 2.19.0...2.19.1
Release 2.19.0
✨ This release includes a number of enhancements and fixes!
Support for major infrastructure and distributed task integrations
As prefect-dask
and other integrations have been added to the prefect codebase, this release adds these integrations as extra requirements of the prefect package, making it easier to install support for everything in your Prefect stack:
pip install 'prefect[dask]'
We loved this community contribution so much, we did it for all our first-party integrations:
pip install 'prefect[aws,kubernetes,dask,dbt,sqlalchemy,slack]'
You can see the full list of Prefect's extra requirements in our setup.py.
Support for timeout seconds in global concurrency context manager
You may want to fail immediately if a global concurrency slot is unavailable. Rather than block and wait, you can now specify a timeout_seconds
argument in the global concurrency context manager and catch a TimeoutError
if a slot is not available within the specified time.
@flow
def fail_immediately_flow():
try:
with concurrency("there-can-be-only-one", occupy=1, timeout_seconds=0.1):
do_something_resource_intensive()
except TimeoutError:
return Cancelled(message="Another flow run is already running")
Manage global concurrency limits via the CLI
Global concurrency limits let you control how many operations can run simultaneously-- now you can create, read, edit, and delete global concurrency limits via the Prefect CLI!
To create a new concurrency limit, use the prefect gcl create
command. You must specify a --limit
argument, and can optionally specify a --slot-decay-per-second
and --disable
argument.
prefect gcl create my-concurrency-limit --limit 5 --slot-decay-per-second 1.0
You can inspect the details of a concurrency limit using the prefect gcl inspect
command:
prefect gcl inspect my-concurrency-limit
To update a concurrency limit, use the prefect gcl update
command. You can update the --limit
, --slot-decay-per-second
, --enable
, and --disable
arguments:
prefect gcl update my-concurrency-limit --limit 10
We also have many more bug fixes and in-flight work! See the release notes for details!
Release 2.18.3
Experimental
Engine
- Wire up new engine to deployment runs — #12914
Fixes
- Fix parameters becoming unresponsive and disappearing in Prefect UI — PrefectHQ/prefect-ui-library#2355
All changes: 2.18.2...2.18.3
Release 2.18.2
💡 Providing a deployment name to flow.serve
is now optional
When running flow.serve
, you can now omit the deployment name. If you do not provide a deployment name, the deployment name will default to the name of the flow. This change makes it easier to run flows without needing to specify a deployment name each time:
@flow
def etl_flow():
pass
if __name__ == "__main__":
etl_flow.serve()
results in:
Your flow 'etl-flow' is being served and polling for scheduled runs!
To trigger a run for this flow, use the following command:
$ prefect deployment run 'etl-flow/etl-flow'
🛠✨ We've also released a few important fixes to our deployment parameter form when creating a run in the Prefect UI! 🧑🎨
🚀 This release also includes a number of other fixes and in-flight feature work. See the release notes for more details!
Release 2.18.1
Release 2.18.1
Fixes
- Fix improper context access for nested async task outside of flow — #12810
- Fix using default interval schedule in
prefect deploy
— #12833 - Handle case in
validationUpdate
schema where definitions are falsy — #12880 - Allow
prefect cloud login
to override current workspace — #12867 - Remove extra quotes in
prefect deployment run --watch
— #12894
Experimental
Events and Automations
- Support filtering by automation name:
- Add support for using the "normal" Trigger classes for
flow.serve
and.deploy
— #12789 - Add an account-level event subscriber — #12808
- Emit flow run state change events — #12825
- Emit deployment status persistence and events — #12853
- Enable event streaming from
PrefectCloudEventSubscriber
via CLI — #12796 - Update the
prefect automation delete
CLI — #12876
Engine
- Add new experimental engine for tasks and flows with improved readability and extensibility — #12856
Documentation
- Improve installation instructions — #12783
- Improve quickstart — #12798
- Migrate
prefect-azure
docs to Integrations section of the Prefect docs — #12794 - Update storage guide credentials blocks — #12819
- Remove
server
import recommendations — #12823 - Remove link to removed API page — #12824
- Add Azure Container Instances worker guide — #12846
- Improve wording on integrations index page — #12852
Prefect UI Library
- Add
FormattedDate
component to display accessible, long-form timestamps consistently - Update modal buttons and add auto-close to the parameters and job variable modals — PrefectHQ/prefect-ui-library#2320
- Add flow run list information density — PrefectHQ/prefect-ui-library#2321
- Fix "Run a deployment" action not populating the default parameters from the deployment — PrefectHQ/prefect-ui-library#2322
- Fix schema form properties with no default value from defaulting to
null
(None
) — PrefectHQ/prefect-ui-library#2323 - Update date-fns and date-fns-tz — PrefectHQ/prefect-ui-library#2319
- Use correct icon colors for non-destructive actions in the UI — PrefectHQ/prefect-ui-library#2328
Integrations
Prefect CGP
- Remove API ref to nonexistent Google Cloud Run V2 page — PrefectHQ/prefect-gcp#260
- Fix VPC access for Cloud v2 worker — PrefectHQ/prefect-gcp#266
- Handle case where
vpc
isn't in job template — PrefectHQ/prefect-gcp#267
New Contributors
- @keizobabybear made their first contribution in #12852
Release 2.18.0
Breaking Changes
- Remove deprecated ability to use
deployment.yaml
inprefect deploy
— #12731 - Remove deprecated ability to pass
-f/--flow
as option toprefect deploy
— #12732 - Remove deprecated
projects
fromprefect deploy
— #12737 - Remove deprecated
--ci
option fromprefect deploy
— #12740
Enhancements
- Improve account selection in
prefect cloud login
andworkspace set
— #12717
Full Changelog: 2.17.0...2.18.0
See the release notes for more!
Release 2.17.1
Fixes
All changes: 2.17.0...2.17.1
Release 2.17.0
🧮 Manage Prefect variables via the Python SDK
Prefect variables are useful for storing and reusing data and configuration between and across workflows; and previously you could only create and update variables via the Prefect UI. With this release, you can now get and set Prefect variables directly in your Python code with the new Variable.set
and Variable.get
methods!
For an example of reading and writing variable values in Python see the following example:
from prefect.variables import Variable
# set a variable
variable = Variable.set(name="the_answer", value="42")
# get a variable
answer = Variable.get('the_answer')
print(answer.value)
# 42
# get a variable with a default value
answer = Variable.get('not_the_answer', default='42')
print(answer.value)
# 42
# update a variable
answer = Variable.set(name="the_answer", value="43", overwrite=True)
print(answer.value)
#43
Refer to the docs for more information and see the PR for implementation details: #12596
Other Enhancements 🌟
- Allow flows inside tasks
— #12559
— #12607 - Add
User-Agent
header containing the running Prefect version — #12601 - Adds deployment version to the flow run object — #12591
... and numerous 🐛 fixes!
Full Changelog: 2.16.9...2.17.0
See the release notes for more!
Release 2.16.9
This release includes a number of enhancements and in-flight feature work.
🛠✨ One such enhancement helps streamline our CLI by adding a -jv/--job-variable
option to prefect deploy
, on par with the option available in prefect deployment run
.
🔄🔗 In terms of enhancing existing Prefect concepts, we've removed a constraint that prevented tasks from being called from other tasks. For example, this allows you to call tasks within tasks within a flow.
🗿 📉 We no longer create artifacts for unpersisted results, which should prevent an influx of entries to the artifact
table. Retried flows without persisted results will now have an error message stating that the "State data is missing" rather than referencing an "unpersisted result".
See the release notes for more details!