Weekly Release Candidate 3.0.0rc19

What's Changed

New Features 🎉

• allow overwrite in create_work_pool by @zzstoatzz in #14967
• Add ability to configure isolation level for a transaction by @desertaxle in #14939
• Add RedisRecordStore by @desertaxle in #14974
• Add FileSystemRecordStore by @desertaxle in #14966
• Adds concurrency_limit to flow.deploy by @jeanluciano in #15001
• Adds concurrency_limit to prefect.yaml by @jeanluciano in #15007
• Adds concurrency limit options to prefect deploy by @jeanluciano in #15005
• Adds the ability to run a server in the background by @desertaxle in #15024
• Add concurrency helpers that use v1 task run concurrency limits by @abrookins in #15037

Bug Fixes 🐞

• Ensure Python executable is quoted in runner command on Windows by @desertaxle in #15023
• Add Block Type registration for its nested block types in field annotations by @GalLadislav in #15011

Integrations & Dependencies 🤝

• Update griffe requirements and corresponding imports by @cicdw in #14999
• Add some missing upper bounds to requirements by @cicdw in #15002
• Update websockets requirement from <13.0,>=10.4 to >=10.4,<14.0 by @dependabot in #15026

Development & Tidiness 🧹

• Remove importlib-resources as an explicit requirement by @cicdw in #14960
• Migrates prefect-redis to the core repository by @desertaxle in #14968
• Bump vue from 3.4.37 to 3.4.38 in /ui by @dependabot in #14988
• Bump vite from 5.4.0 to 5.4.1 in /ui by @dependabot in #14989
• Bump tailwindcss from 3.4.9 to 3.4.10 in /ui by @dependabot in #14990
• Add a few more label rules to help keep things organized by @cicdw in #15000
• Go through some known dead code and remove it by @cicdw in #15006

Uncategorized

• enable client side task run orchestration by default by @jakekaplan in #14913
• log message while events are still draining by @jakekaplan in #14964
• disable task run recorder in tests by @jakekaplan in #14969
• Remove Markdown stubs for SDK docs by @discdiver in #14971
• Remove duplicate griffe requirement by @desertaxle in #14982
• Ensures that server automations schemas are parsed with only server models by @chrisguidry in #14995
• use less clients by @jakekaplan in #14986
• Adds concurrency limit support to client.create_deployment by @jeanluciano in #14997
• README: fix link to Slack community by @mitchnielsen in #15004
• Add section on client-side task run orchestration to upgrade guide by @aaazzam in #15016
• Add section on client-side task run orchestration to write tasks guide. by @aaazzam in #15017
• Adds docs redirects by @discdiver in #15030
• Fix link in docs by @discdiver in #15036
• Adds acquire/release endpoints for task run concurrency limits by @chrisguidry in #15038
• Adds a 2.x version to the v3 docs with a page that directs visitors to docs-2.prefect.io by @discdiver in #15040
• Remove Redis record store settings by @desertaxle in #15039
• Performance: Add defer_build to PrefectBaseModel by @aaazzam in #15046

Full Changelog: 3.0.0rc18...3.0.0rc19

2.20.3: Background to the future

2.20.3: Background to the future

Two items of note in this release:

A note on dependencies: Last week, a major version upgrade of one of Prefect's dependencies (griffe) caused quite the headache for some users' environments, and we are sorry for the inconvenience this caused; in general, we strive to keep major version upper bounds on all of our dependencies, but an oversight caused this dependency to go without an upper bound. This release fixes that issue and added in a few other upper bounds that weren't present.

prefect server start --background: By popular request, this release features a new flag on prefect server start that runs the server in a background process. When starting a background server, a server.pid file is written to the $PREFECT_HOME directory with the process ID of the background server. This lets prefect server stop know which process to interrupt. Currently, only one background server can be started at a time.

New Features 🎉

• Adds the ability to run a server in the background by @desertaxle in #15013
• Backport --overwrite for create_work_pool by @zzstoatzz in #14977
• Add Block Type registration for its nested block types in field annotations by @GalLadislav in #15032

Bug Fixes 🐞

• Ensure Python executable is quoted in runner command on Windows by @desertaxle in #15015
• Remove duplicate griffe requirement by @desertaxle in #14983

Integrations & Dependencies 🤝

• Unbound griffe to 2.0 by @cicdw in #14998
• Upper bound critical requirements by @cicdw in #15003
• Update websockets requirement from <13.0,>=10.4 to >=10.4,<14.0 by @dependabot in #15025
• Update importlib-resources requirement from <6.2.0,>=6.1.3 to >=6.1.3,<6.5.0 by @dependabot in #15041

Full Changelog: 2.20.2...2.20.3

Weekly Release Candidate 3.0.0rc18

What's Changed

New Features 🎉

• Add MemoryRecordStore by @desertaxle in #14919
• Add is_null_ filter to logs filter endpoint by @znicholasbrown in #14930
• Add a deployment concurrency limit field by @jeanluciano in #14929

Development & Tidiness 🧹

• Update @prefecthq/prefect-design to version 2.12.0 by @marvin-robot in #14940
• Update @prefecthq/prefect-design to version 2.12.1 by @marvin-robot in #14941
• Update @prefecthq/prefect-ui-library to version 3.7.1 by @marvin-robot in #14942
• clearer message upon no-op prefect block register by @zzstoatzz in #14949
• add constrained types for select WorkQueue action fields by @zzstoatzz in #14953
• use obfuscate instead of obfuscate_string by @zzstoatzz in #14959

Uncategorized

• Fix error code when work pool name not found by @nialloriordanroo in #14935
• Docs slug redirect and consistency work by @discdiver in #14956
• Fixes two broken links in docs by @discdiver in #14958
• let dynamic key be more dynamic for client side task run orchestration by @jakekaplan in #14962

New Contributors

• @nialloriordanroo made their first contribution in #14935

Full Changelog: 3.0.0rc17...3.0.0rc18

Weekly Release Candidate 3.0.0rc17

What's Changed

New Features 🎉

• Avoid __getitem__ calls on a PrefectFuture or State in checking Flow parameter serialization by @Andrew-S-Rosen in #14900
• Allow for defaults on txn.get by @cicdw in #14925
• Transactions should inherit data from their parents by @cicdw in #14924

Bug Fixes 🐞

• correctly call is_interactive function during server preflight check by @zzstoatzz in #14909
• fallback TaskSource key computation if no source code by @zzstoatzz in #14926
• Deep merge env vars by @cicdw in #14932

Development & Tidiness 🧹

• Update @prefecthq/prefect-ui-library to version 3.5.12 by @marvin-robot in #14907
• Update @prefecthq/prefect-ui-library to version 3.5.14 by @marvin-robot in #14910
• Update @prefecthq/prefect-ui-library to version 3.5.17 by @marvin-robot in #14911
• Update @prefecthq/prefect-ui-library to version 3.5.18 by @marvin-robot in #14912
• Update @prefecthq/prefect-ui-library to version 3.6.0 by @marvin-robot in #14915
• Update @prefecthq/prefect-ui-library to version 3.6.1 by @marvin-robot in #14916
• Update @prefecthq/prefect-ui-library to version 3.7.0 by @marvin-robot in #14928
• Fix flaky log tests by @desertaxle in #14931

Uncategorized

• enable client side orchestration for test_futures.py by @jakekaplan in #14902
• enable cstro for test_task_results.py by @jakekaplan in #14904
• enable cstro for task runner tests by @jakekaplan in #14867
• Chore: Bump axios to v1.7.4 by @znicholasbrown in #14917
• enable cstro in test_tasks.py by @jakekaplan in #14905

Full Changelog: 3.0.0rc16...3.0.0rc17

2.20.2

2.20.2: Save the Environment (Variables)

This release contains many bug fixes and small enhancements; two that I want to highlight are:

• a fix to KubernetesJob.publish_as_work_pool: this method helps users migrate from agents to workers by exposing a method to convert an infrastructure block to a structured work pool. Work Pools allow for better governance of infrastructure, and will be easier to observe and audit than agents.
• a fix to how environment variables on work pools are handled: specifically, users who provided per-flow-run environment variable overrides found that any other environment variables explicitly set on their work pool or deployment were being removed. This fix ensures that all environment variable configuration is carefully merged, preventing the loss of any variables at runtime.

New Features 🎉

• Flow version in runtime by @zzstoatzz in #14898

Bug Fixes 🐞

• Allow for null command when building k8s job by @cicdw in #14920
• First order deep merge of flow run overrides with deployment overrides by @cicdw in #14923
• Avoid raising errors on database passwords that contain a $ character by @desertaxle in #14888

Development & Tidiness 🧹

• Smoke test prefect-client across all supported Python versions by @desertaxle in #14872
• Update 2.x codeowners by @cicdw in #14891
• Docker step caching 2.x by @zzstoatzz in #14894

Integrations & Dependencies 🤝

• allow passing stream_output to dbt tasks by @zzstoatzz in #14866

Full Changelog: 2.20.1...2.20.2

Weekly Release Candidate 3.0.0rc16

What's Changed

New Features 🎉

• prefect-dbt - Clarify missing profile error by @seanpwlms in #14868
• Sharing state with transaction hooks by @cicdw in #14895
• Make flow version available in runtime by @zzstoatzz in #14897

Bug Fixes 🐞

• Avoid raising errors on database passwords that contain a $ character by @desertaxle in #14876

Development & Tidiness 🧹

• Update @prefecthq/prefect-ui-library to version 3.5.11 by @marvin-robot in #14847
• remove Deployment.is_schedule_active and Deployment.schedule by @jakekaplan in #14794
• Consolidate where transaction hooks are run by @cicdw in #14877
• Bump tailwindcss from 3.4.7 to 3.4.9 in /ui by @dependabot in #14885
• Bump vue from 3.4.35 to 3.4.37 in /ui by @dependabot in #14882
• Bump vite from 5.3.5 to 5.4.0 in /ui by @dependabot in #14884
• Bump vue-router from 4.4.2 to 4.4.3 in /ui by @dependabot in #14883

Uncategorized

• fix docs typos by @zzstoatzz in #14874
• Allow schedules: null in prefect.yaml by @zzstoatzz in #14873
• Typo fix: say_hello.submit(name="Marvin) by @Andrew-S-Rosen in #14880
• update CODEOWNERS by @cicdw in #14889
• Update link text and remove unnecessary folder by @discdiver in #14887
• tweak task runner docs by @zzstoatzz in #14879
• Minor updates to Quickstart by @discdiver in #14860
• Ensure global codeowner reviews are accepted for specific files by @cicdw in #14893

Full Changelog: 3.0.0rc15...3.0.0rc16

2.20.1: pip, pip hooray!

This release is small and includes fixes for various dependency issues, as well as a fix for submodule handling in git-based deployments.

Python 3.11 introduced the concept of "exception groups" for situations in which multiple unrelated exceptions need to be raised - for example, in concurrent execution models or retrying an operation across distinct error modes.
Prefect has begun to leverage this capability, and we rely on a separate package (exceptiongroup) which backports
this functionality to lower supported versions of Python. This dependency was missing from client-side requirements in 2.20.0.

Bug Fixes 🐞

• Persist include-submodules flag by @cicdw in #14839
• Move exceptiongroup to requirements-client.txt by @desertaxle in #14871

Integrations & Dependencies 🤝

• Fix integrations for AnyIO 4 by @abrookins in #14827

Development & Tidiness 🧹

• Backport nullable flow_run_id in Log table by @zzstoatzz in #14830

Weekly Release Candidate 3.0.0rc15

What's Changed

New Features 🎉

• Optimize value of checkpoint_every by @bunchesofdonald in #14834
• Allow starting an ephemeral server when emitting or subscribing to events by @desertaxle in #14826
• fix task-run.Scheduled event emission by @jakekaplan in #14849

Bug Fixes 🐞

• Kubernetes worker config fixes by @jeanluciano in #14813
• Persist include-submodules flag when using GitStorage by @cicdw in #14840
• Remove wrapt and fail on incorrect await usage by @zzstoatzz in #14837
• Flush logs when exiting prefect_test_harness to avoid error logs by @desertaxle in #14854
• Enable tasks to be submitted and mapped from tasks submitted to Dask or Ray by @desertaxle in #14829

Development & Tidiness 🧹

• Bump vue-router from 4.4.0 to 4.4.2 in /ui by @dependabot in #14841
• Bump @vitejs/plugin-vue from 5.1.1 to 5.1.2 in /ui by @dependabot in #14843
• Bump autoprefixer from 10.4.19 to 10.4.20 in /ui by @dependabot in #14842
• Bump vue from 3.4.34 to 3.4.35 in /ui by @dependabot in #14844
• Update pytest requirement from <8,>7 to >7,<9 by @dependabot in #14806
• Fix syntax for client test suite pytest command by @desertaxle in #14856
• A couple minor typing fixes by @zzstoatzz in #14859

Uncategorized

• make flow_run_id nullable in Log table by @zzstoatzz in #14828
• Add test profile to default profiles by @desertaxle in #14836
• Client side task run orchestration - link states to results by @jakekaplan in #14822
• Allows for specifying database connection elements in separate settings by @chrisguidry in #14825
• Cleaner exit when Ctrl-C'ing Flow.serve by @zzstoatzz in #14848
• update task run event tests for client side orchestration by @jakekaplan in #14850
• ensure flow engine tests support client side task run orchestration by @jakekaplan in #14852

Full Changelog: 3.0.0rc14...3.0.0rc15

Weekly Release Candidate 3.0.0rc14

What's Changed

New Features 🎉

• Adds as_completed utility for PrefectFuture by @jeanluciano in #14641
• Create an AsyncTaskRunEngine and wire it up to run async tasks by @bunchesofdonald in #14743
• Move tag-based concurrency management into clients by @abrookins in #14382
• Flow run logs csv download by @pleek91 in #14703
• Allow blocks in deployment params by @GalLadislav in #14741
• Transition rolled back tasks into Completed(name="RolledBack") by @collincchoy in #14721
• Move try/finally block to allow events to be emitted when concurrency is released by @bunchesofdonald in #14793
• Return early from concurrency() and rate_limit() without limit names by @abrookins in #14789
• Automatically run a server in a subprocess if no API URL is specified by @desertaxle in #14722

Bug Fixes 🐞

• Microsoft Teams Webhook Support for Power Automate (Workflows) by @zzstoatzz in #14771
• Add support for custom flow decorators to prefect deploy by @desertaxle in #14694
• Fix:load incluster config await removed by @jeanluciano in #14797
• Fix setting of service account on Vertex AI jobs by @desertaxle in #14802
• Waits for task runs in as completed test by @jeanluciano in #14820
• fix: test-harness was breaking w 3.0 RC (closes #14483) by @glesperance in #14798
• Remove PrefectHttpxSyncEphemeralClient by @zzstoatzz in #14823

Development & Tidiness 🧹

• Bump CodSpeedHQ/action from 2 to 3 by @dependabot in #14778
• Bump @vitejs/plugin-vue from 5.1.0 to 5.1.1 in /ui by @dependabot in #14777
• Bump tailwindcss from 3.4.6 to 3.4.7 in /ui by @dependabot in #14776
• Update @prefecthq/prefect-ui-library to version 3.5.7 by @marvin-robot in #14792
• Update @prefecthq/prefect-ui-library to version 3.5.8 by @marvin-robot in #14796
• Rename the /download-logs/csv endpoint to /logs/download by @pleek91 in #14817
• Update @prefecthq/prefect-ui-library to version 3.5.9 by @marvin-robot in #14824

Uncategorized

• Pause compat tests by @cicdw in #14773
• only use concurrency when tasks have tags by @jakekaplan in #14781
• Adds Prometheus metrics for API and client by @chrisguidry in #14783
• Adds metrics for client-side event emission and subscription by @chrisguidry in #14785
• Make flow/task engine release slots after a timeout by @bunchesofdonald in #14787
• add prefect profiles populate-defaults CLI command by @zzstoatzz in #14749
• Remove extra backtick in deployment docs by @mitchnielsen in #14791
• Follow up on prefect profile populate-defaults by @zzstoatzz in #14795
• Remove extra future docs folder by @discdiver in #14811
• Add note about PRAGMA temp_store = memory in sqlite setup by @zzstoatzz in #14812
• Update prefect_test_harness to run a test server in a subprocess by @desertaxle in #14818

New Contributors

• @GalLadislav made their first contribution in #14741
• @glesperance made their first contribution in #14798

Full Changelog: 3.0.0rc13...3.0.0rc14

2.20.0: CVE you later

Continuing the 2.0 lineage, Prefect's 2.20 release includes a major compatibility upgrade with anyio: previous versions of anyio were subject to a critical race condition that many CVE scanners are flagging as a high severity vulnerability. Because of the changes necessary in this upgrade, please be aware that new versions of many Prefect integration packages such as prefect-dask and prefect-ray will require 2.20.0 to function properly.

(For those who have already upgraded to 3.0, don't worry - Prefect 3.0 release candidates are already anyio>=4.4.0 compatible).

Bug Fixes 🐞

• refactors how cluster_config is implemented by @jeanluciano in #14801
• Add support for custom flow decorators to prefect deploy by @desertaxle in #14782

Integrations & Dependencies 🤝

• Bump anyio on Prefect 2.x by @abrookins in #14599
• Fix setting of service account on Vertex AI jobs by @desertaxle in #14816
• Backport teams notif update for 2.x by @zzstoatzz in #14774
• fix teams webhook block syntax for 3.8 by @zzstoatzz in #14790

Development & Tidiness 🧹

• Remove unnecessary config for releases by @cicdw in #14763

Full Changelog: 2.19.9...2.20