3.0.3: Cache Rules Everything Around Me

This release introduces improvements to cache policy configuration, offering developers more control and flexibility in managing task result caching:

• Customizable Storage Location: Specify where cache entries are stored using the key_storage parameter.
• Configurable Isolation Level: Set the isolation level for cache access with the isolation_level parameter.
• Locking Mechanism: Implement concurrent access control using the lock_manager parameter.

These enhancements allow for more sophisticated caching strategies, improving performance and resource management in complex workflows.

To learn more about these changes, check out the task caching docs.

New Features 🎉

• Add key_storage, isolation_level and lock_manager configuration to cache policies by @desertaxle in #15382

Enhancements ➕➕

• Adds settings to control server CORS configuration by @desertaxle in #15396
• Add ability to specify ConcurrencyOptions by @jeanluciano in #15291
• Add concurrency_limit field to flow.serve by @jeanluciano in #15376
• Allow a rich object to be passed to concurrency_limit in client methods by @jeanluciano in #15425
• Add support for insecure webhooks by @westford14 in #15402
• Improve deployment concurrency GCL management by @collincchoy in #15426

Bug Fixes 🐞

• Preserve flow.name with RunnerDeployment.from_storage by @zzstoatzz in #15393
• Lazy import ray to fix server startup issues by @desertaxle in #15392
• Handle unnamed lambdas for consistency by @zzstoatzz in #15400
• Fix task linking by @desertaxle in #15418
• Fix ephemeral server start command for Windows by @desertaxle in #15421
• Fix task run count when a task is configured with retry_delay_seconds by @desertaxle in #15424
• Fix bug where task cache storage is misconfigured by @desertaxle in #15433
• Update task engine to increment run_count when entering RUNNING state by @desertaxle in #15436

Integrations & Dependencies 🤝

• Add environment variables from GCP secrets to cloud run job v2 workers by @Ultramann in #15423
• Prepare prefect-redis for 0.2.0 release by @desertaxle in #15409
• Add AzureBlobStorageContainer.list_blobs method by @westford14 in #15390
• Add oracledb driver support in prefect-sqlalchemy by @Ben-Taarit in #15385

Development & Tidiness 🧹

• Add docs for configuring storage and isolation for cache policies by @desertaxle in #15397
• Improve type hinting docs for sync_compatible by @benjamincerigo in #15327
• Refactors __add__ and __sub__ on cache policies by @desertaxle in #15379
• Add mypy type checking to /models by @bunchesofdonald in #15062
• Only load collections once by @abrookins in #15344
• Bump vue from 3.5.4 to 3.5.5 in /ui by @dependabot in #15386
• Bump typescript from 5.5.4 to 5.6.2 in /ui by @dependabot in #15387
• Bump tailwindcss from 3.4.10 to 3.4.11 in /ui by @dependabot in #15388
• Bump vite from 5.4.2 to 5.4.5 in /ui by @dependabot in #15372
• Bump vue-router from 4.4.3 to 4.4.5 in /ui by @dependabot in #15389
• Update @prefecthq/prefect-ui-library to version 3.9.5 by @marvin-robot in #15411
• Bump vite from 5.4.5 to 5.4.6 in /ui by @dependabot in #15410
• Update @prefecthq/prefect-ui-library to version 3.10.0 by @marvin-robot in #15419

New Contributors

• @benjamincerigo made their first contribution in #15327
• @Ben-Taarit made their first contribution in #15385
• @westford14 made their first contribution in #15390
• @Ultramann made their first contribution in #15423

Full Changelog: 3.0.2...3.0.3

2.20.8: A name to remember

Bug Fixes 🐞

• Preserve flow.name with RunnerDeployment.from_storage by @zzstoatzz in #15394

Full Changelog: 2.20.7...2.20.8

3.0.2: Groundlog day, tomorrow finally came

3.0.2:

New Features 🎉

• Add a ResumeFlowRun automations action by @abrookins in #15269

Enhancements ➕➕

• Notification security controls by @cicdw in #15272
• Create FilesystemLockManager by @desertaxle in #15307
• Concurrency limit UX enhancements: add strict mode by @cicdw in #15297
• Make prefect_test_harness startup timeout configurable by @desertaxle in #15354
• Fix deployment-concurrency-limiting-GCL lifecycle management by @collincchoy in #15365
• Runner enforces Deployment.concurrency_limit by @jeanluciano in #15085
• Disable HTTP2 by default by @zangell44 in #15311

Bug Fixes 🐞

• Move expensive filter to AND by @cicdw in #15286
• Fix deep merge for existing env on work-pools by @zzstoatzz in #15325
• Ensure all tags are used in concurrency limiter by @cicdw in #15349
• Set propagate to false for extra loggers by @cicdw in #15348
• Allow child transactions to access parent values set after child transaction opens by @desertaxle in #15342
• Fix resolution of implicit relative import when using safe_load_namespace by @desertaxle in #15358
• Correctly handles recurring automations and future events by @chrisguidry in #15355

Integrations & Dependencies 🤝

• reintroduce job model for prefect-databricks by @zzstoatzz in #15313
• Update DbtCoreOperation docstring by @zzstoatzz in #15317
• Move import from google into import guard by @desertaxle in #15334
• fix S3Bucket.download_object_to_path by @zzstoatzz in #15293
• Fix issue where DaskTaskRunner was attempting to create a duplicate cluster by @desertaxle in #15341

Development & Tidiness 🧹

• Add dedicated enhancements section by @cicdw in #15265
• Fix labeler job by @cicdw in #15114
• Update @prefecthq/prefect-ui-library to version 3.8.2 by @marvin-robot in #15283
• Refactor some concurrency utilities to be sync_compatible by @cicdw in #15273
• Add some more nuance to labeler by @cicdw in #15288
• Use ResultStore by default for task transactions by @desertaxle in #15264
• Update @prefecthq/prefect-ui-library to version 3.8.3 by @marvin-robot in #15305
• Remove feature flag for nested task runs on the run graph by @pleek91 in #15304
• Add deprecation warnings to all RecordStore implementations by @desertaxle in #15314
• Remove some ancient deprecations by @cicdw in #15310
• Update @prefecthq/prefect-ui-library to version 3.9.1 by @marvin-robot in #15335
• Replace RedisRecordStore with RedisLockManager by @desertaxle in #15320
• Add deprecation warnings to results classes and methods by @desertaxle in #15323
• Update @prefecthq/prefect-ui-library to version 3.9.2 by @marvin-robot in #15346
• Update @prefecthq/prefect-ui-library to version 3.9.4 by @marvin-robot in #15356
• Bump vue from 3.4.38 to 3.5.4 in /ui by @dependabot in #15306
• Bump @vitejs/plugin-vue from 5.1.2 to 5.1.3 in /ui by @dependabot in #15173
• Bump vue-tsc from 2.0.29 to 2.1.6 in /ui by @dependabot in #15281
• Update transactions docs by @discdiver in #15299
• Add unit test for accessing self as normal arg when setting task run name by @jlowin in #15324
• add for_each trigger example by @zzstoatzz in #15326
• Update logging.mdx to fix title display by @discdiver in #15360
• quote type checked value by @zzstoatzz in #15362
• fix: move the github conditional to the appropriate level by @jamiezieziula in #15366
• Fix broken image link and update formatting in prefect-dbt by @discdiver in #15367
• Update integrations docs by @zzstoatzz in #15303
• Rewrite block page in docs by @cicdw in #15254
• chore: only trigger helm release workflows on 3.x.x version releases by @jamiezieziula in #15268
• better common error section by @zzstoatzz in #15271
• fix inconsistent word in upgrade guide by @zzstoatzz in #15277
• Minor edits to the upgrade and what's new docs by @discdiver in #15285

Full Changelog: 3.0.1...3.0.2

2.20.7: Big Data London is Next Week, see you there?

Enhancements ➕➕

• Dont propagate extra loggers through root logger by @cicdw in #15357
• 2.x add cached completion hooks by @derekahuang in #15270
• update environment variable deep merge logic for env by @zzstoatzz in #15363
• Disable HTTP2 by default by @zangell44 in #15312

Bug Fixes 🐞

• Fix query for subflow cancellation by @cicdw in #15292
• Fix resolution of implicit relative import when using safe_load_namespace by @desertaxle in #15361

Integrations & Dependencies 🤝

• Update DbtCoreOperation docstring 2.x by @zzstoatzz in #15318
• [2.x] add function name to prefect-docker cache key by @zzstoatzz in #15263
• Move import from google into import guard by @desertaxle in #15333
• reintroduce job model for prefect-databricks in 2.x by @zzstoatzz in #15315

Development & Tidiness 🧹

• Fix labeler job by @cicdw in #15113
• Reduce test matrix by @cicdw in #15294

New Contributors

• @derekahuang made their first contribution in #15270

Full Changelog: 2.20.6...2.20.7

3.0.1: The Director's Cut

3.0.1

This release contains a large collection of relatively small improvements and bug fixes on the back of our major 3.0.0 release. One particular set of improvements that I'd like to focus on are enhancements to server side logging and event handling. We received a few reports of task run states not showing up in the UI/API. It's important to note that events - and now task run states! - require a websocket connection to the API (the good news is that this will not disrupt workflow execution). Websockets are sometimes blocked by default in firewalls and network configuration, so we are working to raise failures to establish such a connection more prominently and potentially allow for alternative configurations. If you notice any issues with this, please let us know!

Enhancements ➕➕

• update prompt_select_from_table by @zzstoatzz in #15202
• Adds ability to write result metadata separate from data when using ResultStore by @desertaxle in #15214
• Remove CausalOrdering from task-run-recorder by @jakekaplan in #15244
• Disable griffe logging entirely when used by @cicdw in #15193
• Allow using a ResultStore with a Transaction by @desertaxle in #15247
• Introduce LockManager protocol and add locking abilities to ResultStore by @desertaxle in #15225
• check api compatibility when entering a client context by @jakekaplan in #15252
• Add debug logging for server side event handling by @jakekaplan in #15245

Bug Fixes 🐞

• Fix task key comp by @cicdw in #15190
• Fix for Github clone on Windows by @WWM-jschuba in #15143
• remove duplicate pull_code instruction by @zzstoatzz in #15215
• add function name to prefect-docker cache key by @zzstoatzz in #15262
• Deployment.concurrency_limit cannot be zero. by @collincchoy in #15228

Integrations & Dependencies 🤝

• Add KeepAliveClientRequest class for k8s async client by @kevingrismore in #15220
• Kubernetes worker uses enable_socket_keep_alive by @jeanluciano in #15142
• prefect-kubernetes: add missing config loaders by @srsapient in #15209
• fix prefect-azure tests by @zzstoatzz in #15211

Development & Tidiness 🧹

• Fix test flake and add uv dev dependency by @cicdw in #15251
• Fix release pipeline for integration packages by @desertaxle in #15189
• Touch up variable class inheritance by @cicdw in #15188
• Update @prefecthq/prefect-ui-library to version 3.8.1 by @marvin-robot in #15243
• Fix test failures for pydantic==2.9.0 by @desertaxle in #15249
• use is not for type comparison instead of != by @zzstoatzz in #15253
• Updates the worker integration tests for post-GA by @chrisguidry in #15199

New Contributors

• @WWM-jschuba made their first contribution in #15143
• @srsapient made their first contribution in #15209

Full Changelog: 3.0.0...3.0.1

2.20.6: An extra release to fix a problem with extras

This release resolves an important issue that can occur when installing 2.x versions of prefect: because all prefect-{integration} packages also have 3.0 compatible versions, it's important to restrict the upper bounds of those packages alongside the needed upper bound on prefect itself. Please reference this list for a full catalogue of upper bounds that keep your integration package 2.x compatible.

To help alleviate this potential issue, this release applies these upper bounds automatically when installing packages via extras, i.e., prefect[kubernetes]<3.0.0 or prefect[aws]<3.0.0.

In addition, it fixes an issue with -kubernetes flavored Docker images on the 2.20.5 release that was affected by this incompatibility.

Bug Fixes 🐞

• Add upper bounds to extras in setup.py by @desertaxle in #15256
• Use kubernetes extra when building Docker image by @desertaxle in #15257

Full Changelog: 2.20.5...2.20.6

2.20.5: Several Types of Small Code Fixes Gathered Together in a Release and Grooving with a K8s Pod

Several Types of Small Code Fixes Gathered Together in a Release and Grooving with a K8s Pod

Bug Fixes 🐞

• Disable griffe logging entirely when used by @cicdw in #15194
• Fix task key computation by @cicdw in #15191
• backport windows quote fix by @zzstoatzz in #15201

Integrations & Dependencies 🤝

• restore job submission handler in jobs_runs_submit_and_wait_for_completion by @zzstoatzz in #15183
• prefect-kubernetes: add missing config loaders by @desertaxle in #15218
• Use prefect extras when adhoc installing integrations by @desertaxle in #15238
• Add KeepAliveClientRequest class for k8s async client 2.x by @kevingrismore in #15230

Development & Tidiness 🧹

• Fix tests for new apprise version by @cicdw in #15186
• Add active slots to concurrency docs by @abrookins in #15161
• 2.x variable return value fix by @Samreay in #15070
• Update docs header link and language for 3.0 GA by @discdiver in #15166
• Update install instructions in 2.x docs by @discdiver in #15213

New Contributors

• @Samreay made their first contribution in #15070

Full Changelog: 2.20.4...2.20.5

3.0.0

Prefect 3.0.0

For more information see our blog post on 3.0.

New Contributors

Thank you to all the new contributors who helped make 3.0 happen!

@jaraics, @mitchnielsen, @swadhinbiswas, @elisalimli, @LarryUllman, @djsauble, @Waiwait, @mthatt, @rooperuu, @tonykipkemboi, @ravi77o, @ZhipengXue97, @bjorhn, @50Byte, @taranl, @peterbygrave, @s, @jongbinjung, @GalLadislav, @glesperance, @nialloriordanroo, and @tmylt

Full Changelog: 2.19.0...3.0.0

Weekly Release Candidate 3.0.0rc20

What's Changed

New Features 🎉

• allow JSON values in Secret block by @zzstoatzz in #14980
• Workers enforce Deployment.concurrency_limits by @jeanluciano in #15022
• Improve logging on result serialization errors by @desertaxle in #15084
• record task run followers by @jakekaplan in #15100
• Switch to profile with matching PREFECT_API_URL on server start by @desertaxle in #15122

Bug Fixes 🐞

• Revert change to defer building the server-side version of PrefectBaseModel by @desertaxle in #15072
• Fixes prefect server start for Windows machines by @desertaxle in #15103
• Fix CloudUnauthorized case in prefect dashboard open by @zzstoatzz in #15106
• ensure on_failure hook runs upon parameter validation error by @zzstoatzz in #15109
• Update model_rebuild in main.py to explicitly pass namespace types by @desertaxle in #15117

Development & Tidiness 🧹

• Avoid result factories if persistence is turned off for a task by @cicdw in #15055
• Remove UnpersistedResult by @cicdw in #15056
• Replace PersistedResultBlob with ResultRecord by @desertaxle in #15064
• Update @prefecthq/prefect-ui-library to version 3.7.2 by @marvin-robot in #15082
• a couple typing and error improvements by @zzstoatzz in #15081
• Bump vite from 5.4.1 to 5.4.2 in /ui by @dependabot in #15068
• Bump micromatch from 4.0.5 to 4.0.8 in /ui by @dependabot in #15083
• Update @prefecthq/prefect-ui-library to version 3.7.3 by @marvin-robot in #15097
• Updates warning message when result persistence fails by @desertaxle in #15099
• fix typing and docstring for concurrency ctx managers by @zzstoatzz in #15108
• Remove unused Manifest class by @desertaxle in #15111
• Update @prefecthq/prefect-ui-library to version 3.7.5 by @marvin-robot in #15116
• Update @prefecthq/prefect-ui-library to version 3.7.6 by @marvin-robot in #15125
• Update @prefecthq/prefect-ui-library to version 3.7.7 by @marvin-robot in #15126
• annotation type fixes by @zzstoatzz in #15124

Uncategorized

• Test experimental_defer_build_mode with defer_build=True by @abrookins in #15052
• Readme updates: formatting typo and messaging updates by @sarahmk125 in #14952
• bugfix for sorting scheduled flow_runs by @tmylt in #14993
• Add whats new and upgrade guide by @jlowin in #14996
• add prefect dashboard open to open dashboard from CLI by @zzstoatzz in #14985
• simplify on_crash hook example by @zzstoatzz in #15079
• improve Manage concurrency docs by @zzstoatzz in #15088
• Quotes the components of a PostgreSQL connection URL when built from components by @chrisguidry in #15094
• minor docs formatting fixes by @zzstoatzz in #15105
• Update artifacts doc page by @discdiver in #15093
• Update language in quickstart by @cicdw in #15115
• Update write tasks page for clarity by @desertaxle in #15118
• Updates to logging page for clarity by @desertaxle in #15119
• Go through contributing docs by @cicdw in #15120
• Add Prefect Cloud IP Allowlist CLI by @collincchoy in #15087
• bubble up hydration placeholders by @jakekaplan in #15132
• Update work pools doc for flow and clarity by @desertaxle in #15127

New Contributors

• @tmylt made their first contribution in #14993

Full Changelog: 3.0.0rc19...3.0.0rc20

2.20.4: We Held a Funeral for the Bugs. No One Came

2.20.4: We Held a Funeral for the Bugs. No One Came

Bug Fixes 🐞

• Fix prefect server start for Windows machines by @desertaxle in #15104
• ensure on_failure hook runs upon parameter validation error by @zzstoatzz in #15128

Integrations & Dependencies 🤝

• backport prefect-kubernetes fix by @zzstoatzz in #15095

Full Changelog: 2.20.3...2.20.4