From 7319ea6e5387c022fb24560c73a8d34c25ea009a Mon Sep 17 00:00:00 2001 From: Maran Palaniappan Date: Fri, 17 Feb 2023 14:57:29 -0800 Subject: [PATCH] Update ReadMe and Package information --- README.md | 174 +++++++++++++----- ...soft.Azure.WebJobs.Extensions.Redis.csproj | 23 ++- 2 files changed, 149 insertions(+), 48 deletions(-) diff --git a/README.md b/README.md index 2f7f721..c2a1c7a 100644 --- a/README.md +++ b/README.md @@ -2,66 +2,154 @@ # Redis Extension for Azure Functions ## Introduction +This repository contains the triggers and bindings to use in your [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-get-started) and [WebJobs](https://learn.microsoft.com/azure/app-service/webjobs-sdk-how-to). +There are three triggers in the Azure Functions Redis Extension: +- `RedisPubSubTrigger` triggers on [Redis pubsub messages](https://redis.io/docs/manual/pubsub/) +- `RedisListsTrigger` triggers on [Redis lists](https://redis.io/docs/data-types/lists/) +- `RedisStreamsTrigger` triggers on [Redis streams](https://redis.io/docs/data-types/streams/) -This repository contains the Redis trigger bindings to use in your [Azure Functions](https://learn.microsoft.com/azure/azure-functions/functions-get-started) and [WebJobs](https://learn.microsoft.com/azure/app-service/webjobs-sdk-how-to). The trigger binding enables invoking a function when message is received on a [Redis PubSub channel](https://redis.io/docs/manual/pubsub/) or on a [KeySpace or KeyEvent Notification](https://redis.io/docs/manual/keyspace-notifications/). +## Getting Started +1. [Set up an Azure Cache for Redis instance](https://learn.microsoft.com/azure/azure-cache-for-redis/quickstart-create-redis) or [install Redis locally](https://redis.io/download/). +1. Install the [Azure Functions Core Tools](https://learn.microsoft.com/azure/azure-functions/functions-run-local). +1. Create a function project for .NET: + ```cmd + mkdir RedisFunctions + cd RedisFunctions + func init --worker-runtime dotnet + ``` +1. Install the Redis Extension using `dotnet add package Microsoft.Azure.WebJobs.Extensions.Redis --prerelease`. +For private preview, these are the following steps to add the nuget package to your project: + 1. Create a `NuGet.Config` file in the project folder (`RedisFunctions` in the above step): + ``` + + + + + + + ``` + 1. Add the following line to the `` section of the csproj. + ``` + $(RestoreSources);./local-packages;https://api.nuget.org/v3/index.json + ``` + 1. Create a folder `local-packages` within the project folder, and download the latest NuGet package from [GitHub Releases](https://github.com/Azure/azure-functions-redis-extension/releases) to this `local-packages` folder. + 1. Install the package: + ``` + dotnet add package Microsoft.Azure.WebJobs.Extensions.Redis --prerelease + dotnet restore + ``` -The Azure Functions Redis Extension allows you to trigger functions based on Redis pub/sub messages, Redis Keyspace notifications, and Redis Keyevent notifications from your Azure Redis Cache all via the [Azure functions interface](https://docs.microsoft.com/azure/azure-functions/functions-create-your-first-function-visual-studio). +## Usage +### `RedisPubSubTrigger` +The `RedisPubSubTrigger` subscribes to a specific channel pattern using [`PSUBSCRIBE`](https://redis.io/commands/psubscribe/), and surfaces messages received on those channels to the function. -To get started with developing with this extension, make sure you first [set up an Azure Cache for Redis instance](https://docs.microsoft.com/azure/azure-cache-for-redis/quickstart-create-redis). Then you can go ahead and begin developing your functions. +> **Warning** +> This trigger is not fully supported on a [Consumption plan](https://learn.microsoft.com/azure/azure-functions/consumption-plan) because Redis PubSub requires clients to always be actively listening to receive all messages. +> For consumption plans, there is a chance your function may miss certain messages published to the channel. Functions with this trigger shuld also not be scaled out. -## Usage -### RedisTrigger Parameters -Input: RedisTrigger takes in three arguments: -- `ConnectionString`: connection string to the redis cache (eg `.redis.cache.windows.net:6380,password=...`) -- `TriggerType`: This is an `RedisTriggerType` enum that can be one of three values: `PubSub`, `KeySpace`, or `KeyEvent` -- `Trigger`: - - If `TriggerType` is `PubSub`, this is the name of the pubsub channel on which to trigger - - If `TriggerType` is `KeySpace`, this is the key on which to trigger - - If `TriggerType` is `KeyEvent`, this is the keyevent on which to trigger - -The RedisTrigger returns a `RedisMessageModel` object that has three fields: +> **Note** +> In general, functions with this the `RedisPubSubTrigger` should not be scaled out to multiple instances. +> Each instance will listen and process each pubsub message, resulting in duplicate processing. + +#### Inputs +- `ConnectionString`: connection string to the redis cache (eg `.redis.cache.windows.net:6380,password=...`). +- `Channel`: name of the pubsub channel that the trigger should listen to. + +#### Sample +The following sample listens to the channel "channel" at a localhost Redis instance at "127.0.0.1:6379" ```c# -namespace Microsoft.Azure.WebJobs.Extensions.Redis +[FunctionName(nameof(PubSubTrigger))] +public static void PubSubTrigger( + [RedisPubSubTrigger(ConnectionString = "127.0.0.1:6379", Channel = "channel")] RedisMessageModel model, + ILogger logger) { - public class RedisMessageModel - { - public RedisTriggerType TriggerType { get; set; } - public string Trigger { get; set; } - public string Message { get; set; } - } + logger.LogInformation(JsonSerializer.Serialize(model)); } ``` -- `TriggerType`: The `RedisTriggerType` that the function uses -- `Trigger`: The pubsub channel, key, or keyevent on which the function was triggered -- `Message`: - - If `TriggerType` is `PubSub`, this will be the message received by the pubsub channel - - If `TriggerType` is `KeySpace`, this will be the keyevent that occurred on the key - - If `TriggerType` is `KeyEvent`, this will be the key on which the keyevent occurred -> **Note** -> -> If the input `Trigger` value is a glob pattern, the output `Trigger` value will be the exact pubsub channel, key, or keyevent, not the input `Trigger` value. -> -> For example, if the inputs to the function are `TriggerType=PubSub, Trigger="test*"`, and a message is published to the channel `"test2"`, the `Trigger` in the `RedisCacheMessageModel` will be `"test2"`. +### `RedisListsTrigger` +The `RedisListsTrigger` pops elements from a list and surfaces those elements to the function. The trigger polls Redis at a configurable fixed interval, and uses [`LPOP`](https://redis.io/commands/lpop/)/[`RPOP`](https://redis.io/commands/rpop/)/[`LMPOP`](https://redis.io/commands/lmpop/) to pop elements from the lists. -### Example Function -The following example shows a [C# function](https://learn.microsoft.com/azure/azure-functions/functions-dotnet-class-library) that gets invoked (by virtue of the trigger binding) when a message is published to a Redis channel queue named `channel`. +Inputs: +- `ConnectionString`: connection string to the redis cache (eg `.redis.cache.windows.net:6380,password=...`). +- `Keys`: Keys to read from, space-delimited. + - Multiple keys only supported on Redis 7.0+ using [`LMPOP`](https://redis.io/commands/lmpop/). + - Listens to only the first key given in the argument using [`LPOP`](https://redis.io/commands/lpop/)/[`RPOP`](https://redis.io/commands/rpop/) on Redis versions less than 7.0. +- (optional) `PollingIntervalInMs`: How often to poll Redis in milliseconds. + - Default: 1000 +- (optional) `MessagesPerWorker`: How many messages each functions worker "should" process. Used to determine how many workers the function should scale to. + - Default: 100 +- (optional) `BatchSize`: Number of elements to pull from Redis at one time. + - Default: 10 + - Only supported on Redis 6.2+ using the `COUNT` argument in [`LPOP`](https://redis.io/commands/lpop/)/[`RPOP`](https://redis.io/commands/rpop/). +- (optional) `ListPopFromBeginning`: determines whether to pop elements from the beginning using [`LPOP`](https://redis.io/commands/lpop/) or to pop elements from the end using [`RPOP`](https://redis.io/commands/rpop/). + - Default: true +#### Sample +The following sample polls the key "listTest" at a localhost Redis instance at "127.0.0.1:6379" ```c# -[FunctionName("PubSubTrigger")] -public static void PubSubTrigger( - [RedisTrigger(ConnectionString = ".redis.cache.windows.net:6380", TriggerType = RedisTriggerType.PubSub, Trigger = "channel")] - RedisMessageModel result, ILogger logger) +[FunctionName(nameof(ListsTrigger))] +public static void ListsTrigger( + [RedisListsTrigger(ConnectionString = "127.0.0.1:6379", Keys = "listTest")] RedisMessageModel model, + ILogger logger) { - logger.LogInformation(JsonSerializer.Serialize(result)); + logger.LogInformation(JsonSerializer.Serialize(model)); } ``` -## Getting Started -First, [create a Redis cache](https://learn.microsoft.com/azure/azure-cache-for-redis/quickstart-create-redis). +### `RedisStreamsTrigger` +The `RedisStreamsTrigger` pops elements from a stream and surfaces those elements to the function. +The trigger polls Redis at a configurable fixed interval, and uses [`XREADGROUP`](https://redis.io/commands/xreadgroup/) to read elements from the stream. +Each function creates a new random GUID to use as its consumer name within the group to ensure that scaled out instances of the function will not read the same messages from the stream. + +Inputs: +- `ConnectionString`: connection string to the redis cache (eg `.redis.cache.windows.net:6380,password=...`). +- `Keys`: Keys to read from, space-delimited. + - Uses [`XREADGROUP`](https://redis.io/commands/xreadgroup/). +- (optional) `PollingIntervalInMs`: How often to poll Redis in milliseconds. + - Default: 1000 +- (optional) `MessagesPerWorker`: How many messages each functions worker "should" process. Used to determine how many workers the function should scale to. + - Default: 100 +- (optional) `BatchSize`: Number of elements to pull from Redis at one time. + - Default: 10 +- (optional) `ConsumerGroup`: The name of the consumer group that the function will use. + - Default: "AzureFunctionRedisExtension" +- (optional) `DeleteAfterProcess`: If the listener will delete the stream entries after the function runs. + - Default: false + +#### Sample +The following sample polls the key "streamTest" at a localhost Redis instance at "127.0.0.1:6379" +```c# +[FunctionName(nameof(StreamsTrigger))] +public static void StreamsTrigger( + [RedisStreamsTrigger(ConnectionString = "127.0.0.1:6379", Keys = "streamTest")] RedisMessageModel model, + ILogger logger) +{ + logger.LogInformation(JsonSerializer.Serialize(model)); +} +``` + +### Return Value +All triggers return a [`RedisMessageModel`](./src/Models/RedisMessageModel.cs) object that has two fields: +```c# +namespace Microsoft.Azure.WebJobs.Extensions.Redis +{ + public class RedisMessageModel + { + public string Trigger { get; set; } + public string Message { get; set; } + } +} +``` +- `Trigger`: The pubsub channel, list key, or stream key that the function is listening to. +- `Message`: The pubsub message, list element, or stream element. + +## Known Issues +- The `RedisPubSubTrigger` is not capable of listening to [keyspace notifications](https://redis.io/docs/manual/keyspace-notifications/) on clustered caches. + -# Contributing -This project welcomes contributions and suggestions. Most contributions require you to agree to a +## Contributing +This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. diff --git a/src/Microsoft.Azure.WebJobs.Extensions.Redis.csproj b/src/Microsoft.Azure.WebJobs.Extensions.Redis.csproj index 999857b..9ddb26c 100644 --- a/src/Microsoft.Azure.WebJobs.Extensions.Redis.csproj +++ b/src/Microsoft.Azure.WebJobs.Extensions.Redis.csproj @@ -1,17 +1,30 @@  + Library netstandard2.0 - Microsoft.Azure + Microsoft.Azure.WebJobs.Extensions.Redis + Microsoft.Azure.WebJobs.Extensions.Redis + + Microsoft.Azure.WebJobs.Extensions.Redis + Microsoft Microsoft - Azure Redis Cache Extension for Azure Functions - Azure redis cache extension for azure functions. + © Microsoft Corporation. All rights reserved. + + Redis Extension for Azure Functions + This package contains binding extensions for Redis. MIT - Azure, Azure Redis Cache, Redis, Redis Cache, Azure Redis Cache Extension, Azure Functions + true https://github.com/Azure/azure-functions-redis-extension + https://github.com/Azure/azure-functions-redis-extension + Microsoft Azure WebJobs AzureFunctions Redis + + true + true + embedded + true false -