Skip to content

An application for easy handling of Step Function tokens as callback URLs

License

Notifications You must be signed in to change notification settings

bbernays/sfn-callback-urls

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

sfn-callback-urls

AWS Step Functions lets you create serverless workflows in the form of state machines. Step Function supports long-running tasks, where Step Function gives out a token, which you later send back along with the result of the task. The sfn-callback-urls application is designed to make it easier to use these tokens in callback-based situations, like sending an email with a link to click, or passing a callback URL to another service.

How does it work?

You can create a long-running task in Step Functions in two ways. One is activities, which require a polling worker. The other is callback tasks, in which the token is sent out in an event (to a Lambda function, SQS queue, etc.).

Once you have the token, you call the sfn-callback-urls with the token and a set of potential outcomes. You receive a URL for each outcome, which can then be passed on to something deciding the outcome. The chosen outcome URL can then be POST'd or GET'd, and will send that outcome on to Step Functions, completing the task.

Test it out

# *** Do this part if you are deploying from SAR ***

# Visit https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-2:866918431004:applications~sfn-callback-urls
# and deploy the application. Note the stack name you used, and set it below.

STACK_NAME=TODO_DEPLOYED_APP_STACK_NAME

# *** Do this part if you are deploying from source ***

# Set this value
CODE_BUCKET=TODO_YOUR_S3_BUCKET

STACK_NAME=SfnCallbackUrls

sam build && sam package --output-template-file packaged-template.yaml --s3-bucket $CODE_BUCKET && sam deploy --template-file packaged-template.yaml --capabilities CAPABILITY_IAM --stack-name $STACK_NAME

# *** Now, let's get to it ***

# Set these values
NAME=TODO_YOUR_NAME
EMAIL=TODO_YOUR_EMAIL

# This gets the Lambda function we call for creating callback URLs
FUNC=$(aws cloudformation describe-stacks --stack-name $STACK_NAME --query "Stacks[0].Outputs[?OutputKey=='Function'].OutputValue" --output text)

# Deploy the example stack
aws cloudformation deploy --template-file example/template.yaml --stack-name SfnCallbackUrlsExample --parameter-overrides Email=$EMAIL CreateUrlsFunction=$FUNC --capabilities CAPABILITY_IAM

# Go to your email and confirm the SNS subscription

STATE_MACHINE=$(aws cloudformation describe-stacks --stack-name SfnCallbackUrlsExample --query "Stacks[0].Outputs[?OutputKey=='StateMachine'].OutputValue" --output text)

# Run the example state machine
aws stepfunctions start-execution --state-machine-arn $STATE_MACHINE --input "{\"name\": \"$NAME\"}"

# Now you will get an approve/reject email, followed by a confirmation of the same

Security

sfn-callback-urls is entirely stateless; your token is not stored in it anywhere. Instead, the token is encoded into the callback URL payload, along with the output. This means that if you lose the URLs and have not stored the tokens somewhere else, you cannot cause the corresponding state machine task to complete or fail. However, this does not mean you should store the tokens elsewhere. Instead, you may be able to provide a sensible timeout on your task, or simply stop the execution entirely and start a new one. Try these approaches before squirreling away tokens.

By default, sfn-callback-urls creates a KMS key to encrypt callback payloads, so that having a callback URL neither allows you to inspect the payload nor modify it before using it. The default costs money!. There are two alternatives. If you have your own KMS key, you can put the key ARN in the EncryptionKeyArn stack parameter, and it will use that instead of creating one.

If you want to disable encryption entirely, you can set the DisableEncryption stack parameter to true. The consequence of disabling encryption is that the contents of a callback URL, including the token and the output you want to send to the state machine, are inspectable. Additionally, somebody who has gotten a token they should not have could construct a callback URL use it, and since the callback are unauthenticated this would constitute a privilege escalation. However, it still requires that the token have previously leaked, and that the meaning of the token (the state machine it corresponds to, etc.) is known.

The action name and type are put as query parameters on the callback URLs for convenience, to make the URLs more easily distinguishable, but they are not trusted. The name and type are also stored in the payload, and when the callback is processed, the two are compared and the callback is rejected if they don't match.

The callback method is unauthenticated, it will always result in a Lambda invocation. With encryption enabled, no valid input should be able to be provided that hasn't already gone through the authentication URL creation call. However, it is still susceptible to denial-of-wallet attacks from someone who knows the endpoint (such as by having seen a callback URL). If this is a concern, a good course of action is to enable AWS WAF on the API Gateway.

Creating URLs

There are two ways to invoke the service: through the API, or direct to a Lambda. Both take identical input payloads. The API base url can be found as the Api output of the CloudFormation stack. To create callback URLs with the API, POST the JSON payload (with theContent-Type header set to application/json) to the /urls path. Or, simply invoke the Lambda function found as the Function output of the stack. Permissions for both of these are given by the IAM managed policy found as the Policy output of the CloudFormation stack.

Input

{
    "token": "<the token from Step Functions>", // required
    "actions": [ // you must provide at least one action
        {
            "name": "<a name for this action>",
            "type": "success", // this action will cause SendTaskSuccess
            "output": { // required, can be any JSON type
                "<your>": "<content>"
            },
            "response": {} // optional, see below
        },
        { // can have as many actions of the same type as you want
            "name": "<name2>",
            "type": "success",
            "output": "<a different output>"
        },
        {
            "name": "<name3>",
            "type": "failure",  // this action will cause SendTaskFailure
            "error": "<your error code>", // optional
            "cause": "<your error cause>" // optional
        },
        {
            "name": "<name4>",
            "type": "heartbeat" // this action will cause SendTaskHeartbeat (can invoke this type of callback more than once)
        }
    ],
    "expiration": "<ISO8601-formatted expiration>", // optional
    "enable_output_parameters": true // optional, and must be enabled on the stack, see below
}

Actions

For each action you define, you will get a callback URL. Each action you provide has a name and a type. The name is your label for the action. The type corresponds to what Step Functions API the callback will cause to be invoked, and must be one of success, failure, or heartbeat, corresponding to the SendTaskSuccess, SendTaskFailure, and SendTaskHeartbeat API calls, respectively.

For success actions, you must provide an output field, whose value will be passed to the same field in SendTaskSuccess.

For failure options, you may optionally provide error and cause fields whose values are strings, which will be passed to the same fields in SendTaskFailure.

Callback response specification

In every action, you can provide a response specification in the response field with an object like this:

{
    "redirect": "https://example.com", // if the callback is successful, redirect the user to the given URL
}

or this:

{
    "json": {"hello": "world"}, // choose the JSON object returned by the callback for the application/json content-type
    "html": "<html>hello, world</html>", // choose the body returned by the callback for the text/html content-type
    "text": "hello, world" // choose the body returned by the callback for the text/plain content-type
}

All fields are optional, and are only used when the callback is successfully processed; all errors return fixed content. redirect takes precedence over the other fields.

Expiration

You can optionally provide an expiration value as an ISO8601-formatted datetime; if a callback is made after then, it will be rejected.

Parameterizing callbacks

If you've got a lot of different potential successful outputs, you may find it easier to parameterize your callbacks. This feature is disabled by default due to the security considerations described below; you have to set the EnableOutputParameters stack parameter to true. Then, you must also opt-in when creating URLs by setting the enable_output_parameters field to true in your request. Any URLs created without enable_output_parameters set to true will not use parameterized output when the callbacks are processed. If EnableOutputParameters is changed back to false, any previously-created callbacks with parameters enabled will be now rejected.

Once set, any strings in the output field for a success action, the error and cause fields for a failure action, and all the strings in the response object are passed through the Python string.Template.substitute function, using all the query parameters except for the payload. In addition to the action and name query parameters that are already there, you can use your own field name in your strings, and append values to the callback URL query string. You can therefore create many outputs from one callback URL returned by the service. Note that failure to provide all of the necessary parameters will cause the callback to be rejected.

Parameterized callback security

Note that these extra query parameters are inherently unvalidated by the service, and therefore when enabled, someone could modify the query parameters to send unexpected output.

Output

On success:

{
    "transaction_id": "<a unique id>", // for correlation
    "urls": {
        "<action name>": "<url>"
    },
    "expiration": "<ISO8601-formatted datetime>" // only if you provided an expiration
}

On error:

{
    "error": "<error code>",
    "message": "<error description>"
}

Invoking the callback

You can either GET or POST the callback. The response respects the Accept header, supporting application/json, text/html, and text/plain, defaulting to JSON otherwise. As outlined above, the response can be customized when the callbacks are created.

The JSON response on success:

{
    "transaction_id": "<the same id returned by the create call>",
    "action": {
        "name": "<the action name>",
        "type": "<the action type>"
    }
}

The JSON response on error:

{
    "error": "<error code>",
    "message": "<error description>"
}

POST actions

If you'd like to use the body of a POST callback to send output for your task, for example with a webhook, you can do this with post actions. A post action has a non-empty list of outcomes, which use the same form as actions with a few extra fields. Each outcome has a name and a type, where the type is one of success, failure, or heartbeat.

Each outcome has a schema, which must be a JSON Schema that will be evaluated against the POST body. The first outcome whose schema validates against the body will used. If no schema matches the POST body, the callback results in an error.

Like in an action, a success outcome can include an output field, and a failure outcome can have error and cause; these are fixed values. To use the entire body of the request as the output for a success outcome, use "output_body": true in your outcome. To select information from the request body, you can use output_path to specify a JSONPath. Because JSONPath expressions can return multiple values, the output will always be an array; if you expect your expression to return a single object, you must select it from the array in your state machine. Similarly, you can use error_path and cause_path; if these return paths return a single string, it will be used, otherwise the resulting JSON array of matches will be stringified.

Outcomes can contain responses. POST actions disable output parameters, even if the create URLs call requests that they are enabled (other actions in such a call will have them enabled).

A sample request to create a POST action URL looks the following:

{
    "token": "<the token from Step Functions>", // required
    "actions": [ // you must provide at least one action
        {
            "name": "<a name for this action>",
            "type": "post",
            "outcomes": [
                {
                    "name": "<a name for this happy outcome>",
                    "type": "success",
                    "schema": { // require an object that looks like {"result": "good"}
                        "type": "object",
                        "properties": {
                            "result": {
                                "const": "good"
                            }
                        },
                        "required": [ "result" ]
                    },
                    "output_body": true
                },
                {
                    "name": "<a name for this sad outcome",
                    "type": "failure",
                    "schema": { // require an object that looks like {"result": "bad", "reason": "..."}
                        "type": "object",
                        "properties": {
                            "result": {
                                "const": "bad"
                            },
                            "reason": {
                                "type": "string"
                            }
                        },
                        "required": [ "result", "reason" ]
                    },
                    "error_path": "$.reason"
                }
            ]
        },
        { // can have other actions in addition to POST actions
            "name": "<name2>",
            "type": "success",
            "output": "<a different output>"
        }
    ]
}

This feature is disabled by default due to the security considerations described below; you have to set the EnablePostActions stack parameter to true. If EnablePostActions is changed back to false, any previously-created POST action callbacks will be now rejected.

POST action security

POST actions allow arbitrary output to be passed into an unauthenticated endpoint, and are therefore disabled by default. Users are required to provide a JSON schema to validate the body, but this can be the empty schema.

About

An application for easy handling of Step Function tokens as callback URLs

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%