Merge pull request #386 from globaldatanet/4.3.1

4.3.1

CHANGELOG.md

@@ -2,6 +2,41 @@
 
 ## Released
 
+## 4.3.1
+### Added
+- [Issue#365](https://github.com/globaldatanet/aws-firewall-factory/issues/365) UnutilizedWafs - Implemented automated identification and notification system in Firewall Factory to manage unused WAFs, leveraging Lambda and notification services to streamline infrastructure, optimize costs, and enhance security by addressing WAF sprawl proactively and ensuring efficient resource utilization.
+- Added example IAM Role which can be used for [ci-cd](./static/roles/ci-cd-role.yaml) deployments
+
+### Fixed
+- [Issue#380](https://github.com/globaldatanet/aws-firewall-factory/issues/380) Fixes on the CloudWatch dashboard.
+- Restructure Lambda code with ShareComonents to reduce code duplicates
+- Using [cdk-sops-secrets](https://github.com/dbsystel/cdk-sops-secrets) now for all Webhooks - see WebHookSecretDefinition:
+ ```
+  {
+    WebhookUrl: string
+    Messenger: "Slack" | "Teams"
+  }
+```
+- Adding missing: Optional Lambda function to prerequisite Stack that send notifications about potential DDoS activity for protected resources to messengers (Slack/Teams) - [AWS Shield Advanced] - this was removed while migrating lambdas from python to typescript
+- Bump @aws-sdk/client-cloudformation from 3.554.0  to 3.556.0
+- Bump @aws-sdk/client-cloudfront from  3.568.0 to  3.577.0
+- Bump @aws-sdk/client-cloudwatch from 3.554.0 to  3.556.0
+- Bump @aws-sdk/client-config-service from  3.568.0 to  3.577.0
+- Bump @aws-sdk/client-ec2 from  3.568.0 to  3.577.0
+- Bump @aws-sdk/client-fms from 3.554.0 to  3.577.0
+- Bump @aws-sdk/client-pricing from 3.554.0 to  3.556.0
+- Bump @aws-sdk/client-s3 from  3.569.0 to  3.577.0
+- Bump @aws-sdk/client-service-quotas from 3.554.0 to  3.577.0
+- Bump @aws-sdk/client-shield from 3.554.0 to  3.556.0
+- Bump @aws-sdk/client-ssm from 3.554.0 to  3.577.0
+- Bump @aws-sdk/client-wafv2 from 3.554.0 to  3.556.0
+- Bump aws-cdk from  2.137.0 to  2.142.0
+- Bump aws-cdk-lib from 2.137.0 to  2.142.0
+- Bump @typescript-eslint/eslint-plugin from 7.6.0 to 7.9.0
+- Bump @typescript-eslint/parser from 7.6.0 to 7.9.0
+- Bump @types/lodash  from 4.17.0 to 4.17.1
+
+
 ## 4.3.0
 ### Added
 - Allow reusing ipsets with same name. This commit differentiate ipsets from different FMS configs by adding the name of the webacl to it. Without this commit, trying to run aws-firewall-factory for two configs which uses a ipset with the same name would give a error on CloudFormation ('IpSet with name x already exists') - (Add Name of web application firewall to the IPSet Name) - ⚠️ Existing IPsets will be replaced during next update.
@@ -333,7 +368,7 @@ The documentation will be updated regularly to provide you with the most current
 - Fix counter in package.json for versioning
 ## 3.1.2
 ### Added
-- Feature [Issue#52](https://github.com/globaldatanet/aws-firewall-factory/issues/52) - Added Regex for FMS Description Pattern: ^([\p{L}\p{Z}\p{N}_.:/=+\-@]*)$. -> Thanks to [@vboufleur](https://github.com/vboufleur)
+- Feature [Issue#52](https://github.com/globaldatanet/aws-firewall-factory/issues/52) - Added Regex for FMS Description Pattern:  ([\p{L}\p{Z}\p{N}_.:/=+\-@]*)$. -> Thanks to [@vboufleur](https://github.com/vboufleur)
 - Allow a list of resource types to apply firewall -> Kudos to [@vboufleur](https://github.com/vboufleur) for implementing this feature.
 
 ### Fixed

Deployment.md

@@ -4,13 +4,15 @@
 1. [Organizations trusted access with Firewall Manager](https://docs.aws.amazon.com/organizations/latest/userguide/services-that-can-integrate-fms.html)
 2. [Taskfile](https://taskfile.dev/)
 3. [AWS CDK](https://aws.amazon.com/cdk/)
-4. [cfn-dia](https://www.npmjs.com/package/@mhlabs/cfn-diagram?s=03)
-5. Invoke `npm i` to install dependencies
-6. ⚠️ Before installing a stack to your aws account using aws cdk you need to prepare the account using a [cdk bootstrap](https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html)
-
-7. (Optional) If you want to use CloudWatch Dashboards - You need to enable your target accounts to share CloudWatch data with the central security account follow [this](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html#enable-cross-account-cross-Region) to see how to do it.
-8. Assume AWS Profile `awsume PROFILENAME`
-9. (Optional) Enter `task generateprerequisitesconfig`
+4. [Sops](https://github.com/getsops/sops)
+5. [cfn-dia](https://www.npmjs.com/package/@mhlabs/cfn-diagram?s=03)
+6. Invoke `npm i` to install dependencies
+7. ⚠️ Before installing a stack to your aws account using aws cdk you need to prepare the account using a [cdk bootstrap](https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html)
+
+8. (Optional) If you want to use CloudWatch Dashboards - You need to enable your target accounts to share CloudWatch data with the central security account follow [this](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Cross-Account-Cross-Region.html#enable-cross-account-cross-Region) to see how to do it.
+9. (Optional) If you want to use the UnutilizedWafs Feature -  You need to enable your target accounts with a Cross Account Role - You can find an example CfnTemplate you can use [here](static/roles/cross_account_roles_unutilized_wafs.yaml).
+10. Assume AWS Profile `awsume PROFILENAME`
+11. (Optional) Enter `task generateprerequisitesconfig`
 
   | Parameter  | Value |
   | ------------- | ------------- |

Features.md

@@ -78,3 +78,4 @@ See example:
 
 22. Centralized management of RegexPatternSets - No longer will there be a need for manual updates of RegexPatternSets across multiple AWS accounts. These can now be defined in code and replicated for use by WAF rules wherever needed.
 
+23. Automated identification and notification system in Firewall Factory to manage unused WAFs, leveraging Lambda and notification services to streamline infrastructure, optimize costs, and enhance security by addressing WAF sprawl proactively and ensuring efficient resource utilization.

Taskfile.yml

@@ -2,7 +2,8 @@ version: '3'
 #output: prefixed
 env:
   SKIP_QUOTA_CHECK: true
-  WAF_TEST: false
+  WAF_TEST: TRUE
+  WAF_TEST_PASS_STATUS_CODES: 200 # HTTP response status code that WAF uses while passing requests (default [200,404])
   CREATE_DIAGRAM: false
   TOOL_KIT_STACKNAME: CDKToolkit
   LASTEST_FIREWALLFACTORY_VERSION:
@@ -38,7 +39,7 @@ tasks:
   diff:
     desc: CDK Diff
     cmds:
-      - cdk diff --toolkit-stack-name {{.TOOL_KIT_STACKNAME}}
+      - cdk diff
     silent: true
     interactive: true
     env:
@@ -47,7 +48,7 @@ tasks:
   cdkdestroy:
     desc: CDK Destroy
     cmds:
-      - cdk destroy --require-approval never --force
+      - cdk destroy --force
     vars:
       ACCOUNT:
         sh: aws sts get-caller-identity |jq -r .Account
@@ -59,7 +60,7 @@ tasks:
   cdkdeploy:
     desc: CDK Deploy
     cmds:
-      - DOCKER_BUILDKIT=1 cdk deploy --require-approval never {{.TAGS}} --toolkit-stack-name {{.TOOL_KIT_STACKNAME}}
+      - cdk deploy --require-approval never {{.TAGS}} --toolkit-stack-name {{.TOOL_KIT_STACKNAME}}
     vars:
       ACCOUNT:
         sh: aws sts get-caller-identity |jq -r .Account
@@ -86,7 +87,7 @@ tasks:
         items=$(ts-node ./gotestwaf/gotestwaf.ts | jq -r '.[] | .SecuredDomain[]?')
         for item in ${items[@]}; do
             echo "Using fqdn in 🖥  url : $item"
-            ./gotestwaf/gotestwaf --url https://$item --workers 50 --blockConnReset  --wafName="$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Prefix')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.WebAcl.Name')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Stage')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.DeployHash')" --configPath=./gotestwaf/config.yaml --testCasesPath=./gotestwaf/testcases --skipWAFBlockCheck --reportPath "./waf-evaluation-report/$(date '+%Y-%m-%d')" --reportName "$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Prefix')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.WebAcl.Name')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Stage')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.DeployHash')"
+            ./gotestwaf/gotestwaf --url https://$item --workers 50 --blockConnReset  --wafName="$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Prefix')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.WebAcl.Name')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Stage')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.DeployHash')" --configPath=./gotestwaf/config.yaml --testCasesPath=./gotestwaf/testcases --skipWAFBlockCheck --reportPath "./waf-evaluation-report/$(date '+%Y-%m-%d')" --reportName "$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Prefix')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.WebAcl.Name')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.Stage')-$(ts-node ./gotestwaf/gotestwaf.ts| jq -r '.General.DeployHash')-$item" --passStatusCodes {{.WAF_TEST_PASS_STATUS_CODES}} --blockConnReset
         done
     silent: true
     env:

lib/_prerequisites-stack.ts

@@ -2,8 +2,11 @@ import * as cdk from "aws-cdk-lib";
 import { Construct } from "constructs";
 import { Prerequisites } from "./types/config";
 import { RuntimeProperties } from "./types/runtimeprops";
-import { aws_s3 as s3, aws_kms as kms, aws_iam as iam, aws_lambda as lambda, aws_lambda_nodejs as NodejsFunction, aws_logs as logs, aws_glue as glue} from "aws-cdk-lib";
+import { aws_s3 as s3, aws_kms as kms, aws_iam as iam, aws_lambda as lambda, aws_lambda_nodejs as NodejsFunction, aws_logs as logs, aws_glue as glue, aws_stepfunctions as sfn,
+  aws_stepfunctions_tasks as tasks, aws_sns as sns, aws_fms as fms} from "aws-cdk-lib";
+import { EventbridgeToStepfunctions, EventbridgeToStepfunctionsProps } from "@aws-solutions-constructs/aws-eventbridge-stepfunctions";
 import * as path from "path";
+import { SopsSyncProvider, SopsSecret } from "cdk-sops-secrets";
 
 export interface StackProps extends cdk.StackProps {
     readonly prerequisites: Prerequisites;
@@ -17,26 +20,27 @@ export class PrerequisitesStack extends cdk.Stack {
     const accountId = cdk.Aws.ACCOUNT_ID;
     const region = cdk.Aws.REGION;
 
+    // Create SOPS SecretProvider Construct
+    const sopsSyncProvider = new SopsSyncProvider(
+      this,
+      "SopsSyncProvider"
+    );
+
+
     if(props.prerequisites.Information){
       console.log("📢  Creating Lambda Function to send AWS managed rule group change status notifications to messengers (Slack/Teams)");
-      let Messenger:string = "";
-      let WebhookUrl:string = "";
-      if(props.prerequisites.Information.SlackWebhook) {
-        Messenger="Slack";
-        WebhookUrl=props.prerequisites.Information.SlackWebhook;
-      }
-      if(props.prerequisites.Information.TeamsWebhook) {
-        Messenger="Teams";
-        WebhookUrl=props.prerequisites.Information.TeamsWebhook;
-      }
+
+      const InformationSecret = new SopsSecret(this,  "InformationSopsSecret", {
+        sopsFilePath: props.prerequisites.Information.WebhookSopsFile,
+        sopsProvider: sopsSyncProvider,
+      });
       const ManagedRuleGroupInfo = new NodejsFunction.NodejsFunction(this, "AwsFirewallFactoryManagedRuleGroupInfo", {
         architecture: lambda.Architecture.ARM_64,
         entry: path.join(__dirname, "../lib/lambda/ManagedRuleGroupInfo/index.ts"),
         handler: "handler",
         timeout: cdk.Duration.seconds(30),
         environment: {
-          "MESSENGER": Messenger,
-          "WEBHOOK_URL": WebhookUrl,
+          "WEBHOOK_SECRET": InformationSecret.secretName,
         },
         runtime: lambda.Runtime.NODEJS_20_X,
         memorySize: 128,
@@ -45,6 +49,7 @@ export class PrerequisitesStack extends cdk.Stack {
         },
         description: "Lambda Function to send AWS managed rule group change status notifications (like upcoming new versions and urgent security updates) to messengers (Slack/Teams)",
       });
+      InformationSecret.grantRead(ManagedRuleGroupInfo);
 
       new logs.LogGroup(this, "AWS-Firewall-Factory-ManagedRuleGroupInfo-LogGroup",{
         logGroupName: "/aws/lambda/"+ManagedRuleGroupInfo.functionName,
@@ -62,6 +67,126 @@ export class PrerequisitesStack extends cdk.Stack {
       });
     }
 
+    if(props.prerequisites.UnutilizedWafs){
+      console.log("📢  Creating StepFunction to send notification about unutilized Firewalls to messengers (Slack/Teams)");
+
+      const UnutilizedWafsSecret = new SopsSecret(this,  "UnutilizedWafsSopsSecret", {
+        sopsFilePath: props.prerequisites.UnutilizedWafs.WebhookSopsFile,
+        sopsProvider: sopsSyncProvider,
+      });
+      const unutilizedWafsBucket = new s3.Bucket(this, "AWS-Firewall-Factory-Unused-Resources", {
+        blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
+        encryption: s3.BucketEncryption.S3_MANAGED,
+        enforceSSL: true,
+        removalPolicy: cdk.RemovalPolicy.RETAIN,
+        bucketName: props.prerequisites.General.Prefix.toLocaleLowerCase() + "-afwf-unutilized-resources-" + accountId + "-" + region
+      });
+
+      const GetMemberAccountsofFms = new NodejsFunction.NodejsFunction(this, "GetMemberAccountsofFms", {
+        architecture: lambda.Architecture.ARM_64,
+        entry: path.join(__dirname, "../lib/lambda/GetMemberAccountsofFms/index.ts"),
+        handler: "handler",
+        logRetention: logs.RetentionDays.ONE_WEEK,
+        timeout: cdk.Duration.seconds(30),
+        runtime: lambda.Runtime.NODEJS_20_X,
+        memorySize: 128,
+        bundling: {
+          minify: true,
+        },
+        description: "Lambda Function to get all member accounts of AWS Firewall Manager",
+      });
+
+      GetMemberAccountsofFms.addToRolePolicy(new iam.PolicyStatement({
+        actions: ["fms:ListMemberAccounts"],
+        resources: ["*"],
+      }));
+
+
+
+      const CheckUnusedWebApplicationFirewalls = new NodejsFunction.NodejsFunction(this, "CheckUnusedWebApplicationFirewalls", {
+        architecture: lambda.Architecture.ARM_64,
+        entry: path.join(__dirname, "../lib/lambda/CheckUnusedWebApplicationFirewalls/index.ts"),
+        handler: "handler",
+        timeout: cdk.Duration.seconds(900),
+        runtime: lambda.Runtime.NODEJS_20_X,
+        memorySize: 128,
+        logRetention: logs.RetentionDays.ONE_WEEK,
+        bundling: {
+          minify: true,
+        },
+        environment: {
+          "BUCKET_NAME": unutilizedWafsBucket.bucketName,
+          "CROSS_ACCOUNT_ROLE_NAME": props.prerequisites.UnutilizedWafs.CrossAccountRoleName,
+          "REGEX_STRING": props.prerequisites.UnutilizedWafs.SkipWafRegexString || "",
+          "AWS_ACCOUNT_ID": cdk.Aws.ACCOUNT_ID,
+        },
+        description: "Lambda Function to get usage of AWS WAFv2 WebACLs",
+      });
+
+      CheckUnusedWebApplicationFirewalls.addToRolePolicy(new iam.PolicyStatement({
+        actions: ["sts:AssumeRole"],
+        resources: ["*"],
+      }));
+      unutilizedWafsBucket.grantReadWrite(CheckUnusedWebApplicationFirewalls);
+
+      const SendUnusedResourceNotification = new NodejsFunction.NodejsFunction(this, "SendUnusedResourceNotification", {
+        architecture: lambda.Architecture.ARM_64,
+        entry: path.join(__dirname, "../lib/lambda/SendUnusedResourceNotification/index.ts"),
+        handler: "handler",
+        timeout: cdk.Duration.seconds(30),
+        runtime: lambda.Runtime.NODEJS_20_X,
+        logRetention: logs.RetentionDays.ONE_WEEK,
+        memorySize: 128,
+        bundling: {
+          minify: true,
+        },
+        environment: {
+          "WEBHOOK_SECRET": UnutilizedWafsSecret.secretName,
+          "BUCKET_NAME": unutilizedWafsBucket.bucketName,
+        },
+        description: "Lambda Function to send notifications about unused AWS WAFv2 WebACLs",
+      });
+      SendUnusedResourceNotification.addToRolePolicy(new iam.PolicyStatement({
+        actions: ["pricing:Get*","pricing:Describe*","pricing:List*","pricing:Search*"],
+        resources: ["*"],
+      }));
+      SendUnusedResourceNotification.addToRolePolicy(new iam.PolicyStatement({
+        actions: ["fms:ListPolicies"],
+        resources: ["*"],
+      }));
+      SendUnusedResourceNotification.addToRolePolicy(new iam.PolicyStatement({
+        actions: ["ec2:DescribeRegions"],
+        resources: ["*"],
+      }));
+      unutilizedWafsBucket.grantReadWrite(SendUnusedResourceNotification);
+      UnutilizedWafsSecret.grantRead(SendUnusedResourceNotification);
+
+      const chain = sfn.Chain.start(new tasks.LambdaInvoke(this, "GetMemberAccountsofFmsLambda", {
+        lambdaFunction: GetMemberAccountsofFms,
+        retryOnServiceExceptions: true,
+      })).next(new sfn.Map(this, "Map", {
+        itemsPath: sfn.JsonPath.stringAt("$.Payload"),
+        resultPath: sfn.JsonPath.DISCARD,
+      }).itemProcessor(new tasks.LambdaInvoke(this, "CheckUnusedWebApplicationFirewallsLambda", {
+        lambdaFunction: CheckUnusedWebApplicationFirewalls,
+        payloadResponseOnly: true,
+        retryOnServiceExceptions: true,
+      }))).next(new tasks.LambdaInvoke(this, "SendUnsuedResourceNotificationLambda", {
+        lambdaFunction: SendUnusedResourceNotification,
+        retryOnServiceExceptions: true,
+      }));
+
+      const constructProps: EventbridgeToStepfunctionsProps = {
+        stateMachineProps: {
+          stateMachineName: "aws-firewall-factory-unutilizedWafs-"+ accountId + "-" + region,
+          definitionBody: sfn.DefinitionBody.fromChainable(chain),
+        },
+        eventRuleProps: {
+          schedule: props.prerequisites.UnutilizedWafs.ScheduleExpression,
+        }};
+      new EventbridgeToStepfunctions(this, "eventbridge-stepfunction-invoke", constructProps);
+    }
+
     if(props.prerequisites.Logging) {
       if(props.prerequisites.Logging.FireHoseKey) {
         console.log("🔑  Creating KMS Key for Kinesis FireHose.");
@@ -228,5 +353,49 @@ export class PrerequisitesStack extends cdk.Stack {
       }
     }
 
+    if(props.prerequisites.DdosNotifications) {
+      console.log("📢  Creating Lambda Function that send notifications about potential DDoS activity for protected resources to messengers (Slack/Teams)");
+
+      const DdosFmsNotificationSecret = new SopsSecret(this,  "DdosFmsNotificationSopsSecret", {
+        sopsFilePath: props.prerequisites.DdosNotifications.WebhookSopsFile,
+        sopsProvider: sopsSyncProvider,
+      });
+
+      const DdosFmsNotification = new NodejsFunction.NodejsFunction(this, "AWS-Firewall-Factory-FMS-Notifications", {
+        architecture: lambda.Architecture.ARM_64,
+        entry: path.join(__dirname, "../lib/lambda/FmsDdosNotification/index.ts"),
+        handler: "handler",
+        timeout: cdk.Duration.seconds(60),
+        runtime: lambda.Runtime.NODEJS_20_X,
+        memorySize: 128,
+        logRetention: logs.RetentionDays.ONE_WEEK,
+        bundling: {
+          minify: true,
+        },
+        environment: {
+          "WEBHOOK_SECRET": DdosFmsNotificationSecret.secretName,
+        },
+        description: "Lambda Function that send notifications about potential DDoS activity for protected resources to messengers (Slack/Teams)",
+      });
+
+      DdosFmsNotificationSecret.grantRead(DdosFmsNotification);
+
+      const snsRoleName = iam.Role.fromRoleName(this, "AWSServiceRoleForFMS", "aws-service-role/fms.amazonaws.com/AWSServiceRoleForFMS").roleArn;
+      const FmsTopic = new sns.Topic(this, "FMS-Notifications-Topic");
+      FmsTopic.addToResourcePolicy(new iam.PolicyStatement({
+        actions: ["sns:Publish"],
+        principals: [iam.Role.fromRoleArn(this, "AWSServiceRoleForFMS",snsRoleName)],
+      }));
+      DdosFmsNotification.addPermission("InvokeByFmsSnsTopic", {
+        action: "lambda:InvokeFunction",
+        principal: new iam.ServicePrincipal("sns.amazonaws.com"),
+        sourceArn: FmsTopic.topicArn,
+      });
+      new fms.CfnNotificationChannel(this, "AWS-Firewall-Factory-FMS-NotificationChannel", {
+        snsRoleName,
+        snsTopicArn: FmsTopic.topicArn,
+      });
+    }
+
   }
 }