AWS Compute Blog
Implementing a serverless architecture to detect absence of Guardrails in HAQM Bedrock inference API calls
This post is written by Sayan Chakraborty, Senior Solutions Architect, AWS
Implementing a serverless architecture to detect absence of Guardrails in HAQM Bedrock inference API calls
In today’s rapidly evolving artificial intelligence (AI) landscape, organizations are increasingly harnessing the power of foundation models through HAQM Bedrock to build sophisticated generative AI applications. Although this technology opens up exciting possibilities, it also brings forth important considerations around responsible AI implementation and content safety.
HAQM Bedrock Guardrails serve as a crucial safeguard, helping organizations filter out harmful content, prevent prompt injection attacks (LLM01:2025 from OWASP Top 10 for generative AI), and maintain ethical AI practices. These configurable safeguards are essential for enterprises committed to responsible AI development, especially when scaling their applications across various use cases.
However, there’s a critical consideration: although Guardrails are powerful, they’re optional by default in HAQM Bedrock inference API calls. For organizations that mandate the use of Guardrails as part of their responsible AI strategy, a solution is needed to make sure of consistent implementation across all API requests.
In this post, we explore how to build a serverless architecture that automatically detects when Guardrails are absent in HAQM Bedrock inference API calls. We demonstrate how enterprises can implement automated monitoring and alerting systems to maintain compliance with their AI safety standards, making sure that Guardrails are properly implemented wherever needed. This solution is particularly valuable for organizations prioritizing secure and responsible AI deployment at scale.
Prerequisites
Before proceeding with the implementation, make sure that you do the following:
1.Create an AWS account if you do not already have one, and log in. The AWS Identity and Access Management (IAM) user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
2.Have AWS Command Line Interface (AWS CLI) installed and configured.
3.Have Git Installed.
Architecture
The following diagram shows an event-driven architecture of this solution.
Figure 1: Solution architecture diagram
HAQM Bedrock supports model invocation logging. When enabled, it collects the full request data, response data, and metadata associated with all model invocation calls performed in your AWS account. Logging can be configured to send the logs to supported destinations such as HAQM CloudWatch Logs and HAQM S3. This solution uses an S3 bucket to collect these logs. Note that this solution supports the below HAQM Bedrock inference APIs:
As logs get stored in the S3 bucket, an HAQM S3 event notification is generated to an HAQM EventBridge event bus. A rule that matches “Object Created” events from HAQM S3 routes these events to an AWS Step Functions state machine, which defines the orchestration logic to inspect the model invocation logs for missing Guardrails, and sends out an alert to a monitored email address when applicable.
Walkthrough of the orchestration
As mentioned previously, the Step Functions state machine is the orchestration engine that performs the business logic for this solution, as events are received from new logs created in the S3 bucket. When opened in the Workflow studio in the Step Functions console, you should observe the following diagram.
Figure 2: Step Functions state machine diagram as seen in workflow studio
1.The first step in the state machine is to call an AWS Lambda function to get the logs from the S3 bucket using the bucket name and object key supplied in the event object received from EventBridge.
2.If the log shows that the HAQM Bedrock API invocation was successful, then the state machine collects the output object of the API response from the log that is needed for further evaluation.
3.The next step is to check if HAQM Bedrock Guardrails was used. This is done by looking for specific objects in the HAQM Bedrock API output that was captured from the logs.
4.If a Guardrail was detected, then the flow completes successfully, and no further action is needed.
5.If a Guardrail was not detected, then the next step in the state machine collects a few pieces of information from the log file that is necessary to record the transaction and adds the transaction date. Then, the transaction is logged in to the transactions table in HAQM DynamoDB.
6.A user or a role may be making a lot of API calls to HAQM Bedrock each day. Therefore, the solution implements a mechanism to prevent the monitored email address from being swamped by emails reporting the same user or role more than once each day. This is done in parallel to Step 5, where the flow checks if the principal’s identity (user ID/IAM role) is recorded as notified in the current date, by querying the notifications table in DynamoDB. If no results were found, meaning that a notification hasn’t been sent yet, then an email is sent out to a monitored email address through an HAQM Simple Notification Service (HAQM SNS) topic. Furthermore, an item is inserted into the notifications table in DynamoDB to prevent sending more notifications on the same day for the same principal.
Solution deployment
For deployment instructions, follow along in the GitHub repo or use this post. An AWS CloudFormation template is provided to deploy the solution.
1.Create an S3 bucket to store the model invocation logs from HAQM Bedrock. Under bucket Properties, turn the EventBridge notifications to On. This enables HAQM S3 to send an event notification to the EventBridge default event bus whenever a log file is created in the bucket by HAQM Bedrock.
2.Go to the HAQM Bedrock console and enable Model invocation logging under Bedrock Configuration > Settings, from the left navigation pane. Specify the bucket created in Step 1 under S3 location.
Figure 3: HAQM Bedrock settings for Model invocation logging
3. Create two more S3 buckets: one that is used by the Step Functions state machine to store Bedrock model invocation errors detected from the log, and the other that stores the Lambda function code for this solution. Inside the latter bucket, create a Folder called code (or any other preferred name) and upload the ZIP archive under the lambda-code folder of this repository, into that HAQM S3 folder. Note the names for these two S3 buckets and the HAQM S3 object key for the Lambda ZIP file. These must be specified as input parameters to the CloudFormation template.
4. From the CloudFormation console or using CLI, create a stack using the template provided in this repository called bedrock-guardrails-detection-template.yaml. For inputs, specify the BedrockLogsBucket (from Step 1), BedrockLogsErrorBucket (from Step 3), LambdaFunctionCodeBucket (from Step 3), LambdaFunctionCodeBucketKey (S3 object key for the ZIP file uploaded in Step 3, for example code/get-bedrock-logs-from-s3.py.zip), and NotificationEmailAddress (email address to subscribe to the SNS topic). It may take a few minutes to complete deployment of the CloudFormation stack.
5. When deployment is complete, access the email inbox for the email address specified during the CloudFormation stack deployment, and confirm the subscription using the email sent from the HAQM SNS topic. The email should be titled: AWS Notification – Subscription Confirmation. Choose the Confirm subscription link inside the email to complete the subscription process. The email account is now ready to receive notifications from this solution.
Scaling to multiple AWS accounts
The architecture discussed previously shows how Guardrails can be detected from within the same AWS account where HAQM Bedrock APIs are invoked. However, in most production environments, there are multiple AWS accounts where independent teams may be deploying their own generative AI workloads using HAQM Bedrock in their own accounts. To collect model invocation logs from all those accounts, EventBridge can be configured to send events from event buses in separate source workload accounts to a central event bus deployed in a central destination governance account. This central event bus can have a rule to route events to the Step Functions state machine deployed in that central governance account. The deployment model looks like the following diagram.
To learn more about sending and receiving events between AWS accounts in EventBridge, refer to the documentation.
Figure 4: Cross-account guardrail detection solution
Further considerations and clean up
HAQM Bedrock model invocation logging captures requests and responses from model invocations and stores the logs in the destination of your choosing. In this sample it is in an S3 bucket that you create. The following are some more security considerations.
1.To protect information, you may choose to use to encrypt the contents using server-side encryption with AWS KMS keys (SSE-KMS) on the S3 bucket, and specify a customer managed encryption key. More details are in this HAQM Bedrock user guide.
2.Perform regular cleanup of the model invocation logs bucket using an HAQM S3 lifecycle configuration rule as mentioned in this post.
To avoid ongoing charges, clean up your environment by following these steps to delete the resources you created by following this post, if they are no longer needed:
1.Delete the stack:
aws cloudformation delete-stack –stack-name STACK_NAME
2.Confirm the stack has been deleted:
aws cloudformation list-stacks –query “StackSummaries[?contains(StackName,’STACK_NAME’)].StackStatus”
3.Empty contents of the S3 buckets created manually as a prerequisite to deploying the CloudFormation stack and delete the buckets.
4.Turn off model invocation logging from under Settings in the HAQM Bedrock console, if it’s not desired any longer.
Conclusion
This post discussed implementing a serverless event-driven architecture to detect the absence of Guardrails in HAQM Bedrock inference API calls. As organizations increasingly use foundation models through HAQM Bedrock for generative AI applications, making sure of responsible AI implementation becomes crucial.
The solution presents an event-driven architecture that automatically detects when Guardrails are missing in API calls. It uses the HAQM Bedrock model invocation logging, storing logs in an HAQM S3 bucket. When new logs are created, an HAQM S3 event notification triggers an HAQM EventBridge event bus, which routes events to an AWS Step Functions state machine. Then, the state machine inspects the logs for missing Guardrails and sends alerts through HAQM SNS to a monitored email address.
The architecture includes features to prevent notification flooding and can scale across multiple AWS accounts. The post provides detailed deployment instructions using AWS CloudFormation and includes security considerations and cleanup procedures. With this solution you can help your organization maintain compliance with AI safety standards while scaling generative AI applications.