AWS Machine Learning Blog

Protect your DeepSeek model deployments with HAQM Bedrock Guardrails

The rapid advancement of generative AI has brought powerful publicly available large language models (LLMs), such as DeepSeek-R1, to the forefront of innovation. The DeepSeek-R1 models are now accessible through HAQM Bedrock Marketplace and HAQM SageMaker JumpStart, and distilled variants are available through HAQM Bedrock Custom Model Import. According to DeepSeek AI, these models offer strong capabilities in reasoning, coding, and natural language understanding. However, their deployment in production environments—like all models—requires careful consideration of data privacy requirements, appropriate management of bias in output, and the need for robust monitoring and control mechanisms.

Organizations adopting open source, open weights models such as DeepSeek-R1 have important opportunities to address several key considerations:

  • Enhancing security measures to prevent potential misuse, guided by resources such as OWASP LLM Top 10 and MITRE Atlas
  • Making sure to protect sensitive information
  • Fostering responsible content generation practices
  • Striving for compliance with relevant industry regulations

These concerns become particularly critical in highly regulated industries such as healthcare, finance, and government services, where data privacy and content accuracy are paramount.

This blog post provides a comprehensive guide to implementing robust safety protections for DeepSeek-R1 and other open weight models using HAQM Bedrock Guardrails. We’ll explore:

  • How to use the security features offered by HAQM Bedrock to protect your data and applications
  • Practical implementation of guardrails to prevent prompt attacks and filter harmful content
  • Implementing a robust defense-in-depth strategy

By following this guide, you’ll learn how to use the advanced capabilities of DeepSeek models while maintaining strong security controls and promoting ethical AI practices. Whether developing customer-facing generative AI applications or internal tools, these implementation patterns will help you meet your requirements for secure and responsible AI. By following this step-by-step approach, organizations can deploy open weights LLMs such as DeepSeek-R1 in line with best practices for AI safety and security.

DeepSeek models and deployment on HAQM Bedrock

DeepSeek AI, a company specializing in open weights foundation AI models, recently launched their DeepSeek-R1 models, which according to their paper have shown outstanding reasoning abilities and performance in industry benchmarks. According to third-party evaluations, these models consistently achieve top three rankings across various metrics, including quality index, scientific reasoning and knowledge, quantitative reasoning, and coding (HumanEval).

The company has further developed their portfolio by releasing six dense models derived from DeepSeek-R1, built on Llama and Qwen architectures, which they’ve made open weight models. These models are now accessible through AWS generative AI solutions: DeepSeek-R1 is available through HAQM Bedrock Marketplace and SageMaker Jumpstart, while the Llama-based distilled versions can be implemented through HAQM Bedrock Custom Model Import.

HAQM Bedrock offers comprehensive security features to help secure hosting and operation of open source and open weights models while maintaining data privacy and regulatory compliance. Key features include data encryption at rest and in transit, fine-grained access controls, secure connectivity options, and various compliance certifications. Additionally, HAQM Bedrock provides guardrails for content filtering and sensitive information protection to support responsible AI use. AWS enhances these capabilities with extensive platform-wide security and compliance measures:

Organizations should customize these security settings based on their specific compliance and security needs when deploying to production environments. AWS conducts vulnerability scanning of all model containers as part of its security process and accepts only models in Safetensors format to help prevent unsafe code execution.

HAQM Bedrock Guardrails

HAQM Bedrock Guardrails provides configurable safeguards to help safely build generative AI applications at scale. HAQM Bedrock Guardrails can also be integrated with other HAQM Bedrock tools including HAQM Bedrock Agents and HAQM Bedrock Knowledge Bases to build safer and more secure generative AI applications aligned with responsible AI policies. To learn more, see the AWS Responsible AI page.

Core functionality

HAQM Bedrock Guardrails can be used in two ways. First, it can be integrated directly with the InvokeModel and Converse API call, where guardrails are applied to both input prompts and model outputs during the inference process. This method is suitable with models hosted on HAQM Bedrock through the HAQM Bedrock Marketplace and HAQM Bedrock Custom Model Import. Alternatively, the ApplyGuardrail API offers a more flexible approach, allowing for independent evaluation of content without invoking a model. This second method is useful for assessing inputs or outputs at various stages of an application, working with custom or third-party models outside of HAQM Bedrock. Both approaches enable developers to implement safeguards customized to their use cases and aligned with responsible AI policies, ensuring secure and compliant interactions in generative AI applications.

Key HAQM Bedrock Guardrails policies

HAQM Bedrock Guardrails provides the following configurable guardrail policies to help safely build generative AI applications at scale:

  • Content filters
    • Adjustable filtering intensity for harmful content
    • Predefined categories: Hate, Insults, Sexual Content, Violence, Misconduct, and Prompt Attacks
    • Multi-modal content including text and images (preview)
  • Topic filters
    • Capability to restrict specific topics
    • Prevention of unauthorized topics in both queries and responses
  • Word filters
    • Blocks specific words, phrases, and profanity
    • Custom filters for offensive language or competitor references
  • Sensitive information filters
    • Personally identifiable information (PII) blocking or masking
    • Support for custom regex patterns
    • Probabilistic detection for standard formats (such as SSN, DOB, and addresses)
  • Contextual grounding checks
    • Hallucination detection through source grounding
    • Query relevance validation
  • Automated Reasoning checks for hallucination prevention (gated preview)

Other capabilities

Model-agnostic implementation:

  • Compatible with all HAQM Bedrock foundation models
  • Supports fine-tuned models
  • Extends to external custom and third-party models through the ApplyGuardrail API

This comprehensive framework helps customers implement responsible AI, maintaining content safety and user privacy across diverse generative AI applications.

Solution Overview

  1. Guardrail configuration
    • Create a guardrail with specific policies tailored to your use case and configure the policies.
  1. Integration with InvokeModel API
    • Call the HAQM Bedrock InvokeModel API with the guardrail identifier in your request.
    • When you make the API call, HAQM Bedrock applies the specified guardrail to both the input and output.
  1. Guardrail evaluation process
    1. Input evaluation: Before sending the prompt to the model, the guardrail evaluates the user input against the configured policies.
    2. Parallel policy checking: For improved latency, the input is evaluated in parallel for each configured policy.
    3. Input intervention: If the input violates any guardrail policies, a pre-configured blocked message is returned, and the model inference is discarded.
    4. Model inference: If the input passes the guardrail checks, the prompt is sent to the specified model for inference.
    5. Output evaluation: After the model generates a response, the guardrail evaluates the output against the configured policies.
    6. Output intervention: If the model response violates any guardrail policies, it will be either blocked with a pre-configured message or have sensitive information masked, depending on the policy.
    7. Response delivery: If the output passes all guardrail checks, the response is returned to the application without modifications

Prerequisites

Before setting up guardrails for models imported using the HAQM Bedrock Custom Model Import feature, make sure you meet these prerequisites:

  • An AWS account with access to HAQM Bedrock along with the necessary IAM role with the required permissions. For centralized access management, we recommend that you use AWS IAM Identity Center.
  • Make sure that a custom model is already imported using the HAQM Bedrock Custom Model Import service. For illustration, we’ll use DeepSeek-R1-Distill-Llama-8B, which can be imported using HAQM Bedrock Custom Model Import. You have two options for deploying this model:

You can create the guardrail using the AWS Management Console as explained in this blog post. Alternatively, you can follow this notebook for a programmatic example of how to create the guardrail in this solution. This notebook does the following :

  1. Install the required dependencies
  2. Create a guardrail using the boto3 API and filters to meet the use case mentioned previously.
  3. Configure the tokenizer for the imported model.
  4. Test HAQM Bedrock Guardrails using prompts that show various HAQM Bedrock guardrail filters in action.

This approach integrates guardrails into both the user inputs and the model outputs. This makes sure that any potentially harmful or inappropriate content is intercepted during both phases of the interaction. For open weight distilled models imported using HAQM Bedrock Custom Model Import, HAQM Bedrock Marketplace, and HAQM SageMaker JumpStart, critical filters to implement include those for prompt attacks, content moderation, topic restrictions, and sensitive information protection.

Implementing a defense-in-depth strategy with AWS services

While HAQM Bedrock Guardrails provides essential content and prompt safety controls, implementing a comprehensive defense-in-depth strategy is crucial when deploying any foundation model, especially open weights models such as DeepSeek-R1. For detailed guidance on defense-in-depth approaches aligned with OWASP Top 10 for LLMs, see our previous blog post on architecting secure generative AI applications.

Key highlights include:

  • Developing organizational resiliency by starting with security in mind
  • Building on a secure cloud foundation using AWS services
  • Applying a layered defense strategy across multiple trust boundaries
  • Addressing the OWASP Top 10 risks for LLM applications
  • Implementing security best practices throughout the AI/ML lifecycle
  • Using AWS security services in conjunction with AI and machine learning (AI/ML)-specific features
  • Considering diverse perspectives and aligning security with business objectives
  • Preparing for and mitigating risks such as prompt injection and data poisoning

The combination of model-level controls (guardrails) with a defense-in-depth strategy creates a robust security posture that can help protect against:

  • Data exfiltration attempts
  • Unauthorized access to fine-tuned models or training data
  • Potential vulnerabilities in model implementation
  • Malicious use of AI agents and integrations

We recommend conducting thorough threat modeling exercises using AWS guidance for generative AI workloads before deploying any new AI/ML solutions. This helps align security controls with specific risk scenarios and business requirements.

Conclusion

Implementing safety protection for LLMs, including DeepSeek-R1 models, is crucial for maintaining a secure and ethical AI environment. By using HAQM Bedrock Guardrails with the HAQM Bedrock InvokeModel API and the ApplyGuardrails API, you can help mitigate the risks associated with advanced language models while still harnessing their powerful capabilities. However, it’s important to recognize that model-level protections are just one component of a comprehensive security strategy.

The strategies outlined in this post address several key security concerns that are common across various open weights models hosted on HAQM Bedrock using HAQM Bedrock Custom Model Import, HAQM Bedrock Marketplace, and through HAQM SageMaker JumpStart. These include potential vulnerabilities to prompt injection attacks, the generation of harmful content, and other risks identified in recent assessments. By implementing these guardrails alongside a defense-in-depth approach, organizations can significantly reduce the risk of misuse and better align their AI applications with ethical standards and regulatory requirements.

As AI technology continues to evolve, it’s essential to prioritize safety and responsible use of generative AI. HAQM Bedrock Guardrails provides a configurable and robust framework for implementing these safeguards, allowing developers to customize protection measures according to their specific use cases and organizational policies. We strongly recommend conducting thorough threat modeling of your AI workloads using AWS guidance to evaluate security risks and implementing appropriate controls across your entire technology stack.

Remember to regularly review and update not only your guardrails but all security controls to address new potential vulnerabilities and help maintain protection against emerging threats in the rapidly evolving landscape of AI security. While today we focus on DeepSeek-R1 models, the AI landscape is continuously evolving with new models emerging regularly. HAQM Bedrock Guardrails, combined with AWS security services and best practices, provides a consistent security framework that can adapt to protect your generative AI applications across various open weights models, both current and future. By treating security as a continuous process of assessment, improvement, and adaptation, organizations can confidently deploy innovative AI solutions while maintaining robust security controls.

About the Authors

Satveer Khurpa is a Sr. WW Specialist Solutions Architect, Bedrock at HAQM Web Services. In this role, he uses his expertise in cloud-based architectures to develop innovative generative AI solutions for clients across diverse industries. Satveer’s deep understanding of generative AI technologies allows him to design scalable, secure, and responsible applications that unlock new business opportunities and drive tangible value.

Adewale Akinfaderin is a Sr. Data Scientist–Generative AI, HAQM Bedrock, where he contributes to cutting edge innovations in foundational models and generative AI applications at AWS. His expertise is in reproducible and end-to-end AI/ML methods, practical implementations, and helping global customers formulate and develop scalable solutions to interdisciplinary problems. He has two graduate degrees in physics and a doctorate in engineering.

Antonio Rodriguez is a Principal Generative AI Specialist Solutions Architect at HAQM Web Services. He helps companies of all sizes solve their challenges, embrace innovation, and create new business opportunities with HAQM Bedrock. Apart from work, he loves to spend time with his family and play sports with his friends.