Networking & Content Delivery

Streamlining network deployments using AWS Cloud Control

Introduction

In today’s multi-cloud landscape, organizations face significant challenges in network configuration and resource management. Traditional tools often need deep, tool-specific knowledge, which leads to increased deployment times and configuration errors.

The AWS Cloud Control API addresses these common cloud management challenges by providing a unified, language-agnostic interface for resource management. It offers immediate access to the latest AWS features and a standardized Create-Read-Update-Delete-List (CRUD-L) interface across services. These improvements simplify cloud resource deployments and management. Streamlining the interaction with AWS services allows for Cloud Control API to enable organizations to focus on their architectural design and business logic, rather than grappling with the complexities of various management tools and service-specific APIs. This leads to faster implementation, reduced errors, and more streamlined operations in multi-cloud environments.

This post demonstrates how the Cloud Control API refines networking resource development. We showcase inspection example setup using AWS Transit Gateway and AWS Network Firewall spanning multiple AWS Accounts and AWS Regions, highlighting best practices for deploying and managing these resources with the declarative approach of Cloud Control API. The concepts and building blocks discussed in this blog can be extended to other networking resources such as AWS Cloud WAN or HAQM VPC Lattice.

Real-world scenario with Cloud Control API

In this example scenario (Figure 1), we have a workload operating across two AWS Regions (us-east-1 and us-west-2). An internal application in the shared services account connects to this workload through Transit Gateway, which we use for cross-Region connectivity. All communication between the workload and the internal application undergoes inspection by Network Firewall. The blog provides technical deep dive about centralized deployment with AWS Network Firewall and AWS Transit Gateway.

Cloud Control API streamlining development and deployment of real-world network scenario

Figure 1: Cloud Control API streamlining development and deployment of real-world network scenario

The following table (Table 1) summarizes the building blocks for our real-world scenario and how AWS Cloud Control API feature simplifies its development and deployment.

 
Label# Block description    AWS Cloud Control API feature
1 Centralized connectivity: Use Transit Gateway as the hub for all traffic flows. Provision a Transit Gateway and its attachments with a single create_resource API call.
2 Traffic inspection: Route all traffic through Network Firewall. Configure Network Firewall rules dynamically by using Cloud Control API CRUD operations.
3 Multi-account setup: Ensure isolation between workloads in separate accounts. Use assumed roles with Cloud Control API to provision resources across accounts securely.
4 Dynamic updates: Keep route tables updated with changing workloads. Use the read and update operations to dynamically fetch current attachments and update Transit Gateway route tables.
5 Automation: Minimize manual interventions with CI/CD pipelines. Integrate Cloud Control API into CI/CD pipelines for declarative, automated provisioning and management of network resources.

Table 1 : Building blocks for our real-world scenario with Cloud Control API 

Centralized connectivity

For this multi-Region workload, Transit Gateway acts as a network transit hub, connecting VPCs in us-east-1 and us-west-2, and the shared services account. The following table summarizes how Cloud Control API helps you simplify the implementation of this centralized connectivity architecture, by enabling the provisioning of Transit Gateway and its attachments through a single API call. This is shown in Figure 1 as label 1. Table 2 also provides a comparison with other available automation options, such as AWS CloudFormation, CDK and Terraform.

Feature AWS Cloud Control API   AWS CloudFormation AWS CDK Terraform
Unified and Consistent Interface Cloud Control API provides a consistent interface for managing Transit Gateway and its attachments. Although it needs separate API calls for the Transit Gateway and each attachment, it offers standardized CRUD operations across these resources. This approach streamlines management compared to service-specific APIs, but still needs individual resource provisioning with the create_resource operation. This eliminates the need to learn the details of different infrastructure as code (IaC) tools. It uses a template-based approach, with separate resource definitions for Transit Gateway and attachments, needing strict syntax rules. Code-heavy approach that necessitates defining separate classes for Transit Gateway and attachments. It needs programming expertise. Necessitates defining multiple HashiCorp Configuration Language (HCL) resources, with explicit modules and parameters for each resource.
Real Time Updates Provides a consistent interface for managing resources, which can streamline adoption of new features as they become available through Cloud Control API. Delayed updates, as AWS CloudFormation resource specifications must be updated to include new AWS features. Delayed updates, depends on AWS SDK. Delayed updates, needs provider updates.
Declarative State Management YAML file, auto-resolves dependencies. The API automatically resolves dependencies and provisions resources accordingly, reducing complexity. Fully declarative, needs manual dependency configuration between resources. Code driven, explicit dependencies. Declarative, needs explicit modules and manual resolution for resource dependencies
Error Transparency The API provides detailed feedback on the provisioning status with the get_resource_request_status method, making troubleshooting clear. Errors are logged during stack creation but can be verbose and hard to trace. Error handling depends on programming logic and may need additional debugging steps. Errors are logged in Terraform CLI and need state file inspection for resolution.
Cross-Account and Cross-Region Support Seamless with assumed roles. Cloud Control API works consistently across accounts and AWS Regions when proper AWS Identity and Access Management (IAM) roles are set up. Needs StackSets or further parameters for cross-account and cross-Region setups. It needs custom logic to manage assumed roles and AWS Region-specific resources. It necessitates creating provider configurations for each account and AWS Region, which can get complex.
Example Code snippet

Table 2: A comparison for centralized connectivity with AWS Cloud Control API and other automation options, such as AWS CloudFormation, CDK and Terraform

Traffic Inspection

To secure the multi-Region workload and its communication with the internal application, we route all traffic through Network Firewall. This is shown in Figure 1 as label 2. Cloud Control API offers several advantages in implementing this traffic inspection, as show in the following Table 3.

Feature AWS Cloud Control API   AWS CloudFormation AWS CDK Terraform
Dynamic Rule Configuration Using the create_resource and update_resource operations, you can dynamically define and update Network Firewall rules directly through Cloud Control API. This reduces the need for full stack redeployments, but doesn’t completely eliminate it for all scenarios. Needs stack updates and redeployment for rule changes. Needs code changes and redeployment for rule modifications. Needs file updates and state management for rule modifications.
Multi-Region Support Reuse YAML file or invoke APIs with AWS Region-specific parameters. Separate templates or nested stacks. Needs duplicate or parameterize code. Needs AWS Region-specific provider blocks and manual effort.
Simplified Traffic Routing Declarative configuration providing smooth integrations, but may still need some manual configuration for complex scenarios. Routing rules need manual configuration and dependency resolution in templates. Explicit code definition. Routes must be explicitly declared, and dependencies are resolved manually.
CRUD-L Operations for Updates API calls for updates/audits. Example update_resource to modify firewall rules or list_resource to fetch active firewall configurations. Updates involve full stack redeployments, and audits need external scripts or manual checks. Updates need code modifications and redeployment, and audits need AWS SDK integration. Updates are incremental but need state file handling, and audits need scripting or plugins.
Fewer Manual Errors With a single API interface, the risk of syntax errors or inconsistencies between tools (such as AWS Cloud Development Kit – AWS CDK) or CloudFormation) is minimized. Template-based configuration is prone to dependency errors or misconfiguration. Programming logic increases the risk of syntax or logical errors. Errors can arise from manual state mismanagement and dependencies between resources.
Example Code snippet

Table 3: A comparison for traffic inspection with AWS Cloud Control API and other automation options, such as AWS CloudFormation, CDK and Terraform

Multi-account setup

Our scenario involves multiple accounts: the workload accounts in different AWS Regions and the shared services account. Implementing this multi-account setup is crucial for workload isolation and security. This is shown in Figure 1 as label 3. The following Table 3 shows how Cloud Control API facilitates this setup.

Feature AWS Cloud Control API  AWS CloudFormation AWS CDK Terraform
Assumed Role Support Cloud Control API integrates seamlessly with IAM, allowing you to assume roles across accounts. This lets you deploy and manage resources securely in member accounts without further setup. Needs manual configuration of IAM roles and accounts in templates. Needs explicit AWS SDK logic for assuming roles, increasing complexity. Needs multiple provider blocks with role assumption logic, which adds to complexity.
Resource Sharing Using AWS::RAM::ResourceShare in your declarative configuration, Cloud Control API allows you to share the Transit Gateway with all accounts in your AWS Organization. Needs dedicated templates for AWS Resource Access Manager (AWS RAM) resource sharing, increasing management overhead. Needs code logic to explicitly implement AWS RAM resource sharing. Involves creating resource-sharing modules or manually configuring AWS RAM.
Security Best Practices The API enforces AWS security principles such as least privilege, and streamlines secure deployments across accounts, reducing operational overhead. Manual configuration of templates creates risk for errors. Needs custom logic for adhering to security best practices. Relies on manually configured IAM roles and provider settings for security adherence.
Consistency Across Accounts Use a single YAML file to define resources for all accounts. Consistency must be manually managed through template reuse or nested stacks. Needs parameterized logic and code for consistency across accounts. Needs reusable modules for consistency, needing careful configuration for each account.
Example Code snippet

Table 4: A comparison for multi-account setup with AWS Cloud Control API and other automation options, such as AWS CloudFormation, CDK and Terraform

Dynamic updates

To accommodate for workload growth and changes, we need to keep route tables and firewall rules updated across AWS Regions and accounts. This is shown in Figure 1 as label 4. Cloud Control API streamlines these dynamic updates, as show in the following Table 5.

Feature AWS Cloud Control API  AWS CDK AWS CloudFormation Terraform
Real-Time Resource Awareness Use the read_resource operation to dynamically fetch the live state of resources (for example VPCs, Transit Gateway attachments, and route tables). Relies on AWS SDK or manual querying for resource states. Not natively supported. Depends on the existing stack template for changes. Maintains a state file (local/backend), and needs manual refresh to sync live state.
Incremental Updates Modify only the parts of a resource that need changes by using update_resource, reducing downtime, and avoiding full stack redeployments. Needs explicit logic for modifying attributes and redeploying the resource stack. Entire stack is updated, often redeploying unrelated resources. Incremental updates are supported but need careful handling of the state file.
Automation-Friendly The declarative nature of Cloud Control API makes it easy to integrate updates into CI/CD pipelines, making sure that route tables, attachments, and firewall rules are always in sync with current workloads. Automation needs scripting and logic within CI/CD pipelines, adding complexity. Changes must be committed to the stack template, and stack updates are triggered through CI/CD pipelines. Integrates well into pipelines but needs explicit configuration for state and updates.
Code Clear and few lines of code to manage dynamic infrastructure. Needs further logic and scripts for complex changes. Managing dynamic infrastructure changes necessitates redeploying the stack. Dynamic updates are manageable but need state file consistency and careful configuration.

Table 5: A comparison for dynamic updates with AWS Cloud Control API and other automation options, such as AWS CloudFormation, CDK and Terraform

Automation

To minimize manual interventions and make sure of consistency across our multi-Region, multi-account setup, we integrate our network resource management into CI/CD pipelines. This is shown in Figure 1 as label 5. Cloud Control API offers several advantages for this automation, as shown in Table 6.

Feature AWS Cloud Control API  AWS CDK AWS CloudFormation Terraform
Unified Interface for All Stages From provisioning to updates and deletions, provides the same interface, streamlining integration into CI/CD pipelines such as AWS CodePipeline or GitHub Actions. It needs separate scripts and logic for different lifecycle stages, increasing complexity. Templates can be used across stages but need separate configurations for updates and deletions. Consistent CLI and state management streamline lifecycle operations, but complexity increases for pipelines.
Declarative File Reuse YAML files created for deployments can be reused in pipelines. Code-driven approach necessitates parameterization or further scripting to adapt for pipeline reuse. Templates are reusable but may need parameterization for pipeline adaptability. Configuration files are reusable. However, they need careful management of backend state files for pipelines.
Error Feedback in Pipelines The API provides clear success and failure statuses that can be programmatically checked in pipelines to trigger retries or rollbacks. Error handling depends on custom logic within pipeline scripts. Errors in stack creation/update need to be captured manually or using external tools. Errors during state management or apply steps can be captured but need scripting for retries.
Immediate Availability of New Features Immediate deployment possible. Needs updates to both the AWS SDK and the AWS CDK before new features are available. New features need resource specification updates, which can introduce delays. Needs provider updates to support new AWS features, which can cause delays.
Example Code snippet

Table 6: A comparison for automation with AWS Cloud Control API and other automation options, such as AWS CloudFormation, CDK and Terraform

The rest of blog provides details on how to deploy real-world scenario discussed using Cloud Control API. The solution has few prerequisites listed below.

Prerequisites

The following prerequisites are needed to deploy the solution:

  1. To create an IAM user in the Network Deployment account, log in to the Network Deployment account with a user that is set up for AWS Management Console access and has permissions related to IAM. Follow these steps:
    • Navigate to the IAM console in the Console. Create an IAM policy ‘network_deployment_user_policy’ using the policy document network_deployment_user_policy.json. The policy has necessary permissions to IAM, HAQM S3, CodePipeline, AWS CodeBuild, and HAQM DynamoDB.
    • Create a new IAM user ‘network_deployment_user’ using the steps mentioned in the IAM user guide. Attach the IAM policy ‘network_deployment_user_policy’ created in the previous step to this user.
    • Create an Access key for the user to perform programmatic calls using AWS Command Line Interface (AWS CLI).
  2. Install AWS CLI locally following the installation steps. Configure AWS CLI with the access key credentials of the IAM user ‘network_deployment_user’ created in the previous step.
aws configure

3. Install Git locally, referring to the steps outlined in this link.

Deployment steps

To deploy the solution across accounts in your organization using CodePipeline and Cloud Control API, follow these steps:

  1. Clone and set up a Git repository
    • Clone the Git repository to your local machine
git clone http://github.com/aws-samples/sample-network-deployment-using-aws-cloud-control.git
    • Change into the cloned directory
cd sample-network-deployment-using-aws-cloud-control
    • Remove the original remote repository connection
git remote remove origin
    • Add the repository URL that you plan to use for network deployments as the new remote origin
git remote add origin <your-new-repo-url>
    • Push all the content to your new repository
git push -u origin <default-branch-name>

2. IAM Roles

    • Deployment Account Role: Create a “CodePipelineRole” in the Network Deployment account. The role is used by CodePipeline and CodeBuild stages to assume permissions in member accounts that are needed for resource deployment. To deploy the IAM role used by CodePipeline using Cloud Control API, pass the codepipeline_role.json file to the following command:
aws cloudcontrol create-resource --type-name "AWS::IAM::Role" --desired-state file://codepipeline_role.json --region us-east-1 --profile <deployment-account-profile>
    • Save the HAQM Resource Name (ARN) of the IAM role created using the following command:
aws cloudcontrol get-resource --type-name "AWS::IAM::Role" --identifier CodePipelineRole --profile <deployment-account-profile> --region us-east-1
    • Member Account Role: Create an IAM role in each member account. Update the Principal with the correct ARN of the IAM role that you created in the network deployment account in the previous step. Use the member_account_deployment_role.json from the Git repository, and run the following command in the shared-services account and the network account (These are considered as example member accounts in this example). Use appropriate profiles for each member account.
aws cloudcontrol create-resource --type-name "AWS::IAM::Role" --desired-state file://member_account_deployment_role.json --region us-east-1 --profile <member-account-1-profile>

3. CodeBuild project

    • Use a Python environment as the runtime and use the CodePipelineRole that you created previously. Refer to the buildspec.yml file in the root directory of the repository. 
      aws codebuild create-project--name "MultiAccountCloudControlBuild" --description "CodeBuild project for deploying AWS resources." --source type=CODEPIPELINE --artifacts type=CODEPIPELINE --environment type=LINUX_CONTAINER,image=aws/codebuild/standard:5.0,computeType=BUILD_GENERAL1_SMALL --service-role "arn:aws:iam::deployment-account-id:role/CodePipelineRole" --region us-east-1 --profile <deployment-account-profile>

4. HAQM S3 artifact storage bucket

    • Create the S3 bucket for artifact storage in the Network Deployment Account. This bucket is used by CodePipeline. Use the following command to create the S3 bucket.
aws s3api create-bucket --bucket YourS3BucketName --region us-east-1 --profile <deployment-account-profile>
    • Enable bucket versioning.
aws s3api put-bucket-versioning --bucket YourS3BucketName --versioning-configuration Status=Enabled --region us-east-1 --profile <deployment-account-profile>

5. Set up CodePipeline in the Network Deployment Account

    • Create CodePipeline in the US East (N. Virginia) Region (us-east-1) or in your preferred AWS Region of choice. For this post, use the US East (N. Virginia) Region. Use a branch (for example cloud-control-branch) to trigger the pipeline and link the CodeBuild project (created in Step 3) to execute scripts.
    • Refer to codepipeline.json, and update the RoleArn with the ARN of the CodePipeline role created in the Network Deployment account.
    • Under ArtifactStore, update the name of the S3 bucket that stores the artifacts. Update the Git VCS configuration.Then, run the following command to create CodePipeline.
aws cloudcontrol create-resource --type-name "AWS::CodePipeline::Pipeline" --desired-state file://codepipeline.json --profile <deployment-account-profile>
    • Check the Resource Creation Status:
aws cloudcontrol get-resource-request-status --request-token "RequestToken"

6. Set up the DynamoDB table in the Network Deployment Account

    • Create the DynamoDB table in the us-east-1 Region or in any other preferred AWS Region of choice. For the scope of this post, we work in the us-east1 Region. To deploy the DynamoDB table, refer to dynamodb_table.json and run the following command to create the DynamoDB table:
aws cloudcontrol create-resource --type-name "AWS::DynamoDB::Table" --desired-state file://dynamodb_table.json --region us-east-1 --profile <deployment-account-profile>

7. Update the Configuration Files and invoke the CI/CD Pipeline

    • The deployment script and supporting YAML configuration files for deploying the preceding setup are in the scripts folder. Update the Account ID, VPC CIDR, subnet CIDR, and/or naming conventions.
    • After providing and updating the user information, commit the changes to the Git repository that acts as the source to the CodePipeline deployed in Step 5. The CodeBuild stage of the pipeline references the buildspec file and execute the deployment script. The readme.MD file explains how the deployment script uses the YAML configurations files.

Conclusion

The comparative analysis in this post demonstrates that AWS Cloud Control API addresses many challenges faced by traditional IaC tools such as AWS CloudFormation, AWS CDK, and Terraform. Its consistent Create, Read, Update, Delete, and List (CRUD-L) interface, real-time resource awareness, and streamlined cross-account management capabilities benefit enterprises with complex, multi-account network setups.

Organizations face challenges in scaling and securing their cloud networks. Cloud Control API provides a unified, declarative interface for resource management, streamlining the process of setting up and maintaining network architectures across multiple accounts and AWS Regions. It enables dynamic resource updates, seamless integration with CI/CD pipelines, and instant access to new AWS features without tool-specific updates.

About the Authors

Shiva Vaidyanathan

Shiva Vaidyanathan is a Principal Cloud Architect at AWS. He provides technical guidance, design and lead implementation projects to customers ensuring their success on AWS. He works towards making cloud networking simpler for everyone leveraging cutting edge Generative AI technologies. Prior to joining AWS, he has worked on several NSF funded research initiatives on performing secure computing in public cloud infrastructures. He holds a MS in Computer Science from Rutgers University and a MS in Electrical Engineering from New York University.

Omkar Nyalpelly

Omkar Nyalpelly specializes in network architecture and automation, focusing on AWS Landing Zones. With deep expertise in AWS and a strong background in DevOps methodologies across the SDLC, he leads critical implementation and migration projects. His expertise ensures secure, scalable, and efficient cloud operations. At AWS Professional Services, he builds customer-centric solutions, leveraging Generative AI to enhance cloud infrastructure automation. Outside of work, he enjoys playing cricket, baseball, and photography. Omkar holds an MS in Networking and Telecommunications from Southern Methodist University