Simplify hybrid inspection using AWS Cloud WAN service insertion

Organizations face the complex challenge when managing and securing networks across multiple AWS Regions, cloud environments, and on-premises locations. AWS Cloud WAN helps users manage distributed networks through a unified approach. The service streamlines network management by providing a single network policy framework, segmentation capabilities, dynamic route propagation, and monitoring through a centralized dashboard. AWS Cloud WAN now includes service insertion and a native AWS Direct Connect attachment, which streamline traffic inspection and hybrid connectivity integration in global networks.

Before the service insertion feature, users configured static routes to direct traffic through firewall appliances, as described in the blog posts: Inspecting network traffic between HAQM VPCs with AWS Cloud WAN and Hybrid security inspection architectures with AWS Cloud WAN and AWS Direct Connect. To reduce costs, users often deploy firewalls in select AWS Regions. However, with a single global inspection segment, traffic couldn’t be routed through firewalls in specific AWS Regions. To inspect traffic in respective AWS Regions and maintain symmetric routing, users created a unique global shared inspection segment for each AWS Region or edge location, as detailed in the post, Achieve optimal routing with AWS Cloud WAN for multi-Region networks. This blog post explains how AWS Cloud WAN service insertion streamlines traffic inspection and enables traffic steering for multi-Region global networks by defining simple policy statements within global network policy.

Prerequisites

Before proceeding, we assume that you’re familiar with networking constructs such as HAQM Virtual Private Cloud (HAQM VPC), AWS Direct Connect, and AWS Cloud WAN. Detailed information around AWS Cloud WAN service insertion concepts used in the preceding section of this post can be found in the AWS Cloud WAN Service Insertion user guide. This post focuses on hybrid inspection architecture and traffic flows, but if you would like to read about traffic inspection between VPCs using service insertion, then refer to the post, Simplify Global Security Inspection with AWS Cloud WAN Service Insertion.

Hybrid inspection scenario

In this section, we look at an example multi-Region hybrid inspection architecture with Direct Connect and AWS Cloud WAN and do packet walkthroughs of the following four flows:

1. Packet flow 1: Traffic between Corp Data Center 1 and Prod VPC 1 in us-east-2 (Ohio)
2. Packet flow 2: Traffic between Corp Data Center 1 and Prod VPC 2 in us-west-1 (N. California)
3. Packet flow 3: Traffic between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) without edge-override
4. Packet flow 4: Traffic between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) with edge-override

Architecture

Figure 1: Multi-region Hybrid inspection architecture with AWS Cloud WAN service insertion

(A) Three Production VPCs: Prod VPC 1, Prod VPC 2, and Prod VPC 3 are deployed in AWS Regions, us-east-2 (Ohio), us-west-1 (N. California), and us-west-2 (Oregon) respectively with the following CIDR ranges:

• Prod VPC 1 in us-east-2 (Ohio): 10.1.0.0/16
• Prod VPC 2 in us-west-1 (N. California): 10.2.0.0/16
• Prod VPC 3 in us-west-2 (Oregon): 10.3.0.0/16

(B) An AWS Cloud WAN Core Network is created with three AWS Regions enabled (AWS Cloud WAN Core Network Edge (CNE) in each AWS Region) and a Production segment that spans across the three Regions. The Prod VPCs are associated with the Production segment. In their respective AWS Regions, these VPCs advertise their local VPC CIDR to AWS Cloud WAN CNEs (CNEs 1, 2, and 3) using their respective VPC attachments.

(C) Corporate (Corp) Data Center 1, with CIDR range 192.168.1.0/16 is connected to Direct Connect Location 1 (homed to us-east-2 (Ohio)), and Corp Data Center 2 with CIDR range 192.168.2.0/16 is connected to Direct Connect Location 2 (homed to us-west-1 (N. California)).

(D) A transit virtual interface (VIF) is configured for each Direct Connect connection that terminates at a Direct Connect Gateway.

(E) The AWS Cloud WAN Core Network has a Hybrid segment that spans across the three Regions. The Direct Connect Gateway is associated with the Hybrid segment in all three AWS Regions. Both data centers advertise their respective CIDR ranges to the Direct Connect Gateway over the transit VIF BGP sessions. No BGP traffic engineering is applied on any of the Direct Connect connections.

(F) AWS Regions 1 and 2 have Inspection VPC 1 and Inspection VPC 2 deployed respectively, with the following CIDR ranges:

• Inspection VPC 1 in us-east-2 (Ohio): 100.64.1.0/24
• Inspection VPC 2 in us-west-1 (N. California): 100.64.2.0/24

The AWS Region us-west-2 (Oregon) doesn’t have a local Inspection VPC. The Core Network also has an Inspection Network Function Group (NFG) created where Inspection VPCs 1 and 2 are associated.

(G) The AWS Cloud WAN service insertion feature is used to configure a "send-via" action for traffic going between the Production segment and Hybrid Segment through the Inspection NFG. Due to this action:

• The CIDR ranges of Prod VPCs 1, 2, and 3 are propagated into the Inspection NFG route table of each AWS Region.
• The CIDR ranges of Corp Data Centers 1 and 2 are propagated into the Inspection NFG route table of each AWS Region.
• The CIDR ranges of Corp Data Centers 1 and 2 are propagated into the Production segment route table of us-east-2 (Ohio) with Inspection VPC 1 as the destination. Similarly, the CIDR ranges of Corp Data Centers 1 and 2 are propagated into the Production segment route table of us-west-1 (N. California) with Inspection VPC 2 as the destination. AWS Region us-west-2 (Oregon) doesn’t have a local Inspection VPC, thus the CIDR ranges of Corp Data Centers 1 and 2 are propagated into the Production segment route table of us-west-2 (Oregon) with Inspection VPC 1 (of us-east-2 (Ohio)) as the destination. This is due to the ordered list behavior of AWS Cloud WAN with service insertion where us-east-2 (Ohio) is assigned a higher priority than us-west-1 (N. California).
• The CIDR ranges of Prod VPCs 1, 2, and 3 are propagated into the Hybrid segment route table of us-east-2 (Ohio) with Inspection VPC 1 as the destination. Similarly, CIDR ranges of Prod VPCs 1, 2, and 3 are propagated into the Hybrid segment route table of us-west-1 (N. California) with Inspection VPC 2 as the destination. CIDR ranges of Prod VPCs 1, 2, and 3 are propagated into the Hybrid segment route table of us-west-2 (Oregon) with Inspection VPC 1 (of us-east-2 (Ohio)) as the destination again due to us-east-2 (Ohio)’s higher priority.

The following is an example JSON policy snippet of the service insertion "send-via" action for the Production segment:

"action":"send-via",
"segment":"Production",
"mode":"single-hop",
"when-sent-to":{
   "segments":[
      "Hybrid"
    ]
  },
"via":{
   "network-function-groups":[
      "InspectionNFG"
    ]
  }
}

Use case

• Corp Data Centers 1 and 2 need to communicate with all three Prod VPCs through their respective Direct Connect connections.
• Traffic between the Corp Data Center 1 and the Prod VPCs needs to be inspected by firewalls in Inspection VPC 1 (of us-east-2 (Ohio)).
• Traffic between the Corp Data Center 2 and the Prod VPCs needs to be inspected by firewalls in Inspection VPC 2 (of us-west-1 (N. California)).
• Specifically, traffic between the Corp Data Centers and Prod VPC 3 needs to be inspected by firewalls in us-west-1 (N. California) (Inspection VPC 2) instead of Inspection VPC 1. This is because us-west-2 (Oregon) is geographically closer to us-west-1 (N. California) than us-east-2 (Ohio), and inspecting traffic in Inspection VPC 2 provides a better latency.

Packet walkthrough

The following steps walk you through the packet flow.

Packet flow 1: Traffic between Corp Data Center 1 and Prod VPC 1 in us-east-2 (Ohio)

In this section, we observe the default traffic behavior between Corp Data Center 1 and Prod VPC 1 in us-east-2 (Ohio).

Figure 2: Packet flow between Corp Data Center 1 and Prod VPC 1 in us-east-2 (Ohio)

(1) Traffic from Corp Data Center 1 (192.168.1.0/16) destined for Prod VPC 1 (10.1.0.0/16) is routed through Direct Connect location 1 (through the transit VIF) to the Direct Connect Gateway.

(2) Direct Connect Gateway routes traffic to CNE 1 in us-east-2 (Ohio).

(3) After a route table lookup in the Hybrid segment of us-east-2 (Ohio), traffic is forwarded to Inspection VPC 1.

(4) After Inspection, traffic enters Inspection NFG in us-east-2 (Ohio).

(5) and (6) Inspection NFG has a route to Prod VPC 1 and traffic is forwarded to Prod VPC 1.

(7) and (8) In the reverse direction, traffic from Prod VPC 1 (10.1.0.0/16) destined for Corp Data Center 1 (192.168.1.0/16) is routed through AWS Cloud WAN Production segment to Inspection VPC 1 for inspection.

(9) After Inspection, traffic enters Inspection NFG in us-east-2 (Ohio).

(10) Inspection NFG has a route to Corp Data Center 1, and traffic is forwarded to Direct Connect Gateway.

(11) and (12) Direct Connect Gateway routes traffic to Corp Data Center 1 (192.168.1.0/16) connected to Direct Connect Location 1.

Packet flow 2: Traffic between Corp Data Center 1 and Prod VPC 2 in us-west-1 (N. California)

Traffic between Corp Data Center 1 and Prod VPC 2 in us-west-1 (N. California) follows similar flows in the forward and reverse directions (steps (1) to (12) shown in Figure 3).

Figure 3: Packet flow between Corp Data Center 1 and Prod VPC 2 in us-west-1 (N. California)

Packet flow 3: Traffic between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) without edge-override

In this section, we observe the default traffic behavior between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon).

Figure 4: Packet flow between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) without edge-override

(1) Traffic from Corp Data Center 2 (192.168.2.0/16) destined for Prod VPC 3 (10.3.0.0/16) is routed via Direct Connect location 2 (through the transit VIF) to the Direct Connect Gateway.

(2) Direct Connect Gateway routes traffic to CNE 3 in us-west-2 (Oregon).

(3) After a route table lookup in the Hybrid segment of us-west-2(Oregon), traffic is forwarded to Inspection VPC 1 in us-east-2 (Ohio). This is because us-west-2 (Oregon) doesn’t have a local Inspection VPC, and AWS Cloud WAN has assigned a higher priority to us-east-2 (Ohio) than us-west-1 (N. California) using the ordered lists.

(4) After Inspection, traffic enters Inspection NFG in us-east-2 (Ohio).

(5) and (6) The Inspection NFG has a route to Prod VPC 3 and traffic is forwarded to Prod VPC 3.

(7) and (8) In the reverse direction, traffic from Prod VPC 3 (10.3.0.0/16) destined for Corp Data Center 2 (192.168.2.0/16) is routed through AWS Cloud WAN Production segment to Inspection VPC 1 in us-east-2 (Ohio) for inspection.

(9) After Inspection, traffic enters Inspection NFG in us-east-2 (Ohio).

(10) The Inspection NFG has a route to Corp Data Center 2 and traffic is forwarded to Direct Connect Gateway.

(11) and (12) Direct Connect Gateway routes traffic to Corp Data Center 2 (192.168.2.0/16) which is connected to Direct Connect Location 2.

Traffic between Corp Data Center 1 and Prod VPC 3 in us-west-2 (Oregon) (not shown in Figure 4) follows similar flows in the forward and reverse directions.

From a latency perspective, the desired behavior is for traffic between the Corp Data Centers and Prod VPC 3 in us-west-2 (Oregon) to be always inspected by Inspection VPC 2 in us-west-1 (N. California). However, as seen in the preceding packet walkthrough, when a local Inspection VPC isn’t available in a particular Region, the default routing behavior of AWS Cloud WAN can potentially result in the traffic being inspected by firewalls in a remote Region’s Inspection VPC based on priorities set by AWS Cloud WAN’s ordered list. Therefore, this may result in added latency and cost. This concern can be addressed through the “edge-override” feature of AWS Cloud WAN service insertion, which allows you to specify an edge location (CNE) to be used by a Region. In the next section, we show how "edge-override" can help us achieve the desired routing behavior.

Packet flow 4: Traffic between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) with edge-override

In this section, we observe the traffic behavior between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) using edge-override.

Figure 5: Packet flow between Corp Data Center 2 and Prod VPC 3 in us-west-2 (Oregon) with edge-override

(1) Traffic from Corp Data Center 2 (192.168.2.0/16) destined for Prod VPC 3 (10.3.0.0/16) is routed through Direct Connect location 2 (through the transit VIF) to the Direct Connect Gateway.

(2) The Direct Connect Gateway routes traffic to CNE 3 in us-west-2 (Oregon).

(3) Due to "edge-override" configuration:

• The CIDR ranges of Corp Data Center 1 and Corp Data center 2 are propagated into the Production segment route table of us-west-2 (Oregon) with Inspection VPC 2 as the destination.
• The CIDR ranges of Prod VPCs 1, 2, and 3 are propagated into the Hybrid segment route table of us-west-2 (Oregon) with Inspection VPC 2 as the destination.

The following is an example JSON policy snippet of the service insertion "send-via" action with "edge-override" configured for us-west-2 (Oregon) in the Production segment:

"action":"send-via",
"segment":"Production",
"mode":"single-hop",
"when-sent-to":{
   "segments":[
      "Hybrid"
    ]
  },
"via":{
   "network-function-groups":[
      "InspectionNFG"
       ],
   "with-edge-overrides":[
      {
         "edge-sets":[
            [
               "us-west-2"
            ]
         ],
         "use-edge-location":"us-west-1"
      }
    ]
 }
}

After a route table lookup in the Hybrid segment of us-west-2 (Oregon), traffic is forwarded to Inspection VPC 2 in us-west-1 (N. California).

(4) After Inspection, traffic enters Inspection NFG in us-west-1 (N. California).

(5) and (6) Inspection NFG has a route to Prod VPC 3, and traffic is forwarded to Prod VPC 3.

(7) and (8) In the reverse direction, traffic from Prod VPC 3 (10.3.0.0/16) destined for Corp Data Center 2 (192.168.2.0/16) is routed through the AWS Cloud WAN Production segment to Inspection VPC 2 in us-west-1 (N. California) for inspection.

(9) After Inspection, traffic enters Inspection NFG in us-west-1 (N. California).

(10) Inspection NFG has a route to Corp Data Center 2, and traffic is forwarded to Direct Connect Gateway.

(11) and (12) Direct Connect Gateway routes traffic to Corp Data Center 2 (192.168.2.0/16), which is connected to Direct Connect Location 2.

This approach allows you to make sure that the traffic between Corp Data Centers and Prod VPCs in us-west-2 (Oregon) is always inspected by firewalls in a Region closer to us-west-2 (Oregon) rather than a remote Region.

Considerations

• NFG is global in the sense that you can add attachments to it from any of the Regions belonging to the core network.
• All attachment types are supported (VPC, VPN, Connect, Transit Gateway, and Direct Connect Gateway) for both workload segments and NFGs.
• You can configure multiple NFGs in the same core network. However, you can’t insert multiple NFGs with the same segment or segment pair.
• There are no further charges for using service insertion other than the regular AWS Cloud WAN charges.
• You can associate a Direct Connect Gateway only with a single AWS Cloud WAN segment. You can have multiple Direct Connect Gateways associated with the same segment. You can also have different Direct Connect Gateways associated with multiple AWS Cloud WAN segments. However, you can’t have the same Direct Connect Gateway associated with multiple segments.
• When you associate a Direct Connect Gateway with an AWS Cloud WAN segment using a Direct Connect attachment, you can choose all or a subset of the AWS Cloud WAN CNEs.
• Direct Connect BGP community tags are only relevant to the Direct Connect Gateway, and don’t influence the AWS Cloud WAN core network routing decisions.
• The AWS Cloud WAN route evaluation order is detailed in the documentation.

Conclusion

In this post, we covered how hybrid networks that advertise specific routes from on-premises can use AWS Cloud WAN service insertion to inspect traffic between AWS environments and on-premises networks. We also discussed how the edge-override capability within AWS Cloud WAN service insertion helps achieve optimal performance in multi-Region deployments. The combination of service insertion capabilities with AWS Direct Connect native integration delivers a comprehensive solution that addresses the complex challenges of securing hybrid architectures. This unified approach eliminates the traditional complexities of managing multiple security configurations across different AWS Regions and environments, while making sure that traffic inspection policies are consistently applied, whether traffic flows between VPCs, on-premises locations, or the internet. For more information, see AWS Cloud WAN documentation.

About the authors