AWS Transit Gateway Network Manager
Overview
In last post, we discussed AWS transit gateway and service-linked role. In this article, I plan to explore more about AWS network manager.
AWS transit gateway is regional resource, and has built in high availability, we will set up transit gateway in two regions and connect them through inter-region peering.
To manage and monitor AWS-based networks, AWS Transit Gateway Network Manager leverages other AWS services, specifically Amazon CloudWatch and Amazon VPC Flow Logs, to compile and display near real-time metrics such as bandwidth usage on AWS Transit Gateway attachments, packet flow count, packet drop count, and other information related to IP traffic routed through AWS Transit Gateway.
In this article, we will learn a little bit about CloudWatch, the available metrics for transit gateway, integration with Flow logs and network manager.
We will also discover how Route Analyzer helps us to troubleshoot.
System Diagram
We will skip the steps for setting up VPC and transit gateway, you may refer to my last post for more detail.
We will walk through steps to create transit gateway peering, turn on flow log, configure network manger for monitoring and management.
We will have EC2 instance in both region, install Ngnix in the EC2 instance and access the web server from other region via transit gateway and transit gateway peering.
Install Lab Environment
Inter-Region Transit Gateway Peering
From tgw-1 console, we add transit gateway attachment, the type is “peering connection”. The accepter is us-east-1 tgw2.
We will need to accept from us-east-1 transit gateway attachments console. It might take couple of minutes before the state turns to available.
Transit Gateway Network Interface
Let’s add vpc attachment to tgw2, us-east-1a subnet from our testvpc is attached to tgw-2.
Behind the scene, AWS created network interface for the attachment. You may find the NIC resource from EC2 console.
Log Group and Flow Log
A log group is a group of log streams that share the same retention, monitoring, and access control settings. You can define log groups and specify which streams to put into each group. There is no limit on the number of log streams that can belong to one log group.
Create log group from CloudWatch console (us east 1).
Create flow logs for transit gateway network interface, publish to CloudWatch logs, and we will explore how it is integrated with network manager.
(For IAM role, please refer to https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-cwl.html#flow-logs-iam, I skip steps for IAM and focus on networking for now)
For US West 2 region, I utilize existing log group and send transit gateway attachment network interface flow log to /aws/events/networkmanagerloggroup
CloudWatch
Amazon VPC publishes data points to Amazon CloudWatch for your transit gateways and transit gateway attachments. CloudWatch enables you to retrieve statistics about those data points as an ordered set of time series data, known as metrics.
Follow this user guide to show available transit gateway metrics and transit gateway attachment metrics.
Work with Network Manager
Global networks
Network manager is not regional resource, we wouldn’t need to select a region when create network manager global network.
From VPC console, select network manager, then select “create a global network”.
Transit gateway registrations
From global network console, let’s register both tgw-1 and tgw-2 with the global network.
Metrics and Events
Metrics can be easily monitored from network manager monitoring tab. From the filter drop down box, we could select a transit gateway or specific attachment.
To work with CloudWatch events, we will onboard to CloudWatch Log Insights. Once we onboard, from network manager events tab, it will show events if any.
Route Analyzer
The Route Analyzer analyzes the routing path between a specified source and destination, and returns information about the connectivity between components.
Now let’s investigate the route from 10.0.1.241 (us east 1a) to 172.31.42.15 (us west 2b).
The status is “not connected”, as indicated by the analysis report, missing the route to the destination.
Let’s add the route from US East 1 VPC transit gateway route table console. When destination is US West 2b subnet IP, hop to transit gateway peering attachment.
We fixed Forward path, but return path still has issue.
We fix the issue after add static route to US West 2b VPC transit gateway route table.
(Notes: the route analyzer starts from transit gateway, it does not show the subnet level route and whether it sends the traffic from the subnet to the transit gateway, so you have to make sure the subnet level route table has the route to hop to transit gateway, this is irrelevant to status showed in the route analyzer report.
Make sure configure source side subnet level route table and destination side subnet level route table)
Validation
Create Amazon Linux EC2 instances in both regions, install and start nginx. In case you want to customize index page, go to /usr/share/nginx/html to update index page.
sudo amazon-linux-extras install nginx1
sudo systemctl start nginxFrom US East 1 EC2 instance, let’s open the US West 2 EC2 Nginx home page via its private IP.
curl http://172.31.42.15We got the response.
Same thing when connect to US East 1 Nginx server from US West 2 EC2 instance, http://10.0.1.241 we got the response back.
Interpret Flow Log
I created flow log using default format, now let’s inspect the log entry.
Log entry default format is document in below link.
We could filter flow log by time range, we could also filter by key word. Below is example of log entries which contain 172.31 and time range is within two hours.
Now let’s take a closer look at first entry, it’s an outbound traffic from transit gateway attachment network interface.
2 <aws account #> eni-045091a4de2812c32 172.31.36.202 172.31.42.15 46358 80 6 6 396 1625867134 1625867135 ACCEPT OK- version 2 log
- source IP is 172.31.36.202, source port is 46358
- destination is 172.31.42.15 (IP address for our US West 2 region EC2), source port is 80
- protocol is TCP (number 6 stands for TCP)
- 6 — packets, 396 — bytes
- Start — Unix seconds 1625867134, end — Unix seconds 1625867135
Conclusion
To simplify network operations and administration, AWS Transit Gateway Network Manager provides a centralized and consistent user experience.
I have read through AWS transit gateway network manger user guide, it didn’t mention how the tool is integrated with VPC flow logs. So I assume there is no direct integration. Since network manager is integrated with CloudWatch, I send the transit gateway flow log data to CloudWatch and explored the logs from there.
I couldn’t locate flow logs from network manager, it’s not in the events console and it’s not in the monitoring console.
According to AWS doc, following three network manager events are shown up in network manager dashboard. So it’s not supposed to show flow log event.
- Topology changes
- Routing updates
- Status updates
Anyway from CloudWatch Console, we are able to retrieve transit gateway NIC level flow log.
If you are wondering what’s the Azure equivalent, Azure monitor solution is the closest one to AWS network manager.
References
Transit gateway design best practices — Amazon Virtual Private Cloud
Scenarios for Transit Gateway Network Manager — Amazon Virtual Private Cloud
