Solution:
Root Cause: AWS Stateful Security Group Rule Mismatch or Stateless NACL Return Path Block
AWS networking uses two filtering layers:
Security Groups (stateful, applied to ENIs) and
Network Access Control Lists (NACLs) (stateless, applied to subnets). While Security Groups automatically allow return traffic, NACLs require explicit rules for both Inbound traffic and Ephemeral Outbound return ports (TCP 1024-65535).
# Step-by-Step Fix:
Query CloudWatch Insights on VPC Flow Logs:Execute CloudWatch Log Insights query to isolate REJECT actions: text
fields @timestamp, srcAddr, dstAddr, srcPort, dstPort, action, logStatus
| filter dstAddr = '10.0.1.50' and action = 'REJECT'
| sort @timestamp desc
| limit 20
Execute AWS VPC Reachability Analyzer Path Trace:Run path analysis via AWS CLI from Source ENI/EC2 to Destination ENI/EC2: bash
aws ec2 start-network-insights-analysis --network-insights-path-id nip-0123456789abcdef0
Inspect the output for status NOT_REACHABLE and examine the specific ExplainingCode (e.g., ENI_SECURITY_GROUP_MISMATCH or NACL_INBOUND_ENTRY_NOT_FOUND).Audit Inbound/Outbound Stateless NACL Rules:Ensure the subnet NACL contains an outbound rule allowing ephemeral ports 1024-65535 back to the source subnet.Update Security Group Inbound Rule:Add an explicit ingress rule on the target instance's Security Group targeting the correct source CIDR or Security Group ID.