Remote Access VPN performance recommendations for COVID-19

Obligatory Disclaimer – this is my personal blog and not official guidance from my employer. Before making any changes to your environment based on the information below, below contact your account team, customer support, or professional services representative.

Official guidance from Cisco can be found here:

As the Number of Remote Workers Rises, Cisco Supports Customers with Expansion of Free Security Offerings

AnyConnect Implementation and Performance/Scaling Reference for COVID-19 Preparation

Obtaining an Emergency COVID-19 AnyConnect License

I’ve spent many hours this week discussing this topic with customers, so I thought I’d use this page to summarize some of the common questions and provide configuration examples. I will use this as a living document and update as new/changed information comes in (and people flame me in the comments). Also, this information will be focused on RAVPN for ASA code – please see the above and Cisco docs below for more information regarding RAVPN on FTD.

First up, RTFM never hurts:

ASA VPN Configuration Guide – CLI

ASA VPN Configuration Guide – ASDM

(Book) Cisco ASA: All-in-one Next-Generation Firewall, IPS, and VPN Services, 3rd Edition

(Book) Integrated Security Technologies and Solutions – Volume II: Cisco Security Solutions for Network Access Control, Segmentation, Context Sharing, Secure Connectivity and Virtualization

Top 10 ASA Firewall and VPN Troubleshooting Techniques/Commands – CP-1005

Deploying AnyConnect SSL VPN with ASA (and Firepower Threat Defense) – BRKSEC-2501 (Session PDF here)

How many RAVPN sessions does unit X support?

It’s really, really, really important to note here that the per-session VPN throughput will greatly depend on the traffic profile(s) of the clients. The effective “goodput” per session is not simply the throughput of the box / number of remote sessions; you have to consider the mix of average packet types, sizes and the crypto algos used to estimate the actual number of sessions. You may find that you’ll need to adjust the total amount of connections allowed to something lower than the platform can support to meet the minimum performance you’d like to meet for each session.

Model (-X)ASAv5ASAv10ASAv30ASAv50AWS/Azure
Sessions5025075010000250

Cisco ASAv Data Sheet
Model (-X)55065508551655125515552555455555
Sessions5010030025025075025002500

Cisco Firepower NGFW Data Sheet
Model (FPR)10101120114011502110212021302140
Sessions7515040080015000350007500010000
Cisco Firepower 1000 Series Data Sheet Cisco Firepower 2100 Series Data Sheet
Model (FPR)4110411541204125414041454150
Sessions100001500015000200020000200020000
Cisco Firepower 4100 Series Data Sheet
Model (9300)2436403×4448563×56
Sessions20000200002000060000200002000060000
Cisco Firepower 9300 Series Data Sheet

Will platform X support client Y running software Z?

With the scramble to meet the increased load, you may find that you have to pull out that old unit sitting in a warehouse or a lab. This will inevitably bring questions regarding the software it can support, etc. These resources should help answer some of those questions:

https://www.cisco.com/c/en/us/td/docs/security/asa/compatibility/asamatrx.html

https://www.cisco.com/c/en/us/td/docs/security/asa/compatibility/asa-vpn-compatibility.html

Optimizing Performance

Full Tunnel or Split Tunnel?

Full Tunnel with no hair-pining or Internet access. This one is pretty simple – don’t allow the traffic from the RAVPN pool address range to “hairpin” e.g go back out the firewall to the internet, or proceed through the network to your Internet Gateway.

Full Tunnel with an exclude for SaaS subnets. Ok, this is technically split tunneling. In this example the client has a default route over the tunnel, with exclusions only for the ranges you define. You can find the domain and subnet information for Webex and Office 365 here: How Do I Allow Webex Meetings Traffic on My Network and Office 365 URLs and IP address ranges

access-list WEBEX_SUBNETS standard permit 64.68.96.0 255.255.224.0 
access-list WEBEX_SUBNETS standard permit 66.114.160.0 255.255.240.0 
access-list WEBEX_SUBNETS standard permit 66.163.32.0 255.255.224.0 
access-list WEBEX_SUBNETS standard permit 170.133.128.0 255.255.192.0 
access-list WEBEX_SUBNETS standard permit 173.39.224.0 255.255.224.0 
access-list WEBEX_SUBNETS standard permit 173.243.0.0 255.255.240.0 
access-list WEBEX_SUBNETS standard permit 209.197.192.0 255.255.224.0 
access-list WEBEX_SUBNETS standard permit 216.151.128.0 255.255.224.0 
access-list WEBEX_SUBNETS standard permit 114.29.192.0 255.255.224.0 
access-list WEBEX_SUBNETS standard permit 210.4.192.0 255.255.240.0 
access-list WEBEX_SUBNETS standard permit 69.26.176.0 255.255.240.0 
access-list WEBEX_SUBNETS standard permit 62.109.192.0 255.255.192.0 
access-list WEBEX_SUBNETS standard permit 69.26.160.0 255.255.240.0 
access-list WEBEX_SUBNETS standard permit 207.182.160.0 255.255.224.0

group-policy MY_GROUP_POLICY attributes
 split-tunnel-network-list value WEBEX_SUBNETS 

Full Tunnel with a Filter – Applying a VPN filter at the Group Policy level controls what traffic can traverse the tunnel, either from the client (after decryption) or toward the client (prior to encryption). As this processing is done at the headend I’m not sure of the performance impact – so test, test, test! This example blocks access to the 192.168.0.0/24 network. Also, note that this filter is not shown at the client.

access-list FILTER_AT_ASA extended deny ip any object 192.168.0.0-NET 
access-list FILTER_AT_ASA extended permit ip any any 
group-policy MY_GROUP_POLICY attributes
 vpn-filter value FILTER_AT_ASA

Full Tunnel with a Firewall rule to block selected traffic – with this configuration the client will still have a default route over the tunnel, but a host firewall entry (ACL) will applied to the tunnel interface. You can use this to block traffic at the client versus at the Firewall, thus saving bandwidth/processing at the headend. This example blocks access to the 172.16.100.0/24 network at the host.

access-list HOST_FW_FILTER extended deny ip any object 172.16.100.0-NET 
access-list HOST_FW_FILTER extended permit ip any any 

group-policy MY_GROUP_POLICY attributes
 webvpn
  anyconnect firewall-rule client-interface private value HOST_FW_FILTER

Dynamic Split Tunneling – This really useful feature can be used to provide split-tunneling based on domain. It’s most useful for allowing the client to access SaaS applications such as Webex Meetings or Office365 (or Netflix!) directly instead of over the tunnel. You may also need to add specific routes required by the application to the split tunnel rule if some flows are initiated without a DNS lookup, or the traffic goes to a domain that is not listed in the rule. Here’s an example with Cisco Webex and some of the Office 365 domains. When the client performs a DNS lookup for any of the listed domains, the corresponding route is added to the split-tunnel exclude list we defined earlier.

anyconnect-custom-data dynamic-split-exclude-domains SAAS_DOMAINS webex.com,outlook-dod.office365.us,webmail.apps.mil,attachments-dod.office365-net.us,mail.onmicrosoft.com,autodiscover-s-dod.office365.us,protection.apps.mil,protection.office365.us,dod.teams.microsoft.us,online.dod.skypeforbusiness.us,dps.mil,sharepoint-mil.us,wns.windows.com,g.live.com,odc.officeapps.live.com,officeclient.microsoft.com,oneclient.sfx.ms,dod.online.office365.us,dod.cdn.office365.us

group-policy MY_GROUP_POLICY internal
group-policy MY_GROUP_POLICY attributes
 anyconnect-custom dynamic-split-exclude-domains value SAAS_DOMAINS
 webvpn
  anyconnect firewall-rule client-interface private value HOST_FW_FILTER

Crypto bias

You can change the allocation of cryptographic cores on Symmetric Multi-Processing (SMP) platforms to increase the throughput of AnyConnect TLS/DTLS traffic. These changes can accelerate the SSL VPN datapath and provide customer-visible performance gains in AnyConnect, smart tunnels, and port forwarding.

hostname(config)# crypto engine ?

configure mode commands/options:
accelerator-bias 
Specify how to allocate crypto accelerator processors

hostname(config)# crypto engine accelerator-bias ?
configure mode commands/options
balanced - Equally distribute crypto hardware resources
ipsec - Allocate crypto hardware resources to favor IPsec/Encrypted Voice (SRTP)
ssl - Allocate crypto hardware resources to favor SSL

hostname(config)# crypto engine accelerator-bias ssl


Tunnel Optimization

Like the crypto bias above, this feature enables further performance tuning for tunneled traffic from the AnyConnect VPN client:.

anyconnect-custom-data TunnelOptimizationsEnabled False false
anyconnect-custom-data TunnelOptimizationsEnabled True true

group-policy MY_GROUP_POLICY attributes
 anyconnect-custom TunnelOptimizationsEnabled value True

Per-Tunnel QoS

Per-Tunnel QoS performs egress traffic shaping for tunneled traffic toward the client i.e. the download speed from the remote users perspective. Using this can help ensure each session Tunnel Group gets a fair share of the upstream bandwidth available. The TCP apps running over the tunnel will ultimately settle on on the best rates, but this helps out just a bit.

tunnel-group MY_CONNECTION_PROFILE type remote-access
tunnel-group MY_CONNECTION_PROFILE general-attributes
 default-group-policy MY_GROUP_POLICY

class-map MY_GROUP_MAP
 match flow ip destination-address
 match tunnel-group MY_CONNECTION_PROFILE

policy-map PER_TUNNEL_QOS
 class MY_GROUP_MAP
  police output 1000000 (Bits-per-second)

service-policy PER_TUNNEL_QOS interface outside

Timers

Idle timer – use this to terminate sessions that have gone idle for the specified number of minutes. Note that this should be shorter than the session keepalive interval if enabled.

hostname(config)# group-policy MY_GROUP_POLICY attributes
hostname(config-group-policy)# vpn-idle-timeout 15

Keepalive – enabled by default. Disable this for the idle disconnect to work as expected:

[no] anyconnect ssl keepalive {none | seconds}

hostname(config)# group-policy MY_GROUP_POLICY attributes
hostname(config-group-policy)# webvpn
hostname(config-group-webvpn)# anyconnect ssl keepalive 300

Given the chatty nature of apps today, I don’t see many environments where the keepalive nor idle timers are actually useful, but your mileage may vary.

Dead-Peer Detection – use the recommended settings (300) to limit detection to 5 minutes. Use high limits to lessen the load on the platform at the risk of users having “dead” sessions they are not aware of.

hostname(config)# group-policy MY_GROUP_POLICY attributes
hostname(config-group-policy)# webvpn
hostname(config-group-webvpn)# anyconnect dpd-interval gateway 300
hostname(config-group-webvpn)# anyconnect dpd-interval client 300

Simultaneous logins – if possible, limit simultaneous logins.

hostname(config)# group-policy MY_GROUP_POLICY attributes
hostname(config-group-policy)# vpn-simultaneous-logins 1
hostname(config-group-policy)#

Maximum session time – use this to set the maximum session duration (in minutes) that a user can remain connected. Consider adding a notification to the user before the session is dropped. The following shows a 60 minute session with an alert to the user at the 45 minute mark.

hostname(config)# group-policy MY_GROUP_POLICY attributes 
hostname(config-group-policy)# vpn-session-timeout 60 
hostname(config-group-policy)# vpn-session-timeout alert-interval 15

VPN Load Sharing

This has been a really hot topic with everyone scrambling to maximize the sessions a site can support.

If you have a remote-client configuration in which you are using two or more ASAs connected to the same network to handle remote sessions, you can configure these devices to share their session load. This feature is called load balancing. Load balancing directs session traffic to the least loaded device, thus distributing the load among all devices. It makes efficient use of system resources and provides increased performance an availability. To implement load balancing, you group together logically two or more devices on the same private LAN-to-LAN network into a virtual cluster.”

VPN Load-Balancing Algorithm

The master device maintains a sorted list of backup cluster members in ascending IP address order. The load of each backup cluster member is computed as an integer percentage (the number of active sessions). AnyConnect inactive sessions do not count towards the SSL VPN load for load balancing. The master device redirects the IPsec and SSL VPN tunnel to the device with the lowest load until it is 1 percent higher than the rest. When all backup cluster members are 1% higher than the master, the master device redirects to itself.”

VPN Load-Sharing configuration Guide

  • VPN load-sharing is NOT supported on units running in multi-context mode
  • The algorithm is based on load of the units – it does NOT track the state of the address pools. You must ensure you have adequate IP addresses to service the anticipated load
  • This function is limited to AnyConnect clients using SSL or IPSEC
  • You can mix ASA device types e.g. 5555 and 5585/SSP-10 in a virtual cluster, but each unit should run the same code
  • Ensure that the Virtual IP/FQN and individual cluster member IPs and FQDNs are included in the CN and SAN of the certificate
### Unit #1 ###
ip local pool RAVPN_POOL 10.0.100.100-10.0.100.110 mask 255.255.255.0
prefix-list PFX-RAVPN-POOL description RAVPN_Pool
prefix-list PFX-RAVPN-POOL seq 10 permit 10.0.100.0/24 ge 32
!
crypto ikev1 enable inside
!
vpn load-balancing
 redirect-fqdn enable
 priority 1
 cluster key *****
 cluster ip address 172.16.120.5
 cluster encryption
 participate

### Unit #2 ###
ip local pool RAVPN_POOL 10.0.101.100-10.0.101.110 mask 255.255.255.0
prefix-list PFX-RAVPN-POOL description RAVPN_Pool
prefix-list PFX-RAVPN-POOL seq 10 permit 10.0.101.0/24 ge 32
!
crypto ikev1 enable inside
!
van load-balancing
 redirect-fqdn enable
 cluster key *****
 cluster ip address 172.16.120.5
 cluster encryption
 participate

### Common ###
route-map VPN-TO-OSPF permit 10
 match ip address prefix-list PFX-RAVPN-POOL
 set metric 100
!
router ospf 100
 network 10.2.200.0 255.255.255.0 area 0
 area 0
 log-adj-changes
 redistribute static subnets route-map VPN-TO-OSPF

Can I use the ASAv to quickly ramp up capacity?

Yes! Well, it depends. If you have the compute capacity on-prem, you can quickly deploy a number of ASAv units to scale up your RAVPN session capacity (work with your Cisco account team to obtain the appropriate licenses). Of course, extreme care must be taken to extend your DMZ/VLAN/VRF, etc. to the compute pool, and the appropriate port-groups, etc. created on your virtualization platform to avoid inadvertently allowing the world access to your data center.

If you have access to a cloud provider such as Azure or AWS you could use the ASAv in the cloud to handle the RAVPN connections and leverage a direct or site-to-site VPN connection to your Data Center to backhaul the client traffic. *** This is highly conceptual – again please work your Cisco account team if your interested in this options ***

ASAv in Azure example

Closing

While it’s certainly not an exhaustive list, I hope you find this information useful. Please feel free to comment with additional information and best practices below.

Categories: General, SecOps

Tagged as: , ,

1 reply

  1. Great write up! You do however have a typo in the supported VPN connections on the 2110, 2120, 2130 appliances. It should be 1500, 2500 and 7500 respectively. You have an extra “0” on each of these reading 15000 35000 75000.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s