Configuring Virtual Private Network Connections
In the book A Gift From Earth, author Larry Niven introduced a character with an unusual talent. It seems that whenever this character got nervous, folks would just start looking away from him. They didn't realize they were doing this, and were surprised when it was pointed out to them. All the same, they never failed to look away if the character were nervous or scared enough to trigger the effect. He might as well have been invisible. Niven called this phenomenon "Plateau Eyes" because the book was set on top of a plateau miles above the surface of a planet.
I bring up this example for two reasons. First, I've always thought that working in IT is like being on top of a miles-high plateau where every fall is a long, long fall. Second, in the cruel world of the Internet, it would be nice if your data could induce "Plateau Eyes" on all the nasty folks out there who want to see your secrets. That's the premise of a Virtual Private Network, or VPN.
In a VPN, two entities exchange data over a public network but the data and the identity of its end-points are encrypted so that they cannot be read by someone sniffing the wire. The encrypted packets are assembled at each end and decrypted using cipher keys known only to the end entities.
This process of encrypting data from one pair of entities and stuffing it into the payload of packets handled by another set of entities is called tunneling. Windows Server 2003 supports two tunneling methods: Point-to-Point Tunneling Protocol (PPTP) and Layer 2 Tunneling Protocol (L2TP). Deciding which one to use, and configuring it to get the security you expect, requires knowing some details about their operation.
PPTP encapsulates encrypted messages into a standard PPP frame using a special protocol called Generic Routing Encapsulation, or GRE.
PPTP requires two data streams: a control channel negotiated over TCP port 1723 for setting up and tearing down the VPN connection and a tunnel channel negotiated over IP using protocol 47, which contains the encrypted data inside the GRE packets.
A PPTP connection mimics a PPP dial-up transaction except that the client already has a network connection to the RRAS server. The client sends its PPTP connection request to the RRAS server and is authenticated using MS-CHAPv2 by default, although you can elect to use EAP with smart cards for greater security.
After the PPTP link has been established, all traffic passing to and from the client and the RRAS server is encrypted within GRE packets. This uses a great deal of overhead. For instance, using a simple TYPE command to look at the contents of a 10-byte file across a PPTP connection generates nearly a dozen frames carrying GRE packets. This is because the SMB and TCP preliminaries are encrypted right along with the data.
The end-point of a PPTP connection at the RRAS server requires no special configuration. When you enable remote access at the server, five PPTP virtual WAN interfaces are created by default. You can increase this number if you need more concurrent connections. The setting is in the Properties window of the Ports icon in the RRAS console. Highlight the PPTP protocol and click Configure to open a configuration window. The Maximum Ports setting controls the concurrent connection limit.
Data in PPTP is encrypted using the same Microsoft Point-to-Point Encryption (MPPE) protocol used by PPP encryption. The underlying encryption algorithm is streaming RC4 with a cipher key refresh every 256 frames.
As discussed in the "PPP Authentication" section earlier, the original MPPE protocol was significantly flawed. Windows Server 2003 (and every other current Windows remote access service) uses PPTPv2, which has been strengthened considerably but is still the object of criticism.
As it turns out, the similarity of PPTP to standard PPP turns out to be its greatest weakness. This is because the authentication phase occurs outside of the encrypted tunnel, so it's possible for a bad guy to devise an exploit to shanghai the transaction before the encryption phase begins. This is one reason for implementing smart cards if you decide on using PPTP for your VPN.
L2TP avoids the weaknesses in PPTP by embedding the authentication phase into the encrypted transactions and by digitally signing all message traffic. If PPTP is like a bonded courier service, L2TP is like a Brinks armored courier.
Still, PPTP is not exactly a sieve, and it does have the advantage of simplicity. All you need for building a PPTP-based VPN is two Windows entities (or Linux, if you install the PoPToP package) that can see each other on a network. A Windows 98 desktop can set up a PPTP VPN to an XP desktop, although the reverse is a little trickier to set up. This is not true of L2TP, which is only supported on modern Windows platforms and requires a PKI because it uses certificates.
As you might have guessed from the name, L2TP works at Layer 2 of the OSI stack, as opposed to PPTP, which is an application-layer protocol. L2TP works in concert with IPSec to encrypt and digitally sign traffic between the client and the RRAS server.
L2TP works by packaging an entire packet into the payload of another packet. It is theoretically possible to use L2TP simply to package traffic into a stream of anonymous UDP datagrams. Marrying IPSec with L2TP results in an end-to-end system that meets the fundamental security requirements of privacy, authentication, integrity, and non-repudiation.
IPSec encrypts an entire packet and digitally signs it to prevent man-in-the-middle and spoof attacks. The protocol used to create the encrypted packet is called Encapsulating Security Payload, or ESP. This protocol supports several encryption protocols, including DES, 3DES, and Advanced Encryption Standard (AES). By default, Windows Server 2003 and XP uses 3DES, although this might change to AES after the new standard achieves wide support.
L2TP takes the ESP packet and makes it the payload of an otherwise anonymous UDP packet. This is how it "tunnels." The tunnel is initiated by an exchange of X.509 certificates between the end-point computers before the client name is even announced on the wire. This avoids exposing the authentication stage of the connection, which can use MS-CHAPv2, RADIUS, or EAP with smart cards.
Figure 20.41 shows a packet trace that highlights the various components of an L2TP frame. The SPI designator you see in the trace stands for Security Parameter Index, a number that uniquely ties the contents of the encrypted payload with a clear-text header. This prevents spoofing.
Figure 20.41. Packet trace of L2TP frame containing ESP payload.
The IP header parser identifies the packet as Encap Security Payload (ESP) for IPv6. This is because IPSec was initially developed with IPv6 support. ESP will accept IPv4 packets.
The ESP sequence number is important because L2TP does not use TCP so it has no means of guaranteeing sequential delivery. IPSec must keep track of its own sequence numbers and request resends if there are gaps.
The most difficult part of deploying IPSec is distributing certificates to the computers that participate as end entities. The simplest way to do this in a Windows Server 2003 PKI is to use group policies.
There is a policy called Automatic Certificate Request that defines what certificates will be assigned to a computer. By defining this policy to require Computer certificates, machines that fall under the influence of the policy will auto-enroll without any intervention on the part of an administrator. This policy is located in Computer Configuration | Windows Settings | Security Settings | Public Key Policies.
Group policies can only enforce Computer auto-enrollment for Windows Server 2003, XP, and Windows 2000 machines. These are the only Windows platforms that support L2TP, so the limitation is moot.
L2TP Transactions and IPSec
An L2TP connection transaction uses a protocol called ISAKMP, which stands for Internet Security Association and Key Management Protocol. This protocol resembles a meeting between two neurosurgeons. They identify themselves, agree on terms of the meeting, and then begin speaking in an incomprehensible jargon. Figure 20.42 shows a packet trace of a frame containing an ISAKMP packet.
Figure 20.42. Packet trace of frame containing an ISAKMP packet.
It is worth noting that this packet contains the initial connection transaction from an L2TP client to an RRAS server. This packet starts the negotiation for a mutually acceptable Security Association, or SA. An IPSec SA defines policies under which the two entities can exchange information. An SA is defined in an IPSec policy. An L2TP SA policy file is available by default on all Windows Server 2003, XP, and Windows 2000 computers.
After the entities agree on an SA, they authenticate each other by engaging in a key exchange. ISAKMP supports exchanging X.509 certificates or creating dynamic session keys using Diffie-Hellman. Windows L2TP only uses certificate exchange.
The SA negotiation and certificate exchange are the only bits of clear text sent between the two entities. After they have validated each other's certificates, the entities encrypt all messages. This means that the remaining portion of the authentication phase occurs within the tunnel.
IPSec uses UDP port 1701 and requires only one channel for both data and control. This avoids the separate control channel in PPTP.
L2TP is by far the more secure of the two tunneling protocols supported by Windows Server 2003, but it has a few deployment challenges.
First of all, you need an enterprise PKI with Certification Authorities available to any computers engaging in an L2TP transaction. It does not need to be a Microsoft PKI, but if you use a third-party PKI, you will need to arrange for obtaining computer certificates without the convenience of auto-enrollment.
Second, L2TP is very processor intensive, much more so that PPTP. This is because the 3DES encryption used by IPSec is more complex than the MPPE encryption used in PPTP, and the system must check the digital signature of each packet. You can improve IPSec performance significantly by installing network adapters that have on-board IPSec co-processors. Every major network card vendor makes such a card.
Finally, and probably most important, IPSec is not "firewall-friendly" because ESP exposes no port information that can be used by NAT. There are ways to "tunnel" IPSec through a NAT firewall, but the complexity of this workaround gets out of hand pretty quickly. Check references from your particular firewall vendor and be sure to have its tech support number handy as you try to implement the instructions. Windows Server 2003 includes support for the latest RFCs that are designed to implement IPSec tunneling through firewalls; however, this technology is still in its infancy, and you'll need to do a lot of tweaking to get any results.
You can avoid firewall problems with IPSec by terminating the L2TP tunnel at an RRAS server in a DMZ. Figure 20.43 shows an example of this configuration. This has the added benefit of permitting the firewall to do a stateful inspection of the unencrypted traffic as it enters the main network. This avoids stealth viruses that might come in under the radar inside an encrypted payload.
Figure 20.43. L2TP-based VPN terminating at RRAS server in DMZ.
Setting Up VPN Connections
PPTP and L2TP virtual adapters are created automatically when you enable the remote access portion of RRAS. No additional configuration is required to accept a PPTP connection. For L2TP, the RRAS server and the client both need a computer certificate. If you do not want to enable auto-enrollment for these certificates, obtain one as follows in Procedure 20.16 (the steps assume you have a Windows Server 2003 or Windows 2000 CA available).
Procedure 20.16 Manually Obtaining a Computer Certificate
Log on to the machine using an account with administrator privileges.
Open an empty MMC console.
Load the Certificates snap-in using FILE | ADD/REMOVE SNAP-INS from the main menu. Select the Computer Account option rather than My User Account.
Expand the tree to Certificates | Personal.
Right-click the Personal icon and select REQUEST NEW CERTIFICATE from the flyout menu. The Certificate Request Wizard opens.
Click Next. The Certificate Types window opens. Select Computer.
Click Next. The Certificate Friendly Name and Description window opens. Leave these blank.
Click Next. A summary window opens. Click Finish to obtain the certificate.
After you have obtained Computer certificates at the RRAS server and the client, you can configure the client to make a VPN connection. This configuration varies depending on the platform. The key item is to have network connection available to the RRAS server and to specify the IP address of the RRAS server in the client configuration.