Sunday, January 22, 2012

Troubleshooting IPSEC

IPSec Monitoring Tools
IP Security Monitor
  • Used as a MMC snap-in with Windows XP and higher
  • Administrators can use it locally or remotely to monitor IPSec policy
  • Command-line tool
  • Only available in Windows 2000
Windows Firewall with Advanced Security
New to Windows Vista and Windows Server 2008
  • Perform a trace, located in systemroot\debug\oakley.log
  • Enabled in Windows XP and Windows 2000 with registry modification
IP Security Monitor
The IP Security Monitor snap-in is used to view and monitor IP-Sec policy. The IP Security Monitor can be used to troubleshoot and test IPSec policies you create. 
In previous versions of Windows, we used IP Security Monitor as a snap-in in the MMC. With Windows Server 2008, the IP Security Monitor is now integrated with the Windows Firewall with Advanced Security (WFAS). 

Operates in two modes:
  1. Main mode. Main Mode Internet Key Exchange (IKE) negotiation establishes the initial secure channel between two computers. 
  2. Quick mode. The secure channel, known as the Internet Security Association and Key Management Protocol (ISAKMP) security association (SA), between two computers, is used to protect subsequent key exchanges between the two computers and to give you a quick status as to what is happening with IPSec. This is known as Quick mode.
To enable remote management editing, the HKLM\system\currentcontrolset\services\policyagent key must have a value of 1.
IP Security Monitor lets you search for specific main mode and quick mode filters.
Windows Firewall with Advanced Security
Advanced configuration takes place in the Windows Firewall with Advanced Security (WFAS) MMC snap-in. This snap-in provides an interface for configuring Windows Firewall locally, and on remote computers and by using Group Policy. IPSec protections are now integrated with the Windows Firewall.
Connection Security Rules and Security Associations (SA) nodes are used to monitor IPSec connections.
Items that can be monitored:
  • Security Associations
  • Main Mode
  • Quick Mode

Demo – Using IPSec Monitor and Windows Firewall
Windows Firewall with Advanced Security
  1. Start | Administrative Tools | Windows Firewall with Advanced Security
  2. Select Monitoring
  3. Click on Main Mode (in depth view of SAs)
  4. You can see the Security Association (previously created) between the local machine and the remote machine, and the authentication method. 
  5. You can click on Quick Mode (high-level view of SAs) to see the local and remote address and the Ports and Protocols that are allowed.
  6. Close the Windows Firewall with Advanced Security
IPSec Monitor
  1. Start | Run | mmc
  2. File | Add/Remove Snap-in...
  3. Select IP Security Monitor
  4. Under the IP Security Monitor node, you can see the current Active Policy, Main Mode and Quick Mode
  5. Under Main Mode or Quick Mode, drop-down and look at Statistics and Security Associations
Troubleshooting IPSec
Pretend someone in your organization cannot connect to a server on your network that has IPSec deployed. The client machine has rules configured.
Do we first look at IPSec, or do we check to see if there is a network issue?
Stop the IPSec Policy Agent. You can then look through the Oakley logs for pointers. 
Verify firewall configuration settings locally and externally. Use the ping and Net View command to verify communications to rule IPSec in or out as the cause of the problem. If you are able to connect to the network without the IPSec Policy Agent being enforced, it is a good probability the problem is with IPSec
Start the IPSec Policy Agent and use IP Security Monitor to determine if a security association exists. Look at the IPSec policies that are enforced.
Review the Group Policies and IPSec policies and ensure they are compatible. Look at the authentication methods and make sure they match on both the client and the servers. You can use Resultant Set of Policy (RSOP) snap-in to verify the correct application of policy.
Use IP Security Monitor to ensure any changes are applied.

Troubleshooting the Internet Key Exchange (IKE
  1. Identify the connectivity issues related to IPSec and IKE
  2. Identify firewall and port issues. Make sure UDP port 500 and port 4500 is not being blocked. Ensure that IPSec-protected traffic (IP protocol 50 and 51) is not blocked and that the network devices support IPSec.
  3. Review the Oakley.log file in the system root folder (look in the debug directory). The Oakley.log records up to 50,000 lines of IPSec events. Once the 50,000 lines have been reached, the log is copied to a backup file, and a new file is created. So, you may have to review the backup files. Enable the HKEY-LOCAL-MACHINE\System\CurrentControlSet\Services\PolicyAgent\Oakley\EnableLogging registry setting to 1 (stored in the %systemroot%\Debug\folder)

  4. Determine if any Main mode exchange issues regarding initiating the connection are causing a problem.
  5. Ensure both computers have the same time settings
Common Security Event Viewer log codes:
  • Success:
    • 541 – IKE Main Mode or Quick Mode established
    • 542 – IKE Quick Mode was deleted
    • 543 – IKE Main Mode was deleted
  • Information Log Entries:
    • Usually pertaining to denial of service attacks
    • May not be any errors but resources will run low, affecting performance
    • Quick Mode audit failures denoted with 547 error message
Best Practices
    • Establish an IPSec deployment plan.
    • Create and test IPSec policies.
    • Do not use preshared keys. Microsoft does not recommend the use of preshared key authentication because it is a relatively weak authentication method and are stored in plaintext format. Only use preshared keys for testing. Use certificates of Kerberos V5 protocol in a production environment.
    • Use the Triple Data Encryption Standard (3DES) algorithm for stronger encryption.
    • Use Terminal Services to remotely manage and monitor IPSec on computers with different versions of the Windows OS.

1 comment:

  1. Useful topic since,many are not aware of the usefulness of the firewall security.


"Comment As:" anonymous if you would rather not sign into an account!