If you are doing any type of protocol normalization on your network’s perimeter firewall for application security, please don’t forget that the Internet is changing.

Many networks use a firewall to filter traffic by Internet Protocol address and by TCP or UDP port number. The most simple type of filtering is by IP address or protocol, as the firewall only has to inspect the header of the IP datagram.

To do more targeted filtering, the firewall needs to inspect the header of the encapsulated packets to see any source or destination port numbers for the protocol. Some network firewalls will go even deeper and look inside the encapsulated packets to inspect the application protocols. Parsing the common protocols such as HTTP inside a TCP/IP packet or DNS inside a UDP/IP packet gives a firewall administrator even greater options for policy enforcement.

Many security vulnerabilities in software are simply related to lack of foresight by the software developer. When implementing a protocol, the software developer may program in methods to handle every function implemented in the specifications for the protocol, but forget to address possibilities not specified. For example, if a developer reads that there are three possible options for a setting, the program may support all of the three options correctly, but error on an unhandled exception when more than one option is specified at the same time.

If the firewall knows how the application protocols are supposed to be used, traffic which violates these protocols can be stopped before it reaches your network. This can prevent any Internet services that your network offers from having to deal with anything unexpected and block vulnerability probing from outside of your network, whether you have any vulnerable services or not. This is commonly known as protocol normalization or protocol compliance filtering and is a common application security method.

However, what happens when the standards for these protocols change? When implementing protocol normalization, you need to make sure that your firewall vendor is releasing timely software updates to deal with changes, and that your firewall administrator is applying the updates.

A recent change that is causing problems for some firewall administrators is in RFC 4035, which includes the second implementation of the Domain Name System Security Extensions, also known as DNSSEC. The new security extensions provide protection of DNS transactions with data origin authentication, data integrity verification, and authenticated denial of existence capabilities. However, versions of SmartDefense released in 2006 on Check Point firewalls will, by default, block DNS traffic that contains DNSSEC flags. With an outdated SmartDefense, your perimeter firewall may be blocking all DNS queries from security aware resolvers. You may not even be aware of this, as only a small portion of DNS resolvers are currently security aware. You’re unlikely to notice for some time that a small percentage of people can’t reach your web site or send you e-mail, and since most others aren’t having a problem, you may even blame the other affected party.

A similar issue can happen by implementing SMTP proxy gateways. The Cisco PIX security appliances have an SMTP protocol fixup feature that allows the security appliance to act as an SMTP proxy. The PIX pretends to be your e-mail server, enforcing protocol compliance before passing the e-mail message on to the actual e-mail server behind it. Unfortunately, versions of Cisco IOS prior to 7.2(2.19) and 8.0(2.7) fail to parse DomainKeys in e-mail message headers, causing delivery to fail and resulting in messages being undelivered. This is ironic because development by Cisco was key to the current implementation of DomainKeys Identified Mail.

Doing protocol normalization is certainly taxing on the vendors that offer the feature in their products. Not only is it difficult to keep current with the constantly changing Internet, but it’s also a struggle to get customers to update the products. The vendors are often years behind where they need to be for early adoption of changes, and customers seem to be even further behind.

Protocol normalization is a good idea, but it’s not something that can be enabled and forgotten about, and definitely not something that should be enabled on an old firewall that hasn’t been updated. Ironically, organizations that need the added security the most, such as organizations that don’t do regular firewall maintenance, will have the most problems.