Understanding the Dynamic Update Mechanism Tech Note Revision 0.A 2016, Palo Alto Networks, Inc. www.paloaltonetworks.com
Contents Introduction... 3 Types of Updates... 3 Upgrade Architectures... 3 Download from Update Server to Device... 5 Push Update from Panorama to Device... 5 Pull Update from SCP Server/Client... 6 Pull Update from Private Wildfire to Firewall... 6 Installation of Update on Device... 6 URL Filtering Mechanism... 7 Other Considerations... 7 Revision History... 8 2016, Palo Alto Networks, Inc. [2]
Introduction The purpose of this document is to describe in sufficient technical detail the potential mechanisms to allow the updating of the various Dynamic Update files on the various Palo Alto Devices. It is not intended to provide the full technical details of how the Palo Alto Devices do their updates but provide sufficient evidence to allow a Security Accreditor to make an informed decision on the safety and/or vulnerabilities of allowing the updates to take place and which mechanism should be used for their particular environment. Types of Updates The Palo Alto devices have a number of Dynamic Updates which allow the update of regularly changing information within the Device without having to perform a full software update. On top of this there is also the software update mechanism and URL Seed Database download but these are not considered within this document. The following table lists the various types of Dynamic Update and the devices that make use of them. Update Update Interval Devices Update Description Name Affected Apps and Threats 1-5 days PA Firewalls This update is the one of the main updates for the firewall and contains various elements of information primarily including updated and new APP-ID s and Threat Signatures. However, it can also include other dynamic information such as the Geo-Reference database. Apps 1-5 days Panorama This is a cut down version of the Apps and Threats update that simply contains the names but not the definitions of the APP-ID s and Threat Signatures to allow them to be configured into the Panorama configurations for passing to the Firewalls Anti-Virus Daily PA Firewalls Panorama Public Wildfire Private Wildfire WF-500 Content 5 minutes PA Firewalls Panorama Dependent on Threat Detection on WF-500 Daily Table 1 - Dynamic Update Types PA Firewalls Private Wildfire This update contains the latest Anti-Virus definitions and signatures used for scanning the traffic passing through the firewall. This update contains the latest file hash and verdicts, DNS signatures and Malware URL list for zero-day, unknown and APT threats produced from the cloud version of Panorama. This isn t connectivity to the cloud version of Panorama. This is the equivalent of the Public Wildfire update but is retrieved from the Private Wildfire (WF-500). If security policy allows it is also possible for the Private Wildfire to pass any new signatures to the Public Wildfire Service for the benefit of others. This does not pass any data. This contains Meta-data that the Private Wildfire uses to detect and monitor Zero-Day, APT and Unknown threats in its internal VM environment. Upgrade Architectures The following diagram shows the various paths by which Dynamic Updates can be retrieved and installed on the various Palo Alto Devices. 2016, Palo Alto Networks, Inc. [3]
Figure 1 - Dynamic Update Paths 1. The update servers are normally referenced by the FQDN updates.paloaltonetworks.com. As this is a cloud based service these IP addresses may change over time. For tighter control allowing the IP addresses to be known, and therefore defined in any firewall ruleset, the alternate FQDN staticupdates.paloaltonetworks.com can be used which will always resolve to the same IP addresses; 2. Updates pulled by the PA Firewall. Normally this would be via the Management Interface which could potentially be routed to the Internet via another protected network path. It is also possible to use a mechanism known as a Service Route that allows the Firewall to use an interface other than the Management Interface without exposing the Management GUI/CLI. Good practice recommends that this Service Route interface is a loopback interface in a different Security Zone to the interface that provides the Internet connection. This allows for the connection to be passed through a firewall Security Policy and Source IP addresses to be NAT d providing further protection to the firewall; 3. Updates pulled by Panorama. This is currently always done via Management Interface (although the ability to use an alternate interface is to be available soon). The path from the Management Interface to the PA Update Servers should be via a protected path, ideally via a PA or other Next Generation Firewall that can recognise the Palo Alto Updates as a specific application type. Panorama can not only pull down the updates that it uses itself but also the updates for any firewalls that it is managing; 4. Updates pushed to the Firewall. This is basically the same as 2 except that instead of pulling the updates from the Palo Alto Update servers they are instead pushed from Panorama; 5. Updates pulled by Private Wildfire. This is always done via the Management Interface (although the ability to use an alternate interface is to be available soon). The path from the Management Interface to the PA Update Servers should be via a protected path, ideally via a PA or other Next Generation Firewall that can recognise the Palo Alto Updates as a specific application type; 6. Private Wildfire Updates pulled by Firewall. This is normally done via the Management Interface on the Firewall although a Service Route can be configured to use another interface. However, it is always directed at the Management Interface of the Private Wildfire; 7. The various update files are pulled via some out of band mechanism (e.g. via airgap and sheep dip) and loaded onto an SCP server or client device that the various Palo Alto devices can connect to for manual upload; 8,9,10. Updates manually pulled from SCP server when using the CLI or via HTTP upload when going via the GUI. Files are loaded via the appropriate method and the installation is then manually triggered. The firewall is the critical device to keep up to date with the latest malware and threat signatures as it is the primary defence device that actively uses the updated information to monitor and control the data passing through it. On all devices the pulling down of the latest updates and its subsequent installation can be triggered either by a Scheduled Update (e.g. update every day at 02:00) or a manual check, download and install. 2016, Palo Alto Networks, Inc. [4]
Under default conditions the devices would try and establish a connection to https://updates.paloaltonetworks.com.com through the Management Interface to obtain the latest updates working on the assumption that the Management interface has a path to the Internet and, as updates.paloaltonetworks.com is a cloud service, the destination IP s may change dependent on geographic location and service availability, protection of this connection would not be able to filter on destination IP address. However, additional security can be layered onto this connection as follows: Configure the firewall to use https://staticupdates.paloaltonetworks.com which is a defined list of IP Destination addresses that do not change (they just may not be the best for performance). This allows protection of the connection to be filtered on destination IP address; On the firewall only, configure a Service Route for its updates. This allows the firewall to pull the updates via an interface other than the Management Interface, e.g. the interface connected to the Internet or that has been dedicated to the update process and protected via other means. Doing this does not expose the normal management functions to that other interface.; Ensure the update path runs through a Next Generation Firewall that can recognise Palo Alto Updates as a specific application type. This could be done in a number of ways; ensure the path to the Internet runs through another set of firewalls; connect the Management Network to another data interface on the firewall so that the traffic runs through a separate security policy; or set the Service Route to source from a loopback interface within the Firewall. Security policies can then be applied to this traffic. By using an appropriate Next Generation Firewall it can then ensure that the actual traffic being passed across the connection is recognised as a Palo Alto Update application type; The security policy could have a schedule applied to it that matches the update schedule so that the security policy is only active at a time that updates should occur; Updates can be deferred for a period after their release date in case an emergency update is released to fix any further issues or the update is withdrawn, e.g. only install the update if it has been released for more than 24hrs. Perform outbound NAT/PAT on the update traffic to hide the originating location. Download from Update Server to Device Once the connection to the update servers is established the actual update itself goes through a series of steps and checks before it actually becomes live on the firewall. The first step is to get the update file onto the device itself and this is performed as follows: 1. The Firewall performs a full certificate check of the X.509 server certificate to ensure it is issued by an appropriate authority and that it hasn t been revoked. The Firewall can also be configured to perform this check as the traffic passes through a security policy as a double check; 2. The update server checks the Firewall serial number and licenses to ensure it is entitled to receive that update type; 3. The firewall performs a full certificate chain check (when configured, enabled by default in PANOS 8.0 upwards) of the server X.509 certificate (the certificate being issued by a third party CA). As the firewall checks each certificate in the chain, if the certificate has been resigned due to a Man in the Middle attack this would generally be detected. There is also some embedded second level authentication at the application layer using http digest authentication; 4. When the file is pulled down to the firewall, as it passes through the security policy, it is virus and threat scanned for any malware as far as possible. As well as running over a TLS connection the file itself is encrypted and digitally signed so checking of the contents is limited at this point. The file is pulled into a local store in the Firewall Management Plane; 5. The file contains some basic information in its header about the type of update, the hardware its applicable to and a checksum. As the file is saved it is checked against the checksum to ensure that the contents are valid. If the checksum fails, the file is rejected; 6. Until the installation is triggered the file simply sits in the local store. Push Update from Panorama to Device This is another path that an Update can take to get onto the end Device. Instead of the Device downloading the update directly Panorama downloads the update on the devices behalf using the same basic steps as described in the previous section. However, in this case, either according to a schedule or manually triggered, Panorama pushes the update file to the appropriate device(s) and can optionally trigger the installation of that file. 2016, Palo Alto Networks, Inc. [5]
When following this path, apart from the checks described in the previous section, Panorama does not interact with the file. The purpose of this is primarily to reduce the amount of Internet traffic when managing a large number of firewalls so the file is only downloaded once and then distributed internally. Pull Update from SCP Server/Client If it isn t possible to provide a connected path from the Device being updated to the Update Servers, there is an offline mechanism available. The files can be downloaded via some out of band mechanism from the Palo Alto support website and transferred either onto a client machine for HTTP upload via the GUI or an SCP server for upload via the CLI (Wildfire currently can only be updated via automatically or via the CLI). Either way the upload is manually trigger. In this case it isn t possible for the Devices to verify the certificates of the original location that the file came from nor the validity of the client machine or SCP service. It must be assumed that as the person has access to the firewall to trigger the manual upload then that person, the location the files are uploaded from and the path that was taken to get the files to that point is implicitly trusted. However, as the file arrives on the device it will go through the same checksum test to ensure that it hasn t been tampered with, and will be stored in the devices local store until the installation is manually triggered. Pull Update from Private Wildfire to Firewall This mechanism is used to update the hash and signature tables on the firewall with new entries that have been found by the Private Wildfire. The mechanism used is very similar to that used in pulling updates from the update servers but with a few key differences to take account of the fact that the connection is via an internal network. It should be noted that the Private Wildfire system has been FIPS certified from PANOS 7.0 onwards. 1. The firewall establishes an SSL connection to the Private Wildfire using TLS1.1/1.2 and a strong encryption algorithm. As part of the negotiation both the Firewall and the Private Wildfire pass a certificate for validation. This certificate is embedded in PANOS and can be updated via Software Update and/or Content Update should they be compromised or expire; 2. The Firewall/Private Wildfire performs a full certificate check of the X.509 certificate. These certificates are issued by a third party Certificate Authority and when received by either end the CA, Expiry, Constraints and Key Usage are checked for validity against hardcoded parameters in PANOS; 3. When the file is pulled down to the firewall, if it passes through the security policy, it is virus and threat scanned for any malware as far as possible. As well as running over a TLS connection the file itself is encrypted and digitally signed so checking of the contents is limited at this point. The file is pulled into a local store in the Firewall Management Plane; 4. The file contains some basic information in its header about the type of update, the hardware its applicable to and a checksum. As the file is saved it is checked against the checksum to ensure that the contents are valid. If the checksum fails, the file is rejected; 5. Until the installation is triggered the file simply sits in the local store. NOTE: In PANOS 8.0 onwards this mechanism is enhanced to allow the configuration of user supplied client/server/ca certificates. Installation of Update on Device When an installation is triggered, regardless of whether it is automatic or manual, the Update follows the following process: 1. The relevant update file is decrypted and unpacked in the Management Plane ready for processing. During this process the digital signature of the file is checked to ensure the file is legitimate. This is done using a signing keypair with the keypair embedded in PANOS (this is updated via software or content update should it be compromised or expire). The checksum will once again be checked along with any hardware and version dependencies that are listed in the Update file; 2. The file is then processed to extract the information and format it into the correct form for updating the live environment. If there are any internal errors in the file they would be detected and an error produced at this point; 3. For the Panorama and Wildfire, the updated information is then installed in the relevant locations and accepted as live. For the Firewall the update information is then passed to the Data Plane(s) for installation. The Data Plane(s) will then do a further Sanity check before accepting that update as live. 2016, Palo Alto Networks, Inc. [6]
URL Filtering Mechanism The URL Filtering mechanism works differently to the Dynamic Update mechanisms as it can be much more interactive between the Firewall and the Palo Alto PANDB URL Category database. If all the URL filtering is to be performed via Custom URL Categories (i.e. you define a list of URL s and indicate they are all part of a particular category) then there is no specific requirement for the URL filtering seed or online databases. The seed database is downloaded when the firewall is first initialised and licensed and is used to initially populate the URL caches with the most common URL s. If you wish to make use of the standard URL categories, then some access to the various URL Databases will be required. The basic URL filtering mechanism works using URL Categories and these are determined as follows (assuming a valid URL Filtering License is present): 1. Does the URL match the specification of a Custom URL Category? 2. Does the URL match a URL stored in the Data Plane Category cache (initially populated via the seed database)? 3. Does the URL match a URL stored in the Management Plane Category cache? 4. Send the URL to the configured PANDB Database Server to request the category. Normally the configured PANDB Database Server is a server out in the Internet, it is far too big to store directly on the device. As with the Dynamic Updates this connection can be protected using Service Routes and Security Policies using the Application ID pan-db-cloud. The advantage of using this database is that as new compromised websites or updates to categories are made this is automatically and dynamically reflected in the database and so the URL categories on the device are always up to date providing more up to date protection against compromised sites. If it isn t possible to allow access to the Internet base PANDB URL database it is possible to provide an offline local copy of the database on a dedicated M-100/M-500 device specifically configured to act as a PANDB server. This device is loaded with the main database downloaded from the Palo Alto Support Site currently around 1.2GB in size and updated daily. Other Considerations The Firewall is the critical device to keep up to date with the latest Dynamic Updates as it is the main device to actively use those updates to monitor and control the data passing through it it is the first line of defence in the overall capability. The next most critical device would be the Private Wildfire as it uses the updates to help it try to detect unknown, APT and Zero-Day threats. Panorama is the less critical device (unless it is being actively used to update the Firewalls) to keep up to date and the primary purpose of this is so that it knows about the same APP-IDs, Threats, etc as the Firewall for configuration purposes. Given the update periods for the various Dynamic Updates trying to keep up manually will be a very man intensive process and would mean that the firewall signatures will always be out of date. This process could easily take half a day to manually download and complete all the daily updates. Whilst it is understandable to protect the management interfaces of the devices by not providing any form of Internet connectivity this does need to be balanced against the risk of the devices using out of date signatures, especially as there are so many methods to tie down and protect the connection when it happens. If an out of band update mechanism is used then the validation of the file source is lost and risk is introduced that as the Update source cannot be verified an you must implicitly trust the person doing the update and the out of band update mechanism. It is also unlikely that you will be able to do any further checks on the Update file than would already have been done in the automatic process, i.e. a sheep-dip virus check. Whilst desirable to enable automatic updates throughout the system it is possible to use a combination of methods to keep the system up to date. Some organisations have approved a partially automatic measure for doing updates. The firewalls themselves do their Dynamic Updates automatically from the Internet using Service Routes as they already have an Internet connection for data. However, Panorama and Wildfire are manually updated using an out of band mechanism to ensure that there is no electronic connection between the management network and the internet. 2016, Palo Alto Networks, Inc. [7]
Revision History Date Revision Comment 25/11/2016 0.A Initial Revision 2016, Palo Alto Networks, Inc. [8]