How AppScan explores applications with ABE and RBE

Transcription

1 How AppScan explores applications with ABE and RBE IBM SECURITY SUPPORT OPEN MIC To hear the WebEx audio, select an option in the Audio Connection dialog or by access the Communicate > Audio Connection menu option. To ask a question by voice, you must either Call In or have a microphone on your device. You will not hear sound until the host opens the audio line. For more information, visit: April 4, 2018 NOTICE: BY PARTICIPATING IN THIS CALL, YOU GIVE YOUR IRREVOCABLE CONSENT TO IBM TO RECORD ANY STATEMENTS THAT YOU MAY MAKE DURING THE CALL, AS WELL AS TO IBM S USE OF SUCH RECORDING IN ANY AND ALL MEDIA, INCLUDING FOR VIDEO POSTINGS

2 Scheduled Open Mics: today - How AppScan explores applications (ABE, RBE) April 25th - Advanced scan automation in AppScan Enterprise Recorded Open Mic: Mar 14 th - Introduction to the API for Application Security on Cloud Feb 21 - How to transfer a scan from AppScan Standard to ASE Jan 17 th - How to automate scanning with AppScan Enterprise Dec 6 th, What's new in AppScan Enterprise version IBM Security

3 Panelists: Marek Suchecki Presenter AppScan support engineer Joe Kiggen Moderator, AppScan L2 Manager 3 IBM Security

4 Agenda What is Explore? Two methods of Automatic Explore Request-based Explore (RBE) explained Action-based Explore (ABE) explained RBE Pros and Cons ABE Pros and Cons 4 IBM Security

5 What is Explore? A Dynamic Analysis security scan consists of 2 phases: 1. Explore Phase (aka Crawling or Spidering) is a form of reconnaissance. - its purpose is to prepare the inventory of what needs to be tested - it involves preparing a list of URLs, pages, parameters, cookies, etc. - these elements will be tested in the Test Phase NOTE: What is not covered in the Explore Phase, will not be tested! 2. In the Test Phase, AppScan executes the tests that have been prepared in the Explore Phase. In the Explore Phase you may use: Manual Explore - You open a browser in AppScan and navigate through the web site. - Pages that you do not visit will not be tested! Automatic Explore - AppScan performs the spidering on its own, in unattended mode. - The scan will attempt to visit all pages (within configured limits). A combination of Manual Explore and Automatic Explore 5 IBM Security

6 Two methods of Automatic Explore (#1) Today s session is focused on the Automatic Explore and its two available methods: Request-based Explore (RBE), the traditional method Action-based Explore (ABE), the new method, added in AppScan version Both methods have the same purpose, but they use different mechanisms to ensure the complete coverage for the scan. The concepts are similar to: - Request-based Login, the traditional method, and - Action-based Login, the new method, that was introduced in AppScan version 8.8 (about 4 years ago) The Request-based method focuses on parsing HTTP responses. The Action-based method loads the complete pages, renders them in a browser, parses the structure of the page and simulates user actions on the page itself. You can select the method you want AppScan to use for the Automatic Explore phase of the scan. Each method comes with its own advantages, so it is important to enable the right method for the web application under test. It is recommended to enable both methods. 6 IBM Security

7 Two methods of Automatic Explore (#2) In the Scan Configuration dialogue, on the Explore Options page, you can enable or disable each method, independently from the other at: Scan Configuration > Explore Options > Main 7 IBM Security

8 Two methods of Automatic Explore (#3) If both methods are selected, in the Automatic Explore phase AppScan proceeds as follows: 1. First, the Action-based Explore is started. This method will be used: for a predefined maximum duration, or until the web application is fully explored 2. If the maximum duration is reached, and there are still pages left to be visited, the Automatic Explore will be continued using the Request-based Explore method. The maximum duration of the Action-based Explore can be configured in the Scan Configuration > Explore Options > Advanced. The default duration is 30 minutes. 8 IBM Security

9 Request-based Explore explained With the traditional Request-based Explore method (RBE), AppScan works as follows: 1. Sends the HTTP request for the Starting URL (at the beginning, this is the only known URL of the target web application). 2. Receives the HTTP response from the web application. 3. Parses the HTTP response, and looks for: a. links to other pages (and resources) of the application, and adds any new URLs to the list of URLs that are yet to be visited. b. security entities that are worth testing (URLs, Parameters, Cookies,...), and adds any new entities to the list of things to be tested during the Test phase. 4. Selects a new URL from the list of URLs that are yet to be visited. 5. Sends the HTTP request for that URL. 6. Return to Point 2. The process will continue until the user pauses the scan, or until the list of URLs that are yet to be visited is empty (at this point, the Automatic Explore is complete). 9 IBM Security

10 Action-based Explore explained (#1) With the new Action-based Explore (ABE) method, AppScan works as follows: 1. Spawns the browser process, and loads the Starting URL (at the beginning, this is the only known URL of the target web application). 2. Loads all the dependencies referenced on the page (images, JavaScript libraries, cascaded style sheets, etc). 3. Renders the complete page in the browser. 4. Parses the DOM (Document Object Model) of the page, and looks for: a. links to other pages (and resources) of the application, and adds any new URLs to the list of URLs that are yet to be visited b. security entities that are worth testing (URLs, Parameters, Cookies,...), and adds any new entities to the list of things to be tested during the Test phase. 5. Selects a new URL from the list of URLs that are yet to be visited. 6. Loads the new URL in the browser. 7. Returns to Point 2. The process will continue until the user pauses the scan, or the maximum duration is reached, or the list of URLs that are yet to be visited is empty (at this point, the Automatic Explore is complete). You can think of Action-based Explore as an automated Manual Explore. 10 IBM Security

11 Action-based Explore explained (#2) By default, the Action-based browser gets started in the invisible mode. However, you may change it, if you wish to see the browser with the pages it loads, and you wish to monitor the progress of the Action-based Explore. To make the browser visible: 1. Open AppScan menu: Tools > Options > Advanced 2. Find the setting: SessionManagement.ShowActionBasedPlayerWindow 3. Change its value to True, then click OK to validate the change 11 IBM Security

12 RBE Pros and Cons The advantages of the Request-based Explore method: It is usually faster than Action-based Explore: - It runs with up to 10 threads, performing the explore simultaneously (the number of threads is configured in Scan Configuration > Communication and Proxy). - It usually does not need to load the dependencies of the page being visited. - It does not need to render the page in the browser, which may be CPU-intensive. It may also find any content that is not visible to web users, and that an attacker could find (such as links in comments) The main disadvantage is: limited capacity to handle complex client-side web technologies. In some cases (like AngularJS) the Request-based method cannot be used. The Action-based Explore and Manual Explore can be used in those cases. 12 IBM Security

13 ABE Pros and Cons (#1) The advantages of the Action-based Explore method: This method is particularly effective where new technologies such as advanced JavaScript and Session Storage are used, and for sites that use Rich Internet Application (RIA), Single-page Application (SPA), or AngularJS. Rendering pages in a fully capable modern browser process ensures excellent capacity to handle advanced client-side web technologies. Disadvantages: Generally slower than the Request-based method - It runs with a single thread, hence no simultaneous Spidering. - Rendering complex pages in the browser may require a lot of processing. - The browser process needs to load all the dependencies of the page in order to build its complete DOM. It is likely to miss the content that is not visible to web users, which an attacker could find, for example, links in comments. 13 IBM Security

14 ABE Pros and Cons (#2) If a page has a dependency outside the scope of the scan, Action-based Explore has to load it in order to build the complete DOM of the page. However, such dependencies will not be tested, as explained in technote How to exclude files/folders in AppScan Standard scans. The Request rate limit setting (at Scan Configuration > Communication and Proxy) does not apply to Action-based Exploring (it applies only to Request-based Exploring) This is a limitation of ABE, as explained in technote How to limit number of requests sent by AppScan Standard. The technote suggest an alternative approach, allowing to slow down an Action-based Explore. 14 IBM Security

15 Demo

16 Questions for the panel Now is your opportunity to ask questions of our panelists. To ask a question now: Raise your hand by clicking Raise Hand. The Raise Hand icon appears next to your name in the Attendees panel on the right in the WebEx Event. The host will announce your name and unmute your line. or Type a question in the box below the Ask drop-down menu in the Q&A panel. Select All Panelists from the Ask drop-down-menu. Click Send. Your message is sent and appears in the Q&A panel. To ask a question after this presentation: You are encouraged to participate in the dw Answers forum: < 16 IBM Security

17 Where do you get more information? Questions on this or other topics can be directed to the dw Answers forum: More information you can review: Technote: How to exclude files/folders in AppScan Standard scans How to limit number of requests sent by AppScan Standard AppScan Standard versions available Security Learning Academy: Useful links: Get started with IBM Security Support IBM My Support Sign up for My Notifications FREE learning resources on the Security Learning Academy Follow us: 17 IBM Security

18 IBM Security Learning Academy New content published daily! Learning at no cost! Learning Videos Hands-on Labs Live Events 18 IBM Security

19 THANK YOU FOLLOW US ON: facebook.com/ibmsecuritysupport SecurityLearningAcademy.com securityintelligence.com xforce.ibmcloud.com Copyright IBM Corporation All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. Any statement of direction represents IBM's current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. Statement of Good Security Practices: IT system security involves protecting systems and information through prevention, detection and response to improper access from within and outside your enterprise. Improper access can result in information being altered, destroyed, misappropriated or misused or can result in damage to or misuse of your systems, including for use in attacks on others. No IT system or product should be considered completely secure and no single product, service or security measure can be completely effective in preventing improper use or access. IBM systems, products and services are designed to be part of a lawful, comprehensive security approach, which will necessarily involve additional operational procedures, and may require other systems, products or services to be most effective. IBM does not warrant that any systems, products or services are immune from, or will make your enterprise immune from, the malicious or illegal conduct of any party.