ArcGIS 9.x MXD Layers Scalability Testing

Transcription

1 Environmental Systems Research Institute, Inc., 380 New York St., Redlands, CA USA TEL FAX Environmental Systems Research Institute, Inc. ArcGIS 9.x MXD Layers Scalability Testing Systems Integration Test Report Prepared by: Ray Echevarria ESRI Systems Integration Department Redlands, California June 23, 2005

2 Copyright 2005 ESRI All rights reserved. Printed in the United States of America. The information contained in this document is the exclusive property of ESRI. This work is protected under United States copyright law and other international copyright treaties and conventions. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, except as expressly permitted in writing by ESRI. All requests should be sent to Attention: Contracts Manager, ESRI, 380 New York Street, Redlands, CA , USA. The information contained in this document is subject to change without notice. U.S. GOVERNMENT RESTRICTED/LIMITED RIGHTS Any software, documentation, and/or data delivered hereunder is subject to the terms of the License Agreement. In no event shall the U.S. Government acquire greater than RESTRICTED/LIMITED RIGHTS. At a minimum, use, duplication, or disclosure by the U.S. Government is subject to restrictions as set forth in FAR Alternates I, II, and III (JUN 1987); FAR (JUN 1987) and/or FAR / (Commercial Technical Data/Computer Software); and DFARS (NOV 1995) (Technical Data) and/or DFARS (Computer Software), as applicable. Contractor/Manufacturer is ESRI, 380 New York Street, Redlands, CA , 3D Analyst, ADF, AML, ARC/INFO, ArcAtlas, ArcCAD, ArcCatalog, ArcCOGO, ArcData, ArcDoc, ArcEdit, ArcEditor, ArcEurope, ArcExplorer, ArcExpress, ArcFM, ArcGIS, ArcGrid, ArcIMS, ArcInfo Librarian, ArcInfo, ArcInfo Professional GIS, ArcInfo The World's GIS, ArcLogistics, ArcMap, ArcNetwork, ArcNews, ArcObjects, ArcOpen, ArcPad, ArcPlot, ArcPress, ArcQuest, ArcReader, ArcScan, ArcScene, ArcSchool, ArcSDE, ArcSdl, ArcStorm, ArcSurvey, ArcTIN, ArcToolbox, ArcTools, ArcUSA, ArcUser, ArcView, ArcVoyager, ArcWatch, ArcWeb, ArcWorld, Atlas GIS, AtlasWare, Avenue, BusinessMAP, Database Integrator, DBI Kit, ESRI, ESRI Team GIS, ESRI The GIS People, FormEdit, Geographic Design System, Geography Matters, Geography Network, GIS by ESRI, GIS Day, GIS for Everyone, GISData Server, InsiteMAP, MapBeans, MapCafé, MapObjects, ModelBuilder, MOLE, NetEngine, PC ARC/INFO, PC ARCPLOT, PC ARCSHELL, PC DATA CONVERSION, PC STARTER KIT, PC TABLES, PC ARCEDIT, PC NETWORK, PC OVERLAY, Rent-a-Tech, RouteMAP, SDE, SML, Spatial Database Engine, StreetEditor, StreetMap, TABLES, the ARC/INFO logo, the ArcAtlas logo, the ArcCAD logo, the ArcCAD WorkBench logo, the ArcCOGO logo, the ArcData logo, the ArcData Online logo, the ArcEdit logo, the ArcEurope logo, the ArcExplorer logo, the ArcExpress logo, the ArcFM logo, the ArcFM Viewer logo, the ArcGIS logo, the ArcGrid logo, the ArcIMS logo, the ArcInfo logo, the ArcLogistics Route logo, the ArcNetwork logo, the ArcPad logo, the ArcPlot logo, the ArcPress for ArcView logo, the ArcPress logo, the ArcScan logo, the ArcScene logo, the ArcSDE CAD Client logo, the ArcSDE logo, the ArcStorm logo, the ArcTIN logo, the ArcTools logo, the ArcUSA logo, the ArcView 3D Analyst logo, the ArcView Business Analyst logo, the ArcView Data Publisher logo, the ArcView GIS logo, the ArcView Image Analysis logo, the ArcView Internet Map Server logo, the ArcView logo, the ArcView Network Analyst logo, the ArcView Spatial Analyst logo, the ArcView StreetMap 2000 logo, the ArcView StreetMap logo, the ArcView Tracking Analyst logo, the ArcWorld logo, the Atlas GIS logo, the Avenue logo, the BusinessMAP logo, the Data Automation Kit logo, the Digital Chart of the World logo, the ESRI Data logo, the ESRI globe logo, the ESRI Press logo, the Geography Network logo, the MapCafé logo, the MapObjects Internet Map Server logo, the MapObjects logo, the MOLE logo, the NetEngine logo, the PC ARC/INFO logo, the Production Line Tool Set logo, the RouteMAP IMS logo, the RouteMAP logo, the SDE logo, The World's Leading Desktop GIS, Water Writes, and Your Personal Geographic Information System are trademarks, registered trademarks, or service marks of ESRI in the United States, the European Community, or certain other jurisdictions. Other companies and products mentioned herein are trademarks or registered trademarks of their respective trademark owners. 2

3 Test Objective and Hypothesis The purpose of this report is to determine how performance (response time, memory and network activity) and scalability (i.e. client/server CPU load) is affected by increasing the number of layers in an ArcMap document (MXD). The actual distribution of the CPU load between server and client as layers are added to the MXD will also be looked at. That is, what percentage of CPU time is spent on the client versus the server for a typical map display when accessing data from ArcSDE? Previous studies have shown that majority of the CPU time is spent on the client. It is expected that response time (RT) and CPU loads will increase as layers are added, however, the CPU processing ratio between client and server should remain constant. Test Hardware and Software Configuration The test configuration consisted of two machines; an ArcGIS Desktop workstation and an ArcSDE/DBMS server. The ArcGIS Client used an Application Server connection to access ArcSDE. The test hardware and software configuration is presented in Figures 1 and 2. Figure 1 ArcSDE Server Configuration Item Configuration Make and Model HP DL380 Operating System & Version Windows Server 2003 Number of CPUs & Type Two Intel Xeon 3.2 GHz Memory (GB) 4 Network Access 100 Mbit configured to public network System Disk (C:) Data Disk (E:) 2 x 36-GB/UltraSCSI Disk, RAID-1 4 x 72-GB/UltraSCSI Disk, RAID-5 Internal Storage Array Software ArcSDE 9.1 Application Server Connection Figure 2 ArcGIS Client Desktop Configuration Item Configuration Make and Model HP Workstation xw4100 Operating System & Version Windows XP SP2 Number of CPUs & Type One Intel Pentium MHz Memory (GB) 1.28 Network Access 100-Mbit configured to public network. System Disk (C:) Single 74.5-GB UltraSCSI Disk Software ArcGIS 9.1 Desktop 3

4 Test Data Description The test data descriptions are presented in Figure 3. The data was in an ArcSDE versioned geodatabase. Figure 3 ArcSDE Test Data Description Layer Data Type Feature Type Features Fire Hydrants ArcSDE Simple Layer Point 23,585 Schools ArcSDE Simple Layer Point 1,091 Roads ArcSDE Simple Layer Line 145,027 Train Routes ArcSDE Simple Layer Line 387 Freeways ArcSDE Simple Layer Line 275 Parcels ArcSDE Simple Layer Polygon 783,159 Parks ArcSDE Simple Layer Polygon 5,239 Lakes ArcSDE Simple Layer Polygon 704 Zip Codes ArcSDE Simple Layer Polygon 106 Base Image ArcSDE JPEG Image Raster - Total 959,573 Test Procedure and Tools A total of eight MXDs were created. The first MXD had 9 vector layers and 1 raster layer. Each successive MXD had 10 additional duplicate layers. The number of layers ranged from 10 in the first document, to 80 layers in the eighth document (Figure 4). Figure 4 ArcMap MXD Layer Distribution Document Layers Features Queried MXD MXD MXD MXD MXD MXD MXD MXD An automated ArcMap read-only script was then used to query 100 predefined extents and the RT per map display was recorded. This was done for all eight MXDs. The Windows XP/Server 2003 Performance Monitor was used to record CPU, memory and network acitivity during testing. 4

5 Test Results and Analysis As expected, the overall response time (RT) per map display increased as the number of layers increased for each MXD (Figure 5). Response time nearly doubled in every case that the number of layers doubled. Figure 5 Response Time Average Response Time (sec) 6.0 Average Response Time (sec) Time (sec) Regarding scalability, doubling the number of features increased both server and client CPU service time by 86 and 90-percent (Figure 6). The percentage increase for client CPU time gradually decreased as additional layers were added. Server CPU time decreased as well, however, it increased only slightly when going from 50 to 60 and from 70 to 80 layers. These were the only two occurrences when the server CPU time increased when adding additional layers. The distribution of the CPU load between client and server was relatively consistent as layers were added (Figures 7 and 8). The CPU load ratio started at approximately 79-percent for client and 18-percent for server (ESRI sizing models are based on a 80/20 client/server distribution). Queue time accounted for the remaining three percent. With 80 layers, client-side processing slightly decreased to 77-percent while server-side processing increased to 21-percent. Figure 6 Client\Server CPU Load Change Layers Client CPU Time (seconds) Client Change (%) Server CPU Time (seconds) Server Change (%)

6 Figure 7 Client and Server CPU Load Distribution Percentages CPU Service Time Distribution Queue/Latency Time Data Server CPU Service Time ArcGIS Client CPU Service Time Percentage Distribution % 2% 0% 1% 2% 2% 1% 2% 18% 19% 21% 20% 20% 21% 20% 21% 79% 79% 79% 79% 78% 78% 78% 77% 6.00 Figure 8 Client Server CPU Loads CPU Service Time Distribution Average Response Time Per Map Display (sec) Queue/Latency Time Data Server CPU Service Time ArcGIS Client CPU Service Time Breaking down the CPU service times at a more granular level reveals the processing load per GSRVR and DBMS process (Figure 9). There is a direct correlation between the number of layers and the GSRVR and DBMS processing times. Both GSRVR and DBMS processing times nearly doubled as the number of layers doubled. Further, remaining CPU time (CPU time not 6

7 accounted for by the GSRVR and DBMS processes) grew as layers were added and was most likely related to the additional amount of I/O being processed. The ArcMap.exe memory consumption also appeared to be directly tied to the number of layers in an MXD. As layers were added, ArcMap process memory consumption grew from 71 MB for 10 layers to nearly 164 MB for 80 layers (Figure 10). The additional memory consumption is a result of new layers being treated as objects. Each layer added to an ArcMap document requires a new COM based-object be generated and retained in memory. Figure 9 CPU Service Time Distribution Data Server CPU Service Time Distribution Detail Average Response Time Per Map Display (sec) Data Server Remaining CPU Time Data Server Process (gsrvr) CPU Service Time Data Server Process (dbms) CPU Service Time Data Server Process (giomgr) CPU Service Time Figure 10 ArcMap.exe Memory Consumption Client Process Working Set MB ArcGIS Client*\Process(ArcMap)\Working Set

8 Regarding network activity, the amount of network traffic per map increased as layers were added to the MXD (Figure 11). As seen earlier with CPU service times, doubling the number of layers in an MXD resulted in the doubling of network traffic per map. Unlike the network traffic, the actual throughput is relatively stable from 30 layers and higher (Figure 12). The reason why network traffic leveled off at around 5 Mbps is because this test only used a single client and the fact that ArcMap can only query and display one layer at a time. Using more clients would have increased the network throughput substantially. Figure 11 Network Performance Network Traffic Per Map Display Network Traffic Per Map Display (KB) ArcGIS Client Netw ork Traffic per Map Display (KB) Data Server Netw ork Traffic per Map Display (KB) Figure 12 Network Throughput Network Throughput Performance Network Throughput (Mbps) ArcGIS Client Netw ork Throughput (Mbps) Data Server Netw ork Throughput (Mbps)

9 Conclusions Adding layers to an ArcMap document increases RT, network traffic and ArcMap client memory consumption. For scalability, testing also showed that additional layers had a minimal impact on server and client CPU load distribution. The CPU load ratio was relatively stable with a slight shift in additional processing on the server side, and a slight decrease on client-side processing. This indicates that increasing the number of layers by a factor of 5x, 6x, 7x or even 8x does not have a dramatic impact on either client or server-side processing ratios. However, increasing the loads on the client and server systems will impact scalability as more CPU time is required to service the map requests with increased amounts of layers. That is, as more layers are added to the MXD, the data server will be less likely to service the same number of concurrent users without a slow down in performance. This applies to the client side as well for Terminal Server or Web Application Server environments. This testing has demonstrated that the number of layers in an ArcMap document can have a significant impact on performance and system resources. Therefore, the number of layers in an MXD document should be minimized when possible to achieve optimal map display performance, minimize network traffic, optimize client/server capacity, and minimize ArcMap memory consumption. Configuration Support It is important to realize that for enterprise configurations such as the one described in this paper, ESRI Support will typically be limited to supporting only the ESRI software components (ArcIMS, ArcSDE, etc.). ESRI Support strives to provide the best assistance possible, but problems or questions regarding third-party applications and components may require you to contact the support services provided by the respective vendor. ESRI offers ongoing, dedicated assistance with the design and/or implementation of an enterprise configuration through our Professional Services Department. Professional Services staff has real-world experience with enterprise configurations, as well as expert knowledge of ESRI resources. To learn more about what ESRI Professional Services can offer your organization, please see the contact information posted at 9