Microsoft SharePoint data management solution using IBM Spectrum Scale and AvePoint DocAve

Transcription

1 Microsoft SharePoint data management solution using IBM Spectrum Scale and AvePoint DocAve A technical report Udayasuryan Kodoly IBM Systems ISV Enablement May 2015 Copyright IBM Corporation, 2015

2 Table of contents Abstract... 1 Executive summary... 1 Scope... 2 Intended audience... 2 Prerequisites... 2 Solution components... 3 IBM Spectrum Scale... 3 AvePoint DocAve Storage Manager... 4 DocAve architecture... 4 Solution architecture... 5 BLOB externalization to the IBM Spectrum Scale... 6 DocAve installation and configuration... 6 SharePoint document and metadata storage... 6 Exporting file system of IBM Spectrum Scale using NFS for BLOB externalization... 7 Enabling Client for NFS services on Windows Server 2008 R2 or Windows Server 2012 with DocAve Storage Manager... 8 Storage configuration for BLOB externalization with DocAve Storage Manager... 9 Storage policy Enabling the BLOB provider Rule configuration BLOB content offloading Storage report Summary Appendix A: Resources About the author Trademarks and special notices... 20

3 Abstract This technical report illustrates the essential solution technology integration and configuration best practices details about building an extremely scalable enterprise class infrastructure to protect and manage Microsoft SharePoint deployments using IBM Spectrum Scale storage system and AvePoint DocAve software. Executive summary Microsoft SharePoint is a strategic business platform that allows people to connect seamlessly with each other in terms of centralized content management. Furthermore, as a collaborative tool, SharePoint can be used by anyone, and can be installed and configured quickly. As SharePoint adoption has increased in many enterprises, so has the complexity of many SharePoint deployments. SharePoint is now considered a platform delivering business-critical services to organization using web portals and extranets, enterprise content management (ECM), workflow, search and enterprise social networking. With more and more data being stored in SharePoint, enterprises need storage infrastructure to be able to ensure 24x7availability while keeping up with growing storage requirements. Scaling users and storage in addition to changing or upgrading the hardware or software can interrupt business operations, reducing the availability of critical SharePoint data. Managing and protecting SharePoint data can also be time consuming and complex, significantly increasing administration overhead. For enterprise IT managers and SharePoint administrators, this development poses a number of challenges: Cost-effectively meeting service level agreements (SLA) based on increasing business requirements Managing storage and performance for rapidly growing SharePoint server farms Seamlessly scaling SharePoint deployments to manage large data files and more users Protecting SharePoint data efficiently Meeting archive, compliance, and governance requirements To properly address these challenges, enterprises requires a complete infrastructure solution that is proven, on storing and managing large amount of growing SharePoint data more efficiently and economically, while providing extreme availability and optimum performance. The joint solution of IBM Spectrum Scale and AvePoint DocAve provides powerful combination of manageability, efficiency, data protection, and scalability for SharePoint ECM workloads.as SharePoint 2010 and SharePoint 2013 onpremises become the central repository for enterprise content within organizations, increasing user adoption and integration from data sources can cause unstructured data and binary large object (BLOB) content to consume up to 95% of SharePoint SQL Server space. Larger BLOB content can lead to database performance and user experience degradation. The AvePoint DocAve software seamlessly externalize and store unstructured BLOBs to IBM Spectrum Scale by using Network File System (NFS) shares, outside of Microsoft SQL Server, thereby reducing SQL database size. The IBM Spectrum Scale is an agile storage infrastructure that can readily and cost effectively scale to keep pace with the evolving needs of the enterprises. Microsoft SharePoint Data Management Solution Using IBM Spectrum Scale and AvePoint DocAve 1

4 Scope This technical report: Discusses the solution architecture, vital solution configuration, and the related solution configuration workflows with AvePoint DocAve version 6.5 and IBM Spectrum Scale storage system. This technical report does not: Discuss the installation and basic configuration of Microsoft SharePoint Server. Discuss the installation and basic configuration of AvePoint DocAve version 6.5. Discuss the installation and basic configuration of IBM Spectrum Scale. Replace any already available document that is related to AvePoint DocAve, and IBM Spectrum Scale storage system. Intended audience This technical report is intended for: Customers and prospects looking to implement combined solution using AvePoint DocAve and IBM Spectrum Scale to manage the Microsoft SharePoint deployment. Users and management seeking information to implement solution for Microsoft SharePoint deployment using AvePoint DocAve and IBM Spectrum Scale storage system. Prerequisites This technical paper assumes familiarity with the following prerequisites: Basic knowledge of Microsoft SharePoint 2013 Basic knowledge of AvePoint DocAve version 6.5 Basic knowledge of IBM Spectrum Scale storage system 2

5 Solution components This section briefly describes the essential components used in this solution. IBM Spectrum Scale IBM Spectrum Scale is a proven, scalable, high-performance data and file management solution (based on IBM General Parallel File System (IBM GPFS ), also formerly known as code name IBM Elastic Storage ). IBM Spectrum Scale provides a world-class storage management with extreme scalability, flash accelerated performance, and automatic policy-based storage tiering from flash through disk to tape. IBM Spectrum Scale reduces storage costs up to 90% while improving security and management efficiency in cloud, big data, and analytics environments. IBM Spectrum Scale includes a highly differentiated value: Virtually limitless scaling to 9 quintillion files and yottabytes of data High performance (over 400 GBps) and simultaneous access to a common set of shared data Over 400 GBps performance Software-defined storage allows you build your infrastructure your way. Easy to scale with relatively inexpensive commodity hardware while maintaining world class storage management capabilities Use any combination of flash, spinning disk, and tape Use a variety of cluster models that include storage area networks (SAN), Network Shared Disk and shared nothing clusters Add more storage capacity without affecting the application to greatly simplify administration Information lifecycle management (ILM) tools automatically move data based on policies. This can dramatically reduce operational costs as fewer administers can manage larger storage infrastructures Provides global data access across geographic distances and unreliable wide area network (WAN) connections Proven reliability with use in the most demanding commercial applications Protects data from most security breaches, unauthorized access, or being lost, stolen, or improperly discarded with native file encryption for data at rest and secure erase 3

6 Single Name Space GPFS POSIX NFS CIFS OpenStack Nova Swift Cinder Manila Glance Hadoop GPFS Active File Manager S p e c t r um S c a l e IBM Spectrum Scale Cloud Storage GPFS SSD Fast Disk GNR Tape Policies for Tiering, Data Distribution, Migration to Tape and Cloud Figure 1: IBM Spectrum Scale architecture AvePoint DocAve Storage Manager As SharePoint 2010 becomes the central repository for enterprise content within organizations, increasing user adoption and integration from data sources can cause unstructured data, called binary large objects, to consume up to 95% of SharePoint SQL Server space. If ignored, BLOBs can lead to database performance and user experience degradation. With DocAve Storage Manager, organizations can mitigate the negative consequences of exponential data growth by combining multiple real-time and scheduled business rules to externalize BLOB content based on file size, type, or other document properties, allowing SharePoint administrators to: Reduce SharePoint total cost of ownership by utilizing hierarchical storage management (HSM) systems for BLOB storage. Improve user experience by optimizing SQL Server resources while maintaining seamless user access and interaction to externalized content. Easily comply with information governance policies for content lifecycle management within or across multiple SharePoint farms with highly customizable business rules. Ensure all BLOBs work with SharePoint management functions and most third-party applications by leveraging Microsoft s fully supported external BLOB storage (EBS) or remote BLOB storage (RBS) application programming interfaces (APIs). DocAve architecture DocAve architecture is modular and scalable to meet the customer requirements. A DocAve server consists of the following roles: DocAve manager: Manages the communication and operations of the DocAve agents and servers in the farm. 4

7 DocAve media server: Manages the storage operations for the DocAve platform and coordinates the storing of backup data, audit data, externalized SharePoint content, and all other storage needs required by DocAve. DocAve agents: Manages the content as per the rules defined in the DocAve suite for BLOB externalization, backup, recovery, and archiving the SharePoint content. Solution architecture Upload File Retrieve File DocAve Storage Manager or Connector to RBS Store WFE WFE WFE Agent Agent Agent Load file that has been Stubbed from RBS Store Blob Content to RBS Store Generate Stub / Metadata into SharePoint Content DB Blob Stub To SharePoint Content DB Accessed Blob Stub & Metadata Retrieve Blob Content from RBS Store SQL DocAve Manager Stub DB SP Content DB1 Containing Metadata SP Content DB2 Containing Stubs / Metadata DocAve Application Management Console DocAve Media Service Tier Storage (RBS Stores) Data Flow Storage Manager, Connect Data NFS NFS Retrieve Storage Manager, Connected Data IBM Spectrum Scale Figure 2: Solution dataflow architecture diagram Figure 2 depicts the solution dataflow architecture. The solution may consist of multiple servers installed with SharePoint Server and also known as SharePoint farm. The DocAve agent software is installed on each server in the SharePoint farm. Using DocAve application management console, solution is 5

8 configured for BLOB externalization to IBM Spectrum Scale storage system. SQL Server SharePoint content database stores the stub and metadata information. BLOB content is stored and retrieved using RBSAPIs by using configured NFS shares on the IBM Spectrum Scale storage system. BLOB externalization to the IBM Spectrum Scale This section describes the installation and configuration process of the AvePoint DocAve products to optimize the SharePoint storage with BLOB externalization to the IBM Spectrum Scale storage system. You need to complete the following tasks before configuring the BLOB provider. Ensure that Microsoft Active Directory services are available for authentication. Make sure that SharePoint farm components and SQL server deployments are according to Microsoft recommendations. Install DocAve manager and media service roles. It is not recommended to deploy these services on the SharePoint servers. Install DocAve agents on each server in the SharePoint farm. DocAve installation and configuration For detailed AvePoint DocAve software installation and configuration, refer to the following URL for instructions about installing the DocAve Manager, control service, media service, and DocAve agent components. SharePoint document and metadata storage SharePoint supports documents up to 2 GB in size ( a software boundary that results from a 32-bit pointer used in SQL Server. SharePoint includes a file size upload limitation that is configurable per web application. The default maximum upload size is 50 megabytes (MB) considerably smaller than the 2 GB hard limit. This lower limit reflects practical concerns including network performance, the performance of transferring large files over Hypertext Transfer Protocol (HTTP), and user expectations for performance of file transfer. Many organizations retain this default upload size or raise the limit. If you choose to raise the maximum upload size, do so slowly and after careful testing. Each document in a SharePoint library has metadata associated with it. Some metadata is userconfigured, such as columns in the library. Other metadata is used internally by SharePoint. The amount of metadata associated with a document will vary based primarily upon user-configured metadata. It is easy to understand that scenarios with larger documents see a higher ratio of BLOB-to-metadata storage, and scenarios with smaller documents and more metadata will see lower BLOB-to-metadata ratios. The 80 percent estimate is based on an average across multiple SharePoint environments. But here is the rub: A document is rarely, if ever, stored only once. 6

9 Exporting file system of IBM Spectrum Scale using NFS for BLOB externalization This section lists the essential steps for exporting required solution BLOB externalization file system of Spectrum Scale using NFS. 1. Define the root of the overall exported file system (also referred to as the pseudo root file system) and the pseudo file system tree. In the solution lab testing purpose, defined /exports as the pseudo root and export the directory /gpfs/docaveisvshare using the following commands:. mkdir m 755 /exports /exports/docaveisvshare mount --bind /gpfs/docaveisvshare /exports/docaveisvshare In this scenario, /gpfs/docaveisvshare is bound to a new name under the pseudo root using the bind option of the mount command. This bind mount point should be explicitly unmounted after IBM Spectrum Scale is stopped and bind-mounted again after IBM Spectrum Scale is started. To unmount, use the umount command. Note: Make sure that the root directory has the appropriate permission setting. If required, use chmod command appropriately. 2. Edit the /etc/exports file. For example, see the solution lab test environment /etc/exports file entry: /exports/docaveisvshare *(rw,sync,all_squash,fsid=805) 3. Run the service nfs reload command to update the NFS Server configuration. 7

10 Enabling Client for NFS services on Windows Server 2008 R2 or Windows Server 2012 with DocAve Storage Manager It is essential to enable services client for NFS on Microsoft Windows Server 2008 R2 or Microsoft Windows Server 2012, in order to access IBM Spectrum Scale file system resources to externalize the SharePoint data using NFS with AvePoint DocAve. Click Start All Programs Administrative Tools Server Manager. Click Add Roles as shown in the Figure 3. Figure 3: Add roles In the Select Server Roles wizard, select the File Services check box and click Next. Figure 4: Select the File Services 8

11 Under File Services, click Role Services and then select the Services for Network File System check box. Figure 5: Select Role Services Click Install to finish File Services and Services for Network File System. Storage configuration for BLOB externalization with DocAve Storage Manager DocAve Storage Manager stores the uploaded BLOB content to an external device (leaving only a stub of the data in SharePoint), therefore, it is necessary to first configure one or more physical devices and then set up a logical device. Physical devices allow DocAve to store backup data. In order for DocAve to recognize a physical device for use, it must first be configured. Perform the following steps to create Net Share physical devices: In the DocAve manager interface, click Control Panel Storage Configuration Device Create. Select Net Share from the Storage Type drop-down list, as shown in Figure 6. Physical 9

12 Figure 6: DocAve Physical Device creation In the UNC path field (refer Figure 6), enter the UNC share path of the IBM Spectrum Scale NFS share. Click Save to create the physical device. Verify that the device is listed under Physical Device as shown in Figure 7. Figure 7: List of Physical Devices in DocAve 10

13 DocAve has the ability to treat multiple storage drives as a single logical unit when saving DocAve data. This is especially helpful for very large DocAve plans as many smaller drives can be combined. A logical drive must be defined before creating a DocAve plan.perform the following steps to create Net Share logical device. In the DocAve Manager interface, click Control Panel Storage Configuration Logical Device Create. In the Data Storage Type section, select Standard storage device, and add the appropriate physical devices from the drop-down list as shown in Figure 8. Figure 8: Logical device creation Click Save to create the logical device. Verify that the device is listed under logical devices. Storage policy Use the Storage Policy option to select which logical device to use when saving backup data, and setting up the retention policy for the data saving in the logical device. Perform the following steps to create a storage policy. In the DocAve Manager interface, click Control Panel Storage Configuration Storage Policy Create. Provide the storage policy name and select the appropriate logical device as shown in the Figure 9. 11

14 Figure 9: DocAve storage policy Click Save to create the new storage policy. Verify that the newly created storage policy is listed under storage policy. 12

15 Enabling the BLOB provider By enabling the BLOB provider, you are able to externalize BLOBs from a content database to IBM Spectrum Scale storage system as the external BLOB storage. The BLOB provider feature intercepts SharePoint database traffic and redirects all of the BLOB traffic to external BLOB storage; what remains in SharePoint is a stub of the data. The BLOB provider also creates a stub database and enables the provider on the specified farm or node. Perform the following steps to enable the BLOB provider on the SharePoint content database In the DocAve manager interface, go to the storage optimization, select the scheduled storage manager and then the BLOB provider, to enable EBS or RBS for the SharePoint farm as shown in Figure 10. Figure 10: DocAve BLOB provider configuration 13

16 Configure the stub database as shown in Figure 11. It is recommended to configure the stub database for the entire SharePoint farm. Figure 11: DocAve stub database configuration Enable RBS for the SharePoint farm and configure the schedule for scanning by clicking Configure. Figure 12: Enable RBS 14

17 Click Finish to complete the BLOB provider configuration as shown in Figure 13. Figure 13: Finalize the BLOB configuration 15

18 Rule configuration BLOB content offloading DocAve offloads the content based on the rule criteria defined using document attributes such as size, creation date, modified date, and version. Content that meets these rules will be offloaded from the SharePoint storage and stored in the selected logical device. Perform the following steps to create rules to offload the content from the SharePoint storage. From the DocAve manager interface, go to the storage optimization, select Scheduled Storage Manager, select the web application, and then click Create to create the new rule. Click the Create a new rule option and provide a Name for this rule. Define the criteria for offloading the content. For example, a new rule is created based on the document size to offload the content of size greater than 256 KB. Select the logical device of the IBM Spectrum Scale storage system to apply the rule. Note: It also provides the option to select compression or encryption features. However, you may experience slower backup and recovery performance. Figure 14: BLOB rule criteria definition 16

19 Storage report DocAve provides a dashboard view of the BLOB and stub information collected from the SQL databases and reports about the optimized capacities for the SharePoint farm. You can access the storage report from the DocAve manager interface, select the schedule storage manager, and then select the storage report. Perform the following tasks to generate the storage dashboard: Create the profile by invoking the report profile wizard as shown in Figure 15. Select the whole farm or the relevant web applications for this profile to generate the report. Define the schedule specifying whether to collect the data immediately or specify the frequency to collect the data on a recurring basis. Click Save or Save and Run Now to complete the rule definition. Figure 15: Storage report profile creation 17

20 Summary IBM Spectrum Scale removes data-related bottlenecks by providing parallel access to data, eliminating single filer choke points or hot spots. IBM Spectrum Scale also simplifies data management at scale by providing a single namespace that can be scaled simply, quickly, and infinitely by simply adding more scale-out resources, eliminating filer sprawl and its associated problems. DocAve Storage Manager is based on Microsoft EBS and RBS APIs to improve SQL server performance. With DocAve, administrators can use customizable business rules in DocAve SharePoint storage offerings along with a wide variety of supported storage devices to optimize HSM and achieve truly comprehensive SharePoint storage management. The IBM Spectrum Scale and AvePoint DocAve Storage Manager integrated solution provides a robust and very flexible platform needed for Microsoft SharePoint environments through the outstanding characteristics of the software-defined storage and integration through RBS/EBS API integrations to optimize the SharePoint storage by offloading the larger BLOB content. 18

21 Appendix A: Resources The following websites provide useful references to supplement the information contained in this paper: IBM Systems on PartnerWorld ibm.com/partnerworld/systems IBM Redbooks ibm.com/redbooks IBM System Storage Interoperation Center (SSIC) ibm.com/systems/support/storage/config/ssic/displayesssearchwithoutjs.wss?start_over= yes IBM Spectrum Scale ibm.com/systems/storage/spectrum/scale IBM Techdocs Library ibm.com/support/techdocs/atsmastr.nsf/web/techdocs AvePoint DocAve documentation About the author Udayasuryan Kodoly (Uday) is a Storage Technology Specialist in the IBM Systems and Technology Group (ISV Enablement) Organization. Uday has years of extensive experience in designing, architecting storage solutions, and developing solution best practices for enterprise data centers and cloud infrastructures, and building and sustaining customer and partner relationships. Presently, Uday is engaged in planning, designing and developing IBM Spectrum Scale, software-defined storage and cloud infrastructure solutions integrating with various ISVs. You can reach Uday at uakodoly@us.ibm.com. 19

22 Trademarks and special notices Copyright IBM Corporation References in this document to IBM products or services do not imply that IBM intends to make them available in every country. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Information concerning non-ibm products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-ibm list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-ibm products. Questions on the capability of non-ibm products should be addressed to the supplier of those products. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller for the full text of the specific Statement of Direction. Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here. Photographs shown are of engineering prototypes. Changes may be incorporated in production models. 20

23 Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. 21