Panzura White Paper Panzura Distributed File Locking

Transcription

1 Panzura White Paper Panzura Distributed File Locking Panzura s game-changing Freedom Family of Products finally brings the full power and benefits of cloud storage to enterprise customers, helping to break the unending on-site storage expansion cycle while eliminating islands of storage that inhibit cross-site user interaction and productivity and real-time data protection. Panzura s Distributed File Locking enables applications that were developed for the LAN, such as CAD, PDM, and PLM, to work just as well for the WAN. This white paper explains how

2 Executive Summary From its inception, Panzura has understood that distributed file locking is critical to delivering local NAS features from globally-distributed storage. Storage companies are discovering that providing an elegant, seamless, and robust globalized cloud storage solution that closely mimics a local NAS user experience is far from trivial, especially if trying to retrofit systems not designed for cloud storage. To date, no other enterprise-class global storage platform can match Panzura s distributed file locking technology. The ability to provide global storage with local NAS attributes requires deep integration of locking into the very fabric of the distributed cloud file system and global namespace. The result of this integration into the Panzura Freedom Product Family is outlined in this paper and can be used as a guide to understand the dynamics of Panzura s seamless distributed file locking system. This paper is moderately technical and meant to give an overview of Panzura Freedom Product Family distributed file locking technology. A more in-depth technical exploration of this technology is also available from Panzura. The Panzura Architecture Panzura Software drives a highly-scalable, high-performance global file system that delivers a distributed file sharing architecture. This global file system aggregates many core technologies designed by Panzura that are all natively integrated and tightly coupled. These technologies are: 128-bit transactional object file system Distributed cloud file system snapshots Decoupled metadata and data Global Data Asymmetry Referencing Global namespace Integrated SMB and NFS file locking semantics Distributed file locking engine Panzura Flash Cache appliances Scalable global deduplication The Panzura Freedom Family utilizes unique and proven technologies to ensure that files within Freedom View the distributed cloud file system can be reliably locked with guaranteed data consistency while honoring write-order fidelity and preserving precision of data/metadata file states. These core Panzura Freedom Panzura Freedom Flash Cache Flash Cache operating attributes are attained globally with high performance. The Panzura Distributed Cloud File System allows clients from multiple locations to view and lock files with a global namespace that spans many Panzura Freedom systems. Each Panzura Freedom instance can lock any file locally for writing or modifying. The architecture, fundamental mechanisms and core methods of the Panzura Distributed Cloud File System utilizes mature technologies in unique ways that extended them beyond their original design and implementations. /cc-01 Locally Owned Active File System cc-01 cc-02 cc-03 /FreedomView /FreedomView /FreedomView Panzura Freedom Flash Cache /cc-02 /cc-03 /cc-02 /cc-01 /cc-03 /cc-03 /cc-01 /cc-02 Locally Owned Active File System Remote File Systems (Active Clone Meta-Instances) Global Active File System & Global Namespace (GNS) (All File System Instances are Active & Locally Writable) Locally Owned Active File System Logical GNS View Physical GNS View at Each Freedom Flash Cache 2

3 Panzura Global Namespace At the highest level, the Panzura global namespace is an in-band file system fabric that integrates multiple physical file systems into a single space and is mounted locally on each node. The entire global namespace has the root label of the distributed cloud file system. As an example, the following 2 global namespace paths point to the same directory (\projects\team20) and are visible from both nodes as well as locally on nodes cc1-ca (California) and cc1-hi (Hawaii). H:\ -> \\cc1-ca\cloudfs\cc1-ca\projects\team20 J:\ -> \\cc1-hi\cloudfs\cc1-ca\projects\team20 It is important to note some fundamental differences between the Panzura global namespace fabric and conventional global namespace (GNS) concepts. Unlike Panzura s global namespace, conventional GNS architectures require a database process on each storage system (either in-band or side-band on a separate dedicated GNS system). These distributed databases own and manage all file system metadata transactions. File operations are intercepted in-band, processed and acted on by the distributed database instances before each file operation is allowed to complete. Changes to file metadata anywhere within the GNS require complex out-of-band distributed database replication, synchronization, arbitration and resolution while simultaneously attempting to provide real-time access to the file with guaranteed consistency. The reliance on multiple distributed database processes could introduce complexities, in-band latencies and operation challenges that fail to scale at global multi-site levels. Examples of conventional GNS solutions are Microsoft DFS, F5 ARX and EMC Rainfinity. Additionally, these GNS architectures are somewhat limited in their ability to offer capabilities like global snapshots. The Panzura global namespace has no reliance on underlying distributed databases and avoids common GNS limitations (e.g. speed, transactional data coherence, write order fidelity, open files, precision, in-band operation, global snapshots). At a fundamental level, the file system fabric is the namespace engine. s Overview At the very core of Panzura s distributed file locking architecture is Panzura s innovative global snapshot technology. The Panzura Freedom Family actively takes file system snapshots for a multitude of functional requirements. The snapshot kernel is extremely dynamic, operating without any I/O overhead, and is massively scalable. Some key points are: s can occur at the granularity of 1/second, indefinitely s are virtually unlimited in data address space and number of entities User data state, individual file system state, and total system data are all captured and managed by snapshots s are independent and can be transmitted between Freedom systems transmission is native to, and integrated into, the file system, not a separate replication process System snapshots capture the state of the distributed cloud file system address-space in a single operation with guaranteed data coherence at bytelevel granularity. replication is a bulk data transfer mechanism among Freedom systems. The default system snapshot replication interval is 60 seconds but can vary depending on system activity. Time Metadata Data Block AA Block AB Block AC Block BE Block CB 3

4 s Mechanism Within the Panzura Distributed Cloud File System, data blocks are fundamentally never overwritten. If a new data block is written, then the next snapshot that is created contains a pointer to the new block. If an existing block is modified, this transaction also results in a new block being written and referenced both in the active file system state and in the next snapshot; while the old block is retained untouched within the active file system and referenced in the previous snapshot. This results in no I/O latency. A file exists and its data is referenced by 3 blocks. Block AA, AB, AC. s points to these blocks. New data is written to the file and is stored in the new block BE is triggered and now points to 4 blocks. AA, AB, AC & BE. Existing data within the file is modified (data existed in block AC). The new modified data is written to new block CB now points to blocks AA, AB, BE & CB. Data is deleted from the file (relating to block AB) now points to blocks AA, BE & CB. A snapshot metadata image is generated on each Panzura Freedom Flash Cache and contains an aggregate of the all file information and state, including creation time, modification time, size, access control lists (ACLs), extended attributes, streams and i-node structure. In addition to this, it also contains native information about file locks and lock states. This has two distinct benefits: The operation of updating metadata is global and does not depend upon the number of file changes being tracked by the snapshot or a database. All file locks & lock state information are carried with the snapshot into the cloud. This is critical for consistency for applications that write real-time lock state into the file system. s are transmitted using an efficient delta-differencing scheme where only the deltas between 2 snapshots are streamed over the network, providing optimized bandwidth usage and performance. Data Flow After each snapshot is completed, its metadata and data are immediately sent to the cloud. Each Panzura Freedom system maintains a private region in the cloud that holds its own instance of metadata and data for its locally writable file system. No other node except the owner may write into the private region of a node, but all Freedom systems can read from all Origin private regions. This dynamic snapshot flow from the cloud to each Freedom instance ensures that Panzura Freedom systems always get the latest versions of metadata and data from the cloud; but Freedom systems will communicate peer-topeer to receive the most recent data-block updates for an individual file that have not yet been synched to the cloud. This transaction only happens on an individual file basis for remote writes, as needed. This process is managed by the data asymmetry resolution engine. cc-02 DAR cc-01 Meta Owner DAR Data Owner OPLock DAR cc-03 OPLock 4

5 The following are the main types of communications protocols among Panzura Freedom Flash Cache appliances: 1. Heartbeat (to see if controllers in the Panzura Global File System are alive and addressable) 2. Data ownership (request data ownership to apply a localized write lock) 3. Resolution of incremental block data deltas since the last snapshot whenever data ownership is being transitioned Global Locking The below diagram is used to discuss the lower level details and concepts of data ownership, data mobility and global locking. The following core concepts and symbols are used to describe the locking flow: The Origin the node where the file was originally created. The Data Owner Freedom Flash Cache grants ownership of the data by the Origin. The payload is built on this system and the authoritative data instance is managed from here. Ownership Metadata (OM) the metadata showing the Data Owner (DO) state. The Authoritative Write Node the Freedom system where locks for a data write operation are executed. This will normally be the DO. Writes here always happen at LAN speed. Traditional File Lock a lock issued against a file by a file system, a server or an application. This lock may consist of extensive application specific meta-information and be written into part(s) of the file payload and/or its file system metadata. File Coherency Locks these are file system locks that are issued by applications in order to arbitrate guaranteed consistency between applications writing/reading to a single file. An example is: Microsoft Office Application locks. Opportunistic Caching Locks a delegated right issued by a file server protocol engine for a remote client to cache a file locally to increase client-side performance. This is not necessarily a guaranteed write lock. This delegation may be revoked by a file server at anytime. An example is: Microsoft CIFS OPLOCKS. Data Asymmetry Resolution (DAR) Panzura technology that efficiently resolves differences between remote sets of files and transports the required blocks to the DO. Data Ownership, Data Locking and Data Mobility The Panzura Distributed File System was designed so that data and User 1 User 2 User 3 pz-cc1 pz-cc2 pz-cc3 Cloud metadata are physically New York This decoupling enables the file Create file.doc system to be highly flexible in User 1 Creates file.doc GNS Origin + MO + DO Defined Meta + Data referencing which physical blocks are used to construct a file. Global Meta- distributed file locking leverages this flexibility by assigning a DO Open file.doc for Write User 2 Writes to all files. This DO state is held to file.doc Request Origin within the OM of each file and is MO Updates DO = pz-cc2 Data Locked Locally easily transported via snapshots. A Grant DO Freedom system that wants to be Compute & Read Non-Local Blocks Only Compute Block Asymmetry the new DO communicates with the with last DO Origin, whose location is defined Stream Unsynched Block Deltas Grant File Open, Grant OPLock, LAN Speed I/O Global Consistency by a unique unified namespace Checkpoint path for each file when the file was originally created. 5

6 Within distributed file locking, DO states naturally flow from node-to-node. DO transitions are frequent events and are negotiated via small real-time peer-to-peer communications among Freedom systems. As the DO flows to a new node, that new node instantly becomes the authoritative write node. All new writes to the file will now happen at the new DO node at full LAN Speed. Note: The Origin is never involved in the I/O data-path during a write operation once the DO successfully migrates. The final step in assimilating blocks after a DO transition is to resolve any data asymmetry. This involves a direct peer-topeer communication between the Origin and the new DO, and possibly the current DO (which might not be the Origin). Within this peer-to-peer stream the OM computes a final delta list of real-time changes that may have occurred since the DO changed. This list, which can be as small as a single file system block, is streamed directly to the new DO via a secure optimized data channel. The new DO processes all remaining blocks deltas, making the file current and consistent. All file reads and writes from that Freedom system now happen as local I/O operations on the new DO. The DO retains full read/write ownership until a new DO transition occurs. Below are two diagrams that describe two scenarios showing the transactional details of the distributed file locking architecture detailed above. User Paris User London User New York pz-cc1 pz-cc2 pz-cc3 Cloud A 2-User Transaction In this transaction two users will open the same file for write, from one unique global namespace path. Each user will experience LAN speed I/O for the read and write operations as well as 100% data consistency via the data-ownership locking and datablock mobility. User 1 Creates file.doc User 2 Writes to file.doc Create file.doc Meta- Open file.doc for Write MO Updates DO = pz-cc2 GNS Origin + MO + DO Defined Meta + Data Request Origin Data Locked Locally Grant DO Compute & Read Non-Local Blocks Only User 1 is in Paris and creates a new file called File. Compute Block Asymmetry with last DO doc on the Paris node. This transaction defines the file Origin because it assigns the unique global namespace path to a new file. The metadata is updated and DO is assigned as pz-cc1 in Paris. Grant File Open, Grant OPLock, LAN Speed I/O Meta- Stream Unsynched Block Deltas Global Consistency Checkpoint The Paris node will eventually write the metadata and payload for File.doc to the cloud. This will happen independently of the original write. All nodes in the Panzura Distributed Cloud File System global namespace will independently read from the cloud the metadata of all other nodes from their private regions and update their file systems and namespace view. Later in the sequence, User 2 (in London) tried to write to the same file by accessing the unique global namespace path on his local system in London (pzcc2). This is the start of the global read-write locking transaction. Controller pz-cc2 will request DO from Open file.doc for Write User 3 Writes to file.doc Request Origin DO! = Origin Origin Arbitrates New DO Origin Updates MO with New DO Data Locked Locally Grant DO Compute & Read Non-Local Blocks Only Compute Block Asymmetry with Last DO Stream Unsynced Block Deltas Grant File Open, Grant OPLock, LAN Speed I/O Global Consistency Checkpoint Meta- the Origin. The Origin evaluates the OM state and discovers that it itself is the current DO. Controller pz-cc2 is granted DO and the OM is updated by the Origin. 6

7 From here, pz-cc2 will resolve the location of all blocks. Some may come from the local file system instance and some may be transported in from the cloud. As a final consistency state check, pz-cc2 asks the previous DO if it holds any in-flight data blocks that have not yet been synced to the cloud. This resolves any data asymmetry. The processes occur in parallel to all other block reads and results in the entire state of the file being fabricated on pz-cc2 with guaranteed data consistency. At this point pz-cc2 will serve the file to the client at local LAN speeds. All other nodes are updated via the normal snapshot process. A 3-User Transaction In this transaction three users will open the same file for write, from one unique unified namespace path. Each user will experience LAN speed I/O for the read and write operations as well as 100% data consistency via the data ownership locking and data block mobility. User 3 is in New York and wants to write to File.doc. His application opens the path in the global namespace via his local view, which is consistent. During the initial local file open operation, the New York Panzura Freedom Flash Cache (pz-cc3) evaluates the global namespace path and identifies the Paris Freedom Flash Cache (pz-cc1) as the Origin. The New York Freedom Flash Cache will contact the Paris appliance and request DO. Paris evaluates the OM state and discovers that New York is not the current DO; the London appliance is (pz-cc2). Paris will now arbitrate a DO transition. As London relinquishes the DO, Paris updates the OM with a new DO of New York and advises New York that it is now the new DO. At all times, New York knew that Paris was not the DO and that London was the last DO but in order to enforce consistency, the Origin was leveraged as the authoritative arbiter of the transition since it must also update the OM. New York is now the new authoritative DO and will compute any missing blocks that are not localized in the New York file system. These blocks can come from multiple locations in parallel. As part of this process, pz-cc3 will communicate directly with London (pz-cc2) to compute any in-flight blocks not yet in the cloud from the last Authoritative Write Node (i.e. the previous DO which was pz-cc2, London). Any in-flight blocks are peer-2-peer streamed directly between London and New York to resolve data asymmetry. The result is that the full block structure in New York (pz-cc3) has guaranteed data consistency. At this point pz-cc2 will serve the file to the client at local LAN speeds. All other Freedom systems are updated via the normal snapshot process. Summary This paper provided a high-level overview of the robust, high-performance distributed file locking technology developed by Panzura. Such technology is required in order to have a consistent, responsive, and enterprise-class global collaboration and sharing platform. Yet no one else is able to provide the seamless, distributed file locking capability of Panzura. For a more indepth discussion of our distributed file locking system, please contact Panzura and request a briefing. How to Learn More Please visit or contact us at or sales@panzura.com. Additional detail on how the Panzura Freedom Flash Cache operates in conjunction with a cloud environment is available in Inherent Data Protection How the Panzura System Protects Information without Backups. This white paper explores how the Panzura System can be deployed in conjunction with cloud storage to provide world-class levels of data protection without the need to perform traditional backups. Panzura, Inc. 695 Campbell Technology Pkwy #225, Campbell, CA, USA 855-PANZURA Copyright 2017 Panzura, Inc. All rights reserved. Panzura is a registered trademark or trademark of Panzura, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. WP-Distributed-File-Locking b