Distributed Backup System.Net

Transcription

1 Distributed Backup System.Net world can now backup for free... AUTHORS Martina Babini Guido Angelo Ingenito

2 what we have already seen... According w/ DBS.Net vision you should send your files to peers so if a catastrophe strikes your area, you can recover data from surviving peers. Network Discovery.NET Remoting DBS.Net Web Services MSMQ Microsoft Framework.NET Technologies: Framework.NET v.2.0 MSMQ Remoting Web Services Data Files Chunks Replication Metadata Time: Life Cycle A backup contains: N files (C chunks) A replication level R Defines on how many peers the j-th chunk will be stored; A life cycle Defines how long a backup must be stored on other peers. Metadata Check sum, compression, encryption, digest, signature,... A Scheduler allows DBS.Net to plan backup, according w/ user defined strategies. Throw the dynamic assembly load DBS.Net can change its discovery architecture at runtime. XML is used extensively for configuration of both C/S.

3 backup process N Preconditions (entities to be defined): 1.An RSA key-pair; 2.A set K of symmetric keys (the j-th chunk encrypted w/ k-th key mod K ); 3.An allocation strategy AL (used during the chunk send phase). C Pipe Process: 1.Each file is splitted in C chunks of avg. size CS (default 1Mb); 2.Each chunk is: compressed, encrypted, signed, stored locally (producer, consumer model); 3.Each chunk is sent over the network R times according w/ AL. R A user-sizable thread-pool supports each synchronous and asynchronous execution step.

4 network architecture GetPeerRequest WS GetPeerResponse WS-DLL The network layer has been implemented through a component model approach: a. The usage of MSMQ provides the support for messages and protocol implementations (message-queue, data-queue); b. The discovery architecture is inter-changeable at runtime using: SOA or.net Remoting. Each peer can act as a simple peer or as a relay: a. In the first case the peer asks to known relays informations about other peers in the neighborhood. b. In the second case the relay can be directly addressed on the network, so it can reply to other peers w/ the informations it has collected about other peers. A relay peer query defines a search pattern (ex. to specify a location). Informations are stored in an object oriented database. Through the dynamic assembly loading DBS.Net offers the possibility to extend and provide other discovery implementations.

5 web ser vices DBS.Net uses the power of web services to define one of its modules for network discovery: a. The client, called PeerResolverClient, is defined as an external DLL and loaded on demand; b. The discovery web service, called PeerResolver, is automatically loaded into the web server IIS if the end-user chooses to left its DBS.Net client act as a relay. The UML underlines the CleanTimer and a DataProvider: the first is used to perform periodical cleans of information collected from the various Peers, the second is the data source of such informations and can be changed at runtime. Discovery through Web Service is auspicable in context where firewalls doesn t allow other protocols rather than http. The DataProvider can be loaded on-demand w/ the dynamic assembly load according w/ user preferences.

6 SOAP 1.2 request: <soap12:envelope xmlns:xsi=" xmlns:xsd=" xmlns:soap12=" <soap12:body> <Notify xmlns=" <peeroid>guid</peeroid> <peergroupoid>guid</peergroupoid> <peername>string</peername> <peerdescription>string</peerdescription> <peeripaddress>string</peeripaddress> <peerfreespace>long</peerfreespace> <peermessagequeuename>string</peermessagequeuename> <isrelay>boolean</isrelay> <location>string</location> <sendtime>datetime</sendtime> </Notify> </soap12:body> </soap12:envelope> SOAP 1.2 response: <soap12:envelope xmlns:xsi=" xmlns:xsd=" xmlns:soap12=" <soap12:body> <NotifyResponse xmlns=" /> </soap12:body> </soap12:envelope> The placeholders shown are replaced with actual values at runtime. DBS.Net ws - behind the scenes... The PeerResolver defines a web method called Notify, this method is invoked periodically from each peer when it must inform the discovery service of its presence (ping), it provides also timing informations that allows to compute Distance, LastSeen and other parameters as well! Simple Peer periodically: default 60s POST /PeerResolver/PeerResolver.asmx HTTP/1.1 HTTP/ OK WS Relay

7 msmq Sending Sending Application Application Message Queue Receiving Application MSMQ is a message oriented middleware that: enables applications running at different times to communicate across heterogeneous networks; provides guaranteed message delivery, efficient routing, security, and priority-based messaging. DBS.Net defines two distinct queues on each peer: MessageQueue: used for protocol messages. DataQueue: used to send and receive data messages. DBS.Net uses MSMQ to implement its fundamentals communication strategies between peers.

8 backup protocol Chunk Response received phase: Ask Backup for free send space phase green is green a chunk positive received response message phase red means red a negative chunk not response received

9 restore protocol Response phase: Ask for backup green Ok to is start a positive Sending retrieve response phase red is a message negative phase response A peer that sends a positive response sends also an information about chunks that it has.

10 protocol properties The first phase of each protocol is a lookup phase used also to check which peer is really reachable or not. Negative messages are thought to let peers going on w/ their work w/out wasting time in waiting for responses. Timeouts has been implemented too in order to solve disconnections related problems. The Scheduler allows to define periodic tasks in order to organize the network vision (refresh-lookup).

11 properties Reliability: granted by MSMQ, if a chunk was delivered well it would be certainly stored on the destination peer. Security: only encrypted and signed data is sent on the Internet. QoS: due to our protocol a peer that receives a confirmation message can be sure that its message was delivered. Modularity: it was built to be as modular as possible with a great attention on network modules that could easily changed. Network fault tolerance: the peer resolver has a timer that cleans its list of available peers after a time slice. This feature assures to a peer that the peer list it receives contains only really connected ones. Authenticity: when a peer asks for a backup to another one has to send a signed token containing its id to allow the other one to check if he is really the backup owner. Resources management: the backup expiration date (life cycle) allows a peer to delete an expired backup and free storage space for other peers. DBS.Net has been tested using NUnit Framework and Virtualization Technologies!

12 expansion points Extension of the backup chunk concepts to be able to realize an incremental backup; Network analyzer to module network traffic according to bandwidth usage; User interface improvements; Integration with third party applications; Bug fixes...