perfsonar NC - a NETCONF based perfsonar implementation

Transcription

1 perfsonar NC - a NETCONF based perfsonar implementation Arne Øslebø, UNINETT TNC2011, May , Prague Insert additional logos in this space: go to View/Master/Slide master and insert logo must be white versions of logo

2 perfsonar infrastructure MA MP MA MA LS MA MA Client MP

3 perfsonar NC Based on existing IETF protocols NETCONF and YANG One single data model for all MAs Write generic tools to query and retrieve data Design can easily be extended to all perfsonar services Fast Small code base Easy? to implement new MAs Powerful query mechanisms XPath

4 MA data model +--rw perfsonarnc +--ro info +--ro measurementarchives +--ro mas +--ro ma +--ro name +--ro datasources +--ro datasource +--ro name +--ro obspoints +--ro timeinfo +--ro reportsinfo +--rw maqueries Contains all information a client needs for querying and presenting data from the MA

5 Design PerfSONAR NC daemon Server SSH SOAP Clients

6 perfsonar NC daemon daemon XML Implemented in PHP Small code base rrdma netflowma testma ma.abs XML

7 maquery Generic CLI utility maquery.php -q "//pn:obspoint[startswith(pn:name,'bergen7')]/pn:name" localhost <?xml version="1.0"?> <rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1"> <data xmlns:default= " <default:name>bergen7.bergen-ether-1</default:name> <default:name>bergen7.bergen-ether-1</default:name> </data> </rpc-reply>

8 Retrieve data with maquery Inside gnuplot plot "<./maquery.php -r 1 nav rrd singletable standard -o 462 localhost -f ' :00' -t ' :55' grep ';' cut -d \\; -f1" w lines

9 Other Netconf tools - ncclient python nc03.py localhost "//pn:ma/pn:name" import sys, os, warnings warnings.simplefilter("ignore", DeprecationWarning) from ncclient import manager def demo(host, user, expr): with manager.connect(host=host, port=22, username=user) as m: assert(":xpath" in m.server_capabilities) c = m.get_config(source='running', filter=('xpath', expr)).data_xml with open("%s.xml" % host, 'w') as f: f.write(c) if name == ' main ': demo(sys.argv[1], os.getenv("user"), sys.argv[2])

10 perfsonar vs perfsonar NC Three categories of comparison General performance and query mechanisms Scalability Code base rrdma 1000 interfaces Utilization and error counters Netflow MA 10 routers, 65 interfaces

11 Performance and query mechanisms (rrdma) Get all information about interfaces ps (java): 42.2s ps (perl): 0.66s ps NC: 0.19s Get name of all interfaces ps (java): 42.2s* ps (perl): 0.66s* ps NC: 0.09s Get name of all interfaces starting with bergen ps (java): 42.2s* ps (perl): 0.66s* ps NC: 0.04s Retrieve last link load and error for one specific interface ps (java): 4.1s ps (perl): 0.59s ps NC: 0.06s Retrieve last link load and error for all interfaces ps (java): 82.4s ps (perl): 110.1s ps NC: 4.3s Top 10 interfaces with the most traffic ps (java): 41.2s* ps (perl): 52.1s* ps NC: 3.9s * not possible. Retrieve all information and process on client side

12 Performance and query mechanisms (netflow) Get all information about interfaces ps: N/A ps NC: 0.09s Get name of all interfaces ps: N/A ps NC: 0.07s Get name of interfaces starting with bergen ps: N/A ps NC: 0.04s

13 Scalability - # of interfaces Retrieve last link load and errors for a specific interface perfsonar (Java) perfsonar NC # interfaces Time s Error perfsonar (Perl) # interfaces Time s s # interfaces Time s s s s s s

14 Code base PerfSONAR (Java) rrdma Java: 1609 Netflow Java: 3721 Common Java: 7486 PerfSONAR (Perl) rrdma Perl: 2515 Common Perl: 5317 perfsonar NC rrdma PHP: 174 XML: 54 Total: 228 Netflow PHP: 149 XML: 81 Total: 230 Common PHP: 626 (will increase)

15 Summary and future work perfsonar NC Alternative perfsonar implementation Based on NETCONF and YANG Common MA data model Compared to perfsonar More features Faster Better scalability Less code Future work Implement missing features Production ready version Implement generic Web UI

16 Code availability