Multidimensional Testing

Size: px

Start display at page:

Download "Multidimensional Testing"

Douglas George
5 years ago
Views:

1 Multidimensional Testing QA appoach fo Stoage netwoking Yohay Lasi Visuality Systems 1

2 Intoduction Who I am Yohay Lasi, QA Manage at Visuality Systems Visuality Systems the leading commecial povide of SMB solutions fo all maket sectos such as Embedded IoT Java Stoage Visuality Systems expeience coves poting onto following platfoms Embedded devices Java applications Standalone NAS Cluste Visuality Systems solutions cosses numeous platfoms thus the main challenge fo the Visuality Systems QA team is the divesity of stoage solutions. The at-of-qa is to choose the ight test stategy and continue impoving this stategy by leaning fom customes, patnes and supplies on going maket equies. 2

3 Session Plan This pesentation will cove QA stoies when it comes to testing's a netwok solution fo stoage How to choose key configuations Factos defining stoage configuation Stoage classification fom the QA point of view How to conside use cases Consult Visuality Systems customes, patnes and supplies. The Japanese Round Robin method by holding peiodical intenal Think Meetings How to choose ight tools Diffeent test methodologies equie diffeent tools How to develop a good Test Plan Conside all the above conclusions and then think a little moe. 3

4 Configuations : Factos The factos defining Stoage configuations: Softwae Platfom OS Othe softwae e.g., layes above Linux Netwok stacks: TCP/IP, RDMA, DPDK File Shaing Potocols: SMB and NFS. Coss-potocol option Hadwae Types of CPU Memoy size Netwok infastuctue: 1GB / 10GB / Fibe NIC: Ethenet / InfiniBand / RoCE / WARP Dive(s): HDD / SAS / SSD as well as contolle types PCI / SATA / ISCSI etc. Cluste Numbe of nodes Node oles (AA o AP) 4

5 Configuations : Examples Typical stoage configuations: SOHO NAS: ARM 1.2GHz Dual Coe, HDD Mid-level stoage: Atom 2.13GHz Quad Coe, HDD Top-end stoage: Intel Xeon 3.4GHz Quad Coe/IBM Powe8, SSD It is impossible to cove all types of configuations. The idea is to select the appopiate configuation by using stoage classification 5

6 Stoage classification To choose the key configuations, fist QA has to ceate a stoage classification Stoage fo geneal pupose Basic Stoage pice fist, scalability second Mid-level Stoage balanced Top-end Stoage Dedicated platfom scalability only 6

7 Use cases Aeas Data Centes Clouds Dedicated aeas (Video Steaming again) Famewoks Hype-V SQL Seve SAP Name it Numbe of simultaneous connections Dedicated NAS may ask fo 5-10 Geneal pupose NAS SAN fom 100 to 10,000 and moe 7

8 Use cases (cont.) Load is usually foced by applications but may be also specified explicitly, e.g., - fo bulk file tansfe: Recods pe second Read/wite sizes Some customes cae only about 4-8K chunks. This is common fo SQL Seve File tansfe will be happy with 1MB chunks Video steaming may need even moe Intensity of meta-data opeations Usually customes mostly cae about eads/wites SIEM may think diffeently 8

9 QA techniques QA techniques can be categoized as follows: Functional Stess / Pefomance Negative and secuity 9

10 Functional Potocol level API Level Use expeience Type Explanation Advantages Disadvantages Sample tools Potocol level Geneates potocol (SMB) packets by the spec(s) Fomally coves the spec(s) Does not eflect eal taffic Micosoft test suite / Smbtotue API Level Enumeates (elevant) Win API calls Coves potential taffic geneated by applications Does not cove all possible taffic combinations Visuality SMB test suite Use expeience Runs the most common application Coves eal taffic Cannot cove all applications Manual tests / Ranoex 10

11 Stess / Pefomance Single connection Using full capacity of netwok inteface Video steaming o upload lage files Multiple connections 1000 connections to a single folde on a shae, ead / wite 1 mb file 1000 connection to 1000 foldes on a shae, ead / wite 1 mb file File tansfe Upload / Download 1GB, 5GB and 20GB files Upload 1000 files (1MB each file) Vaious buffe sizes fo Read / Wite Buffe size Usage 4 16 kb Typically fo SQL o Metadata 1 8 mb File tansfe / Video steaming 11

12 Negative and secuity Eoneous tansactions, e.g. Witing to a non-existing file Delete on ead only file And moe Sending malfomed inputs Witing a file using unsuppoted buffes Reading file using unsuppoted buffes And moe The above techniques compliment one anothe. A complete QA test equies all of the above. 12

13 Tools fo functional tests - Potocol Level Windows Potocol Test Suites Developed by Micosoft validates potocols including the File Seve (SMB) tests Build veification tests Non build veification tests Positive Negative Compatibility Invalid define Out of bounday Unexpected context Unexpected fields Smbtotue An open souce test suite that uns seies of tests against an SMB seve 13

14 Tool fo functional tests - API Level Visuality Systems test Suite Test Posix and WinAPI commands on a netwok stoage Posix Testing of open, ename and delete file opeations Testing of ead and wite file opeations Testing of opeations on file attibutes Testing of diectoy ceate, ename, attibutes, and scan opeations WinAPI Testing of ceate, open and delete file opeations Testing of ead and wite file opeations Testing file copy, move and attibutes opeations Testing 16-bit file opeations Testing diectoy ceation, delete, info, copy, move, eplace and scan opeations Testing volume infomation opeations Testing netwok opeations Testing of local use management opeations 14

15 Tools fo functional tests - Use expeience Manual tests By using Windows Clients Visuality Systems can geneate set of manual tests such as: Upload/Download files Ceate files Edit files Rename files Delete files Ranoex Automated use expeience. This tool ecods use expeience scenaios and geneate these scenaios upon demand in an automated envionment Scenaios fo Example Ceate Micosoft Office files: Wod, Excel, Powe Point Open Wite Close Rename Delete 15

16 Tool fo Stess Vitual Instuments (Load Dynamics) - VI is a peace of hadwae that knows to simulate numeous clients. As being scipt-diven it allows custom scipting thus potentially coveing any scenaio Also good fo epoducing abnomal taffics Allows to geneate vaious taffic types (we ae limited to SMB expeience) Stess test cases Ceate lage File and wite to it using lage buffes Run 1000 connections to a shae while eading and witing 1 MB file Run 1000 connections that connects instantly. 16

17 Tools fo Pefomance tests Pefomance tools Tool Method Loads Notes SpecSFS 2014 Load geneato Desktop Steaming Compilation Database I/O mete Read/wite load Database Steaming Mix Only geneic esults, Vey high loads. (in tems of taffic) Dill-down esults Diskspd SQL Seve simulation Database Basic benchmaking only LoadDynamics Hadwae simulato Any + multiple connections Vey high loads (also in tems of connections) Nobody's pefect!.. thee is no single tool that measues eveything Visuality Systems expeience is - each custome has his favoite tool and methods 17

18 Negative and secuity tool Synopsys (Defensics) Defensics geneates malfomed SMB (and not only SMB) taffic in any possible way in ode to cash the seve This allows to pevent attacks and eoneous geneated packets Five yeas ago Visuality Systems wee using Defencis fo the fist time, It took 10ms to cash ou seve Now it uns hous unde vey heavy load 18

19 QA techniques guidelines Pio using the QA techniques and tools the following points should be consideed: Pepae test documents Estimate time to test fo each technique Human facto Availability of hadwae esouce (Lab time) 19

20 Go testing Taking into account the points descibed in this pesentation, Visuality Systems can daw the multidimensional QA testing's as follows Build shot tem test plans fo alpha and beta vesions that will include: Running functional test pe featue Running manual tests pe featue Running stess and pefomance tests pe featue Build long tem test plans fo final elease that will include: Running functional tests fo full system equiements Running manual tests full system equiements Running stess and pefomance full system equiements 20

21 Thank You! Tel: +972 (4)

Accelerating Storage with RDMA Max Gurtovoy Mellanox Technologies

Accelerating Storage with RDMA Max Gurtovoy Mellanox Technologies Acceleating Stoage with RDMA Max Gutovoy Mellanox Technologies 2018 Stoage Develope Confeence EMEA. Mellanox Technologies. All Rights Reseved. 1 What is RDMA? Remote Diect Memoy Access - povides the ability