Arguing With the Machine Analysis of Auto-Generated Code. Jacob Cox, TASC

Transcription

1 Arguing With the Machine Analysis of Auto-Generated Code Jacob Cox, TASC 1

2 Agenda Types of code verification General discussion of code generation Some experiences 2

3 The Simple Life In the beginning people wrote FSW in C 3

4 Code Analysis Static code analysis (Syntactic) Uses industrial analyzers Produces large numbers of false positives Warnings need checked by hand Implementation verification (Semantic) Tracing requirements and design elements to the software implementation Manual process 4

5 Code Generators Take input and produce source code Input can be: UML Models & Diagrams Manual Input (drawing Object diagrams) Text Files (XML) Excel Etc. Input Generator Code 5

6 IV&V Approaches Should we analyze the input instead of the output? This is the way we treat compilers Can you trust the generator? Is the generator a COTS product? Has IV&V been performed on it? Can you identify the generated source from the manually coded source? Is the generated code human readable? 6

7 Matlab/Simulink The code had headers stating not to edit Warning found in these files were ignored The generator was trusted The generated code was hard Lack of comments Difficult algorithms Variables & function names that were machine created 7

8 Dictionaries A common place to store information used by everyone. Commands Telemetry Faults The data is stored in XML files 8

9 XML Text file Hierarchical data store Allows defining schemas Human readable <Dictionary> <Command> <Name>AddStarToCatalog<Name/> <Params/> <Param> <Name>RightAscentionHour<Name/> <Type>UnsignedByte<Type/> <MinVal>0<MinVal/> <MaxVal>23<MaxVal/> <Param/> <Command/> <Dictionary/> 9

10 XSLT Is an XML transformation language Is XML Is naturally recursive Data in XML XSLT Processor Resulting Text File XSLT Script 10

11 Using the Dictionaries All products are consistent since they come from the same source XSLT Converting to HTML XSLT Processor HTML Web Browser XSLT Converting to PDF XSLT Processor Command Manual Command Dictionary in XML Code Generator Source File Source File Source File Source File Source File Source File 11

12 Results of the process Clearly readable code Stubs to place processing of Command Handlers Very efficient in code production Very in-efficient with respect to SLOC 12

13 How Did the IV&V Project React Black box approach on validating the generators Compared input XML patterns to resulting code Static (syntactic) code analysis on the resulting code Traced appropriate requirements to the resulting code (semantic analysis) 13

14 KlocworkRaw Results Warnings from Klocwork are sorted by Autocode generator, or Handcoded When warnings are put into a pivot table, the grouping of error types by Autocoderare apparent. 14

15 Examples Checking parameters to insure they are not greater than maximum value but not checking they were less then minimum. Pointer increments at the end of repeated blocks of code. Generally speaking, static code analysis on auto-generated code did not produce issues and the warnings fell into easily identified patterns. 15

16 Code, Dictionaries and Requirements Project A Cmd Rqmnt Cmd Rqmnt Cmd Rqmnt Cmd Rqmnt Cmd Rqmnt Cmd Rqmnt Cmd Rqmnt Cmd Rqmnt Command Dictionary Generator Code for FSW Plumbing There is a transitive relationship between the requirements, the dictionary entries and the code implementation. Checking the requirement in code also verified the dictionary (at least the parts that generated code). Note: Semantic analysis of a command requirement found that mode restrictions Had disappeared from the dictionary. 16

17 Code, Dictionaries and Requirements Ctd. Project B Parent Rqmnt Parent Rqmnt Parent Rqmnt Parent Rqmnt Parent Rqmnt Parent Rqmnt Cmd Rqmnt shall be in the dictionary Command Dictionary Generator Code for FSW Plumbing There is no relationship between the requirements and the dictionary entries or code implementation. A solution could be a reference to the parent requirement in the dictionary entry. 17

18 Model Generated Code Fully model generated code which generates all the code using Class diagrams Sequence diagrams Activity diagrams, etc. Partially generated code where the generator creates headers and stubs that are filled in by hand Class diagrams Analysis should be on the Inputs to the generator Really just well structured Hand written code requiring the same analysis as regular Hand written code. 18

19 Model Generated Code, Ctd. The Hybrid where the diagrams generate the code BUT there is hand written code embedded in the diagrams that is incorporated into the generated code. Semantic Analysis can be performed on the snippets Does IV&V perform static analysis on the handwritten parts? Would it be performed on the snippets only and how to identify them in the generated output? 19

20 Questions? 20