List of detected issues

Transcription

1 List of detected issues Below we report in detail the issues found while testing each software package (see Supplementary Table 7 below). For concision, we report here only a representative sample of the SBML compatibility problems indicated in Supplementary Notes 2. BASIS Opened files Model2_L1v2.xml and Model2_L2v1.xml, and ran simulations. However, these yielded erroneous results (constant levels of all metabolites over 200 s). This seems to have been due to BASIS overlooking the information about units of measurement contained in the SBML files. After changing the values of the second-order rate constants so that these are expressed as molecule -1 s -1 (kf = kt2 = 1 molecule -1 s -1 ) the simulations stopped with the error Species G80_protein has gone negative in reaction 24. The problem now was that the kinetics for the homobimolecular reactions were not automatically corrected to k*x*(x-1). Once the correction was made manually, BASIS was able to simulate the model. CellWare When implementing the model manually, the Multiple Selection tool often behaved inappropriately, resulting in a zoom in. When the manually implemented model was saved in the CellWare format (.cwm) and then reopened after restarting CellWare, the following problems were found. One of the species (G3D_G80D) was drawn outside the compartment. Initial concentrations of the variable metabolites were sometimes (but not every time the operation was repeated) reset to default

2 values. The Species list box in the Species to monitor pane (reached via the Simulation menu) was empty, preventing the setup of the graphics. All test models CellWare freezed while importing SBML files. Further, it was unable to export models in SBML versions above L1v2, although SBML L2v1 is available on the type of archive option in the SBML Export dialog box. Exported SBML Level 1 files fail validation. Level 1 version 1 files yield the following validation errors: [ ]3: (100) Datatype error: Type:InvalidDatatypeValueException, Message:Value 'Modelxxx_celware.cwm' does not match regular expression facet '(_ [a-z] [A-Z])(_ [az] [A-Z] [0-9])*'. 7: (100) Datatype error: Type:InvalidDatatypeValueException, Message:Value 'moles/l' is not in enumeration. Level 1 version 2 files yield only the first of these validation errors. These errors are easily corrected by manually editing the SBML file. COPASI COPASI failed to open files Model2_L1v1.xml, Model2_L1v2.xml and Model2_L2v1.xml with the following error message: >Exception <date> <hour> <\.SBMLImporter.cpp(438) compiled: Feb :05:55. On manual implementation of the model, COPASI ran both deterministic and stochastic simulations without problems. COPASI also exported an SBML L2v1 file for the manually implemented model. This file passed validation, but COPASI could not interpret it correctly.

3 Model 3 Opened a GEPASI (.gps) file for Model 3 but crashed when we then attempted to run any task. This occurred only for models where some user-defined kinetics where set up in GEPASI. This problem did not occur when the model was exported from GEPASI in SBML L1v1 format and then imported with COPASI. Model 5 COPASI failed to open files Model5_L1v1.xml, Model5_L1v2.xml and Model5_L2v1.xml with the following error message: >Exception <date> <hour> <\.SBMLImporter.cpp(503) compiled: Feb :05:55. Upon manual implementation, COPASI was able to integrate the model and to perform stability and sensitivity analyses of the steady state. However, it crashed frequently while attempting to perform these tasks. All Models We found COPASI version 17 (Release Candidate 1) somewhat unstable, and we advise the use of the more stable version 15, until an improved one becomes available.

4 Dizzy Model 1 Dizzy failed to integrate Model 1 for end times larger than 0.01 s. However, it successfully and accurately integrated the model for smaller end times. Dizzy was unable to parse files Model2_L1v1.xml and Model2_L1v2.xml, returning error org.systemsbiology.util.invalidinputexception: invalid kinetic law expression: RATE *pow(g3_protein,2). After pow was replaced by the appropriate explicit multiplication the model was parsed, but all the simulations failed even after trying a wide range of settings for each algorithm. After the model was manually implemented, Dizzy was able to run it using any of the available integration algorithms (both deterministic and stochastic). Model 4 Dizzy was able to export an SBML L1v2 file, but this file failed online validation with the following error: 5: (1205) A 'volume' UnitDefinition must simplify to a single Unit of kind 'litre' or 'metre' (L2v1 Section 4.4.3).

5 The error was due to the units of compartment volume being defined as dimensionless. However, Dizzy interpreted the exported SBML file correctly and could run simulations from it. All Models It was often necessary to try a number of different accuracy and time-step settings before simulations ran successfully. Dynetica Reactions in which one species appeared both as reactant and as product were modified in the stoichiometry box when the Entity Property Editor window box was reopened. The error resulted in incorrect rate equations, which did not allow simulating the model. The rate expressions in the substance boxes and in the math window continued to take a reaction into account even after this was deleted. This error could be corrected by saving the model, quitting Dynetica and reloading the model. GEPASI Models 1, 2 GEPASI crashed when importing files Model1_L1v1.xml and Model2_L1v1.xml. Models 1, 2, 4 After the models were manually implemented, GEPASI was able to create SBML L1v1 files that passed validation. However, it failed to load these files.

6 Model 5 GEPASI yielded erroneous Lyapunov exponents for steady states 1 and 2. It failed to compute steady state 3, yielding error floating point division by zero. JDesigner/Jarnac Model 3 JDesigner failed to open files Model3_L1v1.xml and Model3_L1v2.xml, returning error Error while trying to read SBML file in LoadSBMLFile: LoadFromStream: Error in ParseXMLDoc: This simulator does not currently support fractional stoichiometries, try decimal stoichiometries instead: in getstoichiometry. Opened file Model3_L2v1.xml and ran correct simulations. However, the exported SBML L1v1 file failed validation with error report 326: (100) Datatype error: Type:InvalidDatatypeValueException, Message:Value '0.5' does not match regular expression facet '[+\-]?[0-9]+'.. Furthermore, the stoichiometric coefficient was not reflected in the model equations displayed through the menu View View model equations. Model 5 JDesigner reported a range check error while calculating steady state 3 and yielded an inaccurate Lyapunov exponent. All models JDesigner displayed very tangled diagrams, which were very cumbersome to rearrange, when complex models were open from SBML files generated by other programs. All exported SBML Level 1 files failed validation, yielding the following error messages:

7 12: (100) Unknown element 'annotation' 876: (100) Element 'annotation' is not valid for content model: '(notes,annotations,( listofunitdefinitions,listofcompartments,listofspecies,listofparameters,listofrules), listofreactions)' Pasadena Twain Model 1 Pasadena Twain yielded a range check error and failed to run the simulation. Virtual Cell Virtual Cell failed to open file Model2_L2v1.xml returning error Unable to convert from: item.m-3.s-1 to: um.s-1. Opened files Model2_L1v2.xml and Model2_L1v1.xml returning the following warnings: The following XML element attributes will be ignored: reaction.reversible, Two consecutive membranes/features, Possible unit incosistency for reaction: null and The species: source units are not compatible with Vcell units: item.m-3 um (multiple instances). Misinterpreted the initial numbers of molecules as if these were expressed in micromole and the units of rate constants of second-order reactions as μm -1 s -1. After the initial values and second-order rate constants were corrected, Virtual Cell was able to run the simulations.

8 Model 3 Virtual Cell failed to open file Model3_L2v1.xml returning error Loading document failed: null. Opened files Model3_L1v2.xml and Model3_L1v1.xml returning the warning: The following XML element attributes will be ignored: reaction.reversible speciesreference.denominator. Model 4 Virtual Cell failed to open files Model4_L1v1.xml, Model4_L1v2.xml and Model4_L2v1.xml returning error Loading document failed: null. Model 5 Virtual Cell opened file Model5_L2v1.xml returning warnings: Two consecutive membranes/features, Possible unit incosistency for reaction and Possible unit incosistency for reaction. The program interpreted the concentrations incorrectly. All models Virtual Cell exported SBML L1v2 and L2v1 files that failed validation with numerous errors. It also did not respect the user-requested time interval when exporting simulation results.

9 Supplementary Table 7: Types of detected issues. A black dot indicates a problem detected for one or more test models. The specific problems are described in the previous sections and in Supplementary Notes 2 and 6. Program Failure to open validated SBML Misinterpretation of validated SBML Export of SBML that failed validation Problems in model definition Unwieldy a or inaccurate layout of imported models Misformating data for exporting Errors in stability analysis Errors in steady-state sensitivity analysis Problems with simulation engines BASIS NA NA CellDesigner NA NA CellWare NA NA NA COPASI NA Dizzy NA NA NA Dynetica NA NA NA NA NA NA GEPASI NA JDesigner/Jarnac JSim NA NA NA NA NA NA Pasadena Twain NA NA NA NA NA NA NA PLAS NA NA NA NA Virtual Cell NA NA a Considered a relevant issue only in programs that lack functionality for automatically laying out the diagrams in alternative ways. NA: Not applicable