open source RCP Eclipse based Visualization analysis Python Workflow

Transcription

1

2 An open source not for profit project built on the Eclipse Rich Client Platform (RCP) framework Eclipse based workbench for doing scientific data analysis. It supports: Visualization and analysis of data in 1D, 2D and 3D Python script development, debugging and execution Workflow tool for the treatment/manipulation of scientific data

3 Client server technology Communication with EPICS and hardware Scan mechanism Acquisition Jython and Python Visualisation Communication with external analysis Analysis tools Data read, write, convert Metadata structure Workflows Analysis DAWN is a collection of generic and bespoke views collated into perspectives. The perspectives and views can be used in part or whole in either the GDA or DAWN.

4 Active code contributors/developers Commercial subcontractors Evaluating or strong interest expressed

5

6

7 RCP is an extension based platform (as opposed to an extensible application) i.e. Emacs, Auto CAD, Eclipse Minimalistic runtime, that includes extension mechanism High level language Extension points mechanism Extension based platform Extensible application Through its Plugin architecture RCP lets you: Decompose your code into loosely coupled units Extend (and update) your product incrementally Enforce contracts between groups in your organization Play nicely with components from other vendors Allow even customers to extend your product EclipseCon Advanced RCP by Kai Tödter

8 Standard Widget Toolkit (SWT) Portable open source widget toolkit for Java Displays GUI elements using native GUI libraries Draw2d Lightweight toolkit for displaying graphical components on an SWT Canvas Nebula visualization library data visualization in science, engineering... originally from CSS project Functions of XY Graph: Line, scatter, bar, step, area chart, zooms, panning, auto Scale, undo/redo etc.

9

10 Apache Commons : reusable Java components Math: Lightweight, self-contained mathematics and statistics components IO: Collection of I/O utilities Net: Collection of network utilities and protocol implementations Jexl: Expression language which extends the Expression Language of the JSTL. HDF5/NeXus libraries to support HDF5 data Passerelle toolkit built on Ptolemy II, (Java packages for heterogeneous, concurrent modeling and design) for workflows SLF4J for logging Jackson for processing JSON data format BoofCV Java computer vision library etc

11 RCP many of the core features, editors, toolbars, views, projects GEF for visualization of pipeline graphs SWT/Jface lazy viewers being used extensively for large trees and tables Draw2D for 1D and 2D plotting (SWT XY Graph) Pydev for python/jython scripting layer used by the scientists Ptolemy 2 (a version known as Passerelle ) workflow and pipelining HDF5 libraries for storing large data sets Apache, Eclipse-WST, springsource, JDK, and many more of course...

12 Hosted by the Eclipse Foundation Collaborative effort involving industry, academia, and government Language and platform agnostic Develop well defined interfaces, models, definitions, algorithms, and reusable software libraries, services, and applications Develop reusable open source software for scientific research Making science software inter-operable and interchangeable Who Oak Ridge National Laboratory IBM Diamond Light Source Kichwa Coders IFP Energies Nouvelles MARINTEK / Itema Clemson University Lablicate (Chemclipse) Facility for Rare Isotope Beams (FRIB) Tech Advantage Uppsala University (Bioclipse) Others not on the steering committee

13 Provides features to allow scientific software to be interoperable and re-usable Defines Java interfaces for data description plotting and plot tools data slicing and file loading It defines an architecture oriented around OSGi services provides a reference implementation Eclipse links

14 Eclipse RCP product built using Buckminster (previously PDE) Usage of Jenkins for continuous integration Unit tests using Squish UI Testing, Junit and Junit plugin tests Agile practices being used where otherwise possible Documentation of designs and code using confluence Source code control using Git/eGit

15 Cross-platform GUI and regression testing tool available as an Eclipse plugin Can test applications based on a variety of GUI technologies Able to record and replay test scripts written in JavaScript, Perl, Python, Ruby or Tcl

16 Eclipse UDC Lightweight framework for collecting usage data information using listeners on various aspects of the Eclipse workbench Be able to provide feedback on how DAWN is used What are the most used views/perspectives What are the systems used to run DAWN (Java version, OS, language, etc)... Potentially useful information for management and development teams

17 Conducted Dec 13-Jan 14 By Software Sustainability Institute (SSI)...cultivating world-class research through software. based at the Universities of Edinburgh, Manchester, Oxford and Southampton Funded by EPSRC (Engineering and Physical Sciences Research Council) Technical review Tutorial-based Evaluation Collaboration review Questions agreed in advance on-site interviews, telecon interviews

18 20 recommendations A reasonably typical profile for such a project Significant development history, scale of development

19 Generally a good feedback profile Agreement on many positive aspects Most suggested improvements constrained to individual beamlines Many stakeholders Complex requirements! A challenge! # observations across area # recommendations across area

20 Positive highlights Development, support a major strength Keen to understand scientific needs of beamlines Timely responses for resolving issues and queries Training useful, applicable, and enjoyable Integration with working process Software many useful features and integration Areas for Improvement Future release planning Software interface Documentation quick start guides, goal-oriented docs Technical writer? Addressing new features prioritised over testing Software testing personnel?

21 Open-source and science work well together It is important to pick a technology that Answers your users specific needs Has a proven track record and is well used Has a good community support Better collaboration leads to better software and in result to better science Industry good practices and tools are a necessity for big projects Having your software reviewed will give you valuable information An external viewpoint What you are doing right Areas that can be improved

22 Eclipse community for providing great tools Java/Python community for its APIs Software Sustainability Institute for constructive reviewing of DAWN Diamond science and software groups All collaborators that participate to DAWN developments

23 And there's more. Scientific uses of Diamond presented in tomorrow s talk Posters...