Open Source in Public Sector and large custom development projects Holger Zobel 2009-10-21 Accenture, its logo, and Accenture High Performance Delivered are trademarks of Accenture.
Agenda 1. Open Source in a large project 2. Licences 3. Public sector and Open Source 4. Spring Batch 2
Agenda 1. Open Source in a large project Project description Architecture Selecting frameworks Bugs in open source 2. Public sector and Open Source 3. Spring Batch 3
The Pension Reform - One of the largest reforms in public sector The Norwegian Labour and Welfare Organisation (NAV) is responsible for government pensions, unemployment benefits, children's allowance, disability and several other government benefits. NAV has 14 000 employees and is administrating one third of the Norwegian governments budget. The Norwegian Parliament decided on the Pension reform in the spring session of 2005. The reform is planned to be effective as of 2011. There is broad political agreement upon the principles in the reform A Government White Paper about Old Age Pensions in the National Insurance was released November 2006 4
The new Pension Application The pension project is developing two front end applications: Internet self service application (all adult Norwegians are potential users) Internal Case Worker Application (2000 users in NAV) The first release of the application was in December 2007. About 150 different web pages, 50 batches, 70 developers, integrating 18 systems, 20 000 test steps, databases of all Norwegians taxable income since 1967 5
Technical Architecture Vision and Strategy Technical Architecture Vision: The Pension Reform Project s platform will establish the foundation for The Client s preferred future technical platform. The solution implemented will be available to anyone, anywhere, anytime. The platform is modern in 2010, allowing room for changes in technology and new functionality within a reasonable timeframe, as seen from the system management perspective. Loose Coupling Business Processes Modern Technology Architecture Strategy: Service Oriented Architecture Layering End to End Operation Virtualisation 6
Technical Architecture Java based custom application Documentation Example Code SOA Platform Service Identification Guidelines and connectivity Developer Training and Certification Security Identity and Access Management (I & AM) Rule engine 7
Selection Criteria Functionality Standards based Future proof Support skilled resources available Price Other 8
Example: Selecting OS Functionality Win2000 HP-UX z/os Linux Standard / Future proof Support / Resources Price Other 9
Example: Selecting ORM framework Functionality EJB2.0 EJB3.0 (JPA) Hibernate Standard / Future proof Support / Resources Price Other 10
Technical Platform Red Had Linux Java Frameworks (next slide) WebSphere Application Server WebSphere Process Server Rational Application Developer (Eclipse based) Tivoli Access Manager / Tivoli Identity Manager Blaze Advisor (rule engine) Content Manager (document archive) VMWare for development and test environments 11
Java Frameworks Java EE 5 Spring Apache Commons JSF (myfaces) Spring Web Flow Hibernate Dozer Ajax4Jsf 12
Fixing bugs in Open Source Frameworks (1/2) Bug in Hibernate Found bug in how Hibernate handled a query Reported bug on mailing list, fixed later that day! But fix was done in HEAD, not in our version We did not back port fix, changed our query instead. 13
Fixing bugs in Open Source Frameworks (2/2) Bug in Spring Web Flow (SWF) Found bug in locking in SWF which led to hangs Was already fixed in later version. No interest in fixing in our version. Upgrade would cause changes in almost ready application Had to make an work around. 14
Agenda 1. Open Source in a large project 2. Licences 3. Public sector and Open Source 4. Spring Batch 15
Licenses Overview The Open Source licenses are many and varied. The obligations they impose need to be taken seriously, since they are enforced. Software licenses, whether open source or proprietary, grant rights to licensees that enable them to use the software. In the case of OSS, those rights include the ability to obtain, modify, use and distribute the software and its source code. In exchange for the rights, the licenses impose obligations on the licensees. Proprietary software typically imposes obligations not to copy or distribute the software. OSS obligations vary widely - from having to make any derivative works available under the same license and rights, through attribution of use to seemingly trivial obligations such as buying the author a drink when you see them. It is this variety of obligations that requires OSS to be treated seriously and not just as "free" software. 16
Open Source Licences Defined by Open Source Initiative (OSI): Free Redistribution Source Code Derived Works Integrity of The Author's Source Code No Discrimination Against Persons or Groups No Discrimination Against Fields of Endeavor Distribution of License License Must Not Be Specific to a Product License Must Not Restrict Other Software License Must Be Technology-Neutral 17
Approved Open Source Licenses from opensource.org Academic Free License 3.0 (AFL 3.0) Affero GNU Public License Adaptive Public License Apache License, 2.0 Apple Public Source License Artistic license 2.0 Attribution Assurance Licenses New and Simplified BSD licenses Boost Software License (BSL1.0) Computer Associates Trusted Open Source License 1.1 Common Development and Distribution License Common Public Attribution License 1.0 (CPAL) Common Public License 1.0 CUA Office Public License Version 1.0 EU DataGrid Software License Eclipse Public License Educational Community License, Version 2.0 Eiffel Forum License V2.0 Entessa Public License Fair License Frameworx License GNU General Public License (GPL) GNU General Public License version 3.0 (GPLv3) GNU Library or "Lesser" General Public License (LGPL) GNU Library or "Lesser" General Public License version 3.0 (LGPLv3) Historical Permission Notice and Disclaimer IBM Public License ISC License Lucent Public License Version 1.02 Microsoft Public License (Ms-PL) Microsoft Reciprocal License (Ms-RL) MIT license Motosoto License Mozilla Public License 1.1 (MPL) Multics License NASA Open Source Agreement 1.3 NTP License Naumen Public License Nethack General Public License Nokia Open Source License Non-Profit Open Software License 3.0 (Non-Profit OSL 3.0) OCLC Research Public License 2.0 Open Group Test Suite License Open Software License 3.0 (OSL 3.0) PHP License Python license (CNRI Python License) Python Software Foundation License Qt Public License (QPL) RealNetworks Public Source License V1.0 Reciprocal Public License 1.5 (RPL1.5) Ricoh Source Code Public License Simple Public License 2.0 Sleepycat License Sun Public License Sybase Open Watcom Public License 1.0 University of Illinois/NCSA Open Source License Vovida Software License v. 1.0 W3C License wxwindows Library License X.Net License Zope Public License zlib/libpng license 18
Licenses Overview Each license type defines the usage, modification and distribution of the related sources Type of License Strong Copyleft *) Licence Weak Copyleft License Non Copyleft License Attributes The copyleft governing a work is considered to be "stronger", to the extent that the copyleft provisions "flow down" and apply to all kinds of derived works. "Weak copyleft" refers to licenses where not all derived works inherit the copyleft license; whether a derived work inherits or not often depends on the manner in which it was derived. Weak copyleft" licenses are generally used for the creation of software libraries, to allow other software to link to the library, and then be redistributed without the legal requirement for the work to be distributed under the library's copyleft license. Only changes to the weak copylefted software itself become subject to the copyleft provisions of such a license, not changes to the software that links to it. This allows programs of any license to be compiled and linked against copylefted libraries, and then redistributed without any re-licensing required Under these licenses, the software can be used for any purpose and there are no obligations on licensees to distribute the source code of derivative works. *) http://en.wikipedia.org/wiki/copyleft: Copyleft is a play on the word copyright and describes the practice of using copyright law to remove restrictions on distributing copies and modified versions of a work for others and requiring that the same freedoms be preserved in modified versions. 19
Agenda 1. Open Source in a large project 2. Licences 3. Public sector and Open Source 4. Spring Batch 20
Public Sector Policies The ministry responsible for ICT is Fornyings- og kirkedepartement. Key policies: enorway 2009 Mandatory Open Standards Established Difi The Agency for Public Management and egovernment (Difi) aims to strengthen the government s work in renewing the Norwegian public sector and improve the organisation and efficiency of government administration. 21
enorway 2009 enorway 2009 has three target areas: The individual in the digital Norway Innovation and growth in business and industry A coordinated and user-adapted public sector 22
Use of Open Standards and open Source Applications Initiatives should be undertaken to develop open source skills in the public sector. A recommendation concerning use of open source software in the public sector will be developed in 2005. By the end of 2006, all public sector agencies shall have drawn up plans for use of open source applications. Goals By 2009, all new ICT and information systems in the public sector shall use open standards. By 2006, a set of administration standards for data and document exchange shall have been established. By 2006, all public sector agencies shall have incorporated how they are going to use open standards, service-oriented architecture and open source applications in the relevant planning documents. By 2008, data and document exchange in the public sector shall satisfy administration standards. By 2008, all official forms shall be available electronically and built round a common user interface. 23
Mandatory Open Standards HTML - 4.01/ XHTML - 1.0 is recommended for public sector web sites In special circumstances, like need to keep formatting or documents for further editing, the following formats shall be used: PDF 1.4 or PDF/A - ISO 19005-1 is mandatory for documents which should not be edited ODF 1.1 (Second Edition), ISO/IEC 26300:2006 is mandatory, but it is suggested that other formats be used in parallel. OOXML is observed and will be evaluated later 24
Open sourcing software owned by Public Sector Almost no software is open sourced by public sector organisations. Reasons include: No culture for sharing Few people with experience with open source Very specialised applications like Child Support administration Security 25
Licences Public sector seldom has constraints on keeping trade secrets But still have to consider licences to make sure that they are compatible. Example: Using proprietary source code and GPL in the same project. 26
Agenda 1. Open Source in a large project 2. Licences 3. Public sector and Open Source 4. Spring Batch 27
Why Batch? Large volumes Import/export files Convert data Archiving Not a part of Java EE today 28
Alternatives for Java developers today Do it in Java: Develop an in house architecture Implement business logic in a language you know Use an ETL tool Expensive Implement business logic in another language 29
Spring Batch background Why is Accenture contributing to open source? Collecting many years of experience with batch Want to standardize batch implementation Why Spring? Established and active community with regular releases. A part of many/most Java projects Goal A standarized, scalable batch architecture that is easy to use. 30
Spring Batch: Scenarios Periodical commitment of chunks Manual or planned restart after errors Sequential processing of dependent steps Parallel processing 31
Spring Batch: Features Built in support of different formats fixed length, delimited, XML Automatic retry (policy driven) Execution status and statistics while batch is running and historical Different ways to start batch (JMX, command line) Support functions like logging, resource management, restart, skip, etc. 32
Spring Batch: Scalability Same application code can be used at all levels Application = Batch Container Specialized Batch Container Layer Implementations Workload Multiple Clustered JVMs Multiple JVMs Single JVM Multi-Threaded Application changes are not required when changing the deployment approach Infrastructure Common Infrastructure Single JVM Single Process 33