DALA Project: Digital Archive System for Long Term Access

Transcription

1 2010 International Conference on Distributed Framework for Multimedia Applications (DFmA) DALA Project: Digital Archive System for Long Term Access Mardhani Riasetiawan 1,2, Ahmad Kamil Mahmood 2 1 Master of Information Technology Program, Universitas Gadjah Mada Jalan Grafika No 2, Kompleks UGM Yogyakarta, Indonesia 1 mardhani@mti.ugm.ac.id 2 Computer and Information Science Department, Universiti Teknologi PETRONAS Bandar Seri Iskandar 31750, Perak, Malaysia 2 kamilmh@petronas.com.my Abstract The scientific environment have face the current issues in the data landscape change. The volume, type, formats of scientific data now has being exploding. The industry standard require the long term access and use of digital data for many reasons such as proof of scientific experiments, evidence and proof of findings. It is need to be interactive, reproducible, collaborative, and dynamic and have the reputation and influences for collaborative research. In this research, we introduce DALA (Digital Archive System for Long Term Access) project as the scientific collaboration framework. The project has built for manage and preserve scientific data based on GForge system, and implement several data process. The frameworks have capabilities in handling data collection, data processing, and data archiving. The research aimed is to the new framework for scientific collaboration works. Keywords Scientific, Framework, DALA, GForge System. I. INTRODUCTION The world of science is very dynamics. One innovations or discovery in one area is also will have many discoveries in other area. The innovation itself can count in every minute and every second. The innovation and scientific works have involving many data from many resources. Interdisciplinary or multidisciplinary research has needs on data collection, for example the needs on data share and use of data. The need of long term access and use of digital scientific data has become the major interest in every organization. The data also has become very sensitive when the data need to share and access for another organization. The data is also need to change because of the change on the data landscape, technological matters, and user needs of dynamic data. The digital scientific data have previous and future challenges. The data that produce by research work need to answering the demand on data collaboration. The research need to work with other research, it can be similar or multidisciplinary research. It is a must, because the world now needs the problem solving research and high quality research. Lee Dirks [1] giving the sample of the future needs of scientific digital data, for example is research reporting. The research reports now have characteristics. It must have multi perceptions and dynamically can be customized to each user. The report also can have an access and use, in the term of seeing or following the research workflows and outputs from the lab experiments. The report can be exported into an electronic lab workbench in order to reconstruct the same or similar experiment. It is also giving capabilities to researcher working with multiple reports and having the ability to mash up data and workflows across experiments. The researches have facilities to implement new analyses methods and visualizations and to perform new experiments. The digital scientific data is a proof of evidence. We need it as evidence of research experiments. We use digital data as proof of findings that presenting the high quality results. The data representing the work of the research, open opportunities for another research, and enable to enhance the previous research by developing better research in the future. In this paper, we explain the development of the DALA (Digital Archive System for Long Term Access) project for Digital Scientific Framework collaboration. The research goal is establishing the framework for collaborative research by developing the framework technology to collecting the scientific data, processing the data, analysing the data, visualizing the data and archiving the data. The framework developed using the GForge system, the collaborative development environment. We also developed and combined with preservation system to provide the dynamic digital data in storage and access systems. The paper is presented in several sections. In the section which title Science 3.0 and Scientific Digital, we explain the main concepts of scientific collaboration and needs on preservation system. In the section 3, we explain the development features, and system architectures of the framework. The conclusions and future directions are presenting in the last section II. A COLLABORATIVE SCIENTIFIC WORK The word Science 3.0 is following the term of web 2.0, web 3.0. It is representing the collaborative work in many scientific works. Lee Dirks [2] giving the sample of the needs on high quality reporting. He give sample in the live reporting with characteristics such as have multi perceptions and dynamically can be customized to each user. The report gives capability to access and use research workflows and experience materials. We can analyze the similarity of the research with another. The researches have capabilities to enhance the current research to solve the future problems..

2 Several projects regarding the infrastructure of scientific collaboration actually have implemented. CERN [3] as the European Organization for Nuclear Research is the project that developing new models of data collaboration and supporting with the high technology infrastructures. San Diego Super Computing (SDSC) [4] also giving the models how the high infrastructures of computation giving support to analyse the high volume of data to analyse and get the information in advance issues such as earthquakes prediction, bio-fuel process, and molecules extractions. Open source software is one of the good samples how the collaboration can be from early stage of development until the release. It is also show us on how the software is always keeping a live with a lot of contribution from other researcher. It is can found in the Sourceforge website ( that giving us a lot of software resources to support the researcher interest. III. DESIGN ARCHITECTURES The design architecture will explain the system architecture of the frameworks, the software architectures, and implementation. In the Figure 1, we were showing the system architecture design for the collaboration framework of scientific work. The architecture adapted from Science-Forge research that introduced a collaborative scientific framework design [9]. The system architectures is consist of three components; there application, framework application, PC based cluster, and storage. Storage is the component that handling the data archives process. The storage works under the framework to manage and execute the dynamic data presentation. The main component is the framework application. It is handled the data collection activities, data processing and give the direction the data archive to storage. Access interface is use for managing the data administration from users to access the data. It is functionally important to make sure the data is accessible secure methods. Access node is components to verify and the data presenting to users as needed. We use PC Based Cluster in the implementation to run the system design architectures. We use MySQL Cluster [7] as storage, and GForge system for framework application. The implementation of PC based cluster is using the MPI configuration to allow the workload of the process and allocates the activities. The preservation system is build as the framework components. We developed the preservation components in Gforge. The component functions are to ensure the data have capabilities in access and use as in original formats. A. Software Architecture The software architecture consist of three layer that shown in Figure 3. The storage is the basic layer. The layer used. We installed the MySQL Cluster in the one machines/pc in the same environment with PV Cased cluster. Figure 1. System Architectures (adapted from Science-Forge Framework) [9] Application Layer Collection s Framework Layer PC based Cluster Storage Layer processing Framework Application Cluster based Storage The Framework Application layer is the major component in the design. GForge framework is used because have capabilities in data handling, data processing and data archiving. The Gforge features framework is shown in the Figure 2. It is having several features; there are collaboration, authentication, access, control, output, and source code. We developed the preservation system for data handling and management process. The preservation systems manage the data collection process, data processing, data analysis, data visualization, and data archive. The preservation system has build based on Open Archival Information System [8] to ensure the standards acceptance in scientific digital data subjects. Figure 2. GForge Framework [6] (source: Archive The preservation system component consist of several parts of system, there are input, preservation planning, data management, ingest, and archival storage

3 Figure 3. The Software Architecture The implementation of the framework has several flows, shown in Figure 4. collection: using the input component in preservation system, the data will collect based on the originality of the data. The information is about the data version, application of data, operating system, and any other information related with the data preservation. processing: the data processing consist of two steps, there are data analysis and data visualization. In data analysis, we analysed the data based on the context, provenance, appearances, structure, and behaviour of the data. It is very important to understand the preservation needs of each of the data. Based on these analyses, we can result the preservation action to the visualization actions for each data. The information is also important for end user; they will know well the technological environment need to run the data. archiving: the data will store and keep in the data management system that also being digital repositories, digital library and preservation. The data is managed in term of long term use and access in the future for other researcher. The illustration of the data handling implementation is shown in Figure 4. IV. USE CASE The research has developed by following the research flow and data production in each step of the research. In the first steps, we developed the research flow based on common research activities. The research is consisting of problem identification problems, problem formulation and statements, data gathering, data analysis, and reports and publications. These steps based on data perspectives in scientific works. Figure 4. The Implementation Process [9] Every steps of research work is producing the data. The problems identification steps are producing the literatures data such as downloaded paper, journal, and technical reports. The data can be form as PDF and word processing files. The problem statements steps are producing the research questions file, scopes, assumptions and research proposals. The next steps are data gathering. We usually use survey, collecting data from tools, doing in lab-experiments, and make simulation in software. The analysis processes are producing a lot of data, i.e. images, logs, databases, configurations, and formula. The reports and publication are producing the research report such as technical reports, lab reports, and publication in conferences and journals, and presentations file. The picture of research work can be seen in Figure 5. The second step of implementation, we doing implementation scenario for every data we use in the experiments. The implementation scenario consist of tree major steps, there are data collection, data processes and data archives. collection is entering the each data that produced in every steps of research into DALA systems. In data collection every data will generate based on the document types and formats types. The model of data classification is shown in Figure 6.

4 Reports & Publications Technical report, journal, paper, t ti fil Logs, database, configurations, formula, simulations Analysis Problem Identification Digital Literature files: paper, journals, and technical reports Survey materials, raw data, experiments results, software Research questions, scope, assumptions, related research Gathering Figure 5. The Digital in Scientific Flows Problem Statements The third step on the implementation is data processing. The DALA systems have method to processing the data into 2 processes, there are data analysis and data visualization. analysis is processing the data by analysis the metadata and preservation information for each of data. The metadata is capturing the complete information about the digital data,; there are the author, the creation date, the change date, the description of data, file version, the application, and other information that related for the data. The preservation information is collecting the information about the appearances, content, context, structures, behavior, and context. The result of the analyses process is needed to determine the visualization process. The data will have the information that shown the metadata and preservation information as description in data management and archives. The data can be arrange, classify and manage based on the metadata and preservation information, not only the categories. The flow of the data into the data analysis process is shown in Figure 7. Digital data Analysis E-Journal Metadata Preservation Info Theses & Dissertation classifications Management Preservation Document Type Scientific Scientific Format Type Conference, Meeting, Lecturer E-Records Text Images Numeric Video, Audio Output Model Figure 6. The Classification of Digital Figure 7. The Analysis Processes The process on data management and archives are using the GForge framework for managing every data. The data archives are consist of digital repositories, digital library and digital preservation. The data that has stored in the system will get the facilities as repositories; it is meaning that the data will be continuous data. Every update of the data will store and manage based on the latest release to the earlier release. It is very helpful for the researcher to understand the context and the behaviour of the data. The change of every data is showing up to giving user understand ability of use of the data. The digital library is visualization the digital data as like library. Based on the metadata and preservation information every data can search and access easily. The relation between data more easy to detected and made researcher flexible and fast search for every data they wanted. Preservation is keeping the data into original formats, and keeping the information that embedded in the data. The data archive is very important feature for ensure the data that has entering into the DALA system can access and share now and in the future. The Figure 8 is showing the flow and the relation between visualization processes and data archive.

5 Repositories the original data list of data release change of data comments and involvements contributions logs Digital Library Search by data types Search by format types Related projects User based data Figure 8. The Visualization and Archive Features V. RESULT Preservation appearances content context behaviour structure The DALA project has shown the design, components, and implementation of digital data management system with focus on the long term accessibility and usability. DALA used the grid environment to manage and anticipate the large amount of digital data to be handled. The combination between new preservation application, Gforge framework application, and MySQL Cluster is the good practices in maximize the open source resources that available in the scientific world.. The research is implemented the preservation system based on the OAIS standards and implement the new preservation architectures. We also have introduced the methods in preservation system for creating multi functional archive system. We have implemented several functions such as using the flow of data handling process. The process consist of data collection, data processing, data analysis based on several criteria, data visualization, and data archiving with combination of digital repositories, digital library and digital preservation. The other results are on the data processing. DALA system has proceed the metada and preservation system for the data management and data preservation, that needed for the data visualizations and data archives. Metada data information has captured several information such as author information, the creation date, the change date, the description of data, file version, the application, and other information that related for the data. The preservation information is collecting the information about the appearances, content, context, structures, behaviour, and context. DALA system has presenting the core feature such as the original data, list of data release, change of data, comments and involvements, contributions, and logs. The other feature for supporting digital library are search by data types, search by format types, related projects, and user based data. The preservation are supported several features, there are appearances, data content, data context, data behaviour, and data structure. In the future, the challenges come from the different type of data to be handled. The exploding volume of the data also will be the issues in the infrastructure and storage facilities. The advanced issues will very challenging to address collaboratively. The issues such as the integration of the systems, multi sites access, secure access, and technological preservation is currently still under discussion. REFERENCES [1] IDC Report: The Exploding Digital Universe. John Gantz, Chief Research Officer, IDC. Available at [2] L Dirks. eresearch, Semantic Computing, and the Cloud. Avaliable at [3] CERN, The European Organization for Nuclear Research: [4] SDSC. Available at [5] SourceFoerge. Available at [6] Gforge. Available at [7] MySQL Cluster. Available at [8] OAIS. The Open Archival Information System. Available at [9] M Riasetiawan, AK Mahmood. Science-Forge: a Collaborative Scientific Framework Design.IEEE ISIEA 2010.Penang, Malaysia. Unpublished and will be presented at 3-6 th October 2010.