Applications of Grid Computing in Genetics and Proteomics
|
|
- Priscilla Horton
- 6 years ago
- Views:
Transcription
1 Applications of Grid Computing in Genetics and Proteomics Jorge Andrade 1, Malin Andersen 1,2, Lisa Berglund 1, and Jacob Odeberg 1,2 1 Department of Biotechnology, Royal Institute of Technology (KTH), AlbaNova University Center, SE Stockholm, Sweden {jorge, jacob, malina}@biotech.kth.se, lisaber@kth.se 2 Department of Medicine, Atherosclerosis Research Unit, King Gustaf V Research Institute, Karolinska Institutet, Karolinska University Hospital, Stockholm, Sweden Abstract. The potential for Grid technologies in applied bioinformatics is largely unexplored. We have developed a model for solving computationally demanding bioinformatics tasks in distributed Grid environments, designed to ease the usability for scientists unfamiliar with Grid computing. With a script-based implementation that uses a strategy of temporary installations of databases and existing executables on remote nodes at submission, we propose a generic solution that do not rely on predefined Grid runtime environments and that can easily be adapted to other bioinformatics tasks suitable for parallelization. This implementation has been successfully applied to whole proteome sequence similarity analyses and to genome-wide genotype simulations, where computation time was reduced from years to weeks. We conclude that computational Grid technology is a useful resource for solving high compute tasks in genetics and proteomics using existing algorithms. 1 Introduction Bioinformatics is a relatively new field of biological research involving the integration of computers, software tools, and databases in an effort to address biological questions. Areas include human genome research, simulations of biological and biochemical processes, and proteomics (for example protein folding simulations). With an increasing amount and complexity of data in genomics and genetics generated by today s high-throughput screening technologies and the development of advanced algorithms for mining complex data, computational power now sometimes defines the practical limit. High performance computing or alternative solutions are required to undertake the intensive data processing and analysis. Grid computing [1], offers a model for solving massive computational problems by subdividing the computation in a set of small jobs, executed in parallel on geographically distributed resources. However, the current job management process on Grid environments is relatively complex and non-automated. Biologists who want to take advantage of B. Kågström et al. (Eds.): PARA 2006, LNCS 4699, pp , c Springer-Verlag Berlin Heidelberg 2007
2 792 J. Andrade et al. Grid resources face a process of having to manually submit their jobs, periodically check the resource broker for the status of the jobs ( Submitted, Ready, Scheduled, Running, or Finished status), and finally get the results with a raw file transfer from the remote storage area or remote worker to the local file system of their user interface. Different solutions for increasing the usability, scalability and stability in computational Grids have recently been proposed [2], [3]. The presented implementation represents a model by which access and utilization of Grid resources is greatly facilitated, allowing biologist and other non- Grid-experts to exploit the Grid power without necessarily having knowledge of Grid related details and procedures. The utility of this implementation is demonstrated by application to two computationally expensive bioinformatics tasks: Whole proteome sequence similarity analysis and genotype simulations for genome wide linkage analysis 2 Methods In order to make the interaction with the complex computational environments on Grids more straightforward to the biologically oriented scientists, the following tasks were automated: Proxy setup handles the user authentication as a member of a Virtual Organization (VO) and grants the user access to the Grid resources. By default, twelve hours is the time for the proxy to be in effect. After the proxy expires, the task of re-creating new proxy is automatically scheduled in the local Grid client. Job submission involves the remote distribution of the split input data files or databases, as well as the executable binary files to the Grid workers. For each Grid job submitted, a Grid job specification is created using the Resource Specification Language (RSL). Processing. After job submission, a local temporary installation of datasets and executables in the allocated remote nodes is performed. After that, parallel execution is started in remote nodes, and a constant monitoring of the current job s status is performed. Job re-submission in case of job failure or excessive delay in Grid queue systems is also handled. Job collection. When specific Grid jobs are finished, partial results are downloaded from the remote Grid workers to the local computer. This module is also able to handle parallel retrieval of several finished jobs. The figure 1 shows a graphical description of the Grid framework configuration used for this implementation. 3 Implementation A Perl script based Grid broker that ensure unique user authentication was implemented, allowing the user to remotely deploy and execute pre-existing algorithms or software across available Grid resources at submission time. The presented solution is adjusted to NorduGrid ARC [4], but can be easily adapted to any Globus based Grid middleware.
3 Applications of Grid Computing in Genetics and Proteomics 793 Fig. 1. Grid computing Framework for application in Bioinformatics This implementation can be adapted to tasks suitable for parallelization where an existing Linux executable exists. The implementation consists of two Perl scripts: gridjobsetup.pl. Manages two main tasks. Firstly, the big computationally expensive task is partitioned into a user-selected number of smaller equally sized atomistic jobs, each corresponding to a fraction of the total data. Secondly, for each datra fraction, a Grid job specification is created using the resource specification language (RSL). gridbroker.pl. This is the Grid broker. Its function is to manage the submission, monitoring and collection of the Grid jobs. Following node allocation and job submission, gridbroker.pl performs temporary installations of the deployed executable on the Grid nodes/remote workers, and parallel execution of the Grid jobs is started. gridbroker.pl constantly monitors the parallel execution of the distributed tasks, and in the case of job failure or if a job or set of jobs are excessively delayed in the work-queue scheduler, gridbroker.pl manages the resubmission of this job or set of jobs to different available Grid workers. When jobs reach the status of finished, forked download of specific job-results to the user local file system is performed. The partial Grid job results are finally concatenated to generate the output file. A fraction of the Perl implementation of the broker is shown below. The code shows a loop that manages the submission of a user defined number of Grid jobs; a vector of Grid job identifiers is created
4 794 J. Andrade et al. in memory and in an archive. This vector will then be used to mange the monitoring and downloading of the jobs. A log file that registers submission start and finish times is also created. Fraction of the Algorithm that Manage the Submission of Grid Jobs Input: XRSL-specification(s) of a number of Grid jobs; for each Grid job,a set of specific input parameters. Action: Submit the given number of Grid jobs. Output: Vector of Job s id and file with timings. 1. Process XRSL-specification 2. Create a time-log-file and register the start of submission 3. Create and open a job-id-file 4. For each job (a) Select the cluster(s) to which the job will be Submitted (b)submit the job (c)collect the retrieved job-id (d)push the collected job-id in a vector (e)push the collected job-id in a job-id-file 5. Register in time-log-file the end of submission 6. Close time-log-file 7. Close job-id-file Fraction of Algorithm that Manage the Monitoring and Downloading of Finished Grid Jobs (The following algorithm shows the constantly monitoring of job s status using the previously created vector of jobs identifiers; in case of job failure, re-submission of jobs is performed, jobs that have successfully reached the status of finished are downloaded.) Input: job-id vector and job-id-file. Action: Monitoring and collection of Grid jobs and resubmission if "job-failure". Output: Collection of finished Grid Jobs and time-log-file. 1. While number of downloaded jobs <= number of total Grid jobs submitted 2. For each job: (a)monitoring status of vector job-id[i] (b)if status of job-id[i] is "FAILURE" then: i. Re-submit job- id[i] to available Grid cluster ii. Delete old and push new retrieved job-id iii. Delete old and push new job-id in job-id File
5 Applications of Grid Computing in Genetics and Proteomics 795 iv. Register re-submission time in the log-file (c)if satus of job-id[i] is "FINISHED" then: i. Collect job-id[i] and register time ii. Push job-id[i] from vector of Job s id iii. Push job-id[i] from file of Job s id iv. Increase the counter of downloaded jobs 3. Register end of job-collection and close log-file 4 Results XWe have aimed to develop a generic Grid implementation for solving bioinformatics tasks suitable for parallelization where neither pre-selection of available Grid nodes nor pre-installation of software or databases will be necessary. Existing Linux-based executables can be used when scaling up tasks prohibitively time-consuming to perform in single work stations, as our solution will not require re-codification or programming modifications. The implementation is also applicable in situations where the source code is not available. To streamline the process we chose the strategy of making temporary installations of the executable and databases locally at each remote node at submission, followed by un-installation after download and collection of the results. By avoiding the need of predefined run-time environments, this implementation limits the interaction with Grid administrators for installation of applications/software and updates, thereby accommodating for dynamic Grid environments in which available nodes change between submissions. This strategy is however not applicable for instance in cases when a database management system (DBMS) is required, typical examples of DBMSs like Oracle, Microsoft SQL Server or MySQL, will necessary need the use of a specific run-time environments. Our implementation was evaluated in two highly computer intensive real applications in proteomics and genetics: The first application deals with whole proteome protein similarity analysis using a sliding window algorithm [5]. In contrast to ordinary blastp queries aligning full length query protein sequences, the sliding window approach results in a significantly higher number of blast searchers. Using a sliding window size of 51 amino acids, the number of blastp searches for a 1000 amino acid protein increase from 1 to 950. For the entire human Ensembl database [ of close to 34,000 human proteins, this corresponds to about 15,000,000 blastp searches. The time needed to run this number of blastp searches on a single computer was about eight weeks. As the Ensembl database is constantly evolving and being updated, where protein sequences are added, changed or deleted, frequent reprocessing of the database becomes necessary in the HPA program [ in order to work with the most accurate data at any one time. Once a new version of the database is released, the sequence similarity data on which the epitope design is based needs also to be updated. The computational requirements for this task were exceeding in-house resources if the processed results of a database update were to be
6 796 J. Andrade et al. delivered before it was already obsolete. With a Grid implementation where local installations of both the blastp executable and the entire Ensembl database was performed on each node (a total package of a size of 16 MB)[5], runtime was reduced from about eight weeks on one single up-to-date computer, to less than 24 hours using 300 Grid nodes in Swegrid [ The absolute speed-up for this application was calculated as: Sp = T 1 s (1) T p Where T1 s is the sequential run-time, and T p is the execution time in p Grid nodes. Using the complete human Ensembl database as input, speed-up of 56 fold was archived, this was calculated by dividing T1 s = 1344 hours by T p =24 hours (the Grid run-time with same data as input in 300 Grid processors in Swegrid). The expected linear speed-up (300 fold in 300 nodes) was not archived, mainly due to Grid latency. By making a local installation of a database at each submission, the speed of running queries against a local database was obtained together with running against the most recent update. The alternative strategy of storing the database in one single Grid storage resource accessed by all the other nodes, proved to create an I/O overload in the Grid storage server, resulting in a significant increase of the total runtime. The second application was facilitating computer simulations of genotypes using a HMM based software [6], in order to evaluate the significance of genomewide linkage data. This was applied in a study aimed to identify novel genes involved in the pathogenesis of Alzheimers disease (AD) by performing a nonparametric multipoint linkage analysis on AD families from the relatively genetically homogeneous Swedish population. On a genome-wide scale, this task is extremely computationally intensive. In the absence of sufficient computational resources the number of simulations would therefore have to be limited, which could lead to the estimation of insufficient global significance levels and false positive linkage claims. We developed Grid-Allegro [7] which was used in the hypothesis testing to evaluate the statistical significance of the linkage data under the null hypothesis of no linkage using a set of 109 AD families. Serial execution time required to perform the minimum required genotype simulation analyses was reduced from the projected time, more that 3 years on a single up-to-date CPU, to less than 3 days when distributed computing was performed in 600 Grid workers in Swegrid [7]. 5 Discussion There are several computationally demanding algorithms and tasks in bioinformatics that may cause a computational overload when scaled up. To the researcher without access to expensive resources in-house such as dedicated clusters or computer farms, Grids represents a cost-effective and powerful resource. However, a current obstacle especially to the biologically oriented researcher is managing the middleware that is still raw and hardly accessible. For the
7 Applications of Grid Computing in Genetics and Proteomics 797 non-computer scientist, more user-friendly alternative solutions are necessary. One alternative is to develop web-based user front-end services of underlying Grid implementations, which are accessed by third party users. This is the most accessible alternative of exploiting Grid resources, as it is associated with minimal complexity where no necessary previous knowledge of distributed computing is required by the user. Grid resource brokers and job submission services based on Grid and Web services have been previously proposed [8]. However, for our specific purposes, we decided to use a generic, script-based strategy for implementing Grid-aware applications of bioinformatics task that are suitable for parallelisation. Our major concerns were related with security, stability and usability. Although Grid security is based in public key infrastructure (PKI) and this architecture offers strong security levels for the Grid end-user, current PKI implementations suffer from serious usability issues, especially when applied to web-based Grid-services. [9] Strong efforts are required in searching for new mechanisms for increasing the usability of Grid security. [10] Web-based implementations also confine the input submission format to those defined or envisioned by the provider/developer, which may reduce the flexibility for the third party user. Furthermore, Web-based Grid implementations may require re-codification of previously existing single CPU-oriented algorithm implementations. The developer assumes the administrator responsibility for maintaining the availability and updating of the resource. When web-based services are developed and provided through large initiatives [11], this indeed represents a transparent and user-friendly solution. However, new applications depend on continued development and implementation by these providers, and are hence not always available to meet the specific needs in individual third party projects. The alternative generic strategy, although requiring basic computer knowledge by the user, greatly increases the flexibility by enabling the implementation to be applied to similar distributable computation-demanding tasks. In conclusion, our implementation facilitates the biologically oriented scientist s remote deployment and execution of pre-existing codifications of bioinformatics algorithms across multiple Grid resources. By applying this implementation in solving two data and CPU intensive tasks, we have demonstrated the potential utility of Grid technology for addressing highly computational demanding bioinformatics task. References 1. Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the grid: Enabling scalable virtual organizations. International Journal of High Performance Computing Applications 15(3), (2001) 2. Ellert, M., Konstantinov, B., K onya, J., Lindemann, J., Livenson, I., Nielsen, J., Smirnova, O., Wäanänen, A.: Advanced Resource Connector middleware for lightweight computational Grids. Future Generation Computer Systems. The International Journal of Grid Computing: Theory, Methods and Applications 23, (2007)
8 798 J. Andrade et al. 3. Elmroth, E., Tordsson, J.: Grid Resource Brokering Algorithms Enabling Advance Reservations and Resource Selection Based on Performance Predictions. Future Generation Computer Systems. The International Journal of Grid Computing: Theory, Methods and Applications (2007) 4. Ellert, M., et al.: The NorduGrid project: using Globus toolkit for building GRID infrastructure. Nuclear Instruments & Methods in Physics Research Section a- Accelerators Spectrometers Detectors and Associated Equipment 502(2-3), (2003) 5. Andrade, J., et al.: Using Grid technology for computationally intensive applied bioinformatics analyses. Silico Biology, 6 (2006) 6. Gudbjartsson, D.F., et al.: Allegro, a new computer program for multipoint linkage analysis. Nat Genet 25(1), (2000) 7. Andrade, J., et al.: The use of Grid computing to drive data-intensive genetic research. European Journal of Human Genetics (March 21, 2007) 8. Elmroth, E., Tordsson, J.: An interoperable, standards-based Grid resource broker and job submission service. In: First International Conference on e-science and Grid Computing, IEEE Computer Society Press, Los Alamitos (2005) 9. Gui, X.L., et al.: A grid security infrastructure based on behaviors and trusts. In: Grid and Cooperative Computing Gcc 2004 Workshops, Proceedings, vol. 3252, pp (2004) 10. Beckles, B., Welch, V., Basney, J.: Mechanisms for increasing the usability of grid security. International Journal of Human-Computer Studies 63(1-2), (2005) 11. Blanchet, C., et al.: GPS@ Bioinformatics Portal: from Network to EGEE Grd, vol. 2006, pp IOS Press, Amsterdam (2006)
Data Management for the World s Largest Machine
Data Management for the World s Largest Machine Sigve Haug 1, Farid Ould-Saada 2, Katarina Pajchel 2, and Alexander L. Read 2 1 Laboratory for High Energy Physics, University of Bern, Sidlerstrasse 5,
More informationUsage statistics and usage patterns on the NorduGrid: Analyzing the logging information collected on one of the largest production Grids of the world
Usage statistics and usage patterns on the NorduGrid: Analyzing the logging information collected on one of the largest production Grids of the world Pajchel, K.; Eerola, Paula; Konya, Balazs; Smirnova,
More informationATLAS NorduGrid related activities
Outline: NorduGrid Introduction ATLAS software preparation and distribution Interface between NorduGrid and Condor NGlogger graphical interface On behalf of: Ugur Erkarslan, Samir Ferrag, Morten Hanshaugen
More informationEmpowering a Flexible Application Portal with a SOA-based Grid Job Management Framework
Empowering a Flexible Application Portal with a SOA-based Grid Job Management Framework Erik Elmroth 1, Sverker Holmgren 2, Jonas Lindemann 3, Salman Toor 2, and Per-Olov Östberg1 1 Dept. Computing Science
More informationUNICORE Globus: Interoperability of Grid Infrastructures
UNICORE : Interoperability of Grid Infrastructures Michael Rambadt Philipp Wieder Central Institute for Applied Mathematics (ZAM) Research Centre Juelich D 52425 Juelich, Germany Phone: +49 2461 612057
More informationARC-XWCH bridge: Running ARC jobs on the XtremWeb-CH volunteer
ARC-XWCH bridge: Running ARC jobs on the XtremWeb-CH volunteer computing platform Internal report Marko Niinimaki, Mohamed BenBelgacem, Nabil Abdennadher HEPIA, January 2010 1. Background and motivation
More informationDesign of Distributed Data Mining Applications on the KNOWLEDGE GRID
Design of Distributed Data Mining Applications on the KNOWLEDGE GRID Mario Cannataro ICAR-CNR cannataro@acm.org Domenico Talia DEIS University of Calabria talia@deis.unical.it Paolo Trunfio DEIS University
More informationMonitoring the Usage of the ZEUS Analysis Grid
Monitoring the Usage of the ZEUS Analysis Grid Stefanos Leontsinis September 9, 2006 Summer Student Programme 2006 DESY Hamburg Supervisor Dr. Hartmut Stadie National Technical
More informationDelivering Data Management for Engineers on the Grid 1
Delivering Data Management for Engineers on the Grid 1 Jasmin Wason, Marc Molinari, Zhuoan Jiao, and Simon J. Cox School of Engineering Sciences, University of Southampton, UK {j.l.wason, m.molinari, z.jiao,
More informationArchitecture Proposal
Nordic Testbed for Wide Area Computing and Data Handling NORDUGRID-TECH-1 19/02/2002 Architecture Proposal M.Ellert, A.Konstantinov, B.Kónya, O.Smirnova, A.Wäänänen Introduction The document describes
More informationDevelopment of new security infrastructure design principles for distributed computing systems based on open protocols
Development of new security infrastructure design principles for distributed computing systems based on open protocols Yu. Yu. Dubenskaya a, A. P. Kryukov, A. P. Demichev Skobeltsyn Institute of Nuclear
More informationIntroduction to Grid Infrastructures
Introduction to Grid Infrastructures Stefano Cozzini 1 and Alessandro Costantini 2 1 CNR-INFM DEMOCRITOS National Simulation Center, Trieste, Italy 2 Department of Chemistry, Università di Perugia, Perugia,
More informationThe NorduGrid Architecture and Middleware for Scientific Applications
The NorduGrid Architecture and Middleware for Scientific Applications O. Smirnova 1, P. Eerola 1,T.Ekelöf 2, M. Ellert 2, J.R. Hansen 3, A. Konstantinov 4,B.Kónya 1, J.L. Nielsen 3, F. Ould-Saada 5, and
More informationInteroperating AliEn and ARC for a distributed Tier1 in the Nordic countries.
for a distributed Tier1 in the Nordic countries. Philippe Gros Lund University, Div. of Experimental High Energy Physics, Box 118, 22100 Lund, Sweden philippe.gros@hep.lu.se Anders Rhod Gregersen NDGF
More informationMONTE CARLO SIMULATION FOR RADIOTHERAPY IN A DISTRIBUTED COMPUTING ENVIRONMENT
The Monte Carlo Method: Versatility Unbounded in a Dynamic Computing World Chattanooga, Tennessee, April 17-21, 2005, on CD-ROM, American Nuclear Society, LaGrange Park, IL (2005) MONTE CARLO SIMULATION
More informationTowards ParadisEO-MO-GPU: a Framework for GPU-based Local Search Metaheuristics
Towards ParadisEO-MO-GPU: a Framework for GPU-based Local Search Metaheuristics N. Melab, T-V. Luong, K. Boufaras and E-G. Talbi Dolphin Project INRIA Lille Nord Europe - LIFL/CNRS UMR 8022 - Université
More informationIntroduction to Grid Computing
Milestone 2 Include the names of the papers You only have a page be selective about what you include Be specific; summarize the authors contributions, not just what the paper is about. You might be able
More informationAGARM: An Adaptive Grid Application and Resource Monitor Framework
AGARM: An Adaptive Grid Application and Resource Monitor Framework Wenju Zhang, Shudong Chen, Liang Zhang, Shui Yu, and Fanyuan Ma Shanghai Jiaotong University, Shanghai, P.R.China, 200030 {zwj03, chenshudong,
More informationData Mining Technologies for Bioinformatics Sequences
Data Mining Technologies for Bioinformatics Sequences Deepak Garg Computer Science and Engineering Department Thapar Institute of Engineering & Tecnology, Patiala Abstract Main tool used for sequence alignment
More informationHigh Performance Computing Course Notes Grid Computing I
High Performance Computing Course Notes 2008-2009 2009 Grid Computing I Resource Demands Even as computer power, data storage, and communication continue to improve exponentially, resource capacities are
More informationADAPTIVE AND DYNAMIC LOAD BALANCING METHODOLOGIES FOR DISTRIBUTED ENVIRONMENT
ADAPTIVE AND DYNAMIC LOAD BALANCING METHODOLOGIES FOR DISTRIBUTED ENVIRONMENT PhD Summary DOCTORATE OF PHILOSOPHY IN COMPUTER SCIENCE & ENGINEERING By Sandip Kumar Goyal (09-PhD-052) Under the Supervision
More informationA Distributed Media Service System Based on Globus Data-Management Technologies1
A Distributed Media Service System Based on Globus Data-Management Technologies1 Xiang Yu, Shoubao Yang, and Yu Hong Dept. of Computer Science, University of Science and Technology of China, Hefei 230026,
More informationInteroperable and Transparent Dynamic Deployment of Web Services for Service Oriented Grids
Interoperable and Transparent Dynamic Deployment of Web s for Oriented Grids Michael Messig and Andrzej Goscinski School of Engineering and Information Technology Deakin University Pigdons Road, Geelong
More informationPerformance Analysis of Parallelized Bioinformatics Applications
Asian Journal of Computer Science and Technology ISSN: 2249-0701 Vol.7 No.2, 2018, pp. 70-74 The Research Publication, www.trp.org.in Dhruv Chander Pant 1 and OP Gupta 2 1 Research Scholar, I. K. Gujral
More informationA distributed tier-1. International Conference on Computing in High Energy and Nuclear Physics (CHEP 07) IOP Publishing. c 2008 IOP Publishing Ltd 1
A distributed tier-1 L Fischer 1, M Grønager 1, J Kleist 2 and O Smirnova 3 1 NDGF - Nordic DataGrid Facilty, Kastruplundgade 22(1), DK-2770 Kastrup 2 NDGF and Aalborg University, Department of Computer
More informationA Compact Computing Environment For A Windows PC Cluster Towards Seamless Molecular Dynamics Simulations
A Compact Computing Environment For A Windows PC Cluster Towards Seamless Molecular Dynamics Simulations Yuichi Tsujita Abstract A Windows PC cluster is focused for its high availabilities and fruitful
More informationProblems for Resource Brokering in Large and Dynamic Grid Environments
Problems for Resource Brokering in Large and Dynamic Grid Environments Cătălin L. Dumitrescu Computer Science Department The University of Chicago cldumitr@cs.uchicago.edu (currently at TU Delft) Kindly
More informationglite Grid Services Overview
The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) glite Grid Services Overview Antonio Calanducci INFN Catania Joint GISELA/EPIKH School for Grid Site Administrators Valparaiso,
More informationInterconnect EGEE and CNGRID e-infrastructures
Interconnect EGEE and CNGRID e-infrastructures Giuseppe Andronico Interoperability and Interoperation between Europe, India and Asia Workshop Barcelona - Spain, June 2 2007 FP6 2004 Infrastructures 6-SSA-026634
More informationGrid Resource Brokering Algorithms Enabling Advance Reservations and Resource Selection Based on Performance Predictions
Accepted for publication in "Future Generation Computer Systems. The International Journal of Grid Computing: Theory, Methods and Applications". Elsevier. Grid Resource Brokering Algorithms Enabling Advance
More informationPoS(EGICF12-EMITC2)081
University of Oslo, P.b.1048 Blindern, N-0316 Oslo, Norway E-mail: aleksandr.konstantinov@fys.uio.no Martin Skou Andersen Niels Bohr Institute, Blegdamsvej 17, 2100 København Ø, Denmark E-mail: skou@nbi.ku.dk
More informationGrid Scheduling Architectures with Globus
Grid Scheduling Architectures with Workshop on Scheduling WS 07 Cetraro, Italy July 28, 2007 Ignacio Martin Llorente Distributed Systems Architecture Group Universidad Complutense de Madrid 1/38 Contents
More informationBuilding Data-Intensive Grid Applications with Globus Toolkit An Evaluation Based on Web Crawling
Building Data-Intensive Grid Applications with Globus Toolkit An Evaluation Based on Web Crawling Andreas Walter 1, Klemens Böhm 2, and Stephan Schosser 2 1 IPE, FZI Forschungszentrum Informatik, Haid-und-Neu-Straße
More informationDynamic Data Grid Replication Strategy Based on Internet Hierarchy
Dynamic Data Grid Replication Strategy Based on Internet Hierarchy Sang-Min Park 1, Jai-Hoon Kim 1, Young-Bae Ko 2, and Won-Sik Yoon 2 1 Graduate School of Information and Communication Ajou University,
More informationKenneth A. Hawick P. D. Coddington H. A. James
Student: Vidar Tulinius Email: vidarot@brandeis.edu Distributed frameworks and parallel algorithms for processing large-scale geographic data Kenneth A. Hawick P. D. Coddington H. A. James Overview Increasing
More informationGRIDS INTRODUCTION TO GRID INFRASTRUCTURES. Fabrizio Gagliardi
GRIDS INTRODUCTION TO GRID INFRASTRUCTURES Fabrizio Gagliardi Dr. Fabrizio Gagliardi is the leader of the EU DataGrid project and designated director of the proposed EGEE (Enabling Grids for E-science
More informationDatabase Server. 2. Allow client request to the database server (using SQL requests) over the network.
Database Server Introduction: Client/Server Systems is networked computing model Processes distributed between clients and servers. Client Workstation (usually a PC) that requests and uses a service Server
More informationXML in the bipharmaceutical
XML in the bipharmaceutical sector XML holds out the opportunity to integrate data across both the enterprise and the network of biopharmaceutical alliances - with little technological dislocation and
More informationGrids and Security. Ian Neilson Grid Deployment Group CERN. TF-CSIRT London 27 Jan
Grids and Security Ian Neilson Grid Deployment Group CERN TF-CSIRT London 27 Jan 2004-1 TOC Background Grids Grid Projects Some Technical Aspects The three or four A s Some Operational Aspects Security
More informationIEPSAS-Kosice: experiences in running LCG site
IEPSAS-Kosice: experiences in running LCG site Marian Babik 1, Dusan Bruncko 2, Tomas Daranyi 1, Ladislav Hluchy 1 and Pavol Strizenec 2 1 Department of Parallel and Distributed Computing, Institute of
More informationAn I/O device driver for bioinformatics tools: the case for BLAST
An I/O device driver for bioinformatics tools 563 An I/O device driver for bioinformatics tools: the case for BLAST Renato Campos Mauro and Sérgio Lifschitz Departamento de Informática PUC-RIO, Pontifícia
More informationThe Lattice BOINC Project Public Computing for the Tree of Life
The Lattice BOINC Project Public Computing for the Tree of Life Presented by Adam Bazinet Center for Bioinformatics and Computational Biology Institute for Advanced Computer Studies University of Maryland
More informationA Finite State Mobile Agent Computation Model
A Finite State Mobile Agent Computation Model Yong Liu, Congfu Xu, Zhaohui Wu, Weidong Chen, and Yunhe Pan College of Computer Science, Zhejiang University Hangzhou 310027, PR China Abstract In this paper,
More informationFunctional Requirements for Grid Oriented Optical Networks
Functional Requirements for Grid Oriented Optical s Luca Valcarenghi Internal Workshop 4 on Photonic s and Technologies Scuola Superiore Sant Anna Pisa June 3-4, 2003 1 Motivations Grid networking connection
More informationChapter 4:- Introduction to Grid and its Evolution. Prepared By:- NITIN PANDYA Assistant Professor SVBIT.
Chapter 4:- Introduction to Grid and its Evolution Prepared By:- Assistant Professor SVBIT. Overview Background: What is the Grid? Related technologies Grid applications Communities Grid Tools Case Studies
More informationParalyzing Bioinformatics Applications Using Conducive Hadoop Cluster
IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661, p- ISSN: 2278-8727Volume 14, Issue 6 (Sep. - Oct. 2013), PP 89-93 Paralyzing Bioinformatics Applications Using Conducive Hadoop Cluster
More informationMAPREDUCE FOR BIG DATA PROCESSING BASED ON NETWORK TRAFFIC PERFORMANCE Rajeshwari Adrakatti
International Journal of Computer Engineering and Applications, ICCSTAR-2016, Special Issue, May.16 MAPREDUCE FOR BIG DATA PROCESSING BASED ON NETWORK TRAFFIC PERFORMANCE Rajeshwari Adrakatti 1 Department
More informationJULIA ENABLED COMPUTATION OF MOLECULAR LIBRARY COMPLEXITY IN DNA SEQUENCING
JULIA ENABLED COMPUTATION OF MOLECULAR LIBRARY COMPLEXITY IN DNA SEQUENCING Larson Hogstrom, Mukarram Tahir, Andres Hasfura Massachusetts Institute of Technology, Cambridge, Massachusetts, USA 18.337/6.338
More informationMonitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY
Journal of Physics: Conference Series OPEN ACCESS Monitoring System for the GRID Monte Carlo Mass Production in the H1 Experiment at DESY To cite this article: Elena Bystritskaya et al 2014 J. Phys.: Conf.
More informationDay 1 : August (Thursday) An overview of Globus Toolkit 2.4
An Overview of Grid Computing Workshop Day 1 : August 05 2004 (Thursday) An overview of Globus Toolkit 2.4 By CDAC Experts Contact :vcvrao@cdacindia.com; betatest@cdacindia.com URL : http://www.cs.umn.edu/~vcvrao
More informationA Resource Discovery Algorithm in Mobile Grid Computing Based on IP-Paging Scheme
A Resource Discovery Algorithm in Mobile Grid Computing Based on IP-Paging Scheme Yue Zhang 1 and Yunxia Pei 2 1 Department of Math and Computer Science Center of Network, Henan Police College, Zhengzhou,
More informationAssignment 5. Georgia Koloniari
Assignment 5 Georgia Koloniari 2. "Peer-to-Peer Computing" 1. What is the definition of a p2p system given by the authors in sec 1? Compare it with at least one of the definitions surveyed in the last
More informationScientific data management
Scientific data management Storage and data management components Application database Certificate Certificate Authorised users directory Certificate Certificate Researcher Certificate Policies Information
More informationSolexaLIMS: A Laboratory Information Management System for the Solexa Sequencing Platform
SolexaLIMS: A Laboratory Information Management System for the Solexa Sequencing Platform Brian D. O Connor, 1, Jordan Mendler, 1, Ben Berman, 2, Stanley F. Nelson 1 1 Department of Human Genetics, David
More informationWSRF Services for Composing Distributed Data Mining Applications on Grids: Functionality and Performance
WSRF Services for Composing Distributed Data Mining Applications on Grids: Functionality and Performance Domenico Talia, Paolo Trunfio, and Oreste Verta DEIS, University of Calabria Via P. Bucci 41c, 87036
More informationOracle Big Data Connectors
Oracle Big Data Connectors Oracle Big Data Connectors is a software suite that integrates processing in Apache Hadoop distributions with operations in Oracle Database. It enables the use of Hadoop to process
More informationA VO-friendly, Community-based Authorization Framework
A VO-friendly, Community-based Authorization Framework Part 1: Use Cases, Requirements, and Approach Ray Plante and Bruce Loftis NCSA Version 0.1 (February 11, 2005) Abstract The era of massive surveys
More informationTextual Description of webbioc
Textual Description of webbioc Colin A. Smith October 13, 2014 Introduction webbioc is a web interface for some of the Bioconductor microarray analysis packages. It is designed to be installed at local
More informationJuliusz Pukacki OGF25 - Grid technologies in e-health Catania, 2-6 March 2009
Grid Technologies for Cancer Research in the ACGT Project Juliusz Pukacki (pukacki@man.poznan.pl) OGF25 - Grid technologies in e-health Catania, 2-6 March 2009 Outline ACGT project ACGT architecture Layers
More informationGrid Approach to Embarrassingly Parallel CPU-Intensive Bioinformatics Problems
Grid Approach to Embarrassingly Parallel CPU-Intensive Bioinformatics Problems Heinz Stockinger, Marco Pagni, Lorenzo Cerutti, Laurent Falquet Swiss Institute of Bioinformatics, Vital-IT Quartier Sorge
More informationA Federated Grid Environment with Replication Services
A Federated Grid Environment with Replication Services Vivek Khurana, Max Berger & Michael Sobolewski SORCER Research Group, Texas Tech University Grids can be classified as computational grids, access
More informationEvolving SQL Queries for Data Mining
Evolving SQL Queries for Data Mining Majid Salim and Xin Yao School of Computer Science, The University of Birmingham Edgbaston, Birmingham B15 2TT, UK {msc30mms,x.yao}@cs.bham.ac.uk Abstract. This paper
More informationApplication of Virtualization Technologies & CernVM. Benedikt Hegner CERN
Application of Virtualization Technologies & CernVM Benedikt Hegner CERN Virtualization Use Cases Worker Node Virtualization Software Testing Training Platform Software Deployment }Covered today Server
More informationBoundary control : Access Controls: An access control mechanism processes users request for resources in three steps: Identification:
Application control : Boundary control : Access Controls: These controls restrict use of computer system resources to authorized users, limit the actions authorized users can taker with these resources,
More informationInitial experiences with GeneRecon on MiG
Initial experiences with GeneRecon on MiG Thomas Mailund and Christian N.S. Pedersen Bioinformatics Research Center (BiRC), Dept. of Computer Science, University of Aarhus, Denmark, Email: {mailund,cstorm}@birc.dk
More informationS i m p l i f y i n g A d m i n i s t r a t i o n a n d M a n a g e m e n t P r o c e s s e s i n t h e P o l i s h N a t i o n a l C l u s t e r
S i m p l i f y i n g A d m i n i s t r a t i o n a n d M a n a g e m e n t P r o c e s s e s i n t h e P o l i s h N a t i o n a l C l u s t e r Miroslaw Kupczyk, Norbert Meyer, Pawel Wolniewicz e-mail:
More informationMin Wang. April, 2003
Development of a co-regulated gene expression analysis tool (CREAT) By Min Wang April, 2003 Project Documentation Description of CREAT CREAT (coordinated regulatory element analysis tool) are developed
More informationNUSGRID a computational grid at NUS
NUSGRID a computational grid at NUS Grace Foo (SVU/Academic Computing, Computer Centre) SVU is leading an initiative to set up a campus wide computational grid prototype at NUS. The initiative arose out
More informationA 3-tier Grid Architecture and Interactive Applications Framework for Community Grids
A 3-tier Grid Architecture and Interactive Applications Framework for Community Grids Oscar Ardaiz, Kana Sanjeevan, and Ramon Sanguesa Polytecnic University of Catalunya Campus Nord, Barcelona 08034 Spain
More informationMultiple Broker Support by Grid Portals* Extended Abstract
1. Introduction Multiple Broker Support by Grid Portals* Extended Abstract Attila Kertesz 1,3, Zoltan Farkas 1,4, Peter Kacsuk 1,4, Tamas Kiss 2,4 1 MTA SZTAKI Computer and Automation Research Institute
More informationRapid Deployment of VS Workflows. Meta Scheduling Service
Rapid Deployment of VS Workflows on PHOSPHORUS using Meta Scheduling Service M. Shahid, Bjoern Hagemeier Fraunhofer Institute SCAI, Research Center Juelich. (TNC 2009) Outline Introduction and Motivation
More informationA Fast and High Throughput SQL Query System for Big Data
A Fast and High Throughput SQL Query System for Big Data Feng Zhu, Jie Liu, and Lijie Xu Technology Center of Software Engineering, Institute of Software, Chinese Academy of Sciences, Beijing, China 100190
More informationGrid Architectural Models
Grid Architectural Models Computational Grids - A computational Grid aggregates the processing power from a distributed collection of systems - This type of Grid is primarily composed of low powered computers
More informationMonitoring ARC services with GangliARC
Journal of Physics: Conference Series Monitoring ARC services with GangliARC To cite this article: D Cameron and D Karpenko 2012 J. Phys.: Conf. Ser. 396 032018 View the article online for updates and
More informationROCI 2: A Programming Platform for Distributed Robots based on Microsoft s.net Framework
ROCI 2: A Programming Platform for Distributed Robots based on Microsoft s.net Framework Vito Sabella, Camillo J. Taylor, Scott Currie GRASP Laboratory University of Pennsylvania Philadelphia PA, 19104
More informationUpgrading Existing Databases Recommendations for Irrigation Districts
COLLEGE OF AGRICULTURE AND LIFE SCIENCES TR-371 2011 Upgrading Existing Databases Recommendations for Irrigation Districts By: David Flahive, System Analyst and Guy Fipps, P.E., Extension Agricultural
More informationResearch on the Interoperability Architecture of the Digital Library Grid
Research on the Interoperability Architecture of the Digital Library Grid HaoPan Department of information management, Beijing Institute of Petrochemical Technology, China, 102600 bjpanhao@163.com Abstract.
More informationJava-Grid Environment for Bioinformatics Applied Tools (JEBAT)
Java-Grid Environment for Bioinformatics Applied Tools (JEBAT) Noor Maizura Mohamad Noor 1, Ahmad Faiz Ghazali 1, Md Yazid Mohamad Saman 1, Zafarina Zainuddin 2, 1 Computer Science Department, Faculty
More informationGRID COMPUTING IN MEDICAL APPLICATIONS
GRID COMPUTING IN MEDICAL APPLICATIONS P. Cerello, INFN, Sezione di Torino, Torino, Italy. Abstract Medical Applications can exploit GRID Services in many ways: some of them are computing intensive and
More informationIntegrating a Common Visualization Service into a Metagrid.
Integrating a Common Visualization Service into a Metagrid. R. Watson 1, S. Maad 1, and B. Coghlan 1 Trinity College Dublin, Dublin, Ireland, watsonr@cs.tcd.ie, WWW home page: http://www.cs.tcd.ie/ watsonr
More informationOptimizing Parallel Access to the BaBar Database System Using CORBA Servers
SLAC-PUB-9176 September 2001 Optimizing Parallel Access to the BaBar Database System Using CORBA Servers Jacek Becla 1, Igor Gaponenko 2 1 Stanford Linear Accelerator Center Stanford University, Stanford,
More informationNorduGrid Tutorial. Client Installation and Job Examples
NorduGrid Tutorial Client Installation and Job Examples Linux Clusters for Super Computing Conference Linköping, Sweden October 18, 2004 Arto Teräs arto.teras@csc.fi Steps to Start Using NorduGrid 1) Install
More informationInteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial ISSN:
Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial ISSN: 1137-3601 revista@aepia.org Asociación Española para la Inteligencia Artificial España Kus, Waclaw; Burczynski, Tadeusz
More informationA Cloud Framework for Big Data Analytics Workflows on Azure
A Cloud Framework for Big Data Analytics Workflows on Azure Fabrizio MAROZZO a, Domenico TALIA a,b and Paolo TRUNFIO a a DIMES, University of Calabria, Rende (CS), Italy b ICAR-CNR, Rende (CS), Italy Abstract.
More informationAutomatic Job Resubmission in the Nordugrid Middleware
Henrik Thostrup Jensen Jesper Ryge Leth Automatic Job Resubmission in the Nordugrid Middleware Dat5 Project September 2003 - January 2004 Department of Computer Science Aalborg University Fredrik Bajersvej
More informationResearch and Design Application Platform of Service Grid Based on WSRF
DOI: 10.7763/IPEDR. 2012. V49. 27 Research and Design Application Platform of Service Grid Based on WSRF Jianmei Ge a, Shying Zhang a College of Computer Science and Technology, Beihua University, No.1
More informationWas ist dran an einer spezialisierten Data Warehousing platform?
Was ist dran an einer spezialisierten Data Warehousing platform? Hermann Bär Oracle USA Redwood Shores, CA Schlüsselworte Data warehousing, Exadata, specialized hardware proprietary hardware Introduction
More informationScalable Computing: Practice and Experience Volume 10, Number 4, pp
Scalable Computing: Practice and Experience Volume 10, Number 4, pp. 413 418. http://www.scpe.org ISSN 1895-1767 c 2009 SCPE MULTI-APPLICATION BAG OF JOBS FOR INTERACTIVE AND ON-DEMAND COMPUTING BRANKO
More informationReal-time grid computing for financial applications
CNR-INFM Democritos and EGRID project E-mail: cozzini@democritos.it Riccardo di Meo, Ezio Corso EGRID project ICTP E-mail: {dimeo,ecorso}@egrid.it We describe the porting of a test case financial application
More informationHeterogeneous Grid Computing: Issues and Early Benchmarks
Heterogeneous Grid Computing: Issues and Early Benchmarks Eamonn Kenny 1, Brian Coghlan 1, George Tsouloupas 2, Marios Dikaiakos 2, John Walsh 1, Stephen Childs 1, David O Callaghan 1, and Geoff Quigley
More informationScheduling Large Parametric Modelling Experiments on a Distributed Meta-computer
Scheduling Large Parametric Modelling Experiments on a Distributed Meta-computer David Abramson and Jon Giddy Department of Digital Systems, CRC for Distributed Systems Technology Monash University, Gehrmann
More informationReal-time Calculating Over Self-Health Data Using Storm Jiangyong Cai1, a, Zhengping Jin2, b
4th International Conference on Mechatronics, Materials, Chemistry and Computer Engineering (ICMMCCE 2015) Real-time Calculating Over Self-Health Data Using Storm Jiangyong Cai1, a, Zhengping Jin2, b 1
More informationFuncX: A Function Serving Platform for HPC. Ryan Chard 28 Jan 2019
FuncX: A Function Serving Platform for HPC Ryan Chard 28 Jan 2019 Outline - Motivation FuncX: FaaS for HPC Implementation status Preliminary applications - Machine learning inference Automating analysis
More informationMolecular dynamics simulations in the MolDynGrid Virtual Laboratory by means of ARC between Grid and Cloud
Molecular dynamics simulations in the MolDynGrid Virtual Laboratory by means of ARC between Grid and Cloud Andrii Salnikov * NorduGrid 2016 * manf@grid.org.ua MolDynGrid Virtual Laboratory Has been established
More informationA Resource Discovery Algorithm in Mobile Grid Computing based on IP-paging Scheme
A Resource Discovery Algorithm in Mobile Grid Computing based on IP-paging Scheme Yue Zhang, Yunxia Pei To cite this version: Yue Zhang, Yunxia Pei. A Resource Discovery Algorithm in Mobile Grid Computing
More informationQoS-aware resource allocation and load-balancing in enterprise Grids using online simulation
QoS-aware resource allocation and load-balancing in enterprise Grids using online simulation * Universität Karlsruhe (TH) Technical University of Catalonia (UPC) Barcelona Supercomputing Center (BSC) Samuel
More informationEnterpriseLink Benefits
EnterpriseLink Benefits GGY a Moody s Analytics Company 5001 Yonge Street Suite 1300 Toronto, ON M2N 6P6 Phone: 416-250-6777 Toll free: 1-877-GGY-AXIS Fax: 416-250-6776 Email: axis@ggy.com Web: www.ggy.com
More informationHigh Throughput WAN Data Transfer with Hadoop-based Storage
High Throughput WAN Data Transfer with Hadoop-based Storage A Amin 2, B Bockelman 4, J Letts 1, T Levshina 3, T Martin 1, H Pi 1, I Sfiligoi 1, M Thomas 2, F Wuerthwein 1 1 University of California, San
More informationEvaluating Algorithms for Shared File Pointer Operations in MPI I/O
Evaluating Algorithms for Shared File Pointer Operations in MPI I/O Ketan Kulkarni and Edgar Gabriel Parallel Software Technologies Laboratory, Department of Computer Science, University of Houston {knkulkarni,gabriel}@cs.uh.edu
More informationComprehensive Guide to Evaluating Event Stream Processing Engines
Comprehensive Guide to Evaluating Event Stream Processing Engines i Copyright 2006 Coral8, Inc. All rights reserved worldwide. Worldwide Headquarters: Coral8, Inc. 82 Pioneer Way, Suite 106 Mountain View,
More information