Minutes of WP1 meeting Milan (20 th and 21 st March 2001).

Transcription

1 Minutes of WP1 meeting Milan (20 th and 21 st March 2001). Meeting held INFN Milan, chaired by Francesco Prelz. Welcome (F. Prelz) Report from GGF1 (M. Sgaravatto) ( Massimo gave a summary of the First Global Grid Forum held in Amsterdam ( The meeting had been attended by over 300 people. Future GGF meetings will be held alternately in Europe and the US. In particular Massimo discussed the activities of the scheduling and resource management work group of the GGF. This group has produced 3 documents on: Superschedulling, Advanced reservation API, Attributes for communication about scheduling instances, Suggested for future documents include: Reservation protocol, Dictionary. Massimo also reported the existence of a distributed accounting group within the GGF. There was much discussion on what (if any) involvement WP1 should have in the GGF. It was felt that WP1 had different timescales to the GGF (the GGF timescales being longer than those available to WP1) and so the WP1 contribution to the GGF will be limited. Areas where it was felt that WP1 should contribute include the dictionary and accounting. Ludek Matyska agreed to follow developments in the area of accounting. It was felt that WP1 could help with standardising the Job Description Language. It was suggested that the WP1 broker should have the appropriate hooks to allow for future use of the GARA advanced reservation facilities. Report on DataGrid Meeting (M. Sgaravatto) ( Massimo gave a report of the DataGrid Meeting that followed the GGF1 in Amsterdam. There was much discussion on issues raised at the DataGrid meeting including: Concern was expressed about the time taken to for the GLOBUS team to incorporate bug fixes etc that people had found. The GLOBUS team had suggested that WP1 by part of their alpha release mechanism. Francesco Prelz had since tried to make become involved and had found getting a response difficult.

2 Job partitioning and checkpointing was considered. Whilst this maybe easy to achieve with most HEP applications it is, in general, a difficult problem. It was decided that WP1 should consider this problem on a one year timescale. The present submission mechanism uses Condor-g. This has proved to be a valuable tool offering much functionality. However Condor-g is not open source and the EU contract requires all software used to be open source. After much discussion it was decided that Condor-g will be used at least as far as the Project Month 9 (PM9) release when further consideration will be given to this question. It was also strongly suggested that WP1 developers should not look at the Condor-g source code to avoid legal complications if a replacement should have be written. Architecture Report (F. Giacomini) Francesco gave a talk first given by Steve Fisher at the DataGrid meeting in Amsterdam ( In which he outlined the architectural decisions made by the task force. It was decided that much of what is required by WP1 depends on the other WPs. So questions for the other WP leaders should be sent to Francesco Prelz. Areas in which questions were expected included: Mappings between logical and physical file names How much software should a job require to be preinstalled and how much should be tarred up and sent with the job. ftp access to individual nodes in large clusters/farms. Suggested Architecture for WP1 (F. Giacomini) ( Francesco outlined a suggested architecture for WP1. He identified the components list needed. He defined a job to be an executable to be an executable with the required input and output files. There was some confusion as to the precise meaning of "service index" Job Submission Interface (F. Pacini) DAT pdf Fabrizio outlined the status of the job submission design and the user interface. All user commands will start with the prefix "dg-". There was much detailed discussion as to the exact syntax to be used for some of these commands (these modifications have already been included in the attached document). It was felt that pre-booking facility would be desirable.

3 The nature of the feedback mechanism required by the user was discussed and it was felt that both a fast (via sockets) and slow (via ) mechanism were required, although only the slow mechanism will be implemented by PM9. It was decided that there would be no job re-submission in cases were the user had specified the individual queue. It was also decided that by PM9 there would be no multiple job submission, and job submission will only be implemented on Linux and Solaris. There had been no feedback to the JDL document, and it was decided to advertise it further. Coordination between the JDL developed by WP1 and those developed by the US GRID projects was viewed as important. It was decided that Massimo would put together an appropriately wrapped version of Condor-g in order to connect it to the Resource Broker. Status of the Broker (S. Monforte) (Link to slides to be added here) Salvo described the present state of the broker design and implementation. The broker is being written in C++. There was much discussion as to how information would reach the broker, whether by polling or by pushing. A GRID resource was defined as a collection of GRID information, there was some discussion as to the nature of a composite resource, which was then defined to be a GIIS connected to another GIIS (or even GIS). It was decided that the DoMatch method should be "hot pluggable". After much discussion it was decided that crude versions of broker, user services, submission mechanism, and logger would be available for testing in early June. This version would then be tested at 3 sites in Italy (and maybe Imperial College). Datamat to specify what kind of job status information they need. Initially this information will be provided by a tool provided by Cesnet, later it will be provided by information services. Information Logging and Monitoring (L. Matyska) milano.pdf Ludek gave a short talk describing the logger, which will be based on NetLogger. The logger will require SQLDatabaseServices (or at least some sort of SQL database).

4 There was some discussion as to whether or not WP1 would like access to network weather services information. It was decided that this could be useful. There were then discussions as to whether this would be provided by information services, and whether or not it should be pushed into the GIS. Proposal for an IS schema (M. Sgaravatto) ( Massimo outlined what information he thought was needed for the brokering to work well. Further suggestions of information that might be useful include: Typical Disk footprint, Physical memory, maximum job size, how many CPUs/node, number of SN8 processors and whether it is possible to contact the outside world from within the cluster. There was also discussion as to whether or not it was possible to provide analogous properties for interactive jobs. It was decided that it maybe, however no attempt at this would be made by PM9. Hierarchy After much discussion about the hierarchy of the scheduler it was decided that the WP1 official line statement was "Our implementation will work at personal and community level, however we recommend the community level". Accounting (F. Prelz and C. Anglano) Francesco outline the GGF work on accounting ( They are advocating an economic model. Mino then described a model that included electronic money and cheques, possibly using prepaid card model. This model would require guarantees that a user gets what they pay for. Final It was decided that the next meeting would be in Milan on the 8 th and 9 th of May. It was decided that the broker will be written in C++, however the user interfaces would be through Java, Python and C. Contacts were allocated to the different WPs of the EU DataGrid project and to the US GRID Projects. These were: WP2 Flavia Donno WP3 Zoltan Balaton WP4 Francesco Giacomini WP5 WP6 Antonia Ghiselli WP7 Tiziana Ferrari Applications Silvia Resconi Security Miroslav Ruda

5 Us Projects David Colling