IFREE data center: Seismological data archiving and distributing system

Transcription

1 IFREE data center: Seismological data archiving and distributing system Hiromitsu Mizutani, Seiji Tsuboi, Yasushi Ishihara and Takao Koyama Research Program for Data and Sample Analyses, Institute for Research on Earth Evolution (IFREE) 1. Introduction IFREE Pacific Region Geophysical Network Data Center (IFREE DC) is now in operation and distributing geophysical observation data around the western Pacific region. The Pacific geophysical network consists of seismic observation, electromagnetic observation, GPS observation, superconductive gravimeter, and marine geophysical network. In IFREE DC, we are now archiving and distributing data of OHP (Ocean Hemisphere Project) geophysical observation network to the public. OHP network has 16 broadband seismological stations and 14 stations are now in operation mainly in the western Pacific region (Fig. 1). Observed data are obtained by both offline media and online via internet. 6 stations are now connected to internet and now we can achieve observed data in real time (See Tab. 1). The archived data are provided to researchers via web user interface. We have developed the Java-based networked data distribution system called Ninja (New Interface for Networked Java Applications) to distribute observed geophysical data and the system is now working in IFREE DC and OHP data management center (OHP DMC). In this report, we introduce the system for data archiving and distributing system of broadband seismological data which we are operating in IFREE DC and OHP DMC. 2. Data archiving system The online and offline data are sent from each observatory to each data center in various format, and they are converted to oneday-length international standard format, SEED format (Standard for Exchange of Earthquake Data; see Reference manual) and archived in IFREE DC and OHP DMC. The list of data and estimated usage of disk space are shown in Table 2. In this section, I show the outline of data flow in two data centers. The schematic view of flow diagram of networked data is summarized in Figure Online data To archive online data in real time, we install software (Antelope ( or earthworm ( to sun workstation in OHP DMC. The data packets are provided from the data loggers (Q330) settled on each station. We operate this task by two different software, and both systems are working well. But Antelope system seems more robust and more extensible. In the future, we will integrate online data archiving system to Antelope system. The processed real time data is visible via Antelope application (orbmonrtd) and waveform of archived data can be visible via webpage (Fig. 3). The webpage is published mainly for monitoring of waveforms to check the condition of network, data loggers, and seismometers. The data of long period data (LL? and LH?) components should be treated carefully. In the previous OHP system, LH? component is low gain output acceleration (m/s/s) data from the sensor (STS-1). In the real time system, LH? component is downsampled data of broadband and high sampling data (BH?:20Hz) to 1Hz, and LL? is acceleration data. 2-2 Offline data Offline data is obtained by various media type (MO, TAPE, CD, HDD and so on) and various data format. In IFREE DC, both original data and converted data are stored in HDD for the security. The data conversion to one-day-length SEED files is now operated in IFREE DC. The format conversion software are originally developed in OHP DMC and now these software are transplanted and operated in IFREE DC. The rate of incoming data depends on the recording system (e.g. data logger, media type) working at the station. Q330 data should be carefully treated. Q330 data is archived both by online system as mentioned in previous section, and offline as a HDD data. We should check the difference of these data carefully when we integrate these data. HDD data is generally reliable and online data might be overwritten. But it is possible that the HDD crashes and the data is lost. In such a case, we should treat online data as only one usable data. It is very sensitive problem whether we overwrite the data or not. To avoid this problem, we are planning to archive and publish both online data and offline data by labeling different location id (locid) in the SEED file. Users can select these data by themselves. 2-3 Data sharing between data centers OHP data is shared both in OHP DMC and IFREE DC. The online data is originally archived in miniseed format in OHP DMC, and offline data is archived in miniseed format in IFREE DC. We synchronize the data set of data centers for keeping the security for some trouble. The miniseed data files and database file (mentioned below) are synchronized by unix command (rsync) once a day. In Figure 4, the schematic view of data sharing is written. 3. Database and event data We prepare the database files for archived waveform data files. The database has information for archived SEED files (such as locations, effective times, response information, channel sample rates, and channel orientations). The database files are created by using PDCC (Portable Data Collection Center) toolkit (see PDCC toolkit user's manual, 1998) which is provided by IRIS. We are now using PDCC version 1.0. In this version, the information of data files (date, time, sensor information and so on) are written in large text file. The file named wfdisc becomes larger and larger as archived data increases. The current size of wfdisc is about 70 MB, and increasing day by day. To find data files which is requested by a user, say for some date, some duration, some station, some components, we (or the data distribution software) 1

2 should access wfdisc file and find appropriate wfdisc entry of data file. This process for data searching takes long time because wfdisc is large plain text file. New version of PDCC toolkit uses MySQL to construct tree-structured database. The upgrade to new version PDCC might decrease the time of searching task, and it is a future work. Event data (OHP FARM) is also created by appropriately cutting continuous waveform data using PDE (Preliminary Determination of Epicenters) or QED (Quick Epicenter Determinations) earthquake event catalogue provided by USGS. Created FARM data are archived in Full SEED format (data+ sensor information) and provided via web interface. See Takeuchi et al. [2002] for the detail of making FARM event data. User can access the event data by using web interface shown below. FARM data is created in IFREE DC and the data is synchronized to OHP DMC. 4. Data distributing system We are now distributing OHP data via Web user interface system called as Ninja, which can be accessible from any users (see Fig. 5). Both continuous waveform data and event data are accessible by using Ninja's user request interface. Besides waveform data itself, we provide waveform image of one day length, 3 components via webpage as well, for the purpose of monitoring. We can find some event or trouble of the observatory by using this monitor ( Ninja system is now working in IFREE DC, OHP DMC and NIED ISN (National Research Institute for Earth Science and Disaster Prevention, International Seismic Network). The provided data in each data center is summarized in Table 3. In our system, data centers are connected to one unified interface and user can access data set of each data center by the same user interface. In our new interface, a user can transit from one data center to another, and the user interface for sending request is almost same for each data center. Because the user interface for another data center is invoked in another window, the independence of each data center is preserved but user should re-input their request in newly created window. It is not convenient for users. It is still remaining as a problem, and the development of both user-friendly and data center-friendly system is a future work. We can provide Ninja system to each data centers and by installing Ninja system, the data center can provide their seismological data to the public user. user's local PC. A user send request to each data center from in one unified user interface, but the access to each data center is dependent from each other. In this new application, we consider the dependence of each data center is preserved. References Gorbatov, A. and S. Tsuboi, Database of short-period waveforms and times of seismic phase first arrivals, Frontier research on earth evolution (IFREE report), 1, , PDCC TOOLKIT Portable Data Collection Center toolkit User's manual, Incorporated Research Institutions for Seismology, Standard for the Exchange of Earthquake Data, Reference Manual, SEED Format Version 2.3, Incorporated Research Institutions for Seismology, Takeuchi, N., S. Watada, S. Tsuboi, Y. Fukao, M. Kobayashi, Matsuzaki, Y. and T. Nakashima, Application of distributed object technology to seismic waveform distribution, Seism. Res. Lett., 73, , Appendix: Hinet data archived in IFREE HINET is seismological observation network which is operated by NIED. We are now achieving HINET data via satellite telemetering system. HINET data is very huge so we can only archive continuous data on local RAID about a week and other data are archived in SDLT tape. As shown in Table. A1, we need 10TB disk space to archive one year continuous data. Major event data in the world (PDE, M>6) are archived in network HDD (\\it4p16\hinet_data\: access restricted) only for IFREE researchers. We are also operating automatically seismic phase picking system for earthquake event data [Gorbatov and Tsuboi, 2003]. The auto-picked first arrival traveltime data is also archived in our IFREE DC. We should check the quality of autopicked data carefully. 5. Future work Online data archiving system (Antelope) is stably working now. We will integrate realtime data archiving system from earthworm to Antelope. Real time data will be distributed via Ninja in the nearest future. At moment, Antelope system is working only in OHP DMC. We have a plan to install Antelope in IFREE DC mainly for back up system working in OHP DMC. As mentioned in section 2, the quality comparison between Q330 online data and offline data should be done. The quality control of both online and offline OHP data and associate with the database is also a future work. We are now developing new application to obtain data sets of different data centers without losing independence of each data center and user convenience. Our new application works on a 2

3 Figure 1. OHP station map. SYO is outside the plotted region. Figure 2. Data and request flow of IFREE/OHP DC 3

4 FRONTIER RESEARCH ON EARTH EVOLUTION, VOL. 2 Figure 3. Example of image data of online data. This page is updated every 30 minutes. Figure 4. Data synchronizing between IFREE DC and OHP DMC 4

5 Figure 5. Web interface of data distributing system (Ninja) Table 1. OHP data achieving format and system (2005/01/28) Online Software or Input Format/Byte length Archive Format/Byte length Station System Earthworm MSEED steim2/512byte MSEED steim1/4096byte PATS, ISG (INU, TJN, PSI, JAY) Antelope MSEED steim2/4096byte MSEED steim2/4096byte INU, TJN, PSI, JAY Offline Markrand(M O) YAMADA Format MSEED steim1/4096byte PALU, KMS SYO(TAPE) MSEED steim2/512byte MSEED steim1/4096byte SYO PMG(CD) SEED steim2/4096byte MSEED steim2/4096byte PMG Q330(HDD) MSEED steim2/4096byte MSEED steim2/4096byte OGS, PATS, ISG, INU, TJN, PSI, JAY Table 2. OHP data and its properties Component Sampling rate Unit Disk usage (total) BH? 20Hz m/s 5.5MB/day/station (20GB/yr) LH? 1Hz m/s 0.6MB/day/station (2.4GB/yr) LL? 1Hz m/s/s 0.3MB/day/station (1.2GB/yr) Table 3. Public data of each data center Data center URL Available data OHP OHP, EOC, BATS etc IFREE OHP NIED SPANET, JISNET (access restricted) Table A1. HINET data Component Sampling rate Unit Disk usage (total) EH? 100Hz m/s 36MB/day/station (10TB/yr) 5