A NOVEL LOW COST TELEMEDICINE SYSTEM USING WIRELESS MESH NETWORK

Transcription

1 A NOVEL LOW COST TELEMEDICINE SYSTEM USING WIRELESS MESH NETWORK E. Supriyanto 1, H. Satria 2, I.H. Mulyadi 2 and E.H. Putra 2 1 Faculty of Biomedical Engineering and Health Science 2 Faculty of Electrical Engineering Universiti Teknologi Malaysia ABSTRACT Telemedicine promises an improvement of health care service quality in rural, urban, dense and mobile areas. In order to implement the telemedicine in these areas, a low cost telemedicine system with acceptable quality for medical data transfer is required. This paper discusses simulation and implementation results of a low cost telemedicine system including wireless medical interface and communication infrastructure. A simulation has been done to investigate the network quality of service. The infrastructure has been also implemented using low cost 5.8 GHz transceiver for backhauls and low cost 2.4 GHz transceiver for clients. Test result shows that the low cost telemedicine system is able to do real time communication between patient and medical staff with medical data rate up to 2 Mbps. It shows that the low cost telemedicine system using wireless mesh network can be implemented in remote area with acceptable medical data transfer quality. 1. INTRODUCTION Telemedicine is an emerging technology which combining telecommunication and information technology for medical practice. It gives a new way to deliver health care services when the distance between doctor and patient is significantly away. Telemedicine can deliver health care services to the patient even in remote area. Pavlopoulos et al. (1998) has presented an example of telemedicine advantage with implementation on ambulatory patient care at remote area. Another application has been done by Sudhamony et al. (2008) for cancer care in rural area. High technology telemedicine application in surgery has already been developed by Xiaohui et al. (2007). Currently, the telemedicine uses available wired and wireless infrastructures. Telemedicine infrastructures with wired network have been proposed by Al-Taei (2005) using Integrated Service Digital Network (ISDN), Asynchronous Transfer Modes (ATM) by Cabral & Kim (1996), Very Small Aperture Terminal (VSAT) by Pandian et al. (2007) and Asymmetric Digital Subscriber Line (ADSL) by Ling et al. (2005). Telemedicine has also been implemented in wireless network using Wireless LAN (WLAN) proposed by Kugean et al. (2002), Worldwide Interoperability for Microwave Access (WIMAX) by Chorbev et al. (2008), Code Division Multiple Access (CDMA) 1X-EVDO by Yoo et al. (2005), General Packet Radio Switch (GPRS) by Gibson et al. (2003) and 2G Groupe Spécial Mobile (GSM) by Pavlopoulos et al. (1998). Each infrastructure has its own obstacle, in particularly when they are implemented in a remote area. For example, Asynchronous Transfer Mode (ATM) and Multi Protocol Label Switching (MPLS) had some mobility and scalability limitation, even both networks provide high Quality of Service (QoS) and have stability on delivering data (Nanda et al., 2007). The fragility of 3G UMTS network for telemedicine has been explored by Y. E. Tan et al. (2006), where the implementation costs are high and does not provide enough QoS. Comparison of available network infrastructures and technologies for telemedicine system are summarized in Table 1. It can be shown that based on cost, data rate and mobility, the optimal solution for telemedicine system is Wireless Mesh Network (WMN). In order to investigate the quality and the possibility of using WMN as a telemedicine infrastructure, a simulation of data communication in WMN using NS2 network simulator has been done. Besides, the infrastructure and the medical data interface have been also developed and tested. This is required to create a low cost telemedicine system that has an acceptable quality of service. 382

2 Table 1 Comparison of available network infrastructures/technologies for telemedicine system infrastructure/ bit rate investment cost* operational cost user device mobility technology (Mbps) cost Fiber Optics 2,550,000 H M VH No ATM 155 VH H VH No ADSL 8 H M M No ISDN H M M No VSAT 4.09 VH M VH No GPRS H L L Yes EDGE H L L Yes 3G UMTS H M M Yes HSDPA 3.6 VH M M Yes WMN 54 L VL L Yes WiMAX 54 M VL L Yes Note: *investment cost = cost of investment number of user node that can be serviced VH = very high, H=high, M=medium, L=low, VL=very low 2. MEDICAL QUALITY OF DATA AND SERVICE An application example of telemedicine system can be seen in Fig. 1. It involved patients and doctors that are equipped with medical data assistant (MDA). The MDA acquires the medical data using Medical Data Interface (MDI). MDI retrieves medical data from various medical devices. Typical medical devices are electrocardiography machine (ECG), Doppler instrument, blood pressure monitor, ultrasound machine and stethoscope. The MDA are connected through wireless router and transceiver for data transmitting between patient and doctor. Two-way communication between patient and doctors is supported by camera and microphone. All of patient medical data are continuously recorded inside the server for diagnostic purposes. For this application, the minimum data rate for acceptable (good) quality and excellent quality are listed in Table 2. Table 2 Desired output data rate devices data rate good excellent ECG 2 kbps 12 kbps Doppler Instrument 40 kbps 160 kbps Blood Pressure Monitor 1 kbps 1 kbps Ultrasound Machine 100 kbps 400 kbps Camera 100 kbps 2,000 kbps Stethoscope 40 kbps 160 kbps Microphone 40 kbps 160 kbps Total 323 kbps 2,893 kbps The minimum data rate for acceptable service in telemedicine system is 323 kbps, whereas for excellence quality minimum data rate 2,893 kbps is required. Up to now, there is no specific rule defining QoS provisions of telemedicine application. In addition, parameterized QoS is a clear QoS bound which are stated in terms of quantitative values such as throughputs, end delay, jitter and packet loss. These QoS bounds are explained in Table WIRELESS MESH NETWORK FOR TELEMEDICINE SYSTEM Hardware configuration for low cost telemedicine communication infrastructure is shown in Fig. 2. This infrastructure is divided into three areas of implementation; client and indoor network, backhaul outdoor network and medical central service. Fig. 1 An application example of low cost telemedicine system 383

3 Table 3 QoS for low cost telemedicine system parameter definition requirement throughput packet arrival rate min 323 kbps delay the time taken by a packet to reach its destination max 100 ms jitter time of arrival deviation between packets max 50 ms packet loss percentage of non-received data packets max 5 % Fig. 2 Hardware configuration for low cost telemedicine communication infrastructure In indoor network, a wireless router is connected to medical devices through MDI and connected to outdoor network via switch. Outdoor network is main structure of wireless mesh network. This configuration enables a communication between client and central service or medical doctor via Doctor-Medical Data Assistant (D- MDA). This enables also mobile communication between patient and medical doctor. Characteristic of WMN outdoor backhaul are described in Table 4. The operational frequency of GHz has been used to overcome distance and obstacles between backhaul connections. Parameters for the WMN indoor router are described in Table 5. The operational frequency of WMN indoor router is GHz. Table 4 WMN outdoor backhaul characteristic Parameter value antenna Type directional antenna gain transmitter 10 dbi gain receiver 10 dbi receiver sensitivity -100 dbm line of sight range 20 km network interface input power (max) 1 W channel frequency GHz data rate 54 Mbps Table 5 WMN indoor router characteristic Parameter value antenna Type omni directional gain transmitter 1 dbi gain receiver 1 dbi receiver sensitivity -104 dbm network interface input power (max) 10 mw data rate 11 Mbps channel frequency GHz NS2 version 2.33 network simulator has been used to simulate the QoS parameter including delay, throughput, jitter and packet loss. Three protocols have been implemented in the simulation. The simulation method and results are reported by Supriyanto et al. (2008). Fig. 3 shows throughput as function of hop number. Simulation result shows variations of throughput for each routing protocol. The bigger hop number, the smaller throughput. It is due to delay and waiting time in routing mechanism. DSDV routing protocol inferred its throughput better than the other protocols. Simulation result shows that the network able to transfer the medical data up to 2 Mbps, even still more than 200 kbps for 6 hops. This result shown the ability of network to fulfill the qood quality medical data transfer requirement. 384

4 to the patient and MDIzb. Table 6 shows MDIz characteristics. MDIzb has the same operational frequency as MDIz, but the modulation is slightly different. Table 7 shows MDIzb characteristics. Fig. 3 Throughput comparison of each routing protocol in EPT system. 4. WIRELESS MEDICAL INTERFACE In order to enable the connection between common medical devices and network infrastructure, a wireless medical interface has been developed. The wireless medical interface is implemented using Zigbee and Bluetooth technology. Two kinds of wireless medical interface has been developed; MDIz and MDIzb. MDIz is a device which acquires data from medical device and transmits it out through Zigbee network, while MDIzb coordinates the Zigbee network as well as communicates with Medical Data Assistant through Bluetooth network. Medical data is acquired by MDIz and then added with a frame as shown in Fig. 4. The data is then transmitted to MDIzb through Zigbee network. MDIz is able to send data up to bps. The bandwidth is more than enough for vital sign data. The network has a PAN ID to differ with the other Zigbee networks. Table 6 MDIz characteristic parameter Value Antenna type omni antenna gain transmitter 1 dbi gain receiver 1 dbi receiver sensitivity -92 dbm indoor coverage range 30 m line of sight range 100 m Zigbee module channel frequency GHz bandwidth MHz data rate bps Table 7 MDIzb characteristic parameter Value Antenna type omni antenna gain transmitter 1 dbi gain receiver 1 dbi average power -89 dbm line of sight range 100 m Bluetooth module channel frequency GHz bandwidth GHz data rate bps Fig. 5 shows block diagram of MDIz. Analog signal from medical sensor is conditioned, digitized and processed by a processor. The data from processor is converted to Zigbee and then transmitted out. The data is arranged in a frame as shown in Fig. 6. It consists of device ID, status, length of frame, data and checksum. The Zigbee module is bidirectional. MDIz can receive the command through Zigbee module. Fig. 5 MDIz Block Diagram Fig. 6 Data Frame of MDIz Fig. 4 Wireless medical interface configuration The MDIz was designed for short range communication, between the embedded device attached MDIzb coordinates the Zigbee network. It decides which medical device will communicate to in that time and give them priority based on their status. Fig. 7 shows block diagram of MDIzb. It consists of Zigbee module, two processors, D-latch, SRAM, and Bluetooth module. Data from MDIz is received by Zigbee module and processed by processor 1. Delay may ignore the data. To overcome this matter, a 256kb SRAM is added as a data buffer. The data is then sent to processor 385

5 2 to add a frame as shown in Fig. 8 before transmitted out by Bluetooth module. Bluetooth data is acquired by mobile data assistant. The Bluetooth module is bidirectional. MDIzb can receive the command through Bluetooth module. Fig. 10 MDIzb Bluetooth signal strength Fig. 7 MDIzb block diagram Fig. 8 Data frame of MDIzb Fig. 11 shows the signal strength of communication between P-MDA and WLAN router using WiFi communication. This result shows that the cover of 10 meters still give high Signal to Noise Ratio (SNR) from wireless router sensitivity. 5. TESTING AND ANALYSIS The testing has been done to investigate the network performance and the reliability of system including wireless medical interface. Fig. 9 shows the test result for MDIz signal strength. MDIz signal strength is the MDIz power to receive the data between receiver at medical interface and transmitter at medical equipment communication using Zigbee communication. It should be higher rather than its receiver sensitivity (Signal Strength Rx sensitivity) from the specification of MDIz. Test result shows that for the distance up to 10 meters, the MDIz Signal strength gives the result higher than its receiver sensitivity at -92 dbm. Fig. 9 MDIz signal strength The measurement has also been done for the data transmission between wireless medical interface and P- MDA client. Fig. 10 shows the signal strength of communication between medical interfaces through the P-MDA client using Bluetooth communication. Fig. 11 Wireless router WiFi signal strength The network performance test result is shown in Table 8. The measurement is done for backhauls 1 km distance. The result shows the noise rate varies from its receiver sensitivity. The average signals for each backhaul in open space area with clear weather are in range of -26 and -28 dbm. This gives the SNR values of 72 db for low SNR and 82 db for high SNR. Table 8 Network performance testing of each backhaul at 1 km Location antenna noise noise signal band high low strength (MHz) (dbm) (dbm) (dbm) Location Location Location The testing has been done to transfer the medical data up to 2 Mbps. Test result shows the network is still able to fulfill the minimum requirement stated in Table 3 with excellent signal strength. 386

6 Table 9 Connection characteristic per hop between backhaul distance SNR data rate hop (km) (db) (Mbps) Table 9 shows the average values from connection measurement in each number of hops. Hop is counted from the user in location 1 to the destination in other location. The data rate was calculated from the average of data rate of each user. Testing results shows that the low cost telemedicine infrastructure within 3 hop count with distances more than 3 km fulfills the requirement of medical data rate of more than 2.8 Mbps for excellent quality of medical data. Moreover, the attenuation of weather and obstacles, such as building and tree has decreased the SNR value into low than 72 db in distance of 1.2 km. CONCLUSION A novel low cost telemedicine system using WMN has been successfully developed and tested. The system consists of wireless medical interface and communication infrastructure. The wireless medical interface using Zigbee and Bluetooth technology has been implemented to enable the connection some medical devices to communication infrastructure. The communication infrastructure has been implemented using low cost 5.8 GHz transceiver for backhauls and low cost 2.4 GHz transceiver for clients. Simulation and test results show that the system is able to transfer the required medical data up to 2 Mbps. In order to enable the implementation of this system, a data encryption and safety monitoring units of medical devices should be integrated in the system. REFERENCES Al-Taei, M., Telemedicine needs for multimedia and integrated services digital network (ISDN), Proc. ICSC Congress on Computational Intelligence Methods and Applications, IEEE, Amman, Jordan Cabral, J.J. & Kim, Y,. Multimedia systems for telemedicine and their communications requirements. IEEE Communications Magazine, Chorbev, I., et.all., WiMAX supported telemedicine as part of an integrated system for e-medicine. Proc. 30th International Conference on Information Technology Interfaces, IEEE, Dubrovnik, Deng, L. & Poole, M.,. Learning through telemedicine networks, Proc. the 36th Annual Hawaii International Conference on System Sciences, IEEE, Gibson, O.J., Cobern, W.R., Hayton, P.M. & Tarassenko, L., A GPRS mobile phone telemedicine system for self-management of type 1 diabetes, Proc. 2nd IEEE EMBSS Conference on Biomedical Engineering and Medical Physics, Birmingham, Kugean, C. et al., Design of a mobile telemedicine system with WLAN, Proc. Asia-Pacific Conference on Circuits and Systems, IEEE, Singapore, Ling, L. et al., A multimedia telemedicine system, Proc. 27th Annual International Conference of the Engineering in Medicine and Biology Society, IEEE, Nanda, P. & Fernandes, R., Quality of Service in Telemedicine, Proc. First International Conference on the Digital Society, IEEE, Guadeloupe Pandian, P. et al., Internet Protocol Based Store and Forward Wireless Telemedicine System for VSAT and Wireless Local Area Network, Proc. International Conference on Signal Processing, Communications and Networking, IEEE, Chennai, Pavlopoulos, S. et al., A novel emergency telemedicine system based on wireless communication technology-ambulance, IEEE Transactions on Technology in Biomedicine, vol.2, no.4, p , Sudhamony, S., Nandakumar, K., Binu, P. & Niwas, S., Telemedicine and tele-health services for cancer-care delivery in India, IET Communications, vol.2, no.2, p , Supriyanto, E. et al., Simulation of Emergency Prenatal Telemonitoring System in Wireless Mesh Network, Proc. Third international conference on modeling, simulation and applied optimization, IEEE, Sharjah, Xiaohui, X., Ruxu, D., Lining, S. & Zhijiang, D., Internet based telesurgery with a bone-setting system, Proc. IEEE International Conference on Integration Technology, Shenzhen, Y. E. Tan, N.P. & Istepanian, R.S.H., Fragility Issues of Medical Video Streaming over e-WLAN m- health Environments, Proc. the 28th IEEE EMBS Annual International Conference, Yoo, S.K. et al., Prototype Design of Mobile Emergency Telemedicine System, Proc. Computational Science and Its Applications, Springer, Berlin,

7 Eko Supriyanto received M.Eng in Biomedical Engineering from Institut Teknologi Bandung and Dr.-Ing (2003) from University of Federal Armed Forces Hamburg, Germany. He is currently a senior lecturer in Faculty of Biomedical Engineering and Health Science, Universiti Teknologi Malaysia. Muhammad Haikal Satria received the B.E. (2002) from Universitas Indonesia and M.Sc. (2006) from Universität Duisburg Essen, Germany. He is a PhD student in Faculty of Electrical Engineering, Universiti Teknologi Malaysia. Indra Hardian Mulyadi received the B.E. (2004) from Institut Teknologi Bandung. He is a Master student in Faculty of Electrical Engineering, Universiti Teknologi Malaysia. 388

8 DEVELOPING 3D GEOSPATIAL DATABASE FROM TERRESTRIAL LASER SCANNED DATA Alias Abdul Rahman 1, and Gurcan Buyuksalih 2 1 Department of Geoinformatics, Faculty of Geoinformation Science and Engineering, Universiti Teknologi Malaysia, UTM Skudai, Johor, Malaysia alias@utm.my 2 Istanbul Greater Municipality (IBB, BIMTAS), Tophanelioglu cad. ISKI Binasi, No. 62, K:3-4, Altunizade, Istanbul, Turkey gbuyuksalih@yahoo.com ABSTRACT Most major cities are being managed spatially by 2D geo-dbms, however, unlike Istanbul city it is in the verge of 3D spatial information system especially for the historical peninsular region. Thousands of its 3D objects (mainly buildings old and new) are in the process of 3D database development. Istanbul city municipality is currently busy with various GIS projects including an initiative towards a 3D database for the city. This paper describes the development of 3D database from terrestrial laser scanned data where a leading geo-dbms, i.e. Oracle Spatial 10g were utilised. Useful 3D geoinformation could be generated from such 3D database, thus the initiative would provide an excellent tool for the city s decision making process e.g. for the city planning purposes, and etc. The 3D geo-database approach described in this paper is certainly would be very beneficial to the city and other GIS users since the current and commercial GIS software face some difficulties in providing such GIS operations and information to the relevant authorities. The paper also highlights new 3D spatial operations developed for possible spatial analysis of 3D objects (surface and underground objects). 1. INTRODUCTION Three-dimensional (3D) spatial database will become one of the important components in any near future spatial information systems. The trend shows that more and more GIS users would like to incorporate 3D spatial objects in such systems, thus the more realistic spatial database is inevitable. The problem is that current databases are still 2D in nature and very cumbersome for the 3D objects and still many issues need to be addressed before we could really be able to manipulate real world 3D objects as most users would like to have (Abdul-Rahman et al, 2006). The same sentiment on 3D geo-dbms was also reported by Arens et al (2005) where the unavailability of 3D primitives in the existing geo-dbms is the main drawback. This scenario motivates us to do some investigation works especially by make use of the available 3D datasets, e.g. buildings and other type of objects for the 3D geo-dbms operations and analysis. Therefore, the aim of this paper is to review few recent research works especially on 3D spatial data modeling and describes our recent work on the management of 3D laser scanner data using geo-dbms. In this paper, 3D data acquired from new acquisition technology, i.e. terrestrial laser scanned datasets will be investigated and experimented for the development of 3D spatial database. Initial results have been presented recently, see Abdul-Rahman et al (2008), however, more tests are needed to be carried out for a better understanding of the problem. Section 2.0 describes the study area, the terrestrial laser scanning data acquisition in Section 3.0 3D spatial database development description is discussed in Section 4 and visualization of retrieved 3D objects in Section 5, and finally the conclusion in Section 6 with some issues that still need to be looked further. 2.0 STUDY AREA Suleymaniye, Istanbul, Turkey is the study area. It is a place of historical objects and monuments, which 389

9 contains approximately 48,000 buildings, and an area of 1500 hectare. In this area, we assume the following scenario: the user has 3D data organized only in a database (a quite common case for real world data), i.e. no file with graphical information (e.g. DGN) exists. We have experimented with a set of buildings from the Suleymaniye area. Planar rectangular faces constitute each building. The data are built in CAD and further converted to the geometry representation of Oracle Spatial 10g. The conversion is completed with a topology-geometry of the 2D and 3D spatial object. Since the Oracle Spatial 10g geometry does not maintain a true 3D object, we represented every building as a set of faces (walls, flat roofs and foundations). The faces are stored as polygons with 3D coordinates. In this experiment table, namely LOT stored 2D object, table BUILDING represent 3D object stored face-by-face and table BODY_BUILDING stored the whole 3D object as one object. scanners (four HDS4500 and one HDS3000), four ILRIS-3D scanners from Optech (Figure 2), four Topcon total stations for geodetic control point measurements and pre-calibrated SLR cameras Nikon D70 with 14mm and 28mm lenses for digital photogrammetric documentation. The technical specification of the terrestrial laser scanning systems used for this project are summarised in Table 1. Figure 2 The Optec laser scanning system Figure 1 The Istanbul peninsula historical area 3.0 TERRESTRIAL LASER SCANNING 3.1 Data acquisition The 3D datasets were captured via Terrestrial Laser Scanning (TLS) system, Leica LS4500 (static and mobile). Initially, static system was used but then a more robust and quicker system was utilised, i.e. mobile laser scanning (MBL) system due to project time constrained. The MBL consists of laser unit, IMU, and GPS and mounted on a vehicle. Huge amount of data were captured, it is the order of more than 50 Terra-byte and the data management system has been properly established by the laser-scanning unit at IMP (Istanbul Metropolitan Planning) and BIMTAS offices. For this project a new production organisation was built up at BIMTAS using modern 3D mapping and computer technology. The terrestrial laser scanning group includes 24 staff, which uses the following technical equipment for data acquisition: five Leica The three scanner types use two different principles of distance measurement: Leica HDS4500 uses phase shift method, while Leica HDS3000 and ILRIS-3D scan with the time-of-flight method. In general it can be stated that phase shift method is fast but the signal to noise ratio depends on distance range and lighting conditions. If one compares scan distance and scanning speed shown in Table 1, it is obvious, that the scanner using the time-of-flight method can measure longer distances but is relatively slow compared to the phase shift scanner. The HDS4500 measures distances up to 53m, while the HDS3000 and the ILRIS can measure up to 100m and 1500m, respectively. Due to the limited speed of 1500 or 4000 points per second and due to the limited field of view it quickly became clear that the ILRIS scanners and the HDS 3000 are not useful for the busy and narrow streets of the project area. These scanners are more suitable for the documentation of landmarks. Thus, all buildings were scanned with a scan resolution of ~15mm at the object using four HDS4500. For data processing of the scanned point clouds, which includes registration, geo-referencing and segmentation of the point clouds, five licenses of Cyclone 5.2 and four licenses of Polyworks 4.1 were used in the office. For the scanning of the buildings, targets were used as control points for registration and geo-referencing of 390

10 the scans from different scan stations as illustrated in Figure 3 (right). The targets have black-white quarters of a circle with a diameter of 126mm. To obtain centre positions of the targets, the targets were automatically fitted in the point cloud after manual pre-positioning using algorithms of the Leica Cyclone software. 3.2 Static terrestrial laser scanning The data acquisition by static terrestrial laser scanning started in September During application in the Historic Peninsula streets it turned out that only the HDS4500 were able to scan in this special environment. As mentioned before the ILRIS-3D and HDS3000 scanner could not scan efficiently in the narrow streets due to the limitation in the field of view, distances that were too short and insufficient scanning speed. Furthermore, the registration of the point clouds of the ILRIS-3D caused problems with tilted scans from the same scan stations, and required matching with the Iterative Closest Point (ICP) algorithm and needed initial values for its computation. Consequently, the daily laser scanning was carried out with four, or sometimes with three, HDS4500. Figure 4 shows an example of a coloured point cloud of building facades at the Historic Peninsula. therefore it was decided to increase the production rate by the integration of a mobile system. 3.3 Mobile terrestrial laser scanning As a consequence the scan progress was significantly increased by the introduction of a mobile mapping van (Gajdamowicz et al. 2001) from the Swedish company VISIMIND AB (Figure 4) in June 2007 using a hybrid sensor system on the vehicle consisting of a terrestrial laser scanning system HDS4500, supported by GPS/IMU and digital cameras. The sensor integration and the calibration of the system in the streets of Istanbul took some weeks, but the data acquisition in the field was working by the end of June The laser scanner s orientation was fixed in the horizontal direction, scanning only in the profile perpendicular to the direction of movement of the vehicle. It has been operated with 25 scan profiles/second, later improved to a speed of up to 40 profiles/second (possible maximum by instrument specification: 50 profiles/second). The distance between neighbouring profiles was 2-3cm in the beginning, corresponding to a van speed during scanning of 0.5m/sec up to 0.75m/sec or 1.8 km/h up to 2.7km/h. Figure 3 Example of a coloured point cloud of building facades at the Historic Peninsula In general, a satisfying spatial (geometrical) distribution of the targets on the object or around the object was guaranteed for the required description of the detailed object. The coordinates of all targets were determined by geodetic methods using total stations. The targetbased registration and geo-referencing of the point clouds, which are acquired by the HDS4500 scanners, worked without any problems using five Cyclone software components as following: (a) registration of all scans and quality control of the result (check of residuals), and (b) geo-referencing using all control points including quality control by checking residuals. 80ha of the project area (of in total 1500ha) could be scanned within the first six months using the existing production capacity, which clearly indicated, that the scanning would need more than eight years for the entire area of the project, if this current scan rate of approximately 0.7ha per day could not be increased. It was obvious that the project deadline could not be met; Figure 4 Sensor configuration on the mobile mapping van of VISIMIND AB Due to problems with the reception of the GPS signal in the narrow streets of the Historic Peninsula control points were marked on the buildings every five meters along each side of the street. Some targets were removed or destroyed before scanning and were replaced by natural points such as window corners. Some targets have been destroyed after scanning, but before the geodetic determination of the object coordinates, they also had to be replaced by natural points. The sticking on of the targets was carried out by BIMTAS staff (4-5 people), while the determination of the target coordinates was performed by BIMTAS staff and additional subcontractors. BIMTAS staff measured additional natural ground control points, well distributed on the facades, in order to stabilise the inhouse data processing of the mobile mapping system, 391

11 while the subcontractors only measured the targets. Not all control points have been identified correctly in the point clouds causing geometrical problems for the direct geo-referencing and some geometric deformation of the point clouds (Figure 5: misfit at block corners, swinging building façade, etc.). Nevertheless, the technical parameters of the hybrid systems were optimised on the job due to these problems with the quality of the pre-processed point clouds. Figure 5 Geometric problems from direct georeferencing of point clouds (from left to right: swinging façade, misfit at block corner, and deformation of a façade) For problematic facades where control points were missing, VISIMIND recently developed with the so called image tracking tool an automatic photogrammetric bundle adjustment enabling a bridging of longer distances without control points (Figure 6). Figure 6 Image tracking tool for problematic facades without control points The speed of data acquisition by terrestrial laser scanning with the mobile mapping system of VISIMIND could be increased significantly. 33 blocks could be scanned in 33 working days until the end of August Usually, scanning could be carried out five days per week (Mo-Fr, plus Sunday) starting at 6.30 am until 2 pm of each working day. Consequently, the laser scanning of the remaining 50 blocks could be completed with the mobile system with the improved total production rate of ~600m per hour, while post processing of the multiple sensor data took until early The production rate was mainly 1:10, i.e. for one hour scanning 10 hours post processing was needed. In total, 12 operators of the laser-scanning group were supporting the data post processing of the mobile mapping system during the major processing phase. Nevertheless, approximately 2% of the area (30ha) could not be scanned by mobile terrestrial laser scanning (TLS) due to traffic restrictions and environmental conditions. This remaining 2% of the total area must be scanned by static TLS at the end of the project in order to complete the data acquisition. At least two months will be needed for scanning by static TLS using all available laser-scanning systems. 3.4 Digital photogrammetry For photogrammetric documentation of the building facades as mentioned before, pre-calibrated SLR cameras Nikon D70 with 14mm and 28mm lenses were used. The acquired images were processed in combination with the static terrestrial laser scanning data. When the mobile system was used for data acquisition, only the images of the integrated oblique and horizontal cameras were used for mapping. The upper sideward looking camera is vertically rotated against the lower camera by approximately 34, enlarging the vertical field of view of the camera system to approximately 86, so that the camera system starts at an angle looking down to the street. 3.5 Mapping of facades The geo-referenced point clouds from the laser scanning group were used for line mapping of the facades in a plot scale 1:200. The point clouds were segmented by two people using Cyclone software before mapping to eliminate unnecessary points and to reduce the data volume to the requested minimal portions for the mapping software. In this project generation of façade maps with 1:200 plot scale is required. This extreme demand corresponds to a standard deviation of the positions with 0.2mm in the map and 4cm in the object space, but this extreme accuracy is required only as relative accuracy; for the absolute accuracy a standard deviation of 0.5mm in the map, corresponding to 10cm in object space should be sufficient. As a tolerance limit three times the standard deviation has been accepted. Therefore, the control point configuration and accuracy must always be checked to obtain this accuracy. While all problems of static and mobile scanning were solved, the delay in the control point determination was a bottleneck in the production. Figure 7 Segmentation of point clouds The facade mapping group consists of 34 operators using 34 licences of the Menci-software Z-MAP Laser from Italy, which is able to process laser scan data and rectified photogrammetric images simultaneously for 392

12 line mapping with limited AutoCAD functionality. It was estimated that approximately 5 million m² of facades have to be mapped. The production rate was similar to the static laser scanning group: 80 ha with 32 operators in approximately 6 months. With regards the facade area, in total 81,000 m² could be finished in 39 days, which corresponds to 65 m² per person per day. The production rate could be increased from 60 m 2 of facade/day/operator (March 2007) on average to 140 m 2 /day (October 2007), which is more than a factor of a 2 time increase. If one assumes in total 5 million m 2 façade area for mapping of the Historic Peninsula, it corresponds to an estimated mapping time of approximately five years with 34 operators working on 210 days per year. This estimation indicated that the mapping could not be finished before the deadline of the project. For data processing in Z-MAP all related data of the segmented part (point cloud, Nikon image(s), camera calibration file) was saved in one directory using the name of the block plus a suffix, e.g. 900_01. This block name is defined in the cadastre map. The HP workstations xw8200 used are equipped with dual XEON Processors (3.6 GHZ), 4 GB RAM and nvidea Graphic Cards with 256 MB RAM. For facade mapping the point cloud and one oriented image of the façade were used. Thus, the orientation of the photogrammetric image (usually recorded with the 14 mm lens) had to be determined by resection in space using at least five well distributed corresponding points (usually corners of windows) in the point cloud and in the image. For the adjustment of the spatial resection the calibration data of the pre-calibrated NIKON D70s are used. Usually the residuals of the control points were in the range of some millimetres, which indicated that sufficient results have been achieved. To carry out mapping with Z-MAP the images had to be rectified to the main plane of the facade. Therefore, the plane was defined by more than three points, which were measured in the point cloud and in the image. Thus, the photos were rectified to the main plane of the facades and shifted to parallel planes based on the point clouds. Based on the dense point clouds from the Leica HDS4500 scanners, the mapping was often possible without support of the photos, using just the grey values of the point cloud. Nevertheless, the colour photos are a significant support particularly for the detailed mapping of bricks and stones. One major problem is the very detailed mapping of bricks and stones, which reduced the speed of mapping significantly. Unfortunately, the architects as the major clients could not be convinced to use digital orthophotos of the facades instead of the detailed maps in the scale of 1:200. Final product could be derived from 3D polylines as illustrated in Figure 9. Currently, the mapping of the building facades is still not finished. Figure 8 Detailed mapping of building facades based on laser scanning data and photogrammetric image Figure 9 Mapped 3D polylines of facades of a building block 4.0 3D SPATIAL DATABASE Operational 3D GIS still require extensive efforts as revealed from the recent research output and workshop (see Abdul-Rahman, et al. 2006). It was interesting to note that works on fundamental aspects like 3D spatial database management still not much have been addressed up to the level where an operational 3D system could be realized. Recent research development shows that 3D spatial modeling is becoming very important for many advanced GIS applications and the scenario is being enhanced by the advancement in computer graphics (hardware and software), visualization, etc. and also influenced by development in OpenGIS consortium (OGC, 1999;OGC, 1999a; OGC, 2001). 3D visualization environments such as Google Earth (2008) or 3D navigation software have already made some contribution where more and more users could utilize such visualization tools. Until very recently, only very specialized applications were able to manage and analyse 3D spatial data. The third dimension has been mostly used for visualization and navigation. However, we have seen users are looking for applications that have one or more 3D GIS functionality. Due to the complexity of real-world spatial objects, various kinds of representations (e.g. vector, raster, constructive solid geometry, etc.), spatial data models (topology, and geometry) have been investigated and developed, including e.g. Pilouk, 1996; Zlatanova, 2000; and Kada et al, The management of 3D spatial objects has also been discussed in Orenstein (1986), Penninga (2005), 393

13 Penninga et al. (2006), Penninga, and van Oosterom (2007), Pu (2005), Pu, and Zlatanova (2006), and Rigaux et al. (2002). A universal and practical spatial data model that capable of addressing more than one application are rather hardly available a model normally serves well for an application. One of the reasons is due to the complexity of the real world objects and situations. On the other hand, different disciplines emphasize different aspects of information e.g. including different in requirements and output. Thus, a data model could be considered good for a certain application but not so appropriate for other tasks. Different aspects and characteristics of real objects have led to the existence of several variations in object definition. The solution for these problems has directly referred to GIS standardization. Current 3D GIS offer predominantly 2D functionality with 3D visualization and navigation capability. However, promising developments were observed in the DBMS domain where more spatial data types, functions and indexing mechanism were supported (PostGIS, 2008; Oracle Spatial, 2008). In this respect, DBMS are expected to become a critical component in developing of an operational 3D GIS. However, extended research and developments are needed to achieve native 3D support at DBMS level. 4.1 Existing GEO-DBMS for 3D Modeling Existing DBMS provides a SQL schema and functions that facilitate the storage, retrieval, update, and query of collections of spatial features. Most of the existing spatial databases support the object-oriented model for representing geometries. The benefits of this model is that it support for many geometry types, including arcs, circles, and different kinds of compound objects. Therefore, geometries could be modelled in a single row and single column. The model also able to create and maintain indexes, and later on, perform spatial queries efficiently. In the next section, Oracle Spatial will be discussed, in term of their characteristics, capabilities and limitations in handling multidimensional datasets. 4.2 Oracle Spatial Oracle Spatial is designed to make spatial data management easier and more natural to users of location-enabled applications and geographic information system (GIS) applications. Once spatial data is stored in an Oracle database, it can be manipulated, retrieved, and related to all other data stored in the database. Types of spatial data (other than GIS data) that can be stored using Spatial include data from computer-aided design (CAD) and computeraided manufacturing (CAM) systems. The Spatial also stores, retrieves, updates, or queries some collection of features that have both nonspatial and spatial attributes. Examples of nonspatial attributes are name, soil_type, landuse_classification, and part_number. The spatial attribute is a coordinate geometry, or vector-based representation of the shape of the feature. Spatial supports the object-relational model for representing geometries. The object-relational model uses a table with a single column of SDO_GEOMETRY and a single row per geometry instance. The object-relational model corresponds to a "SQL with Geometry Types" implementation of spatial feature tables in the Open GIS ODBC/SQL specification for geospatial features. 4.3 Modeling 3D Solid Using MultiPolygon In the Oracle Spatial object-relational model, a 3D solid object from 3D primitive is not possible. However, it could be done by implementing the MultiPolygon that bound a solid. The geometric description of a spatial object is stored in a single row and in a single column of object type SDO_GEOMETRY in a user-defined table. Any tables that have a column of type SDO_GEOMETRY must have another column, or set of columns, that defines a unique primary key for that table. Tables of this sort are sometimes referred to as geometry tables. Oracle Spatial defines the object type SDO_GEOMETRY as: CREATE TYPE sdo_geometry AS OBJECT ( SDO_GTYPE NUMBER, SDO_SRID NUMBER, SDO_POINT SDO_POINT_TYPE, SDO_ELEM_INFO MDSYS.SDO_ELEM_INFO_ARRAY, SDO_ORDINATES MDSYS.SDO_ORDINATE_ARRAY); The advantage of implementing the multipolygon in DBMS is that the integration between CAD and GIS is possible for 3D visualization, i.e. Oracle (or called Spatial) spatial schema is supported by Bentley MicroStation (2008) and Autodesk Map 3D (2009). This is due to the geometry column provided by Spatial is directly access to the 3D coordinates of the object, which allow the display tools retrieve spatial information from the geometry column. 4.4 The Implementation of Geo-DBMS The implementation of geo-dbms approach for laser scanner data is the main objective for this study. Thus, the 3D spatial objects were stored within Oracle Spatial environment. A 3D data type, i.e. MULTIPOLYGON was implemented for the spatial object. Figure

14 denotes one of the buildings captured using the laser scanning system (TLS). Although the complete coverage of the spatial data captured for the building could be obtained or acquired via the technique, the result of 3D point-clouds do not provide any semantic related to the building itself, in terms of spatial geometry, object s behaviour (e.g. door is open from inside the building), or related attribute information (e.g. year of construction). Thus, the laser scanner data need to be processed in order to create useful spatial information. The first stage of the process is related to feature extraction and the discussion on this aspect could be found in Oude Elberink and Vosselman (2006); Teunissen (1991); Vosselman (2003). The result from object extraction and reconstruction is shown in Figure 11. The implementation of geo-dbms approach for laser scanner data starts with the database creation. A geometry table was created in order to store the spatial object (see Figure 12). The GEOMETRY column stores the important coordinate structure of 3D spatial objects. For this paper, some of the important attributes, e.g. information related to historical building, material of building construction, etc. will also be stored within the database. The geo-dbms approach could provide spatial query related to the specific condition. For this research, the spatial could be performed using SQL language as shown in Figure 18. In the following spatial query, the experiment defined all related historical building, which is constructed before the year of GEOME TRY column Figure 12 A geometry table describes all related attribute columns The methodology for the complete implementation is given in Figure 13. Raw data f l Feature (a) Object t 1). Building Implemen 3D Impleme ntation CREATE TABLE Solid3D ( ID number(11) not null, shape mdsys.sdo geometr Implem Attach (b) Figure 10 The raw dataset from laser scanner Figure 13 The workflow 5.0 THE VISUALIZATION OF RETRIEVED 3D SPATIAL OBJECTS Figure 11 Feature extraction from laser scanned data (3D view and 2D planar view Any spatial queries from database would only provide geometric and semantic information if the visualization factor is not considered. DBMS only provides a medium for the management of data set, and it certainly requires a front-end tool for visualizing for that information, as one perceives in the real world. The data from DBMS needs to be integrated into visualization tool so that it could be viewed as graphic. The 3D spatial data stores in the spatial column (within DBMS), and a connection needs to be built so that a 395

15 display tool manages to access the spatial column and retrieve the data for 3D visualization. It is also important to note that 3D objects need to be visualized in realism. With the benefit of the computer graphic technology, GIS could provide a good display with textures and colours. Some web application, e.g. Google Earth (GE) maintains the texture of spatial object over the Internet. Here, we used Autodesk Map 3D for displaying the 3D spatial object in this research. 5.1 The initial 3D display of building block only manages to provide standard colours from Map 3D library. However, 3D textures could be attached into the spatial objects. This provides the realistic appearance of 3D building blocks as appeared in various city models. Figure 16 denotes the implementation of textures into the 3D spatial object (simple building without roof). Figure 17 denotes the similar implementation for complex building (with roof). Oracle Spatial and Map 3D schemas In order to visualize Oracle spatial database successfully within the Map 3D environment, synchronization between Oracle spatial and Map 3D schema is required. However, sample spatial datasets within Oracle Spatial is given as follows: Figure 16 3D display of building blocks (without roof) Figure 14 The extraction of geometric datasets After datasets are inserted into Oracle database, the first stage of integration is to login into Map 3D. The second stage of integration is to connect Oracle schema table into Map 3D s schema administration. The Map 3D will prompt user for Oracle database login name and password. From the Oracle database lists, select the appropriate database for visualization. After the integration was included into the Map 3D, databases are imported from Oracle schema table to Map 3D. Features selection must be identified in order to display the appropriate dataset. 5.2 Figure 17 3D display of building (with roof) Real Texture Mapping Real texture mapping provides good realism of the objects and able to enhance the aesthetic appearance and the understanding of the objects such as buildings, etc. Fritsch (2003) describes the concept of texturing of objects from terrestrial images, and basically it is a process of transforming distorted image to rectified image. We used the available tools, Adobe Photoshop Paint module to rectify the distorted images as illustrated in Figure CONCLUDING REMARKS The paper had discussed the implementation of geodbms approach pertaining to the 3D spatial data modelling and management using the data generated from the laser scanning system. The discussions cover the implementation of 3D geometry from Oracle Spatial. However, there are many issues still need to be addressed in order to improve the current situation of 3D spatial modeling of objects from laser scanner data. One of the issues is objects reconstruction from the laser scanning system. At the moment, most systems only provide primitive means for object creation (i.e. manual technique). Our research experience shows that a lot of efforts still need to be done in order to create the 3D objects automatically. The other issue is related to the 3D primitives for the spatial database, e.g. Figure 15 Real texture mapping- rectification process 396

16 polyhedron for building block, and tetrahedron for terrain modeling. This aspect can be solved by implementing the user-defined data type within the geo- DBMS environment. The most important issue for 3D spatial data modeling is the standardization and specification of GIS that related to the Open Geospatial Consortium (OGC standard), from feature extraction to final 3D display. The implementation of 3D spatial operations could also be done in geo-dbms environment. The spatial operators should involve some procedures that able to use, query, create, modify, or delete spatial objects. Other challenges and issues in the near future include the interoperability between different applications, data model, the integration between DBMS and visualization, and the real-time linkage between data model and data acquisition. ACKNOWLEDGMENT Thanks to the Istanbul Metropolitan Planning Unit (IMP) and BIMTAS GIS division especially the data acquisition unit. REFERENCES Abdul-Rahman A, Zlatanova S, Coors V (2006). Lecture Note on geoinformation and cartography Innovations in 3D Geo Information Systems, Springer- Verlag. Abdul-Rahman, A., S. R. Karas, I. Baz, and G. Buyuksalih (2008). Managing 3D Geospatial Objects case study: Istanbul Historical Peninsula Region. Proceedings of International Conference on GIS for Developing Countries (GISDeco) th. June, Istanbul, Turkey. Arens, C., J. Stoter, and P. v. Oosterom (2005). Modelling of 3D spatial objects in a geo-dbms using a 3D primitive. Computers and Geosciences Journal, Vol. 31, No. 2, Elsevier, pp Autodesk Map 3D (2009). Available at Bentley MicroStation (2008). Available at Fritsch, D. (2003). Photogrammetric Week Workshop. Stuttgart, Germany Google Earth (2008). Available at Kada M, Haala N, Becker S (2006). Improving the realism of existing 3D city model. In: Abdul-Rahman A, Zlatanova S, and Coors V (eds), Lecture Note on geoinformation and cartography innovations in 3D Geo information systems, Springer-Verlag. pp OGC (1999). Abstract specifications overview. Available at OGC (1999a). OpenGIS simple features specification for SQL. Available at OGC (2001). The OpenGIS Abstract specification, topic 1: feature geometry (ISO Spatial Schema) Version 5. Oracle Spatial (2008). Available at Orenstein J (1986). Spatial query processing in an object-oriented database system. In: Proceedings of 1986 ACM SIGMOD International Conference on Management of Data, pp Oude Elberink S., and Vosselman G., (2006). Adding the third dimension to a topographic database using airborne laser scanner data. Photogrammetric Computer Vision IAPRS, Bonn, Germany. Penninga F (2005). 3D topographic data modelling: why rigidity is preferable to pragmatism. In: Spatial Information Theory, Cosit 05, Vol of Lecture Notes on Computer Science, Springer. pp Penninga F, van Oosterom PJM, Kazar BM (2006). A TEN-based DBMS approach for 3D topographic data modelling. In: Spatial Data Handling Penninga F, van Oosterom PJM (2007). A compact topological DBMS data structure for 3D topography. In: Fabrikant S, Wachowicz M (eds.), Lecture Notes in Geoinformation and Cartography. ISBN: Pilouk M (1996). Integrated modelling for 3D GIS. PhD Thesis, ITC, The Netherlands. PostGIS (2008). Available at Pu S (2005). Managing freeform curves and surfaces in a spatial DBMS. Msc Thesis, TU Delft. Pu S, Zlatanova S (2006). Integration of GIS and CAD at DBMS level. In: E. Fendel E, Rumor M (eds), Proceedings of UDMS'06 Aalborg, Denmark, TU Delft, pp Rigaux P., Scholl M, Voisard A (2002). Spatial databases - with application to GIS. Morgan Kaufmann Publishers, San Francisco. 397

17 Teunissen P.J.G. (1991). An integrity and quality control procedure for use in multi sensor integration. In: Proceedings of ION GPS-90, September 1990, pp , Colorado Springs, USA. Vosselman G., (2003). 3D reconstruction of roads and trees for city modelling. ISPRS working group III/3 workshop 3-D reconstruction from airborne laser scanner and InSAR data. IAPRS, Dresden, Germany, pp Zlatanova S (2000). 3D GIS for urban development. PhD thesis, ITC, The Netherlands. AUTHORS BIOGRAPHY Alias Abdul Rahman currently is a UTM Professor of 3D GIS and academic staff member of the Faculty of Geoinformation Science and Engineering. He received his PhD and MSc from University of Glasgow, UK in 2000 and ITC the Netherlands in 1992 respectively. 3D GIS is his main research interest and has produced two books with Springer Verlag on the subject. Currently he is a Chair for ISPRS Commission II WG 5 on Multidimensional GIS and Mobile Data Model for 2008 to Gurcan Buyuksalih is consultant for the Istanbul Greater Municipality. He was an academic member of Karaelmas University, Zonguldak, Turkey. He received his PhD from University of Glasgow in 1999 in Photogrammetry. He was a Secretary for ISPRS Commission I WG 2 from His research interest is photogrammetric and laser based mapping. 398

18 Efficient File Storage System with Human-Like Forgetting Mechanism Takumi MIYOSHI Department of Electronic Information Systems College of Systems Engineering Shibaura Institute of Technology Abstract This paper discusses a concept of novel file storage system that gradually loses saved data as men forget their memories. Someone might think that the proposed system looks strange and could be useless. However, are the saved data in your computer really necessary permanently? This might not be always true, since the users sometimes download and save data that they are interested in but do not really want. For instance, some heavy Internet users tend to collect a huge number of content files, independently of their tastes, using peer-to-peer file sharing systems. Some of such contents are tentatively downloaded and would be never played back. They would be thus junk stocks to waste the storage space in their computers. The proposed file storage system can solve this kind of problem by implementing a human-like forgetting mechanism to automatically erase such worthless data. The paper also describes examples for effective implementation. Keywords file storage system, human-like forgetting mechanism, forgetting curve I. Introduction Recent technological innovations in information and communication fields have been rapidly developed. Computing speed and network communication speed are multiplied several times for every couple of years. The amount of memories and file storages also increases steadily. We can thus receive their full benefit: various kinds of large-size contents can be easily downloaded from the Internet and stored on the storage devices. Due to convenient applications such as Web 2.0 and peer-to-peer (P2P) file sharing systems, some users tend to collect many content files independently of their tastes. Their collections include what they want to get, and even what they do not really want but have little interest in. Some of such less interesting files might be tentatively downloaded and would be never played back. They would waste the storage space in their computers like junk stocks. To utilize the storage devices efficiently, unnecessary data, and occasionally including less-used data, should be erased. Additionally, it would be preferable to automatically erase them without users awareness since the number of data is too large to operate by themselves. This paper proposes a concept of novel file storage system that is equipped with a function to automatically erase the saved data. It is based on a human-like forgetting mechanism: the data more commonly used are hardly forgotten and less-used data are gradually erased from the storage drive. Moreover, each disappearing file will lose a part of its content little by little. The rest of this paper is organized as follows. Section II describes the related work for efficient management system on storage devices. Section III explains the proposed file storage system to gradually lose lessused saved data, and also discusses the human-like forgetting mechanism. Effective implementations of the proposed system are described in Sect. IV. The conclusion and future work are given in Sect. V. II. Related Work File storing methods with the functions to automatically delete saved data have been proposed, and some of them are practically used. This section describes typical methods related to the proposed forgetting file storage system. A. Web Caching Mechanism The most typical storage system with an automatic file deleting function is Web caching [1, 2]. Once a Web site is visited by a Web browser, it is temporarily saved in a local storage area, which is called cache. When someone wants to visit the same site, the Web caching mechanism enables to display it quickly by accessing not the real on-line site but the locally cached data. Web caching mechanisms can be deployed in several ways: private caching, proxy caching, and gateway caching [2]. The first one is provided by a Web browser, and generally known and described as Temporary Internet Files in Microsoft Internet Explorer [3] or Page cache in Mozilla Firefox [4]. On the other hand, 399

19 Figure 1. LRU mechanism. latter two caching mechanisms are installed in proxy servers, which obtain Web pages through firewalls instead of users direct Web connections for network security reasons. They are used for multiple users, while the browser cache is used for a single user. Every Web caching system mentioned above has the similar policy to delete the cache. The size of caching area should be decided in advance by users or system administrators. When a new file is saved into the cache and exceeds the predetermined cache size, the least-recently used file will be deleted first. In other words, the caching system will delete the file that no one has accessed for the longest time. This mechanism is well-known as LRU as shown in Fig. 1. For running LRU, the system must collect the last access time from all cached data and select the oldest one. Since the complete LRU mechanism is expensive to be implemented, general implementations introduce age bits that represent how old the cached files are accessed among roughly sliced time periods. B. File Storage with Trash Box Mechanism A data deletion mechanism for audio file storage systems has been proposed in [5]. The method, shown in Fig. 2, is very similar to the trash box mechanism, which is often employed in operating systems and mail applications. This storage system is hereafter called FSTB (File System with Trash Box). If a user tries to delete files, they are moved to the trash box instead of the immediate deletions. For the trash box, a time until the complete deletion should be determined in advance. If the files has never been moved outside of the trash box for the predetermined time, they will be really deleted. To save a new file when the storage system is full, some files in the trash box should be deleted to create available space. This method tries to delete the oldest file in the trash box. It is similar to the Web caching mechanism mentioned above. The method also has an extension to prepare multiple trash boxes that have different time to be deleted for usability improvement. For example, the time to deletion of trash box B can be longer than trash box A. If users want to postpone their judgment to com- Figure 3. Storage system with compression and deletion mechanism. pletely delete their files, they can select trash box B rather than trash box A. It is also possible to automatically select a trash box depending on the playback frequencies. C. File Storage with Compression and Deletion In [6], a file storage system with both compression and deletion mechanisms has been proposed as shown in Fig. 3. This system is hereafter called FSCD (File System with Compression and Deletion). These mechanisms are used when a user wants to save a new file but the storage system does not have enough free space. In such a case, the least-recently used files will be compressed first for creating free space. If there remains no more compressible file, the system offers the user a suggestion that the least-recently used files should be really deleted. The remarkable point of this method is to provide a compression function. Less-used files in the storage system are not suddenly deleted but compressed. That is why the system realizes more flexible storage control than the trash box approach. Reversible compressions such as zip is used for text or document files, while lossy compression is used for large-size multimedia files such as pictures, audio and video files. In the latter case, multimedia files will be re-encoded with higher compression ratio. If all the files have been already compressed enough and the system cannot create any more space available to save a new file, the least-recently used files will be deleted after users confirmation. 400

20 Selected file will be partially erased 0.6 Experimental value 1.84 f(x) = 1.25 (log x) f(x) = x Figure 4. The proposed forgetting mechanism. III. Human-like Forgetting Storage System A. Concept This paper proposes a novel file storage system with a function to automatically lose the saved data. The proposed system is based on a human-like forgetting mechanism. The main purposes of this implementation are: (1) less-used data should be gradually erased from the storage and (2) files to be erased lose a part of its content bit by bit. The proposed forgetting mechanism is shown in Fig. 4. For the first purpose, we might refer to the conventional storage systems with deletion mechanisms as mentioned in the previous section. They are surely deterministic approaches: the files in the trash box are deleted just after the predetermined time in FSTB, and less-used files are chronologically deleted in an LRU manner in FSTB and FSCD. On the other hand, men do not forget their less-used memories in a complete LRU order. They could recollect their old memories and might easily forget recent events. Thus, the order of human forgetting is not deterministic but probabilistic. Hermann Ebbinghaus discovered the human forgetting curve more than 100 years ago [7]. He expressed that a common logarithm function with two constant values well explains the relationship between memory retention and the time since the items were memorized. It is also known that the forgetting curve better fits a power-law function. Namely, there is a linear relationship between the logarithm of a measurement of memory and the logarithm of the elapsed time [8]. Figure 5 shows experimental values of human forgetting and two approximated functions. The proposed storage system is equipped with such a probabilistic approach to efficiently lose the saved data. The second purpose brings about a trigger of change from digital to analog. When the conventional storage systems as mentioned above are about to delete files, the choices are exactly digital: to delete or not. On the other hand, human memory mechanisms are just different from it. Men sometimes forget something entirely, but sometimes partly. This phenomenon can be briefly said to be analog rather than digital. The proposed system is equipped with such an analog mechanism to gradually lose a part of files to be erased. Figure 5. The human forgetting curve. B. Detailed Mechanism There are two important points to implement a human-like forgetting mechanism to the proposed storage system: (1) when the forgetting function is called, and (2) how it loses the saved data partially. A deletion mechanism is commonly called when a new file is about to be saved but the storage system is full. It sounds logical, but it is far from a humanlike forgetting mechanism. In the proposed system, therefore, the forgetting function is called periodically, for example once a day or week, as well as when the storage is full. When the forgetting function is called, files to be forgotten are selected according to a predetermined probability density function reflecting the human forgetting curve. At that time, the last access time of each file is checked, and less-used files are more likely to be forgotten. Next, the proposed system must decide which part of the selected file should be erased. I think that the way of partial loss should depend on file types. For example, plain text or word processing documents lose some pages, paragraphs or lines, while music or video files deteriorate their quality or their playback lengths are shorten. It is possible to change the file formats or encoding such as Word/Excel/PDF to plain text and raw multimedia data to encoded data. Table 1 shows examples of partial loss processes. The processing level can be randomly chosen, or can even be changed according to the last access time of the file. Anyway, even if any level is chosen, the files to be forgotten lose their contents partially. Thus, the less-used data in the proposed storage system will fade away and finally be completely deleted from the storage. IV. Effective Implementation Since the proposed forgetting storage system is unique and strange, someone might think that it could be use- 401

21 Word document compress figures lose lines change to text format PDF file remove figures lose lines change to text format picture file reduce resolution reduce colors crop less as a storage device for computers. However, let us consider again the current Internet trends users will not easily change their convenient environment to obtain high-definition multimedia contents using Web 2.0 and P2P. Especially, heavy Internet users will surely continue to download a tremendous number of rich contents, which include both what they really want to get and what they have little interest in. In that case, the forgetting storage system must be exactly adequate to store the latter type of contents. It is quite natural that the storage should forget the saved contents as humans do, since users might forget downloading them. Thus, the first appropriate usage of the proposed system is a secondary storage for less important data files in the computers. Secondly, the human-like forgetting storage system can be effectively adopted as a shared storage, which has been growing popular since the costs of computers and networking have been lower. In shared spaces, many types of data files will be saved by many users. Once files are saved, they will almost permanently remain: users scarcely delete what others saved. From this viewpoint, the proposed storage system is well adapted to the shared storage. Frequently-used files probably survive in the storage, while less-used ones are likely to forgotten as humans do. V. Conclusions This paper proposed a human-like forgetting storage system, and also discussed the detailed mechanism and effective implementation. It is based on a probabilistic approach that partially erases less-used files as humans do. Although the proposed system might be strange, it would be inartificial depending on situations since it is men who download and save the contents. To realize the proposed system, there still remain some details to be decided. For example, a probability density function for selecting files to be forgotten should be clearly formulated reflecting the human forgetting curve. Further study for the human forgetting mechanism is required. Moreover, I will also consider the way to partially lose the saved data. References [1] D. Wessels, Web caching, O Reilly and Assoc. Inc., [2] Web cache, Wikipedia. cache [3] Internet Explorer, Microsoft. winfamily/ie/default.mspx [4] Firefox web browser, Mozilla. [5] Sony Corp., Storage mechanism and data deletion method, JP , Dec (Japanese) [6] Casio Computer Co.Ltd., Data storage mechanism and condition, JP A, March (Japanese) [7] H. Ebbinghaus, Memory: a contribution to experimental psychology, 1885/ htm [8] S. Sikström, Forgetting curves: implications for connectionist models, Cognitive Psychology, Vol. 45, Issue 1, pp , Aug Takumi MIYOSHI received his B. Eng., M. Eng., and Ph.D. Eng. degrees in electronic engineering from University of Tokyo, Japan, in 1994, 1996, and 1999, respectively. He was a visiting associate from 1994 to 1996 and an Internet technical staff from 1996 to 1997 at the Institute for Monetary and Economic Studies, Bank of Japan. He was also a research associate at Global Information and Telecommunication Institute, Waseda University, from 1999 to 2001, and a research fellow at Telecommunications Advancement Organization of Japan from 1998 to He is presently an associate professor at Department of Electronic Information Systems, College of Systems Engineering, Shibaura Institute of Technology. His research interests include multicast communication, overlay network, ad hoc and sensor networks, and e-learning system. He received the IEICE Young Investigators Award in 2004, the IEICE Information Network Research Award in 2001, 2004, and 2006, and the Ericsson Young Scientist Award in He is a member of IEEE and IEICE. 402