AUTOMATIC RECOGNITION OF RICE FIELDS FROM MULTITEMPORAL SATELLITE IMAGES

Transcription

1 AUTOMATIC RECOGNITION OF RICE FIELDS FROM MULTITEMPORAL SATELLITE IMAGES Y.H. Tseng, P.H. Hsu and I.H Chen Department of Surveying Engineering National Cheng Kung University Taiwan, Republic of China Commission III, Working Group 5 KEY WORDS: Remote Sensing, Classification, Image Interpretation, Multitemporal Images, Temporal Profile. ABSTRACT A technology for automatic recognition of rice fields by using multitemporal satellite images is proposed. The principle of this technology is applying a region-based classification by means of integrating geographical data and domain knowledge with multitemporal Images. Based on the principle, three methods of investigating the temporal NDVI profile to detect rice fields were implemented. They are Profile Matching (PM), Peak Detection (PD), and Difference Classification (DC). All of the methods were tested on a set of multitemporal SPOT XS images (12 epochs) collected during the second rice season of Comparing to the traditional supervised classification using a single image epoch, all the methods can easily improve the accuracy about 20%. The PM and PD methods can also determine the planting time of a rice field, but they do not provide a better result than the DC method. When the number of image epochs is small the PM and PD methods may not work, but the DC method works well even if there are only 2 or 3 epochs. All of the methods do not require any training data for classification. We expect that this approach will dramatically reduce the needs of human work and increase the efficiency of the rice inventory work. 1 INTRODUCTION Land-use and land-cover classification by using remote sensing imagery has been widely employed in many application fields for the past two decades. Monitoring and inventorying agricultural crops is one of the most important applications. Nevertheless, the accuracy of traditional land-use classification using satellite images is generally thought lower than the requirement of our government in monitoring the rice product of each season. This is mainly due to the fact that the traditional classification methods are developed based on single-date images. However, the land cover of an agriculture field is changing rapidly during the growing season of crops. Therefore, this paper proposes the use of multitemporal images to increase the accuracy and automation of detecting rice fields. 1.1 Background Rice-crop inventory is an important mission of the Crop Bureau of Taiwan 1. The purposes of the investigation are to monitor and control the overall rice product and to provide evidences for the government to compensate farmers when planted crops are damaged by disasters such as typhoons or floods. Field investigation was the major method to carry out the mission before Intensive manpower consumption and inefficient process were the major problems to extend the investigation to the whole island. Since 1980, photogrammetric techniques have been extensively employed for the mission. Fieldwork was dramatically reduced. However, photogrammetric procedures still require a large amount of aerial photography and manual photo interpretation. Automatic interpretation of satellite images is then considered to increase the degree of automation and 1 This organization is responsible for inventorying rice crop twice a year. efficiency [Huang, 1984]. The current inventory system is founded on a region-based photogrammetric procedure. Aerial photos will first be visually registered onto the corresponding maps of land ownership. Manual photo interpretation is then implemented to determine each land-ownership polygon is a rice field or not. Polygons of rice field will be marked and counted and their areas will be summed to estimate the rice product. This system is developed based on the fact that most boundaries of rice field coincide with the boundaries of land parcel. There are less than 1% of landownership polygons that are divided into two or more than two agricultural fields. In this case a photo interpreter can visually mark the new boundaries on the map according to the photos, so that the interpretation is still based on regions. Image texture is the principal element for a photo interpreter to identify rice fields. Misinterpretation may happen if the texture of a rice field looks similar to other kind of crops. Due to the land-cover variation of a rice field during a growing season, it is quite possible to confuse with other land covers. Interpreters have to be trained to recognize rice fields that are showing different textures on aerial photos. In order to reduce the probability of errors, the image date should be carefully selected with the considerations of the local crop calendar. However, the unpredictable weather condition commonly does not permit aerial photography to be collected on schedule. Under this circumstance, less reliable data may be obtained. 1.2 The Proposed Idea In order to reduce the cost of the inventory work, using satellite images is proposed to avoid taking aerial photos. Further, an automatic image interpretation should be developed to reduce the need of manual work. The proposed method is a region-based classification by means of integrating land-ownership data and domain knowledge with multitemporal SPOT imagery.

2 The concept of integrating geographical data and remotely sensed images to perform a region-based classification has been promoted by some articles (Johnsson, 1994; Derenyi & Tuerker, 1996). In this application the use of land-ownership data is suggested, because the inventory data related to landowners are more preferable than the data related to image pixels. Janssen et al. (1990) and Zhuang et al. (1991) also reported that the classification accuracy could be improved when a region-based classification is used. In order to use multitemporal images properly, the consideration of domain knowledge is also critical to the classification of remotely sensed images [Argialas and Harlow, 1990]. The knowledge of rice crop including the local crop calendar, time table of rice growing, as well as the variations of spectral reflectance within a rice season should be taken into account. With the knowledge, the spectral variation derived from a time series of multitemporal images becomes meaningful and could be used to detect rice fields automatically. This idea is similar to the use of temporal profiles or temporal differences of spectral properties in image classification proposed by Brisco & Brown [1995], Lo et al. [1986], and Wolter et al. [1995]. Three methods of automatic rice detection were implemented based on the idea. The proposed methods were tested on a set of multitemporal SPOT images. The results show that all of the methods gain two advantages over the classification of using a single image epoch. First, the new methods do not need any training data. Second, the classification accuracy could be improved for about 20%. 2 THEORIES 2.1 Region-Based Classification Two methods have been proposed for region-based classification [Derenyi and Tuerker, 1996]: 1. Performing per-pixel classification first and determining the region class by means of a frequency table; 2. Determining the region class by means of regional statistic measures of spectral data. The first method is basically a modified per-pixel classification. Traditional pixel-by-pixel statistic classification is performed in advance. After that, with respect to each predefined region, the pixels of each class are counted to form a frequency table. The class of a region is then determined by analyzing the frequency table [Janssen et al., 1990]. The easiest way is simply assigning the region class to be the class with largest frequency. However, if there is no dominant class in a region, we may need to consider it a multi-class region. The second method is to calculate statistic measures, such as central tendency, dispersion, and shape of distribution, for the image pixels within each predefined region [Johnsson, 1996]. Then the classification for regions can be done based on these statistic measures. For example, one could treat the spectral mean values of a region as a set of spectral measurements to perform a supervised or unsupervised classification. For this application, we prefer to the second method, because it is a one-step procedure and requires less amount of computation. Furthermore, comparing to the first method, it is a more objectoriented procedure, which makes the procedure more understandable to applicants. 2.2 The Use of Land-ownership Data In order to perform a region-based classification, the applied images should be segmented into regions. Further, because the unit of inventory is land parcel, land-ownership data are appropriate to defining regions. The classification requires that a single spectral class should dominate each region. In most cases, this requirement can be satisfied, because the boundaries of land parcel mostly coincide with field boundaries and each field usually planted only one kind of crop. The exceptions are usually less than 1% of the total number of fields in an agricultural area. There are two steps to segment a satellite image into regions based on the corresponding land-ownership data. First, the image should be registered to the vector data of land ownership. Second, extracting and storing the spectral data of pixels within each polygon as a data set. In order to determine that in which polygon a pixel locates, one can use the center point of the pixel to perform a point-in-polygon check. This procedure can also avoid that a pixel is assigned to more than one polygon. The data arrangement mentioned above transforms raster data structure to object-based data structure. All of the data in a data set are associated with a single region. It is then easier for us to calculate statistic measures of spectral data for each region and to attach the statistic measures to the data set. It is also convenient to add other attribute data for further applications. Furthermore, such data arrangement is suitable to the integration of remote sensing and geographical information system. 2.3 The Use of Domain Knowledge Discipline specific knowledge including information about spectral, temporal, and structural properties of objects is critical to many inventorying and monitoring works using remote sensing. For rice inventory using remote sensing imagery, the following items are the domain knowledge should be taken into account: z The local crop calendar; z The time table of rice growing; z The variations of spectral reflectance of a rice field within a rice season. Most rice fields in Taiwan have two planting seasons a year. Each period of rice season is about four months. In general, the first season starts in spring and harvests in summer. The second season usually begins right after the end of the first season and harvests before winter is coming. Usually the fields in an agricultural area have similar calendars of planting. However, different agriculture areas may have time differences of rice season up to 2 months. It is quite clear that having the knowledge about the local crop calendar is the first requirement of using multitemporal images to detect rice fields. Basically the whole season of the rice growing can be divided into 5 periods: 1. Transplanting: fields are covered by water with little vegetation; 2. Growing: fields are getting vegetation entropy; 3. Reproducing: vegetation entropy reaches the maximum and starts to decrease slowly; 4. Mellowing: vegetation entropy continuously decreases a little; 5. Harvesting: fields become bare soil with a little crop residue. One should keep in mind that the land cover of a rice field is changing during a rice season. On the one hand, knowing this phenomenon is the requirement of choosing appropriate image epochs. On the other hand, this knowledge gives us the clue to distinguish rice fields from other land-use types by using

3 multitemporal images. Knowing the timetable of rice growing, a remote-sensing expert would not have difficulties to imagining the variation of spectral reflectance of a rice field within a rice season. The pattern of the temporal spectral variation of a rice field provides solid evidence to identify rice fields. This is the most important reason that we suggest the use of multitemporal images. 2.4 The Use of Multitemporal Images Using multitemporal images is helpful in many applications of image interpretation. In this study, there are two uses of multitemporal images: 1. Increasing the potential of differentiating rice fields from other land-use types; 2. Reducing the effect of clouded area. The first use is applied based on the domain knowledge mentioned above. Due to the land-cover variation of a rice field within a planting season, the difficulties can be expected in recognizing rice fields by using a single image epoch. If one uses images taken during the period of transplanting, one would not distinguish rice fields from wet lands. Similarly, using images of the growing period we may confuse with other vegetation areas, and using images taken after harvesting we will not be able to differentiate them from regions of bare soil. However, by investigating the variation of spectral responses of multitemporal images, we can minimize the possibility of misinterpreting. Checking temporal variation of spectral responses has proven effective in agricultural land-cover and forest classifications [Lo et al., 1986 and Wolter et al., 1995]. Lo et al. analyzed temporal profiles of ratio or green vegetation index to identify crop types. Based on the similar idea, Wolter et al. utilized temporal NDVI (Normalized Difference of Vegetation Index) differences to distinguish forest types. Therefore the temporal NDVI profile in a rice season is investigated. According to the report of Huang et al. [1985] and some sample data, the temporal NDVI profile of a rice field can be described as in figure 1. Some useful information could be extracted from the temporal profile for distinguishing rice fields, for example the curve shape of the profile or the NDVI differences among image epochs. Vegetation Index Growing Reproducing Mellowing Harvesting SPOT images. They are named profile matching, peak detection, and differences classification. Profile Matching (PM) An expected temporal profile similar to the curve in the figure 1 could be obtained, if an agricultural field is planted rice. By comparing the expected profile with each temporal profile of the agricultural fields, one could distinguish rice fields from the other agricultural fields. Automatic comparison can be completed by matching the curves using the cross-correlation method (figure 2). The threshold value of the cross-correlation function to classify fields into rice and non-rice should determined in advance. Evenly distributive image epochs over the rice season are required to obtain a good result. In addition to detecting rice fields, this method could also determine the planting time of a rice field. NDVI Rice Season A Rice Season B Figure 2. The temporal profile matching using the crosscorrelation method. Peak Detection (PD) The temporal profile of the vegetation index in a rice season is a curve of a mountain shape. The peak represents a growing of vegetation in the field. If a peak that locates within a rice season and whose bottom spans about a time period of a rice-growing cycle is detected in the temporal profile of a field, this field has the evidence of being planted rice. To detect the peak, one can set a horizontal line to intersect with the temporal profile, so that a pair of intersections defines a peak which width and height are estimated as in figure 3. There is no rigorous rule to set the NDVI level of the horizontal line. In general, it should be a little bit larger than the NDVI of bare-soil pixels. At least three image epochs those distribute in the beginning, middle, and end of the rice season are required. This method could also roughly determine the planting time of a rice Transplanting Time of Rice Season (Month) Figure 1. The expected temporal profile of vegetation index in a rice season. The star symbols represent the values of sample data. The second use of multitemporal images is decreasing occlusions of cloud. It is in common that images are partially clouded in Taiwan, so that we frequently need to combine more than one image to get a complete visibility. In order to reduce the effect of land-cover changes, the images to be combined should be taken closely in date. 2.5 Automatic Detection of Rice Fields Three methods were developed to recognize rice fields from field. Heigh Width Figure 3. The method of peak detection. Difference Classification (DC) Instead of checking the shape of the temporal profile, one could also take the NDVI differences between some specific epochs as the features to classify the fields. If a rice season is equally divided into 3 periods, the NDVI

4 of an image epoch in the second period should be larger than that in the first or the third periods. Therefore, the NDVI obtained from an image epoch of the second period minus that of the first or the third periods will provide appropriate NDVI differences for the classification. The features of such NDVI differences will increase the separability between the polygons of rice and nonrice (figure 4), so that it improves the classification accuracy. Positive NDVI differences will be obtained if a field is planted rice. Unsupervised classifying the fields into two groups, the group has the larger NDVI differences should belong to the class of rice field. At least two images that provide an appropriate NDVI difference are required. size. The use of multitemporal Landsat TM images was also considered. However, although Landsat TM images provide better spectral resolution than SPOT images, the image repeatability frequency is generally too low to fit this application. 3.2 Land-Ownership Data and Reference Data The land-ownership data were digitized from a land-parcel map provided by the Crop Bureau of Taiwan. The data contain 2874 polygons of land parcel. Figure 6 shows the land-parcel map. These data provide us the guide to divide the images into the regions with respect to the agricultural fields. Frequency Non-rice Rice NDVI 1 Frequency Non-rice Rice NDVI 2 Frequency Non-rice Rice 1 km Figure 4. NDVI difference increases the separability between rice and non-rice fields. 3 THE STUDY SITE The study site is located in Tainan County, Taiwan. It is a typical agricultural area in Taiwan. The area of the study site is about 7.7 km 2. The average size of the agricultural fields is about 3000 m 2. Rice crops are the major agricultural products in this area. Usually there are more than 70% of the fields being planted rice crops. 3.1 Applied Images NDVI 2 -NDVI 1 Twelve multitemporal SPOT XS images were studied. The images were selected referring to the local calendar of the second rice season in The images are shown with their identification numbers and the collecting dates in figure 5. The land-cover changes are obviously presented in the time series of images. The images were selected from the product list of the collected images by the Center for Space and Remote Sensing Research, National Central University, Republic of China. All of the images were registered to the land-parcel map of the study area, and were resampled to the resolution of 12.5m by 12.5m pixel Figure 6. The land-parcel map of the study site. The Crop Bureau of Taiwan also provides us the data of the ground truth, which will be used to evaluate the results of our experiments. The data were generated by means of manual interpretation of aerial photographs. The data contents are attributes of land parcels that indicate whether a land parcel is a rice field or not. However, some land parcels may be divided into several parts and plant more than one type of crop. Under this circumstance, the attribute becomes a mixed field. Because there are few mixed fields and most of the land-parcel areas are too small to allow us to judge whether a land parcel is a mixed field or not using a SPOT image, we treat all fields as pure fields. If there is more than 50% area of a mixed field planted rice, it is assigned to be a rice field otherwise it is treated as a non-rice field. In the second rice season of 1993, there were 2043 rice fields, 737 non-rice fields, and 94 mixed fields. 4 EXPERIMENTS 4.1 Single-Image Classification In order to provide a base of comparison for the proposed methods, the traditional supervised classification using a single image was carried out first. The test image is I93-6, which was collected when the rice was grown up. A pixel-based classification and a region-based classification were performed by

5 I /07/23 I /06/30 I /07/13 I /07/23 I /08/15 I /08/16 I /09/11 I /10/12 I /10/22 I /10/28 I /12/03 I /12/08 I /12/13 Figure 5. The multitemporal SPOT-XS images used for the study. using the maximum-likelihood classifier. Table 1 shows the classification accuracy of the two tests. Although using the region-based classification provides higher accuracy than using the pixel-based classification, the both tests reveal the fact that it is inefficient to distinguish rice and non-rice fields by using a single image. Table 1. Accuracy assessment for single-image classification. User s acc. User s acc. Overall k hat of rice of non-rice accuracy Pixel based 72.7% 32.1% 62.3% 4.6% Region based 69.7% 87.7% 74.4% 46.6% 4.2 Multitemporal-Image Classification Rice(ref) - Rice Non-rice(ref) - Rice Figure 7. A graphical presentation of the detection results using the DC method. Rice(ref) - Non-rice Non-rice(ref) - Non-rice Profile Matching All of the test images were used to perform the profile matching. The threshold value of the cross-correlation function was 0.9. The detection accuracy is shown in table 2. Peak Detection All of the test images were also used. The threshold values of the peak height and width for detecting rice fields were 40 and 100 (days). The detection accuracy is shown in

6 table 2. Difference Classification Three tests were completed for difference classification with respect to the uses of 2, 3, and 4 images. The first test uses the difference of (NDVI I93-6 -NDV II93-2); The second test uses the differences of (NDVI I93-6 -NDV II93-2 ) and (NDVI I93-6 -NDV II93-10 ); And the third test uses the differences of (NDVI I93-6 -NDV II93-2 ), (NDVI I93-6 -NDV II93-10 ), and (NDVI I NDV II93-11 ). The accuracy was improved when the number of the features of NDVI difference was increased. Table 2. Accuracy assessment for multitemporal-image classification. User s acc. User s acc. Overall k hat of rice of non-rice accuracy PM 95.3% 53.3% 84.1% 54.4% PD 79.9% 83.0% 80.7% 70.3% DC (2 images) 89.8% 70.7% 84.7% 60.7% DC (3 images) 90.3% 75.3% 86.3% 65.2% DC (4 images) 91.3% 76.8% 87.5% 67.9% 4.3 Presentation of Results The results of the region-base rice detection can be easily put into a geographical information system. A graphical presentation of the detection results and the correctness of detection can be shown as figure 7 by using a GIS tool. 5 CONCLUSIONS A technology for automatic recognition of rice fields by using multitemporal satellite images is proposed. The principle of this technology is applying a region-based classification by means of integrating geographical data and domain knowledge with multitemporal Images. This approach has several advantages in comparing to standard classification approaches: z The region-based classification directly generates inventory data related to land owners; z Referring to the domain knowledge, this approach will automatically identify rice fields by using multitemporal satellite images without the need of training data; z The use of multitemporal images will also increase the classification accuracy; z The region-based classification is more efficient in computation. According to the experimental tests, this approach has the potential to generate a comparable accuracy to the traditional photogrammetric approach. Comparing to the traditional supervised classification using a single image epoch, all the methods can easily improve the accuracy about 20%. The PM and PD methods can also determine the planting time of a rice field, but they do not provide a better result than the DC method. When the number of image epochs is small the PM and PD methods may not work, but the DC method works well even if there are only 2 or 3 epochs. Comparing to the current inventory work, this approach will dramatically reduce the needs of human work and increase the efficiency of the inventory work. The use of multitemporal images can be extended to check the number of rice seasons of a year by investigating an annual temporal profile. If the pattern of temporal profile can be linked with other crop knowledge this technology would then be possibly used to recognize various crops. It would also be a very interesting topic that if the temporal relationships are implemented in a knowledge-based classification of satellite images. ACKNOWLEDGMENTS The authors are appreciated that this research project was sponsored by the National Science Council of the Republic of China under the grants of NSC E and NSC E We also would like to thank the Crop Bureau of Taiwan for providing us the test data. REFERENCES [Argialas and Harlow, 1990] Argialas, D.P., and C.A. Harlow, Computational Image Interpretation Models: An Overview and a Perspective, Photogrammetric Engineering and Remote Sensing, Vol. 56, No. 6, pp [Brisco & Brown, 1995] Brisco, B., and R.J. Brown, Multidate SAR/TM Synergism for Crop Classification in Western Canada, Photogrammetric Engineering and Remote Sensing, Vol. 61, No. 8, pp [Derenyi & Tuerker, 1996] Derenyi, E. and M. Tuerker, Polygon Based Analysis of Remotely Sensed Images in an Integrated Geographic Information System, International Archives of Photogrammetry and remote Sensing, Vol. 31, Part B4, Commission IV, pp [Huang, 1984] Huang, T.L., Y.L. Liao, and Z.F. Her, The Applications of Remote Sensing Technologies in the Investigation of Agricultural Resources and Land Use Classification, Agricultural & Forestry Aerial Survey Institute, Taiwan Forestry Bureau, Bulletin No.10. [Huang et al., 1985] Huang, T.L., Y.L. Liao, and S.C. Wang, Study on the Spectral Characteristics of Rice Crops, Agricultural & Forestry Aerial Survey Institute, Taiwan Forestry Bureau, Bulletin No.12. [Janssen et al., 1990] Janssen, L.L., M.N. Jaarsma, and E.T. van der Linden, Integrating Topographic Data with Remote Sensing for Land-Cover Classification, Photogrammetric Engineering and Remote Sensing, Vol. 56, No. 11, pp [Johnsson, 1994] Johnsson, K., Segment-Based Land-Use Classification from SPOT Satellite Data, Photogrammetric Engineering and Remote Sensing, Vol. 60, No. 1, pp [Johnsson, 1996] Johnsson, K., Generalization of Image Data to GIS Polygons for Change Detection and Data Base Revision, International Archives of Photogrammetry and remote Sensing, Vol. 31, Part B4, Commission IV, pp [Lo et al., 1986] Lo, T.H., F.L. Scarpace, and T.M. Lillesand, Use of Multitemporal Spectral Profiles in Agricultural Land-Cover Classification, Photogrammetric Engineering and Remote Sensing, Vol. 52, No. 4, pp [Wolter et al., 1995] Wolter, P.T., D.J. Mladenoff, G.E. Host, and T.R. Crow, Improving Forest Classification in the Northern Lake States Using Multi-Temporal Landsat Imagery, Photogrammetric Engineering and Remote Sensing, Vol. 61, No. 9, pp [Zhuang et al., 1991] Zhuang, X., B.A. Engel, M.F. Baumgardner, and P.H. Swain, Improving Classification of Crop Residues Using Digital Land Ownership Data and Landsat TM Imagery, Photogrammetric Engineering and Remote Sensing, Vol. 57, No. 11, pp