An object-based approach for tree species extraction from digital orthophoto maps

Tree segmentation is an active and ongoing research area in the field of photogrammetry and remote sensing. It is more challenging due to both intra-class and inter-class similarities among various tree species. In this study, we exploited various statistical features for extraction of hazelnut trees from 1:5000 scaled digital orthophoto maps. Initially, the non-vegetation areas were eliminated using traditional normalized difference vegetation index (NDVI) followed by application of mean shift segmentation for transforming the pixels into meaningful homogeneous objects. In order to eliminate false positives, morphological opening and closing was employed on candidate objects. A number of heuristics were also derived to eliminate unwanted effects such as shadow and bounding box aspect ratios, before passing them into the classification stage. Finally, a knowledge based decision tree was constructed to distinguish the hazelnut trees from rest of objects which include manmade objects and other type of vegetation. We evaluated the proposed methodology on 10 sample orthophoto maps obtained from Giresun province in Turkey. The manually digitized hazelnut tree boundaries were taken as reference data for accuracy assessment. Both manually digitized and segmented tree borders were converted into binary images and the differences were calculated. According to the obtained results, the proposed methodology obtained an overall accuracy of more than 85% for all sample images.


Introduction
Photogrammetry and remote sensing techniques have widely been used to obtain spatial distribution information of tree species over large geographic areas.The information can be used for better understanding of the tree species ecology and their contribution to the ecosystem functions (Fassnacht et al., 2016).Moreover, various applications, such as forest management (Schultz, Verbesselt, Avitabile, Souza, & Herold, 2016), biodiversity monitoring (van Ewijk, Randin, Treitz, & Scott, 2014), environment monitoring (Zhou, Divakarla, Liu, Weng, & Goldberg, 2016) etc. are the key drivers that demand the development of new techniques that need both photogrammetry and remote sensing data.This is especially true where data collection using surveys is costly and time consuming over large geographic areas.The reflectance information obtained by sensors provides useful clue about characteristics of objects.Since, each object behave differently to the electromagnetic radiations due to their intrinsic dielectric properties, it produces a different response signal (W.Zhang et al., 2008).This response can be used to distinguish between various types of land surface objects such as build area, vegetation, bare land, water bodies etc.These characteristics have been successfully employed for forest cover type classification e.g.(Immitzer, Atzberger, & Koukal, 2012;Laliberte & Rango, 2006;Y. Zhang, 2001).Vegetation information can be obtained from multispectral images using NDVI index (Adhikari, Heiskanen, Maeda, & Pellikka, 2015).It is a simple index that uses the reflectance information in the red (R) and near infrared (NIR) portions of the spectrum to find the vegetation information.Higher values indicate high vegetation and lower values indicate low or lack of vegetation.This method helps suppress background reflectance and improve the accuracy in extracting vegetation information.Researchers have also employed other types of indices such as greenness (K.Zhang & Hu, 2012), Reduced Simple Ratio (RSR) (Majd et al., 2013;Korhonen et al., 2013;Wu, 2011), various ratios between bands (Pena-Barragana et al., 2004).Generally, the classification of the remotely sensed data can be done either by pixel-or object-based approaches (Blaschke et al., 2014).In pixel-based approach, each pixel is treated as unit of measurement and it is classified into respective classes based on features extracted for each pixel.On the other hand, the object-based approaches first group the pixels into segments based on their spatial relationships with neighboring pixels and then classify these objects into their corresponding classes.The later approach has several added advantages over pixel-based methods.First, it reduces within class variations so that each segment can represent a real world object.Second, further information can be derived from the segments which is not possible with pixel based approach, such as contextual, spatial and geometrical information (Qin, 2014).This made the object-based approach a strong candidate for analysis of remote sensing images.Turkey is among the world's largest Hazelnut producer and it shares high percentage of the total volume of its production across the globe (Europe, 2010).A study concerning the estimation of the area covered by hazelnut trees and its production is important for the top level management in various decision making processes.No study has been conducted for hazelnut tree classification using digital orthophoto maps in Turkey so far to the best of our knowledge.In this study, 1:5000 scaled digital orthophoto maps are employed for identifying hazelnut trees.The data was obtained from Giresun province in Turkey.Since, hazelnut trees are relatively large in size and usually found in groups, therefore, our objective is detecting the groups of trees.First, the vegetation areas are obtained by applying NDVI.Second, the NDVI images are normalized to the range 0-256 and mean shift algorithm was applied to obtain segments based on their spectral similarity.Third, morphological processing and geometrical constraints were applied to remove noisy objects.Finally, a knowledge-based decision tree was constructed to classify hazelnut trees from rest of classes.A manual editing step was also added with the final release of the software as an optional step to improve the classification accuracy by manually selecting and add/delete objects.

Study Area
The study area considered is in central district of Giresun province, which is located in the Northeastern part of Turkey (41°01′29″N 40°31′20″E).The province is known for high quality hazelnut production.The region is generally characterized by relatively high temperature in summer and higher rain fall.The landscape contains both valleys and mountains and it is mostly covered with greenery with both agricultural and forest belts.It produces high quality hazelnuts which are supplied in different part of Turkey and also exported to other countries.In addition, there are other types of vegetation such as grassy lands, other types of trees etc.Moreover, build areas, roads, and bare land can also be seen in the images.Fig. 1 shows the map of the Giresun in (a) and (b) shows a sample image of hazelnut tree area from the study area.

Material and Methods
The aim of this study is to automatically extract borders of hazelnut tree areas from digital orthophoto maps.The data was obtained on April 06, 2013.The images were preprocessed and scaled to 1:5000.The input images were first smoothed with Gaussian filter and then NDVI index was derived from the image.The output images were segmented by applying mean shift clustering algorithm.For each segment a set of features were extracted.Finally, a knowledge-based decision tree was constructed to classify the segments into hazelnut or non-hazelnut trees.Following sections describe the proposed methodology in details.

Image segmentation
NDVI provides a measure of the greenness information by exploiting the near infrared (NIR) and red (R) bands of the multispectral imagery.In the current study, the NDVI measure was applied to obtained vegetation areas.These vegetation areas were further refined to obtain hazelnut tree areas.NDVI values usually lie between -1 and 1, therefore, these values were normalized to 0-255.Higher values indicate high vegetation while lower values indicate lack of vegetation.For the current study, the NDVI values less than 0.0 were discarded since they represent nonvegetation areas.Image segmentation techniques were employed to partition an image into homogeneous segments.The present study proposed an object-based image segmentation technique using mean shift clustering method.The idea is to transform the pixels into an object based representation, which characterize the homogeneous regions in the image (Mustafa & Habeeb, 2014).Mean shift clustering was first proposed by (Fukunaga & Hostetler, 1975), which is a robust nonparametric approach and it can be used for various application such as mode seeking, clustering, and image segmentation (Qin, 2014).It assume the feature space to be a probability density function and finds its local maxima (Comaniciu & Meer, 2002).Suppose data points be represented by  , where i=1,2,…n, then the kernel density estimation function can be written as (1) Here K(x) represents the kernel function that measures correlation between the data point in   ,   is the centroid, h represents bandwidth parameter, and c is a normalization constant.The image is represented as 2D grid of p dimensional pixels (Comaniciu & Meer, 2002).The multivariate kernel can be estimated by: (2) Where ℎ  represent spectral and ℎ  represent spatial domain.For the current study, mean shift parameter were empirically calculated that produced optimal results.The application of mean shift algorithm on NDVI image resulted in an object-based representation of image pixels.Since, the trees in image are compact and appear in groups, so morphological and geometrical filters were applied to remove relatively small isolated objects such as small trees and other types of vegetation.Morphological dilation and erosion was performed on each image in sequence by using a structuring element of size 7 x 7 which was empirically calculated.Various sizes of structuring elements were considered.Higher values resulted in merging noisy elements into hazelnut tree candidate objects and introduced more noise.Very small values did not suppress the noise.Geometrical constraints were then applied to remove object with small size which are not part of hazelnut trees.Objects with area less than 500 pixels, width and height of the bounding box less than 100 pixels were removed.This step resulted in a binary image as output containing white pixels representing candidate hazelnut trees while black pixels represented non-hazelnut trees.

Decision Tree-based Hazelnut Tree classification
Decision tree based classification is a highly efficient approach for tree species classification (K.Zhang & Hu, 2012).In the current study, a knowledge based decision tree was constructed and applied to obtain hazelnut tree objects.Both spectral and textural features were extracted from each object to distinguish between various land cover classes.The spectral information was obtained from all available bands (NIR, R, G, B) and NDVI index image.The spectral values for each object were calculated and averaged as representative feature set for each object.The obtained features were then fed into the constructed decision tree.At first level, NDVI values were used to distinguish between the vegetation and non-vegetation areas.The NDVI values greater than 0.1 were classified as vegetation areas while others as non-vegetation areas.
The vegetation areas were further refined to classify the hazelnut trees.Decision was made on each vegetation object based on their spectral and textural values.If the average spectral values for each object fall in the defined range, then the object is classified as hazelnut tree object, otherwise it is discarded.Fig. 3 shows the constructed decision tree for classification of hazelnut trees.All the segmented objects obtained from previous step are fed into the decision tree one by one.Based on their spectral features, the objects were classified into their respective classes.The leaves of the tree represent the classes while nodes are processing components which take the objects as input.For this study, two classes were considered, one is hazelnut tree and non-hazelnut tree objects (including non-vegetation and other types of vegetation).For the final classification the threshold value   is calculated according to the conditions defined in Table 1.Where   ,   ,   ,   represent mean spectral value for i th object in red, green, blue and near infrared bands respectively.
Fig. 3 Construction of decision tree for hazelnut tree classification

Experimental Results
In this study, a dataset consisting of 10 digital orthophoto maps was used for evaluation of the proposed methodology.Each image was first smoothed by applying Gaussian filter to reduce the impact of noise.All available bands (R, G, B, and NIR) as well as NDVI index were used for classification.Mean shift was employed as an object-based segmentation technique for hazelnut tree species.The decision tree classifier produced a binary image, where white pixels corresponds to hazelnut tree areas and black pixels corresponds to rest of vegetation and classes.Fig. 4 shows the results obtained for hazelnut tree classification.The classification accuracy obtained for hazelnut tree classification is summarized in Table 2.An area-based accuracy assessment was performed to evaluate the accuracy of the proposed method proposed by (Whiteside, Maier, & Boggs, 2014).The ground truth data was prepared by geospatial analysts by manually drawing the boundaries around hazelnut tree areas.These images were binarized and then compared with the output of the proposed method.The evaluation was based on overall accuracy (OA), users' accuracy (UA), producer's accuracy (PA), false positives (FA) and false negatives (FN) as shown in Table 3. Various experiments were performed to obtain the parameters required for mean shift segmentation and decision tree classifier.The optimal bandwidth parameters (ℎ  ,ℎ  )) for mean shift were selected that produced good segmentation results.It was observed that higher values for ℎ  and ℎ  tend to merge the cluster there by producing undesirable results.Moreover, smaller values were unable to merge hazelnut tree regions into possible segments.For the current study the bandwidth parameters were empirically calculated that produced optimal results and were set as (ℎ  =9 , ℎ  =7 ).
The decision tree employed two decision steps.At initial step, a threshold on NDVI values was applied to distinguish vegetation and non-vegetation candidate objects.The threshold value was empirically calculated and set to 0.1.The objects satisfying the first condition were evaluated at step 2. We empirically calculated the possible spectral ranges for hazelnut trees from each band.Fig. 5 shows the spectral range of data in each band for hazelnut trees.Based on these values a range of possible gray values was defined in each band.The classifier correctly classified the candidate objects into hazelnut trees if their mean gray values satisfy the threshold conditions defined for each band.As it can be observed from the box plot in Fig. 5 that there is overlap between spectral signature in blue and near infrared bands for both hazelnut and non-hazelnut areas.To compensate for this problem, NDVI values were also used in the classifier which reduced the possible miss classifications.However, there were some still false positives due to similarity between two classes, especially in the vegetation areas.Therefore, an additional manual editing step was also included to overcome the miss classifications and improve the overall accuracy of the proposed method.
Where C represents the hazelnut tree area obtained by proposed method and G is the ground truth data.It was observed that some other tree species were misclassified as hazelnut trees.In fact, they exhibit similar spectral and textural properties that make the problem even more complex.This problem is alleviated by introducing a manual editing process in the software.Users will be able to select the misclassified regions in the output image and then able to delete/add it from final classification output.This way the overall accuracy of the proposed method can be improved while reducing the misclassifications.The proposed method classified hazelnut trees efficiently.However, in some situation it faced challenges to distinguish between hazelnut trees and other types of vegetation when the spectral signatures matched.In this case, manual editing was done using the software.The falsely classified objects were manually selected by the user and then add or delete them according to their needs.

Conclusions
In this work, an object based approach was employed for delineation of hazelnut tree species from 1:5000 scaled digital orthophoto maps obtained from Giresun province in Turkey.A Gaussian smoothing filter was first applied to the input image as a preprocessing step before further processing was done.Then, vegetation areas were identified by employing NDVI index and mean shift algorithm was applied to obtain segments from the NDVI image.These segments were then used as building blocks for further classification.Moreover, morphological processing and geometrical constraints were applied to remove non-candidate hazelnut trees.Both textural and spectral features were extracted from each object.Finally, a knowledge-based decision tree was constructed to classify objects into hazelnut trees and rest of vegetation.An additional optional step was also included in the software to allow the users to manually edit the output of the classification step to reduce misclassifications.Experiments were performed to assess the performance of the proposed system on the dataset of 10 orthophoto maps.An area-based accuracy assessment showed that the proposed method is effective for hazelnut tree classification.
In future, we would like to integrated support vector machine into the object-based approach for classification.By including the SVM, the approach can further be extended for multiclass classification problem.Moreover, additional features can be extracted such as, shape features, for training and classification.The classification accuracy can be improved and the method can be adopted for any land cover classification problems.

Acknowledgements
This study is a part of TEYDEP Project entitled "Development of Object Based Neural Network Image Processing System Determination of Vegetation and

Fig. 1 .
Fig. 1.Study area.a) Map of Giresun province in Turkey.b) A sample image (R, G, B) showing hazelnut trees

Fig. 4 .
Fig. 4. Final results obtained from proposed method.(a) Sample input image (NIR, R, G) (b) Classified image (c) Classified image overlaid on input image (d) Contours drawn around hazelnut trees

Fig. 5
Fig. 5 Gray value distribution for each band from sample image in (a) vegetation area (b) non-vegetation area