Semantic Concept Detection Using Weighted Discretization Multiple Correspondence Analysis for Disaster Information Management

Transcription

1 Semanti Conept Detetion Using Weighted Disretization Multiple Correspondene Analysis for Disaster Information Management Samira Pouyanfar and Shu-Ching Chen Shool of Computing and Information Sienes Florida International University Miami, FL 3399, USA {spouy, Abstrat Multimedia semanti onept detetion is an emerging researh area in reent years. One of the prominent hallenges in multimedia onept detetion is data imbalane. In this study, a multimedia data mining framework for interesting onept detetion in videos is presented. First, the Minimum Desription Length (MDL) disretization algorithm is extended to handle the imbalaned data. Thereafter, a novel Weighted Disretization Multiple Correspondene Analysis (WD-MCA) algorithm based on the Multiple Correspondene Analysis (MCA) approah is proposed to maximize the orrelation between the feature value pairs and onept lasses by inorporating the disretization information aptured from the MDL module. The proposed framework ahieves promising performane to videos ontaining disaster events. The experimental results demonstrate the effetiveness of the WD-MCA algorithm, speifially for imbalaned datasets, ompared to several existing methods. Keywords-Weighted disretization; Multiple Correspondene Analysis (MCA); imbalaned data; video onept detetion; disaster information management I. INTRODUCTION Nowadays, multimedia data onsisting of audio, text, image, and video has grown tremendously [][][3][4][5]. Soial networks suh as Faebook, Instagram, and Twitter as well as multimedia sharing websites inluding YouTube, Flikr, SlideShare, et. are the main soures of multimedia data widely used by ordinary users and even sientists for researh purposes. With suh an inrease in the amount of multimedia data through the Internet, the main question raised is how one an analyze this high volume and variety of data in an effiient and effetive way. To answer this question, many researh studies have been done reently in multimedia big data analysis [6][7]. Among various multimedia appliations, video onept detetion has attrated lots of attention in both aademia and industry due to the rih ontent and information in the videos [8][9]. In the literature, various data mining approahes have been proposed to detet onepts and interesting events in videos [][][][][3]. Example lassifiers inlude neural networks [4], deision trees [5], Multiple Correspondene Analysis [6], et. However, one main remaining hallenge is that of bridging the gap between the low-level visual features and the high level onepts in the videos. Another ritial hallenge in multimedia data is how to proess data with skewed distributions or in other words, the imbalaned datasets. This an be seen ommonly in real world multimedia appliations where the lasses are not distributed uniformly [7][8][9][][][]. There are usually two lasses: the major lasses (or alled the negative lasses) and the minor one (or alled the positive lass), where we are more interested in deteting the minor lass. For instane, in medial lab results, aner instanes are rare but more important than those instanes for regular diseases. Other appliations of imbalaned data are fraud ativities detetion, bomb detetion, failure preditions of tehnial equipment, et. [3][4]. In suh onditions, onventional mahine learning and data mining algorithms often fail to detet the minor lass, and they are biased toward the negative lasses, whih may have serious effets. Suppose an instane of a medial lab result is predited as non-aner (a negative lass), while in reality the patient has the aner. This error is alled false negative, whih an ause very serious harm. To overome the aforementioned hallenges, in this paper, a new Weighted Disretization Multiple Correspondene Analysis (WD-MCA) is proposed. It ontains a new weighting fator for the disretization algorithm, whih is later utilized in the Multiple Correspondene Analysis (MCA) lassifier. By assigning reasonable weights to the instanes in the minor lass, it would be possible to improve the interesting onept detetion in multimedia data. This observation motivates us to propose a new data mining framework to handle the imbalaned data problem. For this purpose, the supervised disretization funtion introdued in [5] is extended to penalize the negative lasses and bias the learning model toward the positive lass. Moreover, the disretization fator is integrated to the MCA weighting funtion for effetive video onept detetion. The rest of this paper is organized as follows. Setion II disusses the existing work in multimedia data analysis.

2 In setion III, the proposed framework is introdued and eah omponent of the WD-MCA is disussed in details. Setion IV gives the experimental results and observations. Finally, onlusions and reommendations for future work are presented. II. RELATED WORK Regarding the data imbalane issue, onventional approahes an be mainly ategorized into the following groups [3][6]: Sampling methods, ost sensitives learning, and hybrid algorithms. Typially, sampling methods modify the data distribution in order to balane the dataset and improve the lassifiation results. There are two main resampling approahes in the literature: over-sampling the minority (positive) lass [7] or under-sampling the majority (negative) lass [8]. Either way an be used in any mahine learning algorithm as a preproessing phase. Another solution is Cost Sensitive Learning (CSL) whih modifies the learning proess by inorporating the mislassifiation osts of the different lasses [9]. Currently, CSL has been applied in various learning algorithms suh as deision trees [3], AdaBoost [3], and Naive Bayes [3]. Reently, various hybrid methods have been proposed, whih ombine the traditional solutions for data imbalane subjet [33]. In reent years, with the advent of new tehnologies and easy aess of multimedia data in the soial networks and sharing websites, multimedia onept detetion has beome a hot topi, both in industry and aademia [34][35][36][37]. Current video searh engines often use textual desriptions and video tags to retrieve videos. However, due to the limitation and subjetivity of video metadata, suh engines may provide a very low performane. Thus, automati onept detetion is ruial in multimedia analysis [38]. Ha et al. [39] proposed a new framework using two different orrelation-based approahes integrated with a well-known deep learning method alled Convolutional Neural Network (CNN) to automatially detet semanti onepts from NUS- WIDE image dataset [4]. The Positive Enhaned Ensemble Learning (PEEL) framework is presented in [4], whih addresses the video onept/event detetion, speifially for soer videos. By integrating the ensemble learning algorithm with a sampling-based mehanism, it outperforms the existing single models and ensemble lassifiers. The TRECVID data is a very large real world dataset fousing on information retrieval, speifially on video ontent based retrieval [4]. Reently, many researh studies have been done based on the TRECVID dataset, whih made onsiderable ontributions in this area and improved the video semanti onept detetion, espeially for the imbalaned datasets [43]. Despite the fat that many real world appliations deal with ontinuous features, most of the mahine learning algorithms an only be applied to nominal or disrete numerial features [44]. Therefore, disretization ontinuousvalued features are onsidered as a signifiant step in the preproessing phase. Disretization algorithms an be lassified into supervised and unsupervised methods [45]. Fayyad and Irani [5] proposed a supervised disretization algorithm using an information entropy heuristi alled the Minimum Desription Length (MDL) priniple. In this algorithm, first, the ontinuous features are sorted, then the potential utting points are alulated from lasses boundaries based on the MDL prinipal. An unsupervised disretization algorithm based on the Self-Organizing Map (SOM) is presented in [46]. Unlike the K-means lustering whih requires the number of lusters beforehand, SOM only requires the maximum number of requested intervals and an effetively partition the feature values into nominal values. In this study, however, we extend the MDL approah to improve the disretization algorithm, speifially for the imbalaned datasets. In this paper, MCA is used as a lassifier due to its powerful nature whih is able to measure the orrelation between the attributes and lasses [47]. In the literature, MCA is widely applied to several multimedia appliations inluding feature seletion, disretization, data pruning, and lassifiation [48][6][49]. In the urrent studies, MCA analyzes eah instane by using the equal weight funtion for all feature sets. However, in this paper, MCA is extended to inorporate the disretization information to enhane the lassifiation effiieny. This study onentrates on binary lassifiation algorithm. The ontributions are as follows. First, the MDL disretization algorithm is extended to handle the imbalaned datasets using a novel osting fator. Then, the disretization fator is ombined with the MCA weighting funtion to improve the lassifiation performane. III. THE PROPOSED WEIGHTED DESCRITIZATION MULTIPLE CORRESPONDENCE ANALYSIS FRAMEWORK The proposed WD-MCA framework is depited in Figure. The whole framework an be divided in three main steps: the preproessing omponent (the top left panel), the training proess (the right panel), and the testing phase (the bottom left panel). The preproessing phase inludes shot boundary detetion routine, visual feature extration, and data splitting. As this step is domain speifi, other appliations may apply different preproessing routines. For example, eah data type (e.g., audio, speeh, image, text, and video) may require a speifi feature set and various preproessing tehniques. The next step is the training proess where a learning model is trained using the proposed WD-MCA algorithm whih ontains the weighted MDL disretization algorithm and the MCA based disretization fator. On the other side, testing data instanes are disretized using the training disretization information and the WD-MCA model is used as a lassifier to detet the semanti video onepts.

3 A. Preproessing The preproessing phase is domain speifi and eah appliation applies different preproessing routines. Speifially, for video analysis, the preproessing inludes shot boundary detetion, key-frame seletion, and feature extration as explained in more details as follows. In this paper, an automati and effetual shot boundary detetion algorithm desribed in [5] is applied on the raw video. This algorithm is based on an unsupervised method for image segmentation as well as objet traking tehniques. The segmentation algorithm first lusters the feature map of every video frame and groups the frame pixels into several lasses. Then, these segmentation maps are ompared to see how different they are. In addition, an objet traking algorithm is used to detet moving objets and luminane hanges, whih improves the final mathing results. Moreover, it an be further used for other purposes suh as ontent analysis and video indexing. After final shots are extrated, a key-frame is seleted as a representative of eah shot. Key-frames are helpful for video summarization, and therefore, it is important to selet the most distintive one whih represents the ontents of the whole shot. For this reason, the first frame of eah shot is hosen beause it is the ut-point separating suessive shots in the shot boundary detetion algorithm. In this study, several low-level visual features are extrated from raw videos as desribed in [5]. Histogram of Oriented Gradient (HOG) [5] has been proven to be an effetive and robust visual desriptor in many image proessing appliations suh as objet reognition, human detetion, and ation reognition, to name a few. Color and Edge Diretivity Desriptor (CEDD) [53] is another wellknown visual desriptor, whih inorporates a histogram s texture and olor information. Many researh studies have leveraged the CEDD features for image indexing and retrieval. These two feature sets, plus other low-level visual attributes suh as texture wavelet, olor histogram, and olor moment are integrated as the final feature set. Finally, the dataset is split into training set and testing set for further proedures. B. Training Phase The proposed training algorithm inludes two main omponents. First, a weighted disretization algorithm is applied to the training set and then to the disretized dataset. Seond, the disretization fators are used to train the MCA algorithm. The algorithm and its tehnial details are desribed as follows. ) The Weighted MDL Disretization Algorithm: In this omponent, the Minimum Desription Length (MDL) approah [5] is extended to improve the disretization step by onsidering the importane of positive instanes in an imbalaned dataset. For this purpose, a weighting fator is Figure : Illustration of the proposed WD-MCA framework proposed to assign a weight to eah positive instane using Equation (). { + (ps/l) ϑ if = w (i) = () otherwise where w (i) is the weighting fator for the i th instane, is the lass onept (positive= and negative=), ps is the number of positive instanes in the orresponding onept, L is the total number of training instanes, and ϑ is a predefined onstant. For instane, w (i) =.4 for a positive instane i in a dataset with positive instanes out of training instanes, where ϑ=. The purpose of using the onstant fator (ϑ) is to inrease the weight of positive instanes in an imbalaned dataset. As ps/l is a very small number, espeially for a highly imbalaned dataset, it is multiplied by a larger number (ϑ) to inrease the weighting fator for positive instanes. If ϑ is very small, the weighting fator for positive instanes would be very lose to the negative ones. On the other hand, if it is very large (>), the results will be overfitted to the positive lass. Therefore, a number between to 5 (depending on the ps/l fator) is reasonable. In order to find the best ut-point for eah feature, the MDL algorithm is applied as follows. First, all the instanes are sorted. Next, the lass ount Count in the dataset is alulated as shown in Equation (). Count = L w (i). () i= To ontinue the previous example, Count = 8, whih inreases by.4 times for the positive instanes. Afterward,

4 the entropy and information gain [54] of the given dataset is omputed using Count for both positive and negative lasses. Finally, a ut-point of a dataset T, inluding N instanes is evaluated using Equation (3), and Delta is defined in Equation (4). InfoGain > log (N ) N + Delta N. (3) Delta = log(3 CL ) ((CL priorentropy) (CL right entropy right ) (CL left entropy left )), where CL is the total number of lasses (CL = for binary lassifiation), priorentropy is the entropy value before the split, entropy right and entropy left are the entropy values of the right and left subsets, respetively, and CL right, CL left are the total number of lasses of right and left subsets, respetively. The ut-points will be iteratively generated for both left and right sides of the given dataset until the ondition in Equation (3) is true. As a result, all features are disretized into several feature items. Finally, the total number of disretized subsets for eah feature is stored in the DisCount j, where j =,,, M and M is the total number of features. ) MCA based Disretization Fator: Multiple orrespondene Analysis (MCA) is a modified version of original orrespondene analysis whih aptures the orrelation between features and lasses. In this paper, the MCA algorithm is enhaned using the disretization information aptured from the previous omponent. In multimedia databases, rows represent data instanes and olumns represent features as well as the orresponding onept labels. MCA aptures the orrespondenes between rows and olumns whih will be later leveraged in the lassifiation step to bridge the gap between the low-level visual features and high-level onepts. Algorithm illustrates the whole proedure of the Weighted Disretization Multiple Correspondene Analysis (WD-MCA) approah. The WD-MCA input inludes a matrix ontaining all training instanes T and feature values F ; its output is the Weight Matrix (W M j,ϕ ) alulated using the orrelation information. First, as desribed in the previous setion, eah feature is disretized using the weighted MDL disretization algorithm alled W DISC in line in Algorithm, whih generates the disretized data as depited in Table I. Let the total number of feature items for all features be DisCount, new training instanes being disretized into nominal intervals be T, feature items be F j,ϕ, and and be the positive and negative lasses. Afterwards, an indiator matrix (Ind) is onstruted whose dimension is (DisCount + CL) (DisCount + CL) as shown in Table II. This table is a binary representation of the disretized features, where the rows indiate the training instanes and the olumns indiate the feature items (featurevalue pairs). Therefore, eah instane an only belong to one (4) Table I: Disretized data F F F M Class t F, F, F M, t F, F, F M, t 3 F, F,3 F M, t N F, F, F M, Table II: Indiator matrix F, F, F, F M, F M, t t t 3 t N of the feature items. In this ase, the orresponding indiator value equals. After that, Burt matrix (Burt) is alulated by the inner produt of the indiator matrix (Burt = Ind T Ind) as shown in Table III. Eah number in the Burt matrix represents the number of ourrenes of a speifi feature item. For instane, Burt(F,, ) = if there are two instanes with feature item F, that belong to lass. Then, Burt is normalized by the grand total (G) of Ind (Z = Burt/G). Thereafter, Z is transformed to a new projeted spae using the eigenvetors V and eigenvalues E extrated from the Singular Value Deomposition (SVD) and the diagonal matrix D is derived from the singular vetor V. Next, the orrelation (weight Wj,ϕ, as shown in line in Algorithm ) between lasses and feature-value pairs is alulated using the osine value of the angle between them. The smaller the angle value is, the higher orrelated the feature-value pairs and lasses are. The final MCA weight value for eah feature-value pair is then alulated using the orresponding weight value. For more details regarding the MCA proess, please refer to [6]. In this paper, a penalized fator obtained from the disretization algorithm is utilized to redue the weight of the features with higher feature items. In other words, the smaller the disretization ount is, the more valuable information it has. For this purpose, in eah iteration, the final weight value is alulated using the W Mj,ϕ ombined with the penalized fator dw DisCount j, as shown in line 4 in Algorithm, where dw is the disretization weight obtained from the weighted MDL algorithm in Setion III-B, and DisCount j is the number of feature items for eah feature j. Eventually, the final weights are stored in the weight matrix W Mj,ϕ. C. Testing Phase The testing module inludes the weighted disretization and lassifiation steps. First, the testing dataset is disretized into nominal features using the weighted disretization algorithm desribed in Setion III-B. Then, the final

5 Table III: Burt matrix F, F, F, F, F, F, Table IV: Disaster dataset statistis Algorithm WD-MCA Input: Training instanes T {ti, i =,,, N }, feature set F = {fj, j =,,, M }.. Output: Weight matrix W Mj,ϕ : {T, Fj,ϕ, DisCount} WD ISC(T, F ); : for all fj F, (j =,, M ) do 3: Create Indiator matrix Ind; 4: Create Burt matrix Burt; 5: {Z, V, E} M CA(Burt); 6: Create orrespondene matrix CM ; 7: Derive diagonal matrix D from V ; 8: for all Fj,ϕ (ϕ =,, DisCountj ) do 9: for all Cj, ( =,, CL) do : Calulate Wj,ϕ ; : end for : end for 3: for all Fj,ϕ (ϕ =,, DisCountj ) do 4: for all Cj, ( =,, CL) do 5: W Mj,ϕ W Mj,ϕ + Wj,ϕ ; 6: end for 7: W Mj,ϕ W Mj,ϕ /(dw DisCountj ); 8: end for 9: end for : return W Mj,ϕ No Conept damage fire mud-rok lightening snow #Positive Instanes P/N ratio (a) (b) () (d) (e) Figure : Different sample onepts in the disaster dataset: (a) damage, (b) fire, () snow (d) lightening, (e) mud-rok IV. E XPERIMENTAL A NALYSIS features are fed to the lassifiation omponent to predit the onept lass of eah testing instane. The W Mj,ϕ matrix reated in the training phase (depited in Algorithm ) is used in the testing phase to generate the ranking sore for eah instane. This sore is alulated by aumulating all the weights within one instane i and then is normalized by the total number of features (M ) as shown in Equation (5). PM j= ( mw j (i)) Sorei = (5) M After the sore matrix SM is alulated for all training instanes, it an be diretly used to rank the testing data. For lassifying instanes, a threshold needs to be generated based on the training performane as desribed in [55]. In the first step, training sores are sorted by the desending order, and then the andidate thresholds are seleted based on the indexes of the sores with target lass label. Finally, the best threshold is generated by iteratively evaluating the performane of the andidate thresholds. A. Dataset Desription Although the proposed WD-MCA an be used as a general framework for various multimedia appliations (with data inluding video, image, audio, and/or text), in this paper, a speifi task is seleted alled semanti onept detetion from videos ontaining disaster events. Automati disaster detetion from videos, a new and demanding topi, an be benefiial for lassifying videos inluding disaster events from non-disaster ones. For this purpose, the proposed framework is tested using a new disaster dataset. Speifially, it ontains about 8 different YouTupe videos with 5 disaster onepts. Figure depits a key-frame sample extrated from the videos for eah disaster onept. The detailed statistis of the dataset is summarized in Table IV. In total, the dataset inludes 6884 video shots and the average ratio of the positive instanes to the negative ones (P/N) is.5, whih shows the non-uniform distribution of the dataset.

6 B. Evaluation Criteria In the imbalaned datasets, auray may not be the best metri to show the effetiveness of the lassifiation algorithm beause most onventional lassifiers are biased toward the major (negative) lass and may have very high performane on negative lasses. However, as the minor (positive) lass is more important and ritial to be deteted, the proposed WD-MCA framework is evaluated using a ommon measurement metri for imbalaned data alled F sore as defined in Equation (6), where preision and reall are defined as shown in Equations (7) and (8), respetively. Here, TP, FP, and FN refer to the numbers of true positive, false positive, and false negative data instanes, respetively. F = P reision Reall (P reision + Reall) ; (6) T P P reision = T P + F P ; (7) T P Reall = T P + F N. (8) C. Evaluation Results As mentioned earlier, the first step in the proposed framework is preproessing the data whih inludes shot boundary detetion, key-frame seletion, visual feature extration, and finally data splitting. After applying an automati shot boundary detetion approah [5], the first frame of eah shot is seleted as a representative of that shot. Then, several visual features as desribed earlier are extrated from eah key-frame. In total, there are 6884 instanes and 77 features for eah instane. Then, the dataset is divided into three training and testing sets through a 3-fold validation whih ontains approximately equal numbers of positive and negative instanes (P/N ratio is almost equal). In the training phase (see Setion III-B), the training set is disretized using the proposed weighted MDL disretization algorithm and then the testing set is disretized using the same disretization sheme. Afterward, the disretized training instanes are passed to the WD-MCA module to train the model. For evaluating the proposed WD-MCA model, an experiment is onduted using the testing instanes to see how aurate they are lassified. The performane results are ompared to two well-known existing methods: standard MCA [6] and Deision Tree (DT), whih ahieved very high performane for other imbalaned datasets [5]. The detailed omparison results for eah onept and eah framework are presented in Table V. As an be seen from this table, the proposed WD-MCA outperforms other methods in terms of F sore for all disaster onepts. For the fire onept, for instane, it has a promising performane (F=9%) and improves the lassifiation result by about 8% and % ompared to DT and MCA, respetively. For the snow onept, the average F sore is a little bit low (a) () Figure 3: Average omparison results on disaster dataset: (a) F sore, (b) Preision, () Reall (F=67%). However, it is still about % and 4% higher than that of DT and MCA, respetively. In overall, the average F sore of all three folds for all 5 disaster onepts is 85%. In an imbalaned dataset where deteting positive instanes suh as disaster, aner, fraud, and bomb is very vital, suh an improvement (even small for some onepts) is very signifiant. In addition, only positive instanes are used for evaluating the results and the performane of negative instanes is not onsidered. The average omparison results inluding F, preision, and reall for eah onept are shown in Figure 3. As an be inferred from Figure 3b, MCA has higher preision values in three onepts, while its low reall values, as shown in Figure 3, derease its overall performane. The DT algorithm has the lowest preision values in all the onepts, whih redues its overall F sores. All in all, the proposed WD-MCA ahieves the highest average results, demonstrating the effetiveness of integrating the weighting disretization funtion with the standard MCA algorithm. V. CONCLUSION Multimedia analysis has attrated lots of attention in reent years. One of the signifiant appliations in multimedia is video semanti onept/event detetion. In partiular, the data imbalane problem, an open issue in multimedia analysis systems, is seleted beause onventional data mining algorithms are often unable to detet the minor (positive) lass in suh non-uniform data distribution. To overome this hallenge, a Weighted Disretization algorithm based on the MCA lassifier (WD-MCA) is proposed to improve the orrelation between lasses and feature-value pairs. Speifially, the MDL disretization approah is extended to takle the imbalaned data issue by applying a weighting fator to (b)

7 Table V: Detailed omparison results on disaster dataset disaster fold # DT MCA WD-MCA onept preision reall F preision reall F preision reall F fold damage fold fold average fold fire fold fold average fold mud-rok fold fold average fold lightening fold fold averagee fold snow fold fold average the minor (positive) lass. Moreover, the disretization fator is integrated with the MCA algorithm to enhane the multimedia semanti onept detetion. The whole WD-MCA framework is suessfully evaluated on videos ontaining the disaster events. This dataset inludes few positive instanes and has a highly imbalaned P/N ratio. The experimental results show the effetiveness and high performane of the proposed algorithm ompared to several existing data mining algorithms in terms of the F sore. However, there are still some limitations that need to be overome. The proposed framework is tested on a new dataset olleted by our team, whih is not publily available. In the future, this framework will be extended to detet more onepts from various datasets and appliations. Furthermore, in the urrent framework, only low-level visual features are used for video analysis. Some mid-level and high-level features inluding spatio-temporal and textual information (e.g., objet motion features, and video metadata) will be investigated and utilized to improve the onept detetion performane. ACKNOWLEDGMENT For Shu-Ching Chen, this researh is partially supported by DHS s VACCINE Center under Award Number 9-ST- 6-CI and NSF HRD-83393, HRD , CNS- 669, and CNS REFERENCES [] S.-C. Chen, A. Ghafoor, and R. L. Kashyap, Semanti models for multimedia database searhing and browsing. Springer Siene & Business Media,. [] S.-C. Chen and R. Kashyap, Temporal and spatial semanti models for multimedia presentations, in Proeedings of the 997 International Symposium on Multimedia Information Proessing, 997, pp [3] S.-C. Chen, M.-L. Shyu, and R. Kashyap, Augmented transition network as a semanti model for video data, International Journal of Networking and Information Systems, Speial Issue on Video Data, vol. 3, no., pp. 9 5,. [4] M.-L. Shyu, C. Haruehaiyasak, S.-C. Chen, and N. Zhao, Collaborative filtering by mining assoiation rules from user aess sequenes, in Proeedings of the International Workshop on Challenges inweb Information Retrieval and Integration (WIRI). IEEE, 5, pp [5] M.-L. Shyu, S.-C. Chen, and C. Haruehaiyasak, Mining user aess behavior on the www, in Proeedings of the IEEE International Conferene on Systems, Man, and Cybernetis, vol. 3. IEEE,, pp [6] M. Naphade, J. Smith, J. Tesi, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis, Large-sale onept ontology for multimedia, IEEE Multimedia, vol. 3, no. 3, pp. 86 9, july-sept. 6. [7] M. L. Shyu, Z. Xie, M. Chen, and S. C. Chen, Video semanti event/onept detetion using a subspae-based multimedia data mining framework, IEEE Transations on Multimedia, vol., no., pp. 5 59, Feb 8. [8] S.-C. Chen, M.-L. Shyu, and C. Zhang, An intelligent framework for spatio-temporal vehile traking, in Proeedings of the 4th International IEEE Conferene on Intelligent Transportation Systems. IEEE,, pp [9] S.-C. Chen, M.-L. Shyu, C. Zhang, and R. L. Kashyap, Identifying overlapped objets for video indexing and modeling in multimedia database systems, International Journal on Artifiial Intelligene Tools, vol., no. 4, pp ,. [] M. Chen, S.-C. Chen, M.-L. Shyu, and K. Wikramaratna, Semanti event detetion via temporal analysis and multimodal data mining, IEEE Signal Proessing Magazine, vol. 3, pp , Marh 6.

8 [] M.-L. Shyu, S.-C. Chen, M. Chen, and C. Zhang, A unified framework for image database lustering and ontent-based retrieval, in Proeedings of the nd ACM International Workshop on Multimedia Databases, ser. MMDB 4. New York, NY, USA: ACM, 4, pp [Online]. Available: [] X. Chen, C. Zhang, S.-C. Chen, and M. Chen, A latent semanti indexing based method for solving multiple instane learning problem in region-based image retrieval, in Proeedings of the IEEE International Symposium on Multimedia (ISM). IEEE, 5, pp [3] X. Chen, C. Zhang, S.-C. Chen, and S. Rubin, A humanentered multiple instane learning framework for semanti video retrieval, IEEE Transations on Systems, Man, and Cybernetis, Part C: Appliations and Reviews, vol. 39, no., pp. 8 33, 9. [4] A. Karpathy, G. Toderii, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, Large-sale video lassifiation with onvolutional neural networks, in Proeedings of the 4 IEEE Conferene on Computer Vision and Pattern Reognition, ser. CVPR 4. Washington, DC, USA: IEEE Computer Soiety, 4, pp [Online]. Available: [5] S.-C. Chen, M.-L. Shyu, C. Zhang, and M. Chen, A multimodal data mining framework for soer goal detetion based on deision tree logi, International Journal of Computer Appliations in Tehnology, vol. 7, no. 4, pp. 3 33, 6. [6] L. Lin, M.-L. Shyu, G. Ravitz, and S.-C. Chen, Video semanti onept detetion via assoiative lassifiation, in The th IEEE International Conferene on Multimedia and Expo (ICME). IEEE, 9, pp [7] S.-C. Chen, S. Rubin, M.-L. Shyu, and C. Zhang, A dynami user onept pattern learning framework for ontent-based image retrieval, IEEE Transations on Systems, Man, and Cybernetis: Part C, vol. 36, pp , November 6. [8] X. Huang, S.-C. Chen, M.-L. Shyu, and C. Zhang, User onept pattern disovery using relevane feedbak and multiple instane learning for ontent-based image retrieval, in Proeedings of the Third International Workshop on Multimedia Data Mining, in onjuntion with the 8th ACM International Conferene on Knowledge Disovery & Data Mining, July, pp. 8. [9] X. Li, S.-C. Chen, M.-L. Shyu, and B. Furht, An effetive ontent-based visual image retrieval system, in Proeedings of the IEEE International Computer Software and Appliations Conferene, August, pp [] L. Lin, G. Ravitz, M.-L. Shyu, and S.-C. Chen, Video semanti onept disovery using multimodal-based assoiation lassifiation, in Proeedings of the IEEE International Conferene on Multimedia & Expo, July 7, pp [] M.-L. Shyu, T. Quirino, Z. Xie, S.-C. Chen, and L. Chang, Network intrusion detetion through adaptive sub-eigenspae modeling in multiagent systems, ACM Transations on Autonomous and Adaptive Systems, vol., no. 3, Sep. 7. [Online]. Available: [] L. Lin, G. Ravitz, M.-L. Shyu, and S.-C. Chen, Effetive feature spae redution with imbalaned data for semanti onept detetion, in Proeedings of the IEEE International Conferene on Sensor Networks, Ubiquitous and Trustworthy Computing (SUTC8). IEEE, 8, pp [3] V. Ganganwar, An overview of lassifiation algorithms for imbalaned datasets, International Journal of Emerging Tehnology and Advaned Engineering, vol., no. 4, pp. 4 47,. [4] R. Longadge and S. Dongre, Class imbalane problem in data mining review, CoRR, vol. abs/35.77, 3. [5] U. M. Fayyad and K. B. Irani, Multi-interval disretization of ontinuous valued attributes for lassifiation learning, in Proeedings of the 3th International Joint Conferene on Artifiial Intelligene, 993, pp. 9. [6] H. He and E. A. Garia, Learning from imbalaned data, IEEE Transations on Knowledge and Data Engineering, vol., no. 9, pp , 9. [7] H. Han, W.-Y. Wang, and B.-H. Mao, Borderline-smote: a new over-sampling method in imbalaned data sets learning, in Advanes in intelligent omputing. Springer, 5, pp [8] S.-J. Yen and Y.-S. Lee, Cluster-based under-sampling approahes for imbalaned data distributions, Expert Systems with Appliations, vol. 36, no. 3, pp , 9. [9] N. Thai-Nghe, Z. Gantner, and L. Shmidt-Thieme, Costsensitive learning methods for imbalaned data, in The International Joint Conferene on Neural Networks (IJCNN). IEEE,, pp. 8. [3] S. Lomax and S. Vadera, A survey of ost-sensitive deision tree indution algorithms, ACM Computing Surveys (CSUR), vol. 45, no., p. 6, 3. [3] J. Zheng, Cost-sensitive boosting neural networks for software defet predition, Expert Systems with Appliations, vol. 37, no. 6, pp ,. [3] X. Chai, L. Deng, Q. Yang, and C. X. Ling, Test-ost sensitive naive bayes lassifiation, in Fourth IEEE International Conferene on Data Mining (ICDM). IEEE, 4, pp [33] M. Galar, A. Fernandez, E. Barrenehea, H. Bustine, and F. Herrera, A review on ensembles for the lass imbalane problem: bagging-, boosting-, and hybrid-based approahes, IEEE Transations on Systems, Man, and Cybernetis, Part C: Appliations and Reviews, vol. 4, no. 4, pp ,. [34] S.-C. Chen, S. Sista, M.-L. Shyu, and R. Kashyap, Augmented transition networks as video browsing models for multimedia databases and multimedia information systems, in Proeedings of the th IEEE International Conferene on Tools with Artifiial Intelligene, 999, pp [35] X. Li, S.-C. Chen, M.-L. Shyu, and B. Furht, Image retrieval by olor, texture, and spatial information, in Proeedings of the 8th International Conferene on Distributed Multimedia Systems, September, pp

9 [36] M.-L. Shyu, C. Haruehaiyasak, and S.-C. Chen, Category luster disovery from distributed www diretories, Information Sienes, vol. 55, no. 3, pp. 8 97, 3. [37] S. Pouyanfar and H. Sameti, Musi emotion reognition using two level lassifiation, in Iranian Conferene on Intelligent Systems (ICIS). IEEE, 4, pp. 6. [38] M.-L. Shyu, S.-C. Chen, and R. L. Kashyap, Generalized affinity-based assoiation rule mining for multimedia database queries, Knowledge and Information Systems (KAIS): An International Journal, vol. 3, no. 3, pp ,. [39] H.-Y. Ha, Y. Yang, S. Pouyanfar, H. Tian, and S.-C. Chen, Correlation-based deep learning for multimedia semanti onept detetion, in Web Information Systems Engineering (WISE). Springer, 5, pp [4] T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y. Zheng, Nus-wide: a real-world web image database from national university of singapore, in Proeedings of the ACM international onferene on image and video retrieval. ACM, 9, p. 48. [4] Y. Yang and S.-C. Chen, Ensemble learning from imbalaned data set for video event detetion, in 5 IEEE International Conferene on Information Reuse and Integration (IRI). IEEE, 5, pp [4] Tre video retrieval evaluation: Trevid, retrieved at: [Online]. Available: [43] Y. Sun, K. Sudo, Y. Taniguhi, H. Li, Y. Guan, and L. Liu, Trevid 3 semanti video onept detetion by ntt-mddut, in TRECVID 3, 3. [49] Q. Zhu, L. Lin, M.-L. Shyu, and S.-C. Chen, Effetive supervised disretization for lassifiation based on orrelation maximization, in IEEE International Conferene on Information Reuse and Integration (IRI). IEEE,, pp [5] S.-C. Chen, M.-L. Shyu, and C. Zhang, Innovative shot boundary detetion for video indexing, Video data management and information retrieval, pp. 7 36, 5. [5] Y. Yang and S.-C. Chen, Disaster image filtering and summarization based on multi-layered affinity propagation, in IEEE International Symposium on Multimedia (ISM). IEEE,, pp. 3. [5] N. Dalal and B. Triggs, Histograms of oriented gradients for human detetion, in IEEE Computer Soiety Conferene on Computer Vision and Pattern Reognition (CVPR), vol.. IEEE, 5, pp [53] S. A. Chatzihristofis and Y. S. Boutalis, Cedd: olor and edge diretivity desriptor: a ompat desriptor for image indexing and retrieval, in Computer vision systems. Springer, 8, pp [54] R. S. Mihalski, J. G. Carbonell, and T. M. Mithell, Mahine learning: An artifiial intelligene approah. Springer Siene & Business Media, 3. [55] Y. Yang, H.-Y. Ha, F. Fleites, S.-C. Chen, and S. Luis, Hierarhial disaster image lassifiation for situation report enhanement, in IEEE International Conferene on Information Reuse and Integration (IRI). IEEE,, pp [44] M.-L. Shyu, K. Sarinnapakorn, I. Kuruppu-Appuhamilage, S.-C. Chen, L. Chang, and T. Goldring, Handling nominal features in anomaly intrusion detetion problems, in Proeedings of the 5th International Workshop on Researh Issues on Data Engineering (RIDE Workshop of Stream Data Mining and Appliations RIDE-SDMA 5), in onjuntion with The st International Conferene on Data Engineering (ICDE 5). IEEE, 5, pp [45] J. Dougherty, R. Kohavi, and M. Sahami, Supervised and unsupervised disretization of ontinuous features, in Mahine learning: proeedings of the twelfth international onferene, vol., 995, pp. 94. [46] M. Vannui and V. Colla, Meaningful disretization of ontinuous features for assoiation rules mining by means of a som. in ESANN, 4, pp [47] H. Abdi and D. Valentin, Multiple orrespondene analysis, Enylopedia of measurement and statistis, pp , 7. [48] L. Lin, M.-L. Shyu, and S.-C. Chen, Enhaning onept detetion by pruning data with ma-based transation weights, in th IEEE International Symposium on Multimedia (ISM). IEEE, 9, pp