Teaching Performance Evaluation Using Supervised Machine Learning Techniques

Size: px

Start display at page:

Download "Teaching Performance Evaluation Using Supervised Machine Learning Techniques"

Easter Cain
6 years ago
Views:

1 Teaching Perfrmance Evaluatin Using Supervised Machine Learning Techniques Elia Gergiana Dragmir University Petrleum-Gas f Pliesti, Department f Infrmatics Bd. Bucuresti Nr. 39, Pliesti, RO , ROMANIA elia.dragmir@yah.cm Abstract Teaching perfrmance evaluatin can be dne using multiple surces, like students, peers and teachers themselves. Even thugh nly peers have the substantive expertise fr a relevant evaluatin, it is generally well-knwn that students are qualified t assess sme f the classrm teaching aspects: clarity f the presentatin, interpersnal rapprt with students etc. The cre idea f this research is t study if there can be built a cmputatinal mdel that uses past students evaluatin in rder t predict future teaching perfrmance assessments. There can be designed different system based n supervised machine learning techniques. In this paper there are built several mdels based n tw classificatin techniques: K-Nearest Neighbr and Supprt Vectr Machine with the purpse f finding a mdel that has the smaller classificatin errr f the new cases. Keywrds: Teaching perfrmance evaluatin, K-Nearest Neighbr, Supprt Vectr Machine Intrductin The teaching perfrmance evaluatin reviews academic qualificatins, relevant experience, quality f teaching, and prfessinal cntributins. All these aspects can be assessed by the students, peers r by the teachers themselves. In this paper, we will fcus n the students evaluatin. Aleamni sudgests that students are the main surce f infrmatin abut the learning envirnment, including teachers' ability t mtivate them fr cntinued learning, rapprt r degree f cmmunicatin between instructrs and students. They are als the mst cnsistent evaluatrs f the quality, the effectiveness f the learning prcess and satisfactin with curse cntent, methd f instructin, textbks, hmewrk, and student interest (Aleamni, 1981). The results f many evaluatins perfrmed by the Statistics Department f the University f Wiscnsin-Madisn are stred in a dedicated database. This research fcuses n the applicatin f sme machine learning techniques n this data in rder t develp a mdel that can use sme past assessment t determine a future evaluatin. This paper is structured as fllws. The first sectin presents a brief intrductin t the supervised learning technique used t build these mdels, K Nearest Neigbr (KNN) technique and Supprt Vectr Machine (SVM) technique, then the methdlgy applied in this prblem and the data available. The next sectin presents the results, the final discussins and cnclusins are given in the last part. Supervised Machine Learning Techniques It is nt easy t establish the relatinships between multiple features f sme prblem and peple are ften prne t make mistake in their analysis and furthermre t find the slutins t certain

2 The 5 th Internatinal Cnference n Virtual Learning ICVL prblems. In rder t imprve the efficiency f the systems and the designs f the machine, there can be applied machine learning. (Maglgiannis et al, 2007). Related t the type f the data set features recrdings there can be implemented the supervised machine learning techniques, if the instances are given with knwn labels (the crrespnding crrect utputs) and the unsupervised machine learning techniques where the instances are nt labelled. In this paper, there are briefly described tw supervised learning techniques: K-Nearest Neighbr and Supprt Vectr Machine in rder t determine if they can be imprve the teaching perfrmance evaluatin using these methds. K Nearest Neigbr Nearest Neighbr technique is ne f the classificatin methds used in machine learning. It is based n the idea that a new bject is classified based n attributes and training samples, using a majrity f K-nearest neighbr categry. In rder t apply this technique, it is necessary t have a training set and a test sample, t knw the k value (hw many neighbrs are used in classificatin) and the mathematical frmula f the distance calculated between the instances (Hart and Cver, 1967). The k nearest neighbr classifier is cmmnly based n the Euclidean distance (Frmula 1) between a test sample and the specified training samples. n [1] x i y i i= 1 2 The general algrithm f cmputing the k-nearest neighbrs is as fllws: Establish the parameter k that represents the nearest neighbrs number; Calculate the Euclidian distance between the query-instance and all the training samples; Srt the distances fr all the training samples and determine the nearest neighbr based n the k-th minimum distance; Use the majrity f nearest neighbrs as the predictin value. Supprt Vectr Machine The basic idea f Supprt Vectr Machines is t map the riginal data X int a feature space F with high dimensinality thrugh a nn linear mapping functin and cnstruct an ptimal hyperplan in new space. SVM can be applied t bth classificatin and regressin. In the case f classificatin, an ptimal hyperplan is fund that separates the data int tw classes. Whereas in the case f regressin a hyperplan is t be cnstructed that lies clse t as many pints as pssible (Burges, 1998). SVMs revlve arund the ntin f a margin either side f a hyperplan that separates tw data classes. Maximizing the margin and thereby creating the largest pssible distance between the separating hyperplan and the instances n either side f it has been prven t reduce an upper bund n the expected generalizatin errr (Cristianini, 2001). SVM has yielded excellent generalizatin perfrmance n a wide range f prblems including biinfrmatics (Zien et al, 2000), text categrizatin (Jachims, 1998), image detectin (Osuna et al., 1997), frecasting f the air quality parameters (Radhika and Shashi, 2009) etc. Case Study The experiment presented in this paper fcuses n the utility f the past cases in rder t predict sme new evaluatins. Fr that, it is necessary t design sme mdels based n the past

3 392 University f Bucharest and University f Medicine and Pharmacy Târgu-Mureş assessment. In rder t find a mdel that has the smaller classificatin errr f the new cases there are used tw supervised machine learning techniques: K-Nearest Neighbr and Supprt Vectr Machine. Data set The set f data used in this experiment is prvided by the Statistics Department f the University f Wiscnsin-Madisn. It cnsists f evaluatins f teaching perfrmance ver three regular semesters and tw summer semesters f 151 teaching assistant assignments in this Department. It cntains 151 instances with 6 attributes. The characteristics f these attributes as their names, type and pssible values are centralized in table 1. Table 1. Database Attributes Used In This Experiment Attribute name Attribute type Attribute pssible value English_speaker Binary 1=English speaker 2=nn-English speaker curse_instructr Categrical 25 categries curse Categrical 26 categries regular_semester Binary 1=Summer semester 2=Regular semester class_size class_attribute Real 1=Lw 2=Medium 3=High The data was prcessed in rder t be used by Weka, a data mining sftware tl develped at the University f Waikat. It cntains a cllectin f visualizatin tls and algrithms fr data analysis and predictive mdelling, tgether with graphical user interfaces fr easy access t this functinality. Experimental Results and Discussins In the first experiment there is built a mdel based n the KNN technique in rder t determine if a new case can be crrectly classified. The results are written in the secnd clumn f the Table 2. The statistical results f the secnd mdel based n the SVM methd are presented in the third clumn r the same table. Bth mdels are analysed accrding with their values fr sme accuracy measures, such as the crrectly r incrrectly classified instances errrs, Kappa statistic, mean abslute errr that is a quantity used t measure hw clse frecasts r predictins are t the eventual utcmes, rt mean squared errr, which cnstitutes a gd measure f the mdel s accuracy, rt relative squared errr (the average f the actual values), and relative abslute errr that is similar t the relative squared errr. Table 2. The Experimental Results KNN Technique SVM Technique Crrectly Classified Instances % % Incrrectly Classified Instances % % Kappa statistic Mean abslute errr Rt mean squared errr Relative abslute errr % % Rt relative squared errr % %

4 The 5 th Internatinal Cnference n Virtual Learning ICVL The cmparative study f these results reveals that, using the same dataset, a mdel bases n the KNN technique is a better classifier fr the new instances, having nly a % incrrectly classified instances percentage. This accuracy measure is % fr the SVM based mdel. The mean abslute errr is nly fr the KNN mdel cmparative t the value f f the same statistics measure fr the SVM mdel. Table 3. Cnfusin Matrix fr KNN Technique a b c <-- classified as a = b = c = 3 The cnfusin matrix fr each technique reflects the incrrectly classified instances. Thus, in Table 3, it can be seen that fr class a=1 there were crrectly classified 46 instances frm the ttal f 49, the ther three being classified as class b=2 and c=3. In the same manner there can be fund that fr class c=3 there are n incrrectly classified instances, all 51 cases are crrectly classified as class c=3. Table 4. Cnfusin Matrix fr SVM Technique a b c <-- classified as a = b = c = 3 Table 4 cntains the cnfusin matrix fr the SVM mdel. The incrrectly classified errr is reflected in the number f misclassified cases. There are nly 33 frm 49 instances are classified crrectly fr class a=1, 30 frm 50 fr class b=2 and 38 frm 51 fr class c=3. Therefre, fr all the classes the KNN technique perfrms better than the SVM methd in rder t assess the teaching perfrmance. It wrth be mentined that the results are clsed related t the data set used t design these mdels and that if the data set is changed it is mst pssible that the mdel turns ut t be different. Cnclusins The teaching perfrmance evaluatin can be dne using supervised machine learning techniques, such as K-Nearest Neighbr r Supprt Vectr Machine. The mdels are built using sme past assessments stred in a database in rder t autmated classify new cases. Frm this experiment, we can cnclude that, in the cnditins described in this paper, the KNN technique classifies better a new teaching perfrmance evaluatin case than a mdel based n SVM technique. References Aleamni, L. M. (1981): Student ratings f instructin, ed. J. Millman Burges, C.(1998): A Tutrial n Supprt Vectr Machines fr Pattern Recgnitin, Data Mining and Knwledge Discvery, vl 2, Issue 2, June 1998, pg Crtes, C. and Vapnik, V.(1995): Supprt vectr netwrks, Machine Learning, vl 20, pp Cristianini, N. and Shawe-Taylr, J. (2000): An Intrductin t Supprt Vectr Machines, Cambridge University Press

5 394 University f Bucharest and University f Medicine and Pharmacy Târgu-Mureş Hart, P., E. and Cver, T., M.(1967): Nearest neighbr pattern classificatin. IEEE Transactins n Infrmatin Thery, IT-13 Hastie, T. and Tibshirani, R.(1996): Discriminant adaptive nearest neighbr classificatin, IEEE Trans. Pattern Anal. Mach. Intell. 18(6), Jachims, T. (1998): Text Categrizatin with Supprt Vectr Machines: Learning with Many Relevant Features, Prceedings f the Eurpean Cnference n Machine Learning, Springer. Maglgiannis, I., et al (2007), Emerging Artificial Intelligence Applicatins In Cmputer Engineering, Is Press, Pp 14 Radhika, Z. and Shashi, M.(2009): Atmspheric Temperature Predictin using Supprt Vectr Machines, Internatinal Jurnal f Cmputer Thery and Engineering, Vl. 1, N. 1, April 2009, Osuna, E., Freund, R., Girsi, F. (1997) Training supprt vectr machines: an applicatin t face detectin, Prceedings f Cmputer Visin and Pattern Recgnitin, pp Zien, A., et al (2000): Engineering supprt vectr machine kernels that recgnize translatin initiatin sites, Oxfrd University Press, Biinfrmatics Vl. 16 n , Pages

CS 309: Autonomous Intelligent Robotics. Instructor: Jivko Sinapov

CS 309: Autonomous Intelligent Robotics. Instructor: Jivko Sinapov CS 309: Autnmus Intelligent Rbtics Instructr: Jivk Sinapv http://www.cs.uteas.edu/~jsinapv/teaching/cs309_spring2017/ Machine Learning Annuncements Final Prject Presentatins Saturday, May 13, 7:00-10:00