Adaptive Naïve Bayesian Anti-Spam Engine

Size: px

Start display at page:

Download "Adaptive Naïve Bayesian Anti-Spam Engine"

Sheryl Short
6 years ago
Views:

1 Wold Academy of Scece, Egeeg ad Techology Adaptve Naïve Bayesa At-Spam Ege Wojcech P. Gajewsk Abstact The poblem of spam has bee seously toublg the Iteet commuty dug the last few yeas ad cuetly eached a alamg scale. Obsevatos made at CERN (Euopea Ogazato fo Nuclea Reseach located Geeva, Swtzelad) show that spam mals ca costtute up to 75% of daly SMTP taffc. A aïve Bayesa classfe based o a Bag Of Wods epesetato of a emal s wdely used to stop ths uwated flood as t combes good pefomace wth smplcty of the tag ad classfcato pocesses. Howeve, facg the costatly chagg pattes of spam, t s ecessay to assue ole adaptablty of the classfe. Ths wok poposes combg such a classfe wth aothe NBC (aïve Bayesa classfe) based o pas of adjacet wods. Oly the latte wll be etaed wth examples of spam epoted by uses. Tests ae pefomed o cosdeable sets of mals both fom publc spam achves ad CERN malboxes. They suggest that ths achtectue ca cease spam ecall wthout affectg the classfe pecso as t happes whe oly the NBC based o sgle wods s etaed. Keywods Text classfcato, aïve Bayesa classfcato, spam, emal. I. INTRODUCTION HE poblem of Spam (Usolcted commecal emal) has T become vey seous fo the Iteet commuty the last few yeas. I Decembe 2004 oly less tha 5% of mals avg at SMTP gateways of CERN wee legtmate (Souce: statstcs of CERN Mal Sevce ([2]). Smla fomato s head fom all pats of the wold ad oe ca hadly fd a emal use who s ot famla wth ths stuato fom hs o he pesoal expeece. Ths pheomeo s egaded as a theat to emal use poductvty, ases seously the TCO (Total Cost of Oweshp) of mal seves ad fally eve becomes a secuty ssue to may ogazatos. Fom a emal use pespectve, t hghly educes the usablty of emal eveyday lfe. I such ccumstaces t s udestadable that the wold of scece has see may woks elated to spam ecet yeas. Fom the theoetcal pot of vew, spam fghtg s a text categozato task ad a pat of the Natual Laguage Pocessg (NLP) feld. Ths pape poposes usg two depedet aïve Bayesa classfes to detect spam. The fst oe epesets a emal as Mauscpt eceved July 3, Wojcech P. Gajewsk s wth the Isttute of Cotol ad Computato Egeeg at the Wasaw Uvesty of Techology, ul. Nowowejska 5/9, Waszawa, Polad. (e-mal: wojcech.gajewsk@gmal.com). a vecto of sgle wods (ugam ad ts tag set s costat wheeas the othe epesets a emal as a vecto of pas of adjacet wods (bgam ad ts tag set s supplemeted by spam messages epoted by uses. Such a cofguato esults adaptablty of at-spam ege ad the mpovemet of classfcato pefomace. I ths secto, some specfc featues of spam detecto as a text categozato wll be metoed ad the costucto of so called Gaham's veso of aïve Bayesa classfe wll be shotly dscussed. The eade ca also fd hee some commets o elated wok. Next, the teestg featues of used algothm wll be descbed ad test esults wll be peseted. Fally, a shot dscusso wll coclude the pape. A. Spam Fghtg as a Text Classfcato Task At the fst glace, spam detecto mght appea a elatvely easy task fo two easos. Fst of all, thee ae oly two categoes whle a stadad text classfcato poblem ca have tes o moe of them. Secodly, a defto of Spam as a usolcted commecal mal may tself appea vey hemetc ad stctly elated to commece ad cosumpto goods o some vey well defed sectos of actvty as phamacy o poogaphc dusty. Both of them mght mply that we could quckly come out wth classfes whch pefomace s suffcet eough to elmate the poblem. Ufotuately, eve f ths mght have bee tue a few yeas ago, ow t s ot the case aymoe. Spammes ae awae of geeal methods used at-spam systems ad take steps to ovecome mposed baes. Some messages cota a set of adom wods that ca fluece a classfe's decso. I othes the key wods ae dvded by spaces, whch stll make t eadable by humas, but mpossble fo classfes to make a ght decso. I my opo, we could be eve obsevg a classcal amamets ace whee a mpovemet o oe sde s mmedately balaced by a eply fom the othe whch wll be aga ovecome by fst sde ad so o. Fo exteded dscusso of ths matte also cotext of spam detecto see [0]. Futhemoe, most spam messages ae owadays costucted a way that makes them as smla to legtmate emals as possble. They ca esemble a message wth Chstmas wshes fom a fed o cota oly oe ot suspcous setece (e.g. Do you wat a Watch?). These examples ae poofs that the defto of spam categoy s fuzzy. Of couse ths ca be equally sad about ay othe categoy whch wll have a ceta amout of examples suffcetly close to the bode of categoy, makg t patculaly dffcult to gve a cofdet classfcato. I case 45

2 Wold Academy of Scece, Egeeg ad Techology of spam, howeve, the fact that ceatos of those messages ae puposely tyg to make them dstgushable fom legtmates s becomg a seous poblem. All these factos make the spam detecto task moe dffcult tha a classcal task of text classfcato. I the latte case, obody s puposely tyg to make both possble categoes as smla as possble; ethe takes steps to make the classfcato moe dffcult. O the othe had, oe could fd othe featues of spam that could ease spam detecto: spam s usually set lage sees to mass ecpet. That meas that whe ecevg a patcula usolcted emal thee s a vey hgh pobablty of ecevg othe smla o detcal emals at the scope of ogazato. Pesumably, ths mechasm s elated to methods of updatg ecpet lsts used by spammes. Fally, a obsevato could be made that spam s chagg o a daly bass. Ceta malg campags ae stll ogog but othes wll quckly eplace them. To deepe aalyze the last obsevato I poposed the followg expemet. Fom spams epoted by uses at CERN dug a peod of Octobe 2004 I extacted all URLs. Ths set was the made avalable to SpamKlle (moe fomato o SpamKlle ad CERN at-spam achtectue ca be foud [3]) system that couted a umbe of emals comg to CERN cotag oly URLs aleady oted befoe. Dug the tme of the expemet, o ew URLs wee added to the set. Daly statstcs fo the ed of Octobe ae peseted at Fgue. Fg. Numbe of emals cotag URLs extacted fom a set of spams We ca see that afte 0 days of expemet oly up to 20% of the mals wee ot chaged. Of couse, a chage of the URL s possbly the easest possble to execute but ths gves a clea dea how dyamc s a pheomeo of spam. I vew of these agumets t s mpotat to esue ceta featues of evey spam detecto. It should be: Adaptve Spam pattes ae cuetly evolvg a vey dyamc way. At-spam softwae that was petty successful ot a log tme ago could today have a pefomace below the level of acceptablty o smply be useless. Ole A deal classfe wll statly adapt tself to elmate coectly classfed samples. Usg uses' feedback Oly close coopeato wth mal uses ca povde suffcet fomato to allow ole etag of spam classfes. Automatc As lage as possble depedece fom a eed of huma's teveto ca help to educe mateace costs. May at-spam eges use aïve Bayesa classfe based o a sgle wod epesetato of a emal because of ts good pefomace ad smplcty. Ole update of ts spam tag set ca, howeve, deteoate the pecso of classfcato. I popose theefoe to combe such a classfe wth aothe aïve Bayesa classfe epesetg a emal usg pas of adjacet wods. The latte classfe's tag set of spam wll be updated wth ew spam epoted by emal uses. Ths soluto, what s show by the expemets, aswes all of the metoed eeds. B. Gaham's Veso of Naïve Bayesa Classfe May woks ad pactcal mplemetatos (e.g. SpamGuu, see [4]) base o the dea of aïve Bayesa classfe ad Bag of Wods (BOW) epesetato of emals to detect spam. A patcula veso of such classfe, dffeet fom classcal (descbed fo example [9]), was peseted by Paul Gaham []. I geeal, a classfcato doe by aïve Bayesa classfe ca be descbed as choosg ths patcula categoy c fom the set of categoes C whch s the most pobable gve a w featue vecto w = M, costucted of all the wods foud w aalyzed message m: c = ag max c w) () c C c w) s computed usg Bayes fomula: c w) = C k = c) I I w c) k w c ) c ) Navety of the model comes fom the assumpto of wod depedece wth categoes. Theefoe: I w c) = w ) (3) c I spam detecto task, a set of categoes s educed oly to two elemets ( C = { s, l} ) as we ca cosde a emal as ethe beg a spam o legtmate oe. I Paul Gaham's classfe oly the pobablty of belogg to spam categoy gve the featue vecto w s k (2) 46

3 Wold Academy of Scece, Egeeg ad Techology estmated ( s w) ). Equato (3) appled togethe wth Bayes fomula yeld: w ) I w s w ) = (4) = w ) I w l) l w ) = (5) = Equatos (4) ad (5) appled to (2) yeld: s w) = s w ) + s w ) ( ) ( s w )) Gve a umbe of messages tag sets of spam ad legtmate messages ( N ad N espectvely) ad umbe of s occueces of a wod w tag sets ( s ad l ) a followg estmate s computed fo each wod dug tag pocess: s Ns s w ) = (7) s l + Ns Nl Fally, I assume a po that P ( =. Gaham s 2 veso of the classfe s bette suted fo the spam detecto task tha the classcal oe as t cludes fomato about both categoes featue pobablty computg. Ths tu smplfes featue vecto educto to mpove classfcato pefomace. Moe o classcal aïve Bayesa classfes ca be foud [5, 9]. To follow a detaled dscusso about Gaham's aïve Bayesa pobablstc techques cosult [4]. C. Pevous Wok Thee have bee seveal attempts to use addtoal kowledge elated to bgams the aïve Bayesa classfcato. I [7] the authos use bgams togethe wth sgle wods (ugam as featues the NBC the geeal text classfcato task. They ote a se of a umbe of coectly classfed postve documets (spam. Howeve, moe egatve documets wee also classfed coectly. A compaso betwee a aïve Bayesa classfe based o tems ad bgams spam ecogto ca be foud [8]. Thee, fo some paametes of classfes, the oe based o bgams ca have bette oveall pefomace. Howeve, the methodology of tag pocess s uclea. II. ALGORITHM Two modules based o Gaham's veso of aïve Bayesa classfe [] wee vestgated: oe epesetg a emal the fom of Bag of Wods (usg sgle wods - ugam ad the othe epesetg a emal as a set of pas of adjacet l (6) wods (bgam. Betwee the classfes thee ae also two othe mpotat dffeeces: The NBC based o sgle wods geeates the pobablty oly o 5 featues fo whch the followg value s hghest: P ( s w ) 0.5 (8) as t was descbed the ogal Gaham's algothm. I the case of the latte NBC the same fucto s used to choose featues (pas of adjacet wod fo the classfcato pocess. Howeve, the umbe s ot costat ad s popotoal to the message body legth. It ca be descbed as: l( m) ( l( m)) = m l( m),max(5, ) (9) 5 whee l(m) s the umbe of wods the emal body. I the case of the NBC based o bgams the assumpto has bee made that each featue located a patcula emal ad ot ecoded the classfe dctoay (ot foud the tag set) has a po low pobablty. Abtaly a value of 0.03 was chose. The assumpto of default low pobablty fo pevously usee featues case of NBC based o bgams s costucted as a aswe to the way the classfe wll be used. The tag set of spams wll potetally have a lage cadalty tha ths of solcted mals ad the pocess of etag wll oly mply elagemet of the fst set. Theefoe, teatg bgams usee the tag set as belogg to solcted categoy wll esult categozg as spam oly emals vey smla to examples exstg the spam tag set. Vey lage dctoay of pa of wods s ecouagg us to make ths assumpto. To lmt dctoay sze, a smple featue selecto algothm was mplemeted both vesos of algothms. It deleted all wods o pas of wods whch occued less tha 5 tmes the tag sets ad s based o Zpf's Law [6]. Accodg to t we ca assume that vey ae wods would have lttle fomatve stegth ad could theefoe be emoved fom tag pocess wthout much ham to classfcato pefomace. A. Mal Pasg Scheme Much cae ad atteto was dected to assue the pope pasg of a emal message. Fst of all, a multpat message was ufed to a sgle mal body cosstg oly of textual (pla text ad HTML) peces. No pat of message heade s used the classfcato pocess. Also I esue that tasfe ecodg lke `Quoted-ptable' ad `Base64' (see []) ae decoded as they ae vey ofte used to obfuscate emals. I the ext step HTML pasg s used to locate those pats of the text, whch ae vsble o vey dffcult to see by huma eadg the emal. A popula spammes techque s based o cludg some lettes, wods o eve fagmets of text sde the physcal message wth ethe fot colo equal to fot backgoud o mmal fot sze. Ths esults esults hdg pats of a text fom the eye of a use but potetally ot 47

4 Wold Academy of Scece, Egeeg ad Techology fom a classfyg ege. Fally, all the HTML tags o pseudo-html tags (aga used by spammes to obfuscate a emal) togethe wth HTML commets ad othe specal chaactes (lke ASCII Le Feed ad Caage Retu) ae emoved fom mal body leavg oly a sequece of legth of pla wods. It s used to ceate a featue vecto. III. EMPIRICAL RESULTS I ths secto the codtos of expemets ad the esults wll be peseted. A. Tag Sets Both of classfes wee taught wth a basc set of solcted mals ad spams. Solcted mals wee adomly chose fom use malboxes ad caefully vefed ot to cota ay spams. Spam set was collected dug the whole yea Those sets wll be efeeced to as basc tag sets futhe o. Tag set of spams was elaged by 963 spam mals epoted by CERN mal uses dug Decembe 2004 ad begg of Jauay Ths goup of spams wll be late efeeced to as addtoal spam set. Both classfes wee taed wth basc tag set ehaced by a addtoal oe. B. Tests Tests volve vefyg esults povded by both classfes o a gve set of spam o solcted mals. I vefy how spam detectablty could be mpoved by classfyg a emal as spam f ay of the classfes epots t as such. The most popula pefomace measues used the lteatue o spam ecogto ae spam pecso (p) ad spam ecall(): s s p = (9) s s + l s s s = (0) s s + s l whee s meas umbe of spams classfed coectly. l ad s l mea espectvely a umbe of legtmate emals ad spams classfed coectly. To facltate pesetato of esults, a abbevato NBC wll mea a aïve Bayesa classfe based o sgle wods ad NBC 2 a aïve Bayesa classfe based o adjacet pas of wods. ) Spams Repoted by Uses Ths test s pefomed o goup of spam emals epoted by uses betwee Decembe 2004 ad begg of Jauay The set used fo ths test ad the addtoal spam set wee fomed depedetly. Results of the test ca be see Table I. TABLE I SPAM REPORTED BY CERN USERS Classfcato codto s Recall NBC % NBC % NBC NBC % NBC NBC % We ca see that combg both classfes ca mpove the ecall by about 7%. Addtoal mals detected wth help of NBC 2 epeset ealy 34% of all of the false egatves ot detected by NBC. 2) Solcted Mals Extacted fom Uses Spam pecso s much moe mpotat tha spam ecall fom the pot of vew of a emal use who should be cofdet about ecevg solcted mals. Solutos leadg to spam ecall cease should theefoe be tated wth cauto beag md a possble decle of classfcato pecso. Ths ca be moe costly fom a use pot of vew tha a se of ecall ad theefoe questo the whole modfcato. To aalyze ths poblem I extacted adom mals fom CERN malboxes. Ths goup of mals was the classfed wth both NBCs taed wth all avalable tag sets (basc ad addtoal). All coectly classfed examples wee vefed to exclude the possblty of meetg eal spam the mals take fom use accouts. To vefy the mpact o each classfe's pecso, the same set of mals was the checked by NBC taught oly wth basc tag set. The esults ae peseted Table II. TABLE II LEGITIMATE S Classfe l l afte vefcato NBC NBC a NBC a Retaed usg oly basc tag set The esults clealy dcate supeoty of NBC 2 ove NBC tems of pecso. Futhemoe, they show that expadg the spam tag set the case of NBC ca lead to seous deteoato of classfe pecso. I ou case we eceve twce moe of false-postves. 3) Spam fom spamachve.og Ths test was pefomed o 5448 spams fom the begg of Jauay 2005 stoed SpamAchve ( Both NBCs wee taed wth basc ad addtoal tag sets. The esults ae peseted Table III. TABLE III SPAM FROM SPAMARCHIVE Classfcato codto s l Recall NBC % NBC % NBC NBC % NBC NBC % 48

5 Wold Academy of Scece, Egeeg ad Techology The esults clealy dcate that usg NBC 2 aloe wll esult vey poo ecall of spam. Howeve, combg output fom both classfes ca esult detectg ove 4% moe of spam. Moeove, a cosdeable goup of spam s detected by both classfes. Ths fact ca be used to teat those spams wth a specal cofdece. Fo example they could be ejected at the SMTP level wthout a eal sk of ot delveg a eal message. A geeally lowe pefomace of NBCs tha the case of spams collected at CERN ca be explaed by a choce of tag sets. They cosst oly of spams fom CERN, whch ae dffeet fom messages eceved by othe ogazatos. Ths fact udeles the ecessty of adjustg tag sets to patcula ogazato's eeds. 4) Real-Wold Evomet The cuet veso of SpamKlle mplemeted at CERN to flte all comg mals s usg a dual-nbc stuctue descbed ths atcle. Pevously t was based oly o NBC taed wth basc tag set, detcal to ths used above tests. Fom the begg of 2005, all the spams epoted by CERN mal uses automatcally elage a set of spams used to ta NBC 2. So fact evey comg mal s checked by two NBCs ad a decso of ay of them s fact suffcet to teat a emal as a spam. Fgue 2 shows daly statstcs of SpamKlle fo Febuay Fg.2 Pefomace of aïve Bayesa classfes wokg at CERN It s woth otg that SpamKlle aalyzes oly pat of the spam avg at CERN SMTP gateways. The majoty of t s ejected wth ealy level mal classfcato based fo example o blacklsts of teet addesses. Oly the less clea cases of spam ae let ad subject to cotet aalyss. We ca see that dug the whole moth the pefomace of NBC s degadg. Ths s elated to evolvg atue of spam. We should ot foget that ths classfe s ot beg etaed ad ts kowledge s based o costat tag set. Meawhle the spam chaactestc s chagg, povg oce aga the eed to dyamcally ta the classfe. The latte classfe, based o bgams, s costatly etaed wth use-epoted spams. The amout of detected spam s cosdeably lowe tha the case of NBC. Ths s udestadable as ts codtos of classfcato ae much stcte tha case of NBC. Howeve, the amout of detected spam by NBC 2 s ot deceasg; fact s eve slowly gog up dug the moth. It dcates that the classfe s adaptg to chagg spam pattes ad ts kowledge s ot agg. Fally, t s woth metog that dug moe tha two moths of opeatos of NBC 2 thee was ot a sgle sgal fom the use sde about possble false-postves wheeas thee wee seveal cases of false-postves elated to the wok of NBC.Ad stll the spam tag set was at the begg of Mach 2005 about two tmes lage tha the tag set cotag legtmate emals. The tag pocess ca be theefoe descbed as ot eedg opeato assstace. IV. CONCLUSION The esults show that etag aïve Bayesa classfe based o sgle wods ca, apat fom sg spam ecall, deteoate seously a classfcato pecso. I pactce, ths sde effect should eglect such a opto as a espose to chagg pattes of spam. Howeve, suppotg ths classfe wth aothe oe, based o pas of adjacet wods ca esult both ecall cease ad steady pecso. Of couse, the addtoal classfe decsos ae legtmate-based. Oe of the easos fo ths s a teatmet of pevously usee bgams, whch ae a po cosdeed as stogly legtmate-boud. Ths tu allows us to safely beak the balace betwee the stegth of legtmate ad spam tag sets. These featues ca allow ceatg a at-spam system that wll automatcally copoate kowledge about ew spams epoted by uses. Rug of such a system ca be assued wth mmal umbe of huma tevetos as a successful mplemetato at CERN Mal Sevces showed. Nevetheless, stll much could be doe to mpove the spam ecall of the addtoal classfe. Aeas of mpovemet should clude paametezato (e.g. the legth of featue vecto) ad fght wth mal obfuscato whch flueces the decsos made by automatc classfes a egatve way. I my opo, whateve steps wll be pocued, the pecso of the classfe should always be kept o as hgh level as possble. To sum up, some questos coceg ole etag of both of the classfes wll be dscussed the followg sectos. ) Retag NBC based o Sgle Wods Naïve Bayesa classfe ca be easly etaed afte tal tag phase by addg ew examples to ethe the goup of legtmate mals o spams. Howeve, thee s lttle eed to elage a set of legtmate mals as the categoy s geeal statc ad s ot subject to seous modfcatos wth tme. Ths s ot the case of spam as ths tag set ca be expaded wth spam examples that ae coectly classfed by the classfe. It should be oted that etag a NBC based o sgle wods has seous dawbacks: 49

6 Wold Academy of Scece, Egeeg ad Techology A cosdeable amout of examples s eeded to alte the pobablty of featues (wod that wee peset both tag sets. Ths s cotay to the eed of classfe adaptablty as patcula wods peset ewly obseved spam wll stll be cosdeed as legtmate o eutal dug classfcato task. Iceasg a spam tag set too much may esult may eutal ealty featues ecevg a hgh spam pobablty just because of the fact that they wee ot peset the legtmate tag set. Behd evey wod thee s ts sematcs. It becomes evdet that the most tellget spam s costucted wth as ocet as possble vocabulay whch makes the fomatve cotet of a spam blued ad udefed. Two fst pots ae elated to elatvely small featue space. Of couse, the secod pot oe be addessed by equally addg examples to legtmate tag set. Ufotuately, ths ca be dffcult fo easos of pvacy. Futhemoe, smply extactg emals fom use malboxes may clude a umbe of spams to the legtmate tag set whch tu could heavly lowe the classfe pefomace. 2) Retag NBC based o Bgams My expemets show that combg use feedback wth aïve Bayesa classfe epesetg a emal as set of bgams (pas of adjacet wod ca lagely elmate poblems metoed pevous secto. Such soluto ca be used as a complemet of the NBC based o ugams. Usefuless of pas of wods to epeset a emal s elated to a exta dmeso of featue space, whch tu lagely ceases the vocabulay sze. Patcula featues (udestood as pas of adjacet wod have much lowe pobablty of epeatg tself whe ceasg the tag sets. Ths fact makes t possble to easly extact fgepts of a ew type of spam epoted by uses: usually t wll cota a set of bgams usee legtmate tag set. Retag the classfe wth a ceta umbe of examples of udetected spam wll expad the dctoay wth some wod pas wth vey hgh pobablty of epesetg a spam. Moeove, because of addg a featue dmeso, t s possble to dastcally chage the popotos of tag sets. Amout of spam used to ta the classfe ca be ow much hghe tha amout of solcted mals as patcula spam featues wll stll be aely met solcted categoy. Fally, such a classfe, fom the theoetcal pot of vew, wll base ts decsos moe o a wod stuctues used a emal tha o a sematc meag of patcula wods, whch ca be msleadg. Spammes, as dscussed befoe, ted to avod usg suspcous wods ad specal message stuctues wth goal of makg the spam as smla to a oday emal as possble ad to fool ths way the at-spam eges. Bgam epesetato could be egaded to a ceta extet as a aswe to spams costucted to be solcted-lke because t s moe sestve o patcula phases that ca deteme a spam such cases. ACKNOWLEDGMENT The autho would lke to wamly thak d Paweł Cchosz fo valuable commets o ths pape. Specal expessos of gatefuless must be dected towads Iteet Sevces goup at CERN ad especally M. Emmauel Omacey (ceato of SpamKlle) fo makg possble the completo of ths aalyss. REFERENCES [] P.Gaham. (2002, August). A Pla fo Spam [Ole]. Avalable: [2] P. Gaham, Bette Bayesa Flteg, Poceedgs of Spam Cofeece Avalable: [3] I.Adoutsopoulos, J.Koutsas, K.V.Chados, G.Palouas, C.D. Spyopoulos, A evaluato of aïve Bayesa at-spam flteg, Wokshop o Mache Tag the New Ifomato Age [4] R.Segal, J.Cawfod, J.Kephat, B.Leba, SpamGuu: A Etepse At-Spam Flteg System, Poceedgs of Fst Cofeece o Emal ad At-Spam (CEAS) [5] K. Aas, L. Ekvl. Text categozato: A suvey, Techcal epot, Nowega Computg Cete, 999. [6] G. Zpf, Huma Behavo ad the Pcple of Least Effot. Addso-Wesley, 949. [7] C. M. Ta, Y. F. Wag, C. D. Lee, The Use of BGams to Ehace Text Categozato, Joual Ifomato Pocessg ad Maagemet., vol. 30, No. 4, pp , [8] H. Ste. Optmzg Naïve Bayesa Netwoks fo Spam Detecto, CSCI 6509: Natual Laguage Pocessg poject, Dalhouse Uvesty, Halfax, NS, Caada, [9] T. Mtchell, Mache leag. McGaw Hll, 997. [0] N. Dalv, P. Domgos, Mausam, S. Sagha, D. Vema. Advesaal Classfcato, Poceedgs of the Teth Iteatoal Cofeece o Kowledge Dscovey ad Data Mg (pp ), [] Multpupose Iteet Mal Extesos (MIME) Pat Oe: Fomat of Iteet Message Bodes. Request fo Commets 2045, 996. [2] Euopea Ogazato fo Nuclea Reseach. Mal Sevce Web Ste. [3] Euopea Ogazato fo Nuclea Reseach. At-Spam Web Ste. [4] M. Fombege. Bayesa Classfcato of Usolcted E-Mal, upublshed. Avalable: 50

Risk Evaluation in Auto Spare Parts Transport Based on the

Risk Evaluation in Auto Spare Parts Transport Based on the MAEC Web of Cofeeces 100, 05060 Rsk Evaluato Auto Spae Pats aspot Based o the AHP Method Rog Zeg 1 ad Chag Xu 2 1 Wuha Huaxa Uvesty of echology, Wuha, Cha 2 Wuha Uvesty of echology, Wuha, Cha Abstact By