Hidden Markov Model and Chapman Kolmogrov for Protein Structures Prediction from Images

Transcription

1 Hidden Markov Model and Chapman Kolmogrov for Proein Srucures Predicion from Images Md.Sarwar Kamal 1, Linkon Chowdhury 2, Mohammad Ibrahim Khan 2, Amira S. Ashour 3, João Manuel R.S. Tavares 4, Nilanjan Dey 5 1 Eas Wes Universiy Bangladesh, sarwar.saubdcoxbazar@gmail.com 2 Chiagong Universiy of Engineering and Technology, linkoncue@gmail.com 2 Chiagong Universiy of Engineering and Technology, muhammad_ikhancue@yahoo.com 3 Deparmen of Elecronics and Elecrical Communicaions Engineering, Faculy of Engineering, Tana Universiy, Egyp ( amirasashour@yahoo.com) 4 Insiuo de Ciência e Inovação em Engenharia Mecânica e Engenharia Indusrial, Deparameno de Engenharia Mecânica, Faculdade de Engenharia, Universidade do Poro, Poro, Porugal ( avares@fe.up.p) 5 Deparmen of Informaion Technology, Techno India College of Technology, Wes Bengal, , India ( neelanjan.dey@gmail.com). Absrac: Proein srucure predicion and analysis are more significan for living organs o perfec asses he living organ funcionaliies. Several proein srucure predicion mehods use neural nework (NN). However, he Hidden Markov model is more inerpreable and effecive for more biological daa analysis compared o he NN. I employs saisical daa analysis o enhance he predicion accuracy. The curren work proposed a proein predicion approach from proein images based on Hidden Markov Model and Chapman Kolmogrov equaion. Iniially, a preprocessing sage was applied for proein images binarizaion using Osu echnique in order o conver he proein image ino binary marix. Subsequenly, wo couning algorihms, namely he Flood fill and Warshall are employed o classify he proein srucures. Finally, Hidden Markov model and Chapman Kolmogrov equaion are applied on he classified srucures for predicing he proein srucure. The execuion ime and algorihmic performances are measured o evaluae he primary, secondary and eriary proein srucure predicion. Keyword: Hidden Markov Model, Chapman Kolmogrov equaion, Flood Fill, Warshall algorihm, Teriary Srucure 1. Inroducion Proeins are he complex biological organic and macromolecules paerns. For body cells, issues and organs srucure and funcionaliies, porions have a significan role. Proeins consis of amino acids and form a pepide bond by joining differen pepide bond. Differen proein srucures have differen funcionaliies as enzymes or form enzymes subunis of proeins play a srucural or mechanical role. Some proeins are used o ranspor various srucural ligands and some funcion in immune response. Proeins provide he organism wih cerain amino acids ha are no synhesized by ha organism. An amino acid is macromolecule ha conains boh an amino group and a carboxylic acid group. An amino acid forms a pepide bond afer loses a waer molecule. There are 20 differen amino acids in naure ha form proeins. These 20 are encoded by he universal geneic code. Nine sandard amino acids are called "essenial" for humans because hey canno be creaed from oher compounds by he human body. Proeins consis of amino acids joined ogeher by pepide bond o form a polypepide. Resolving he funcional variaion of proein is challenging issue in molecular biology [1]. Srucural knowledge is imporan for proein funcionaliy analysis. Three-dimensional proein srucure predicion is considered he greaes challenge for he srucural biologis. Many compuaional sudies and echniques were conduced for srucural proein analysis. Such echniques include he evoluionary algorihm [2,3], Mone carlo [4,5] and HP model [6,7]. Geneic algorihm (GA) can be chosen o solve he Proein Srucure Problem (PSP). Is performance varies due o he assigned GA parameers. The GA compuaional seps are involved o find he opimal sraegies for large space. Differen GA varians [8-17] are used for solving he PSP. This GA varians performance varies due o he differen inpus parameers. Some GA approaches have less performance due o he manual uning of he GA parameers by rail-and-error mehod. Furhermore, several biological srucures ypes are preprocessed by using image processing approach. Binarizaion approach or hreshold approach are used for pre-processing [18]. Image hresholding or hisogram can be used o 1

2 classify he objecs ino inra-/iner- classes [18]. Muli-level hresholding is anoher pre-processing algorihm ha segmen he color image based enropy [19]. The fuzzy C-mean and rough se approaches are clusering and classificaion approaches [20-21].Osu is anoher hresholding mehod ha used he hreshold globally and conver a color image ino binary image [22]. Afer pre-processing differen ypes of machine approaches are used for proein srucural predicion [23]. Hidden Markov Model (HMM) [24-26], and Suppor vecor Machine (SVM) [27-28] are more accurae mach predicion approaches for proein srucures. Typically, he HMM is a saisical approach, where he probabiliies of cerain predicion of wo consecuive erms are calculaed. I is applied for biological daa analysis such as he secondary proein srucures, Ribonucleic acid (RNA) secondary predicion, muliple sequence alignmen, and Genome/Gene predicion. In addiion, when HMM operaes on proein daa ses under some condiion or parameers, i is known as Markov Chain (MC). Since he RNA conains differen amino acid srucures. Thus, he MC can predic easily cerain properies based on cerain parameers from large proein srucure daases. Furhermore, he Chapman Kolmogrov can predic any erms of consecuive. Consequenly, in he presen work, Hidden Markov Model and Chapman Kolmogrov equaion are applied for proein predicion. The proein srucure classificaion is performed by using Chapman Kolmogrov and Hidden Markov Model. Osu mehod is employed iniially o conver he proein image ino binary images as a preprocessing sep. A srucural predicion approach is proposed o predic differen ypes of proein srucures hrough wo phases, namely: RNA srucure filer phase and predicion phase. In he filer phase, Osu mehod is used as preprocessing o conver he srucural images ino binary marix. Image noise is removed in pre-processing phase. Flood fill and Warshall algorihms are used for classificaion srucures. Furhermore, anoher wo machine learning predicion approaches, namely Hidden Markov Model and Chapman Kolmogrov models are used o find he posiion of he ones and o mach wih he rained daa. The srucure of proeins srucure is prediced based on one posiion. The proposed approach classified primary, secondary and eriary srucures by using he proposed machine predicion approaches. 2. Lieraure Review Compuaional proein design is a challenging issue due o heir single sie muaion and redesign of proein [29-32]. In naure, proein srucural design also challenging due o he proein binding affiniy/specificiy, enzymaic aciviy [33-34], new folds of proein creaion [35] and funcionaliies [36]. Proein design has been used o find he sequence consrains generaion of specific folds or funcions [37-39]. These consrains are used o address he physical evaluaion. Differen ypes of compuaional approaches are used o predic he proein physical srucure. Mainly, hree compuaional approaches are used for predicing srucure of proein: hreading or folding reorganizaion, comparaive modeling and ab iniio predicion. The comparaive modeling or homology modeling requires one or more 3D homologous proein srucures. Proein fold recogniion handles he non-homologous proein srucures. However, all ypes of proein srucures have he similar folds. Several ypes of machine learning algorihm based predicors have been developed o predic proein folds [40-43] and proein srucural classes [44,- 46] using feaures, such as he secondary srucure profile, HMM profile and PSSM (posiion-specific scoring marices) profile. However, ab iniio approach for PSP is he mos challenging, which is no previously solved srucures. The ab iniio predicion of proein srucure can illusrae he 3D srucure of proein ransformaion process from sequence from ino srucure from scrach. To solve PSP, an efficien predicion algorihm along wih an opimal energy funcion [47-49]. A well-defined finess or energy funcion o recognize he final goal from he random conformaion led o he simplified model based PSP problem [50]. Furhermore, he GA, which is an adapive heurisic search and opimizaion algorihm, is used for PSP [51-53]. Nowadays, Eva on-line evaluaion [54] is he op performing mehods include several approaches based on neural neworks, e.g. PSIPRED [55], PROFsec and PHDpsi [56]. Recenly, researches applied he SVM (suppor vecor machine) for secondary srucure predicion [57-59]. Anoher effecive research mehodology for secondary srucure predicion is he Hidden Markov Models (HMM).These models show heir abiliy by allowing an explici modeling of he daa. Asai e al. [60] prediced secondary srucure using HMMs. Four sub-models were applied separaely for paricular local srucures, namely alpha, bea, coil and urns in pre-clusered phase sequences. The sub-models conain four or five hidden saes merged ino a single model. Each model achieved 54.7% accuracy 2

3 rae. In order o represen specific classes of proein, a collecion of HMMs algorihm were used [61-62].These models were consruced a generalized approach ha reduce he conneciviy and surface loops or urn size [61]. This involved wo ypes of disinc helices posiion, namely: he N-cap and C-cap posiions in helices. An explici model also designs for amphipaic helices and β-urns. The HMM was applied o perform he secondary srucure predicion. For several amino acid group predicions, semi-hmm model, which is an exension of he HMM was also applied [63-65]. These models allow an explici consideraion for differen lengh of secondary srucures. Anoher machine learning approaches by using homologous sequence was proposed wih single sequences wih higher accuracy [65-66]. A novel HMM was depiced for proein secondary srucures by using prior biological knowledge [67]. By using biological knowledge, HMM is an ineresing ool o reveal hidden feaures of he inernal archiecure of secondary srucures. Novel HMM firs analyze in deail he model and predicive poenial on single sequences and on muliple sequence informaion. In his approach, an evaluaion daa se of 506 sequences and daa se of 212 sequences obained from he EVA Web sie [68]. Novel HMM more successfully predic secondary srucure han radiional HMM. Differen ypes of biomedical and scienific image processing approach are used for feaure or srucure selecion. SCIFIO (SCienific Image Forma Inpu and Oupu) is a library ha a Bio-Formas o creae a domain-independen image I/O framework [78].The goal of SCIFIO is o inegrae he scienific formas beween Digial image and medicine. DICOM [79], Flexible Image Transpor Sysem (FITS) [80] and necdf [81] are common image I/O framework for scienific and srucural daa analysis. Several graph heory are approaches are applied in order o idenify he proein ineracion. Dynamic PPIs Alignmen Sysem (DPPIsAS) [82] is an algorihm based sysem ha can be employed o find ou he proeins associaed ineracion by calculaing he proein Road Discovery (PRD) in a cerain nework. Bi-parie graph heory and Proein Road Mainenance (PRM) are used for finding he proein pah discovery. Canonical Correlaion Analysis (CCA) is also used o find ou he degree of correlaion among he proeins in a cerain nework. Furhermore, differen ypes of compuaional approaches are carried ou o classify reliable and unreliable PPIs [83]. Two mehods for deecing he reliable PPIs and for evaluaing he performance several mehods have been repored. 3. Mehodology Differen machine learning and srucural predicion approaches are used for proein srucure predicion, where mahemaical and saisical approaches were applied o predic 2D and 3D srucure of proein srucure. Proein images noise as well as machine learning complexiy affecs negaively he proein srucure predicion. In he curren work, an approach combining image processing echniques and machine learning for proein srucure predicion. The proein srucure image is pre-processed and convered ino binary image as an array of 0 s and 1 s Preprocessing Noise removal, sharpness enhancemen and edges blaring are essenial before any furher processes. Differen ypes of filering approaches are used for noise removal and sharpness enhancemen. Several algorihms [69] including Gaussian filering, Weiner filering, frequency domain filering are used for pre-processing. However, such filers suffer from several problems, such as ineviable loss of image, loss of sharpness and ringing effecs. Differen rank algorihms are more popular rank algorihms are used for image filering [70]. Non-local means filering is more popular in rank algorihm approach for noise reducion [71]. I operaes on he average pixels raher han pixels values on heir neighbor saisics [71-72]. The non-local means filering calculaes he average weighing of he neighborhood pixels. In he curren work, he bilaeral filering algorihm is applied. I is a well-known non-local means filering for pre-processing ha performs edges preservaion and image enhancemen [73]. The bilaeral filer is used for edge-preservaion and noise reducion of images. A each pixel, he inensiy value is replaced by he inensiy values weighed average from nearby pixels. The weighs depend on he Euclidean disance of he pixels as well as he radiomeric differences. The bilaeral filer is applied on an image I inpu and generaes filered image I ou, which is formed as a weighed sum funcion from is neighborhood pixels θ. The generaed filered image is given by [73]: 3

4 I ou ( x, y) = 1 w å i, jîq å i, jîq w( x, y, j, i). I( x + j, y + i) The weigh w depends on he geomeric disance and color differences beween pixels ( xyand, ) ( x+ j, y+ i). In bilaeral filering, color and spaial coordinaes beween wo pixels are considered o find he pixels similariy for filering using he following expression: Here, 2 s and i + j (( I x+ j, y+ i) -I(, x y)) Wxyji (,,,) = exp.exp 2 2-2s -2b b are he variance of color and spaial pixels coordinaes ses. The weigh indicaes he similariy of color and spaial coordinaes pixels. Weigh is vary from color inensiy and disance of pixels. Comparing he value of wo pixels based on conen of image pach. The weigh of image pach (small square of images) is calculaed by: (1) (2) wxy (,,,) ji = exp vx ( + jy, + i) -vxy (, 2-2b 2 2 (3) The square norm of pixel wise pich difference ensures he average pixels of surrounding conens. Threedimensional (3D) summaion and calculaion of weigh in equaion (3) is adjused, where he similar paches are searched among he adjacen slices. In he pre-processing phase, sharpens of image is increased by using bilaeral filering algorihm. I reduces he noise and correcs he edges. Aferward, he image is convered o binary for furher enhancemen for he image qualiy as follows Binarizaion Binarizaion is a process of separaing he image pixels ino wo groups, namely whie pixels ha indicae he background pixels and black pixels ha indicae he foreground pixels. Image binarizaion is based on cerain hreshold value using one of he mos common echniques, namely Osu hresholding mehod [74]. Osu hresholding is an ieraive process over all hreshold values ha calculaes and measures he pixels spread as foreground or background. Osu mehod applies global hresholding for generaion binary marix. I is employed in he curren work is o find ou he opimal hreshold a which he sum of foreground and background spread is less. Osu approach generaes an inensiy bi-modal hisogram ha has sharp valley beween wo he peaks represening he foreground and background [75]. In hisogram graphically represen 256 pixels image inensiy in gray scale image and binary image consider wo inensiies (0 and` 1). In order o deploy he binarizaion of he image from gray one, le he pixels of a gray image represened in L gray levels. The number of pixels a level I is denoed by he oal number of pixels N = n + n n. Suppose he pixels are divided ino wo groups: foreground (P 1 2 L f ) and background (P b ) by a hreshold level, where P f denoes he pixels wih level [ 1, 2,.., ] wih level [ 1, 2,, L] and P b denoes he pixels + +.The calculaions o separae he foreground and background using he variance is based on he hreshold value. For background pixels class P b, he weigh, mean and variance are given respecively by: W b n = å (4) i= 1 N 4

5 In similar procedure, he foreground pixels class P f has: i= 1 µ = b å* n ån i= 1 2 å( - µ ) * b 2 i= 1 s = b W f i= 1 ån n N n (5) (6) L = å (7) i=+ 1 L i=+ 1 µ = f L å * n å n i=+ 1 L 2 å ( - µ ) * f 2 i=+ 1 s = f L å n i=+ 1 The wihin he class variance is hen calculaed from he sum of he wo variances associaed by heir weighs, which is expressed by: n w b b f f (8) (9) s = s * w + s * w (10) Finally, he weighed variance (class variance) is compared wih he hreshold value. All pixels wih a level less han he hreshold value are considered background and pixels wih values greaer han he hreshold value are considered foreground. The Osu mehod conver he gray image ino binary image based on hreshold value ha belongs a marix of 1 or 0 (binary marix). In he presen work, Osu mehod is applied o allow global hresholding. Global hresholding generally simple and i allows many variaions of hresholding for neighbor pixels. Bu i is failure when gray scale image range varies in locally. In addiion, oher binarizaion mehods are applied o compare he evaluaion he performance of Osu mehod. These echniques are: i) he local hresholding [76], which is a binarizaion process ha can conver he colored image ino binary image. Local hresholding deermine he hreshold value wih surrounding region. However, i requires more compuaion and i performs on uniforms neighbors. ii) Opimized hresholding [76] is anoher approach of binarizaion by using opimal mahemaical approach. I is more accurae han local hresholding hough is compuaional cos is high. The accuracy of his approach depends on mahemaical opimizaion. In order o evaluae he performance of he differen binarizaion mehods, some merics are measured including he Sandard deviaion, which indicaes he error rae of he binarizaion process. High sandard deviaion means less accuracy and maximum error. In addiion, he signal-o-noise raio (SNR) is also measured, which indicaes he srengh of removing noise. Furhermore, he mean is measured o indicae he average numbers of 1 s (ones). Afer he binarizaion process, he generaed binary marix is processed by hidden markov model and Champman kolmogrov approach. These approaches predic he RNA srucure paern using he 1/0 values Warshall s and Flood Fill Algorihm The Warshall and flood fill mehods are employed o process he binary marix afer he binarizaion process. Boh mehods are applied o measure he oal number of 1 s by couning he number of ones in he binary marix as hey predic he paern of 1 or 0 for he RNA srucure. The binary marix is spil ino differen porions o calculae he number of 1 s. Flood fill is a pixel couning process by using 4 conneced and 8 conneced neighbors. In he presen work, 4 conneced approach is applied for couning in every porioned porion. Furhermore, he Warshall algorihm is also a couning approach ha convers he binary marix ino adjacen marix. Transiive closure or relaion is generaed from adjacen marix from he couning 1 s. Simple Boolean operaion is performed in Warshall approach ha is more efficien han flood fill algorihm. Muliple couning processes are performed in he flood fill algorihm ha reduces he sysem performance. Typically, he Warshall algorihm for couning 1 s in binary marix designs an 5

6 adjacen marix of ransiive dependency. I generaes he ransiive binary marix o calculae he number of 1 s. Differen proein srucures belongs cerain number of 1 s. In he presen approach, an adjacen marix A is used by binary daa processing, where he generaed binary marix from he binarizaion procedure is considered as an adjacen marix. Thus, Warshall s algorihm, which is an efficien algorihm for finding ou he effecive adjacen marix of ransiive closure relaion R is employed. The relaion R derived from finie verices se S, where S be he v, v,..,. v and R is a relaion on S. The adjacen marix A of R is an n*n Boolean marix, which is 1 2 n finie se { } defined by: ìï 1, A = i, j í ï î0, if anedges from v ov if no edges from v ov i i j j (11) An adjacen marix T of he ransiive Closure R is generaed using he following algorihmic approach (Algorihm 1). Algorihm 1: Warshall s algorihm Sar Inpu: Adjacency marix A of relaion R on a se of n elemens Oupu: Adjacency marix T of he ransiive closure of R Iniialize he adjacen marix T:=A for j=1 o n for i=1 o n if T i,j =1 hen Apply Boolean OR operaion, A i = A i A j Endif End End T=A Sop In he presen work, he rained proein srucure convers ino binary adjacen marix. Since he es adjacen marix conains 1 or 0, coun he number of 1 in he adjacen marix. Aferward, he adjacen marix is compared wih he rained daases for cerain proein srucure. In order o coun number of 1 s in he binary marix, he flood fill algorihm is used. The flood fill is a linear searching approach ha sars from a seed as an ineracive flood sysem. Every pixel is conneced by 4 conneced way or 8 conneced way. I specifies a seed by poining o he inerior of he region o iniiae a flood operaion. I sars from a seed and flood he whole region unil he boundary region mee. From he seed, a search for 1 ha is conneced wih seed in 4 ways (Lef, righ, up and down) is performed, aferward anoher seeds are chosen. In he curren work, he recursive approach is applied o selec he seeds ha are called recursively unil he whole couning is being compleed. Generally, he flood fill recursive approach is basically used in pain sysem. In flood fill algorihm user selec a seed and color he conneced neighbor seeds. Flood fill performs in wo differen caegories: 4 conneced and 8 conneced neighbor. In he curren work, 4 conneced approach is used for flood fill algorihm. A seed a he (0,0) posiion in he binary marix is seleced. The conneced 4-neighbor posiion and coun he number of 1 are used. Aferward, he nex posiion of 0 or 1 is seleced as a seed, and hen scanned he 4 conneced neighbors and couned he 0 s or 1 s. This process coninues recursively unil scanning he whole binary marix is compleed. 3.5 Hidden Markov Model and Chapman Kolmogrov Model In order o find he posiions of he ones from binary marix, machine learning approaches, such as HMM and Chapman Kolmogrov (CK) are applied. The HMM is a recursive process in which he 1 s posiions are calculaed by 6

7 using cumulaive disribuion. Cumulaive disribuion probabiliy marixes are generaed for predicing he 1 s posiion. Marix muliplicaion operaions are performed in HMM process o enhance he sysem complexiy, while Chapman Kolmogrov (CK) has less sysem complexiy. Marix muliplicaion operaions are no performed in CK operaion ha enhances sysem performance. The probabiliies of 1 s posiion are calculaed direcly wihou consecuive marix operaions. Generally, he HMM requires consecuive marix operaion for finding cerain posiion of 1, while he CK can direcly find ou he cerain posiion of 1 s. If he posiion of 1 in raining daa is similar o rained dae he srucure of proeins. The predicaed posiions are chosen randomly. Typically, he mos common saisical approach for biological daa analysis is applied, namely he Hidden Morkov Model (HMM). Furhermore, he CK approach is efficienly predics a cerain posiion of 1 s superior o he HMM approach ha operaes on rained daa. I is applied o analyze several biological daa such as secondary proein srucures, RNA secondary predicion, muliple sequence alignmen, Polygeneic analysis and Gene predicion. HMM is based on calculaing he probabiliies of cerain predicion of wo consecuive erms. In some cases, HMM is known as Markov Chain (MC), when i operaes he whole daases under some manners. Since he DNA conains billion on nucleoides, MC can easily predic cerain properies based on cerain facors from large biological daases. Markov chain need limied memory space as well as ime o find specific iem from large and muliple biological iems. I can effecively handle and analyze he disconinuiy in long or very long DNA or proein sequences. The general expressions o define he overall impacs of he Markov Chain are as follows: Where, P + a b = å. P zy (12) a b xy P xz PH { } = P... P a 1 x01 x x b - (13) + 1x b PH { = y h = x} = å PH { = y h = zh, = x}. PH { = z/ h = x} (14) a+ b a a+ b a+ 1 a a+ 1 a z= 0 PH { = y h = x} = å P. P (15) a+ b a zy z= 0 xz b PH { = y h = x} = P (16) a+ b a xy Consequenly, he HMM is used o predic specific binary number posiion by measuring he probabiliy of one s posiion from he proposed binary marix. The probabiliy of one s posiions are varies for differen secondary proein srucure. Chapman Kolmogrov equaion is he opimal predicion characerisic under cerain condiion. Chapman Kolmogrov equaion finds ou he probabiliy from one sep o anoher sep of cerain evens. Transiions marix and ransiion equaion are design for Chapman Kolmogrov equaion. Boh HMM and Chapman Kolmogrov equaion are sochasic (random) process. In Morkov propery, he probabiliy disribuion of he curren value is condiionally independen of he series of pas value. On he oher hand, Chapman Kolmogrov equaion predics he discree value on cerain condiion. HMM perform cross check beween he rained and raining daases ha used for proein srucures predicion Proposed Mehod Generally, several researchers were ineresed wih employing image processing in he predicion of he proein srucures. He e al. [85] adaped a marix-maching algorihm from image processing o offer an effecive sequence maching process. The similariy degree of wo sequences was calculaed using a fas normalized cross-correlaion (FNCC) algorihm adjused from image processing. Cheia and Sarma [86] proposed an approach o predic he proein srucure using sof compuing echniques. The arificial neural nework (ANN), image processing and saisical echniques were carried ou o formulae he srucure for proein predicion. In he presen work, differen machine learning approaches are applied for paern re-organizaion using he procedure in Fig. 1. 7

8 Figure 1: The proposed inegraed predicion approach using machine learning approach The wo-/hree-dimensional RNA images are used as inpu o he proposed sysem as illusraed in Figure 1. These proein images are processed ino binary image by using Osu mehod is a binary marix afer he Bilaeral filering pre-processing sep. The binary marix is handled by using Warshall algorihm, Flood fills, Hidden markov model and Chapman Kolmogrov equaion for proein predicion. Typically, he flood fill and Warshall algorihms are used o classify he srucures by couning he number of one in binary marix ha classifies he srucure based on number of one. Markov chain and Chapman Kolmogrov equaion are used o esimae he probabiliy of he binary marix, where he flood fill and Warsheal algorihms are applied o measure he oal number of 1 s. Thus, hese approaches predic he paern of 1 or 0 for RNA srucure predicion. The final oupu from he Chapman Kolmogrov equaion is considered rained daa ha predic he proein 2D and 3D srucure. The obained oupu is compared wih he esed daa o predic he proein srucure. In addiion, he accuracy rae and sysem performance among he machine learning approaches are evaluaed. The proposed proein srucure predicion approach has been implemened by MahLab and PHP environmens. The MahLab handles he proein image and conver ino binary image, aferward he PHP environmen is designed o analyses he proein srucure by using differen machine approaches. The classified differen proein srucure has differen posiions of 1 and 0 ha opimized he classificaion accuracy. Three daases are used for he presen experimen including primary srucure; secondary srucure and Teriary srucure. Primary srucure is a simple sequence wih differen ypes of amino acids for Drosophila melanogaser spices. Differen ypes of primary srucures are used for srucure predicion. In he curren work, he experimenal of all 2D and 3D images are generaed by using Cyoscape ools [84]. We generae proein image based on differen amino acids ineracions. In primary srucure we selec small number of proein insance from daabase. 2D and 3D srucure proein images are generaed by using large volume of proein daa. Image size varies on differen ineracion of amino acids. When he ineraced amino acid ha generaed secondary and eriary image. We measure he image size from Cyoscape ools. Typically, he size (number of he 1 s and 0 s) of he primary srucure images is no more han 20KB. In he curren work, 85 differen ypes of primary images are used. In addiion, secondary images of size range beween 15 KB and 35 KB. Niney-six differen ypes of secondary srucure are collaed for proein srucure predicion. Furhermore, eriary proein srucure is colleced, where he eriary images have sizes of values beween 45 KB and 90 KB. In he presen work, 105 differen ypes of eriary srucures are generaed from Cyoscape ools for 8

9 proein srucure predicion. The srucure of proein is exraced from he differen ypes of proein images. Typically, he differen proein srucures have indicaed differen ypes of biological and organic funcionaliies in he living organs. Primary proein srucure indicaes he linear sequence of he amino acid ha formed by pepide bonds. In he curren work, he proein folds are exraced in he primary srucure. In he secondary proein srucure, he linear, unfold and helical feaures are exraced in he polypepide chains. Twised and bending feaures are exraced from eriary proein srucures. 4. Resuls Analysis and Discussion Iniially, he Bilaeral filering algorihm is employed as a pre-processing sep o aain sharp image as illusraed in Figure 2. (a) (b) Figure 2: Bilaeral filering algorihm as a pre-processing: (a) 3D images before pre-processing (b) Increase he sharpness afer pre-processing. Aferward he Osu mehod is applied o conver a color image ino binary image as illusraed in Figure 3. (a) (b) (c) Figure 3: Conversion process from gray image o binary image (a) A secondary proein gray image ha conains 256 colors, (b) Binary image ha conains wo color inensiies of values 0 or 1. (c) Hisogram ha separaes foreground and background color based on hreshold. The flood fill and Warshall approaches are used for couning he 1 s in he binary marix. Proein srucures are prediced based on he number of 1. The range of 1 s for differen primary, secondary and eriary proein srucures is assigned from he curren experimenal resuls. In addiion, here exis a difference in he shape of he proein srucures based on he differen number of ones in he differen proein srucures, which is used for he predicaion and classificaion. In primary srucure, he ranges of numbers of 1 are 102 o 114. In he secondary proein srucure, number of 1 s is greaer han 115. These couning performed by flood fill approach on binary marix. We classify primary and secondary srucure based on hese numbers on 1 s ranges. All of hese predicion algorihms operae on binary marix. Osu mehod is used o generae binary marix from a proein image. Markov chain and Chapman Kolmogrov equaion are used o measure he probabiliy of binary marix o classify he proein srucures. 9

10 The hree daases have been used for his experimen including primary srucure; secondary srucure and Teriary srucure as repored in Table 1. Table 1: Differen proein srucure, samples and size for hree daa ses Daase Number of Samples Sample size (KB) Primary Srucure Secondary Srucure o 35 Teriary Srucure o 90 In Table 1, he primary daa srucure image size is minimum han oher wo daa ses.this size refers oal number of 1 and 0 in differen proein srucures. Secondary srucure images size have average image size. In addiion, he eriary (3D) srucures have images of maximum size compared o he oher daases. Generally, using he proposed mehod indicae ha every binary image includes differen binary marices which have more similar or differen number of 1 s and 0 s. The binarizaion process s execuion ime for he differen srucure images is measured. Flood fill and Warshall algorihm is used for couning number of 1 s. I opimizes he sample classificaion and unnecessary srucures. Finally, he HMM and Chapman Kolmogrov approaches are used o predic he final proein srucures Performance analysis of he binarizaion approaches The binarizaion processes are varies based on he hreshold values. In he presen work, differen binarizaion mehods are applied on differen proein srucures, namely he Osu mehod, Local Thresholding (LT) and Opimized Thresholding (OT). The daases conain differen lengh of srucures. Primary srucure conains 85, Secondary srucure conains 96 and a Teriary srucure conains 105 proein srucures; respecively wih differen sizes, where increased he srucures size leads o increased binarizaion execuion ime. Differen performance parameers including he SNR, mean, sandard deviaion (SD) and execuion ime for each srucure are measured o compare he differen binarizaion approaches as illusraed in Table 2. Table 2: Measuremen of performance parameers for differen proein srucures Proein Daa Size Srucures (KB) Primary 0 o 5 6 o o 20 Secondary 15 o o o 35 Teriary 45 o o o 90 Mean Osu Mehod Local Thresholding Opimized Thresholding SNR SNR SNR Sandard Deviaion Mean Sandard Deviaion Mean Sandard Deviaion Table 2 esablishes ha he SD of Osu mehod is less han he ohers wo mehods. The accuracy rae of binary images conversaion is high for Osu mehod. In he primary srucures, he average SD of Osu, local hresholding 10

11 and he opimized hresholding are 10.1, and 13.91; respecively. However, he SD is increased for he secondary and eriary srucures by using Osu mehod. Opimized hresholding and local hresholding have maximum SD compared o Osu mehod for secondary and eriary srucures, which esablishes he superioriy of Osu mehod. Generally, Table 2 repors ha he maximum SD is for 3D proein srucure binarizaion process which daa size belongs 76 o 90 KB, while he minimum SD is 9.87 for primary srucures by using Osu mehod. In addiion, he opimized hresholding achieves average level SD for he secondary srucures. Figure 4 illusraes he performance of he differen binarizaion approaches in erms of he differen evaluaion merics repored in Table2. (a) (b) (c) (d) (e) (f) (a) (b) (c) Figure 4: Evaluaion of performance analysis for differen proein srucures binarizaion. (a) Sandard deviaion measuremen of differen proein srucures. (b) Mean value for differen proein binarizaion mehods. (c) Measuremen of SNR for hree proein binarizaion process. Figure 4 illusraes he performance evaluaion for hree binarizaion approaches for differen proein srucures. Primary srucures primary size sar from 5 o 20KB, secondary srucures belongs 30 o 40 KB and approach and eriary srucures sizes are 55 o 75 KB. Thus, wih he daa size increase, he SD of Osu mehod increases having less values compared o he oher wo binarizaion processes as demonsraed in Fig.4(a). In addiion, he SNR of he opimized hresholding process is higher han he local hresholding as demonsraed in Fig.4(c). Furhermore, he execuion ime of he hree binarizaion processes (Osu mehod, local hresholding and opimized hresholding) for differen proein srucures of binary images conversion is measured as given in Table 3. 11

12 Table 3: Measuremen of execuion for differen proein srucures Proein Srucures Daa Size (KB) Primary 0 o 5 6 o o 20 Secondary 15 o o o 35 Teriary 45 o o o 90 Osu Mehod (nano second) Local Thresholding (nano second) Opimized Thresholding (nano second) Table 3 depics ha he execuion ime is varies for differen proein srucure images, where he binarizaion ime depends on number of pixels and srucures. Thus, wih increased number of images and size, he binarizaion execuion ime is also increased and vice versa. In addiion, he execuion ime is also depends on he srucures of images (primary, secondary and eriary). Hence, for he primary srucure of proein size is 11 o 20 KB, he execuion ime of Osu mehod, local hresholding and opimized hresholding are ns, and ; respecively. The resuls depics ha he Osu mehod is more abou wice imes faser han he Opimized hresholding approach, where Osu process is ( )/ = 39.04% faser han he opimized hresholding for primary hresholding (Figure 4). Osu mehod also faser han oher wo approaches for secondary and eriary images. Osu mehod required less ime because i is used global hresholding for binarizaion. This global hresholding performed on wihin a class variable. The class variable separaes he foreground and background and generaes binary images. Figure 5 illusraes he execuion ime for Osu mehod, local hresholding and opimized hresholding for proein srucures predicion. Figure 5: Execuion ime for Osu mehod, local hresholding and opimized hresholding for differen proein srucures Figure 5 illusraes he execuion ime of Osu mehod, which is varies for secondary daase of differen images size han oher wo processes. For eriary srucure binarizaion execuion imes are varies due o differen images sizes Performance Analysis of Flood fill and Warshall approaches Afer he binarizaion process evaluaion, he assessmen for boh he flood fill and Warshall approaches ha used for he binary marix processing is performed. The execuion ime for oal number of 1 s couning is measured, where he execuion ime increases wih he binary size increase and vice versa as illusraed in Table 4. Binary image size depends on srucure of proein. Since differen proein srucures belongs o cerain number of 1 s. In he 12

13 curren work, he raining daase is used o generae he oal number of 1 in he binary marix and o classify he srucures based on heir 1 s values. Table 4: Execuion ime for flood fill and Warshall approach Proein Srucures Daa Size (KB) Warshall (nano second) Flood Fill (nano second) Primary 0 o 5 6 o o 20 Secondary 15 o o o 35 Teriary 45 o o o Table 4 depics ha he execuion ime of Flood fill and Warshall approaches varies for differen proein srucure lengh. The execuion or couning ime of heses approaches depends on he proein srucures and binary marix size. The execuion ime of boh approaches increases wih he increase in he size of binary marix and vice versa. Warshall approach convers he marix in adjacency marix and coun number of 1. I compares he number of 1 wih rained daa se and classifies he srucures ino primary, secondary or eriary srucures. In addiion, he flood fill algorihm is also employed o coun he number of 1 s. Is couning operaion is performed based on 4-conneced or 8-conneced approach. Flood fill algorihm chooses a posiion in binary marix and coun he number of 1 in 4- conneced neighbor. This process is coninued unil he whole binary marix scanning is compleed. I is also compared wih he rained daase and is used o classify he proein srucures. I required more execuion ime for similar daa operaion han Warshall approach. Figure 6 demonsraes he execuion ime comparison for boh approaches. Figure 6: Execuion ime for Warshall and flood fill algorihm for differen proein srucures Figure 6 illusraes he execuion ime for 1 s couning by using Warshall and flood fill algorihms, where he execuion ime of he Warshall algorihm is less han he flood fill algorihm s execuion ime. All of he approaches linearly increased due o proein srucure daa lengh. In he secondary srucure wih 26KB o 35KB, he execuion ime of Warshall algorihm is ns and flood fill algorihm is ns. is more abou 1.5 imes faser han flood fill approach. Warshall approach is ( )/ =71.94% faser han he flood fill algorihm. 13

14 4.3. Performance Evaluaion of HMM and Chapman Kolmogrov approaches In order o evaluae he used machine learning approaches, namely he HMM and Chapman Kolmogrov (CK), he execuion ime is measured for boh of hem. In addiion, he performance of he proposed approach is compared wih he self-organizaion geneic algorihm (SOGA) [77] echnique. Typically, he self-organizing sysem is a chemical, physical, or biological sysem wihou a cenral conrol. I is applied o obain he paern a global level of a sysem. In order o creae an auomaed opimized soluion, he SOGA has been used for Proein Srucure Predicion (PSP). Furhermore, he execuion ime of SOGA for differen proein srucures is measured. Typically, he performance rae of he SOGA approach depends on he muaion and crossover process. The complexiy of muaion and crossover process is increased wih he differen srucures, which indicaes high complexiy rae of SOGA. Therefore, he proposed approach ouperforms he SOGA as i easily handles he proein srucure variaion as demonsraed in Table 5. Table 5: Execuion ime for Hidden Markov model, Chapman Kolmogrov and SOGA approach Proein Srucures Daa Size (KB) CK (ns) HMM (ns) SOGA (ns) Primary 0 o o o Secondary 15 o o o Teriary 45 o o o Table 5 depics ha he execuion ime of HMM, CK and SOGA approaches are varies for differen proein srucure lengh. The HMM and CK s execuion ime depends on he proein srucures and number of predicion posiions. The predicion posiions indicae he presence 0 s or 1 s in cerain index of binary image. The binary marix is a wo dimensional array. The presence of 0 s or 1 s is found ou in he arge posiion from he binary marix. This arge posiion is seleced randomly. The execuion ime is increased for HMM and CK, when he size of arge posiions and proein srucures are varies and vice versa. The HMM and CK approach compares he number of 1 s wih he rained daase and classifies he srucures ino primary, secondary or eriary srucures. The SOGA measures he execuion ime for proein srucures for muaion and crossover. The execuion ime of SOGA is increased wih he increase of he daa volume. In primary srucure wih 11KB o 20KB, he execuion ime of HMM and CK approaches are ns and ns respecively. SOGA approach needs ns for same daa ses. CK needs less ime han HMM and SOGA. CK and HMM approach are 5.17% and 13.42% less execuion ime respecively han SOGA. Figure 7 demonsraes he comparison of execuion ime for he SOGA and he HMM as well as wih he CK. 14

15 (a) (b) Figure 7: Execuion ime for differen proein srucures (a) Comparison beween CK and SOGA approach based on execuion ime (b) Performance analysis beween HMM and SOGA based on execuion ime. Figure 7 illusraes ha for every daase he CK esimaes less execuion ime han he SOGA. The CK used less execuion ime because CK measure probabiliy from firs sep o anoher sep. On he oher hand, he SOGA uses he geneic algorihm, which is linearly search for he posiion conen. Figure 7 (b) indicaes he execuion ime of 1 s or 0 s predicion, showing ha he HMM requires less execuion ime compared o he SOGA algorihm. All of he approaches are linearly increased due o proein srucure daa lengh. Due o various lenghs wih crossover and muaion approach generaes significan number of processing seps ha need more ime. Furhermore, a comparison beween he hybrid processes wih HMM & SOGA and CK & SOGA is demonsraed in Figure 8. Figure 8: Comparison beween HMM & SOGA and CK & SOGA based on execuion ime Figure 8 depics ha for every daases he hybrid approach wih CK & SOGA esimaes less execuion ime han HMM & SOGA. Hybrid process wih wo approaches generaes high accuracy han single approaches. Since he F-measure indicaes he false predicion of he daa sample and he accuracy rae means he rue predicion rae of daa rae. In he curren work, he F-measure and accuracy rae for differen proein srucures are measured. Differen raining ses are depiced of differen proein srucures based on he rained daa. The HMM and CK approach on differen srucural daases are operaed. I is obvious ha, when he daases is increased and varies, he accuracy raes are also changed. Table 6 repors he execuion ime, sensiiviy and F-measure for HMM, CK and SOGA. 15

16 Table 6: Accuracy rae and F-measure for Hidden Markov model, Chapman Kolmogrov and SOGA approach Proein Srucure s Number of Sample CK (ns) HMM (ns) SOGA (ns) F-measure Accuracy F-measure Accuracy F-measure Accuracy Primary % 88.55% 14.67% 85.33% 19.23% 80.77% Secondary % 86.76% 17.26% 82.74% 22.21% 77.79% Teriary % 83.24% 19.24% 80.76% 24.23% 75.77% Table 6 includes he performance merics measuremens for he 161 primary srucures, 134 secondary srucures and 93 eriary srucures. I is obvious ha he accuracy rae is high for primary srucure compared o he secondary and eriary srucures. Consequenly, when he approaches are applied on eriary daases, he accuracy rae is decreases and F-measure is increased. However, for all he srucures, he CK s accuracy rae is high compared o he HMM and SOGA. The HMM accuracy rae is high and he F-measure is low in primary srucures. On he oher hand, he SOGA accuracy rae is less for every ype of he proein srucures. Figure 9 demonsraes he accuracy and he F-measure values for he differen daase s size. (a) (b) Figure 9: Measuremen accuracy rae and F-measure for differen proein srucures (a) Comparison beween CK, HMM and SOGA approach based on accuracy rae (b) Performance analysis beween CK, HMM and SOGA based on F-measure. The bar chars illusrae he comparison beween Chapman Kolmogrov, HMM approach and SOGA based on accuracy and F-measure in Figure 9. For every daase, he CK esimaes high accuracy and less F-measure han he SOGA. On he oher hand, he SOGA used geneic approach ha linearly searches posiion conen. The HMM also achieves less F-measure compared o he SOGA mehod for every ype of he proein srucures. The preceding resuls esablished ha he HMM is accuraely measure he primary srucure and rae gradually decrease for secondary and eriary approaches. However, he accuracy values decreased wih he increase in he proein srucure dimension as he number of samples for each srucure increased. Consequenly, i is suggesed o apply oher opimizaion algorihms o suppor he classifier more accuraely. Furhermore, in he curren proposed approach he coun of he ones was used, however, in furher works he coun of he number of zeros can be used. 16

17 Conclusion The curren work proposed Hidden markov model and Chapman Kolmogrov for classifying differen ypes of proein srucures. These machine learning approaches are opimal and require less execuion ime for proein srucure predicion. Boh HMM and CK approaches operaed on binary marix. The binary marix is generaed by Osu mehod ha convers proein images ino binary marix. Flood fill and Warshall algorihm were used for couning 1 s from he binary marix for furher proein srucures classificaion. The HMM and CK were used hen were applied o classify he srucures. The simulaion resuls esablished ha he HMM and CK are effecive for proein srucures analysis based on proein srucures images. The resuls esablished ha higher accuracy was obained o classify he primary srucures compared o classifying secondary and eriary srucures. Therefore, he accuracy rae decreased while classifying he eriary daases. For all srucures, he accuracy rae using he CK mehod is higher compared o he HMM and SOGA. Insead, he SOGA accuracy rae is less for all proein srucures. References: [1] Manish Kumar An Enhanced Algorihm For Muliple Sequence Alignmen Of Proein Sequences Using Geneic Algorihm. EXCLI Journal;14: ,2015. [2] Lijun Quan, Qiang Lv, and Yang Zhang. STRUM: srucure-based predicion of proein sabiliy changes upon single-poin muaion. Bioinformaics, 2016, 1 11 [3] Brender,J.R. and Zhang,Y. (2015) Predicing he effec of muaions on proein-proein binding ineracions hrough srucure-based inerface profiles. PLoS Compu. Biol., 11, e [4] Carnevali P, Toh G, Toubassi G, Meshka SN. Fas proein srucure predicion using mone carlo simulaions wih modal moves. J Am Chem Soc 2003;125(47): [5] Lee B, Kurochkina N, Kang HS. Proein folding by a biased Mone Carlo procedure in he dihedral angle space.faseb J 1996;10(1): [6] Mandal S, Jana ND. Proein srucure predicion using 2D HP laice model based on ineger programming approach.in: Proceedings of 2012 Inernaional Congress on Informaics, Environmen, Energy and Applicaions.2012 Mar 17-18; Singapore. p [7] Beniez CM, Lopes HS. Proein srucure predicion wih he 3D-HP side-chain model using a maserslave parallel geneic algorihm. J Braz Compu Soc 2010;16(1); [8] Cui Y, Chen RS, Wong WH. Proein folding simulaion wih geneic algorihm and supersecondary srucure consrains. Proeins 1998;31(3): [9] Zhang X, Wang T, Luo H, Yang JY, Deng Y, Tang J, e al. 3D proein srucure predicion wih geneic abu search algorihm. BMC Sys Biol 2010;4 Suppl 1:S6. [10] Hoque MT, Chey M, Saar A. Geneic algorihm in ab iniio proein srucure predicion using low resoluion model: a review. In: Sidhu AS, Dillon TS, ediors. biomedical daa and applicaions. Heidelberg, Germany:Springer; p [11] Dandekar T, Argos P. Applying experimenal daa o proein fold predicion wih he geneic algorihm. Proein Eng 1997;10(8): [12] Conreras-Moreira B, Fizjohn PW, Offman M, Smih GR, Baes PA. Novel use of a geneic algorihm for proein srucure predicion: searching emplae and sequence alignmen space. Proeins 2003;53 Suppl 6: [13] Goldberg DE. Geneic algorihms in search, opimizaion and machine learning. Reading (MA): Addison- Wesley Publishing Co.; [14] Kaiser CE, Merkle LD, Lamon GB, Gaes GH Jr, Pacher R. Case sudies in proein srucure predicion wih realvalued geneic algorihms. In: Proceedings of he 8 h SIAM Conference on Parallel Processing for Scienific Compuing; 1997 Mar 14-17; Minneapolis, MN. [15] Day RO, Zydallis JB, Lamon GB, Pacher R. Solving h proein srucure predicion problem hrough a muli objecive geneic algorihm. In: Proceeding of he Inernaional Conference on Compuaional Nanoscience and Nanoechnology; 2002 Apr 21-25; San Juan, Puero Rico. p