Fast Intra- and Inter-Prediction Mode Decision in H.264 Advanced Video Coding

Transcription

1 130 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May 2008 Fast Intra- and Inter-Predcton Mode Decson n H.264 Advanced Vdeo Codng Mehd Jafar and Shohreh Kasae, Islamc Azad Unversty, Kerman Branch, Sharf unversty of Technology Summary H.264/AVC, the latest vdeo standard, adopts rate-dstorton optmzaton (RDO) technque to obtan the best ntra- and nterpredcton, whle maxmzng vsual qualty and mnmzng the requred btrate. However, the full RD cost calculaton for all ntra-predcton modes, the exhaustve searches for fndng optmal moton vectors for all block szes, and the multple references frame procedure consderably ncrease ts computatonal complexty wth the allowed number of predcton modes. In order to reduce the complexty, here we propose a new approach for both nter- and ntra-mode decsons, that takes nto account the two effectve parameters, mage content type and the quantzaton parameter. The proposed fast ntra-predcton mode selecton strategy uses some observatons on the nteror and the exteror MB propertes to select a subset of canddate modes at dfferent quantzaton parameters. Also, the fast nter-predcton mode decson approach uses splt/merge procedure based on correlaton of moton vectors and moton detals of vdeo objects. Also, here we use a context-based adaptve method to speed up the multple reference frames moton estmaton that s based on nter- and ntra-predcton resdues and quantzaton parameters. As such, only a subset of nter- and ntra-modes s chosen for RDO calculaton. Expermental results show that the proposed algorthm, reduces the total encodng cost wth neglgble loss n PSNR and a slghtly ncrease n the requred btrate, when compared to RDO and other fast algorthms reported n the lterature. Key words: H.264/AVC, ntra- and nter-predcton, rate-dstorton optmzaton, smlar predcted-pxels, splt/merge. 1. Introducton As recent multmeda applcatons are growng rapdly, vdeo compresson requres hgher performance as well as new features. The newest vdeo codng standard s developed by the jont of vdeo teams of ISO/IEC MPEG and ITU_T VCEG as the nternatonal standard (MPEG-4 part 10) advanced vdeo codng (AVC) [1, 2]. H.264/AVC has ganed more and more attenton; manly due to ts hgh codng effcency, mnor ncrease n decoder complexty compared to exstng standards, adaptaton to delay constrants, error robustness, and network frendlness [1, 2]. Table 1 [3] and Fgure 1 [4] show some performance comparsons usng MPEG-2, MPEG-4 (ASP), and H.264/AVC. To acheve an outstandng codng performance, H.264/AVC employs several powerful codng technques such as drectonal predcton of ntracoded blocks, nter-predcton wth varable block-sze moton compensaton, mult-reference frame moton estmaton, moton vectors wth quarter-pel accuracy, nloop deblockng flter, 4 4 nteger transform, and the forth. Accordng to these new features, the encoder computatonal complexty has extremely ncreased compared to the prevous standards. Ths makes H.264/AVC dffcult for applcatons wth low computatonal capabltes. Thus, untl now, the reducton of ts complexty s a challengng task. Table 1: Average bt-rate reducton compared to pror codng schemes[3]. Standards MPRG- H.263/HLP MPEG-2 4/ASP H.264/AVC 38.62% 48.80% 64.46% MPRG % 42.95% 4/ASP H.263/HLP % Fg. 1 Performance comparson of dfferent vdeo codng standards [4]. Lke prevous standards, H.264/AVC stll uses moton compensated transform codng. The mprovement n codng performance comes manly from the ntra- and nter-predcton part. In partcular, ntra-predcton at dfferent block szes and an mproved number of drectonal mode decsons contrbute to a better vdeo qualty ncrease. Also, nter-predcton usng moton estmaton (ME) at quarter-pxel accuracy wth varable block szes and multple reference frames greatly reduces the predcton error. H.264/AVC employs the Lagrange RDO method to fnd out the best codng mode of ntra-and nter-predcton wth hghest codng effcency. The RDO technque requres a lot of computatons snce t tests the encodng process wth all possble codng modes of ntraand nter-codng, and calculates ther RD costs to choose Manuscrpt receved May 5, 2008 Manuscrpt revsed May 20, 2008

2 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May the mode havng the mnmum requred btrate. The reference software of H.264/AVC, JM [15], adopts full search for both moton estmaton and ntra-predcton. The run-tme percentage of each functon s shown n Fgure 2 [20]. As shown n Fgure 2, moton estmaton s the most computatonally ntensve part. The nstructon profle of the reference software shows that real-tme encodng of CIF 30Hz vdeo (baselne optons, search range [-16.75, ], fve reference frames) requres 314,994 mllon nstructons per second and memory access of 471,299 MBytes/sec. Also, H.264/AVC offers a rch set of predcton patterns for ntra-predcton,.e., nne predcton modes for 4 4 luma blocks and four predcton modes for luma blocks. The ntra-predcton mode decson s very complex and the number of computng RD cost values for luma and chroma of a macroblock s 592 [5]. Thus, the computatonal burden of these types of brute forcesearchng algorthm s far more demandng than any exstng vdeo codng algorthm. Fg. 2 Run-tme percentages of functonal blocks n H.264/AVC baselne encoder [20]. Fast mode selecton for ntra- and nter-predcton s consdered n ths paper; whch s a challengng subject n H.264/AVC. To reduce the computatonal complexty, many algorthms have been proposed. For fast nterpredcton mode decson, the early termnaton technque [23] reduces the number of potental predcton modes. In [24], a classfcaton method s proposed. Recently fast nter-mode selecton algorthms were proposed n [25], [26] and [27] to allevate the encoder complexty due to the moton estmaton and nter-mode decsons. In [28], a fast nter-predcton mode decson based on pre-encodng process s presented. Fast ntra-mode decson algorthms usng edge detecton hstogram and local edge detecton are proposed n [1, 6, 7]. However, ther preprocessng stages stll consume a codng tme to detect the edge drecton and to classfy t nto a lmted drecton. Also, there exst fast algorthms to select the optmal ntra-predcton mode usng smple drectonal masks n [8] wth savng tme of 70%, and statstcal-based methods n [9] wth savng tme of 45%. Another fast ntra-mode decson scheme s proposed n [10], where the encodng speed s approxmately 30% faster than that of the RDO method. A new fast ntrapredcton algorthm based on macroblock propertes (FIPAMP) s presented n [11]. Ths algorthm can acheve 10% to 40% of computaton reducton whle mantanng smlar PSNR and btrate performance of H.264/AVC codes. In [12], an effcent ntra-predcton (EIP) algorthm based on early termnaton, selectve computaton of hghly probable modes, and partal computaton of the cost functon s presented. Also, an mproved cost functon to mprove the codng performance s proposed n [13]. In [14], a fast algorthm based on the local edge nformaton obtaned by calculatng edge feature parameters. Ths paper descrbes a new approach for both nter-and ntra-mode sze selecton, that takes nto account the content of the vdeo and the quantzaton parameter used to encode each macroblock. In fact, the mode selecton also depends on the quantzaton parameter, snce the vdeo qualty due to dfferent quantzaton levels nfluences the choces on dfferent encodng tools. Here, for ntra-predcton mode decson, we mprove Pan s method [1, 6, 7]. As an alternatve method, here we proposed a fast ntra-method usng statstcal propertes of adjacent MBs and reference pxels wth a combnaton of the presented algorthms n Pan s method. Also, for nter-predcton mode decson, the splt/merge procedure s used. In ths method a macroblock s dvded nto equal-sze quarters, and then usng the smlartes of moton vectors of adjacent blocks we show how to merge the sub-blocks for quarter dvsons. In ths paper, an effectve method for acceleratng the multple reference frames ME wthout sgnfcant loss of vdeo qualty s proposed, whch s based on analyzng the avalable nformaton obtaned from prevously processed frames. We have verfed the dfferent parts of the proposed algorthm step by step by mplementng t on JM7.1 [15] reference software and have compared ts performance wth RDO and other avalable fast algorthms. Expermental results show that the proposed method reduces the encodng cost up to 40% wth a neglgble loss n the reconstructed vdeo qualty and a neglgble ncrease n the requred btrate. The remanng parts of the paper are organzed as follows. We revew the full search RDO ntra- and nterpredcton scheme of H.264/AVC n Secton 2. Secton 3 and 4 present the fast ntra- and nter-predcton mode decson algorthms, respectvely. Also, complexty reducton of multple reference frames moton estmaton s consdered n Secton 4. Expermental results are gven n Secton 5 and fnally Secton 6 concludes the paper.

3 132 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May Mode Decson and RDO n H.264/AVC In common wth earler standards, H.264/AVC does not defne the encoder, but defnes the syntax of an encoded vdeo btstream together wth the method of decodng the btstream [16, 17]. The codec combnes ntra-pcture predcton wth nter-pcture predcton to explot the spatal and temporal redundancy, respectvely. 2.1 Intra-Predcton Mode Decson Intra-predcton s based on the observaton that adjacent macroblocks tend to have smlar propertes. Predcton may be formed for each 4 4 luma block (I4MB), luma MB (I16MB), and 8 8 chroma block. For predcton of 4 4 lumnance blocks, the 9 drectonal modes consst of a DC predcton (Mode 2) and 8 drectonal modes; labeled 0, 1, 3, 4, 5, 6, 7, and 8 as shown n Fgure 3. In Fgure 3, the block (values of pxels a to p ) s to be predcted usng A to Q pxel values. Fg. 3 Intra-predcton modes for 4 4 lumnance blocks. Labelng of predcton samples. The DC predcton (mode 2), useful for those blocks wth lttle or no local actvtes, the other modes (1-8) may only be used f all of the requred predcton samples are avalable. For regons wth less spatal detals (.e., flat regons), H.264/AVC supports ntra-codng; n whch one of four predcton modes (DC, vertcal, horzontal and planar) s chosen for predcton of the entre lumnance components of the macroblock[18]. H.264/AVC supports four chroma predcton modes for 8 8 chromnance blocks, smlar to that of the I16MB predcton, except that the order of mode numbers s dfferent: DC (Mode 0), horzontal (Mode 1), vertcal (Mode 2), and plane (Mode 3). 2.2 Inter-Predcton Mode Decson Inter-predcton s based on usng moton estmaton and compensaton to take advantage of the temporal redundances that exst between successve frames. The mportant dfferences from earler standards nclude the support for a range of block szes (down to 4 4), multple reference frames, and fne sub-pxel moton vectors (1/4 pxel n the luma component). H.264/AVC supports moton compensaton block szes rangng from to 4 4 lumnance samples wth many optons between the two. The lumnance component of each macroblock (16 16 samples) may be splt up n 4 ways as 16 16, 16 8, 8 16 or 8 8. If the 8 8 mode s chosen, each of the four 8 8 macroblock parttons wthn the macroblock may be splt n a further 4 ways as 8 8, 8 4, 4 8 or 4 4. These parttons and sub-parttons gve rse to a large number of possble combnatons wthn each macroblock. A separate moton vector s requred for each partton or sub-partton. Each moton vector must be coded and transmtted; n addton, the choce of partton(s) must be encoded n the compressed btstream. Choosng a large partton sze (e.g., 16 16, 16 8, 8 16) means that a small number of bts are requred to ndcate the choce of moton vector(s) and the type of partton; however, the moton compensated resdual may contan a sgnfcant amount of energy n frame areas wth hgh detals. Choosng a small partton sze (e.g., 8 4, 4 4, etc.) may gve a lower energy resdual after moton compensaton, but requres a larger number of bts to sgnal the moton vectors and the choce of partton(s). The choce of partton sze therefore has a sgnfcant mpact on compresson (a small partton sze may be benefcal for detaled areas). H.264/AVC as enhanced reference pcture selecton as H enables effcent codng by allowng an encoder to select, for moton compensaton purposes, among a large number of pctures that have been decoded and stored n the decoder RDO Procedure For I-frames all MBs are predcted as Intra. H.264/AVC encoder encodes the best mode usng all mode combnatons of luma and chroma and chooses the one that gves the best RDO performance. For P-frames, ntra- and nter-predcton s done and RDO s used to fnd the best predcton. The RDO procedure to encode one MB n an I- frame s gven n [5]. Accordng to the RDO procedure of ntra-predcton n H.264/AVC, the number of mode combnatons for luma and chroma blocks n a macroblock s N8 (16 N4 + N16), where N8, N4, and N16, denote the number of modes for 8 8 chroma blocks, and 4 4 and luma blocks, respectvely [5]. Also, accordng to the RDO procedure of nter-predcton, assume that we have M block modes, N reference frames, and ± W search range. Then, we need to 2 check M. N.(2W + 1) postons for a sngle reference frame and sngle block mode. Ths makes the complexty

4 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May of the encoder extremely hgh. In order to reduce the encodng complexty wth lttle RD performance degradaton, fast ntra- and ntra-predcton mode decson methods are proposed. 3. Proposed Fast Intra-Predcton Mode Decson Methods Ths secton presents a new fast ntra-predcton algorthm. The proposed method s based on several facts that we have observed from the statstcs of dfferent sequences as follows: a) Fgure 4 shows the total number of 4 4 and ntra-coded macroblocks at dfferent quantzaton parameters (QPs). As a result, fast detecton of 4 4 ntra-predcton mode can sgnfcantly mprove the encodng speed at low QPs, whle ntra-predcton can mprove the speed at large QPs. b) The predcton modes of each block are correlated wth those of neghborng 4 4 lumnance blocks. Number of MB x16 MB 4x4 MB Quantzaton Parameter Fg. 4 Number of 4 4 and ntra-coded macroblocks at dfferent quantzaton parameters. c) Normally, pxels along the drecton of local edges have smlar values. Therefore, a good predcton can be acheved f we predct the pxels usng those neghborng pxels that le n the same drectons of the edges. d) The optmal mode (found by full-search) and other good (second or thrd best) modes are most lkely to have smlar drectons. e) The drectonal features of 4 4 blocks can be preserved roughly after down-samplng. f) Expermental results show that the reference pxels of a 4 4 luma block are lkely to be smlar to each other [22]. Based on these observatons, we propose a fast ntrapredcton mode selecton algorthm. In ths secton some new deas are combned wth the fast mode selecton algorthm ntroduced n [1, 6, 7] to mprove ther effcency. 3.1 Improved Pan s Method for Fast Decson of I4MB Pan et al. presented a fast mode selecton for ntrapredcton method n [1] n whch the average edge drecton of a gven block s measured. The Sobel operators are frst used to obtan the drectonal vector of each pxel n a block by: v D j = { dx, j, dy,, j where the Sobel operator are:, j + 1, j 1 + 1, j } dx, j = P 1, j P, j P + 1, j + 1 P 1, j 1 2 P, j 1 P + 1, j 1 (2) dy = P + 2 P + P + 1, j + 1 P 1, j 1 2 P 1, j P 1, j + 1 The ampltude and angle of each edge vector can be calculated usng r Amp ( D j ) = dx, j + dy, and,, j (1) (3) r o 180 dy, j (4) Ang ( D, j ) = arctan( ) π dx, j When Ang(.) s ftted nto one of the 8 modes. Then, the edge drectonal hstogram of the block s found. The edge drecton hstogram (EDH) ndcates the number of pxels wth smlar edge drectons. Therefore, the cell k wth the maxmum amount ndcates that there s a strong edge along that drecton n the block and thus assgned as the domnant block drecton. Fgure 5 show the edge drecton hstogram of Fgure 6 and an example of edge domnant drecton, respectvely. Fg. 5 Edge drecton hstogram of Fgure 5. Fg. 6 An example of 4 4 edge patterns and ther domnant drecton. In the Pan s method, for I4MB there are 4 modes (1 DC (mode 2), 1 from maxmum ampltude of EDH and ts 2 neghbors) whle 2 modes (1 DC mode and 1 drectonal) for each luma block and 8 8 chroma block. Here, to mprove the Pan s method, we elmnate the DC mode

5 134 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May 2008 from the canddates f the drecton of the block s obvous, and otherwse, we choose only DC mode. To check whether the DC of the block s clear or not, the dff value, gven n Eq. (5), s obtaned to check whether t s smaller than a predetermned threshold or not, usng = 15 dff = avg p (5) avg = ( = 0 15 = 0 p + 8 ) >> 4 The mproved Pan s method s proposed as follows: 1. Fnd the maxmum value of the edge drectonal hstogram H. Denote the correspondng mode by M1. 2. If dff > T, carred out the RDO procedure for 3 modes at the most (M1 and ts two neghbors). 3. Else, f dff <T, carred out the RDO procedure for two canddate modes at the most. The DC wth maxmum of EDH (M1). 4. For I16MB, based on the same observaton as above, and after down-samplng by a factor of 2, f dff1 > T1, consder only the prmary predcton mode decded by edge drecton hstogram as a canddate for the best predcton mode. The dff1 n ths case s presented as: dff 1 = avg = ( = 64 = 0 64 = 0 avg p p + 32) >> 6 5. If dff1 < T1, choose the maxmum predcton mode and the DC mode. Extract the maxmum predcton mode as I4MB but wth DC and only 3 drectons of ntra For 8 8 chroma block, and after down-samplng by a factor of 2, use the same procedure as I16MB but by usng Eq. (13). Pan s method can reduce RDO calculaton from 592 to 132 tmes. The number of canddate modes and the RDO calculaton n the worst and the best cases are shown n Table 2. Table 2. Number of canddate modes Block Sze 4 4 (Y) (Y) 8 8 (U/V) RDO Pan s Method Proposed Method (mn) (6) Proposed Method (max) or Table 2 summarzes the number of canddates selected for RDO calculaton based on edge drecton hstogram. As can be seen from ths table, the encoder wth the fast mode decson algorthm needs to perform only 33 or 100 RDO calculatons, whch are much less than that of Pan s method (132) and current H.264 vdeo codng, RDO (592). 3.2 Fast Intra-Predcton Mode Selecton based on Statstcal Propertes of Adjacent MBs and Reference Pxels The method proposed n ths secton analyzes the characterstc of reference pxels, and uses the smlarty between adjacent MBs, whle the mproved Pan s method analyzes the characterstc of the 4 4 block tself. As a result, the combnaton of these three knds of methods can acheve better results. The proposed algorthm s as follow: 1. Fnd the maxmum value of the edge drectonal hstogram H. Denote the correspondng mode by M1. 2. If the modes for one of the top or left blocks are M1, then choose M1 as the best canddate mode for the current block. Go to step For 4 4 luma block, compute the mean of absolute dfference (MAD) of ts reference pxels. Then, f ths value s smaller than a predetermned threshold, select M1. Go to step 7. Ths result s yelded from the fact that f the smlarty of reference pxels of a block s hgh, the dfference between dfferent predcton modes wll be very small. For ths case, t s not necessary to check all of 9 predcton modes, but only one predcton mode s enough [19]. 4. If the mean of absolute dfference of horzontal references (MADH) s less than a threshold and M1 s a member of the set {mode 0, mode 3, mode 7}, select M1 s. Go to step Also, f the mean of absolute dfference of vertcal references (MADV) s less than a threshold and M1 s a member the set of {mode 1, mode 8}, select M1. Go to step 7. It s obvous that f the smlarty of horzontal reference pxels of a block s hgh, the dfference between predcton results obtaned wth predcton modes 0, 3 and 7 wll be very small. Also, f the smlarty of vertcal reference pxels of a block s hgh, the smlarty between modes 1 and 8 s hgh. 6. If all of above condtons are unsatsfed, use the mproved Pan s method (explaned n Secton 3.1). 7. Termnate. As such, n the worst case only three dfferent 4 4 ntramode costs wll be evaluated. Also, for I16MB and 8 8 chroma blocks the mproved Pan s method s used. To ncrease the speed of the algorthm, we use the early termnaton of RDO calculatons for all proposed algorthms as n [1]. 4. Proposed Fast Inter-Predcton Mode Decson Method The motvated facts about the nter-predcton are summarzed below.

6 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May a) The block wth hgh moton actvtes, nstead of hgh textural detals, can be better coded usng smaller block szes, whle the block wth less moton actvtes can be more effcently encoded usng larger block szes. b) It s observed that n natural vdeo sequences, when the vdeo objects move, the dfferent parts of the vdeo objects move n a smlar manner.then homogenous regons are encoded usng block szes whle non-homogenous regons are encoded usng smaller block szes. c) The background s not homogeneous, but because of the temporal statonary s coded usng block sze. 4.1 Fast Inter-Predcton Mode Decson usng Splt/Merge Procedure The splt procedure parttons the MB nto varable sze blocks usng a quad-tree approach. In ths method, a macro block s dvded nto equal-sze quarters. Then, usng the smlartes of moton vectors of adjacent blocks we wll show how to merge the sub-blocks for quarter dvsons. The proposed algorthm s summarzed as follows. 1. Subtract the current frame from ts prevous frame, then for any MB compute: N N m = number = number of pxels belong to the set MB of nonzero pxels n dfference MB 2. If N m N s smaller than or equal to a predetermned threshold, choose drect mode as the fnal macroblock type. 3. Otherwse, splt the block nto four 8 8 blocks and conduct a new teraton of block matchng for each of these four descendng blocks. 4. If moton vectors of 8 8 sub-block are equal or three sub-block MV are the same and the forth unequal MV only dffer by one quarter-pxel dstance, choose mode 1(16 16). Termnate. 5. If MV0=MV1 and MV2=MV3 (see Fgure 7), choose the Termnate. 6. If MV0=MV2 and MV1=MV3, choose the Termnate. 7. Repeat the steps 2 to 4 for each 8 8 blocks, except that the sub-blocks are Termnate. MV0 MV1 MV2 MV3 Fg. 7 MB dvson. The mode selecton methodology employed n ths paper s as below: For I4MB, I16MB and P the proposed algorthms are used to extract the best mode among the related category and at last RDO s used to extract the fnal mode. 4.2 Analyss and Complexty Reducton of Multple Reference Frames Moton Estmaton In H.264/AVC, moton estmaton s allowed to search multple reference frames. Therefore, the requred computaton s hghly ncreased, and t s n proporton to the number of searched reference frames. Expermental results show some facts that are used for decdng the number of references frames as[20]: 1. For QP=20, QP=30 and 40, t can be seen that 65%, 79% and 95%, of macroblocks need only one reference frame, respectvely. Therefore, we should proceed the block matchng process from the nearest reference frame to the farthest reference frame. 2. Another nterestng pont s that low btrate cases are more lkely to have best reference frames close to the current frame than hgher btrate cases are. 3. We can see that for QP=20, there are 59.84%, 05.00%, 04.88%, 28.11%, and 02.17% of the macroblocks selected as P16 16, P16 8, P8 16, P8 8, and ntra, respectvely, when only one prevous frame s searched. For QP=30, there are 75.97%, 05.36%, 05.45%, 11.04%, and 02.18% of the macroblocks selected as P16 16, P16 8, P8 16, P8 8, and ntra, respectvely. For QP=40, the correspondng percentages are 89.34%, 03.21%, 03.07%, 01.69%, 02.69%. 4. In H.264/AVC, SKIP mode s a specal case of P The percentages of SKIP macroblocks after one reference frame s searched are 44.57%, 62.69%, and 79.14% for QP=20, 30, and 40, respectvely. Ths result show, a large percent of MBs are coded as or skp mode, that are use only one reference frame, whle for large QP ths fact are amplfed. Accordng to these observatons, n the followng, we lst the steps for each macroblock to check whether t s necessary to search the next reference frame at the end of each reference frame loop. 1. After the predcton procedure, resdues are transformed, quantzed, and then entropy coded. If we can detect the stuaton that the transformed and quantzed coeffcents are very close to zero n the frst reference frame, we can turn off the matchng process for the remanng frames. 2. Calculate the sum of absolute transform dfference (SATD), f t s less than threshold(thsatd), we wll stop the searchng process. 3. If the best reference frame s prevous frame and f the best moton vector s the same as that of SKIP mode or

7 136 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May 2008 Cate gory Intra- predcton (I4MB,I16MB Chroma) Inter- Predcton mode and QP s larger than threshod(thqp), the multple reference frames loop wll be early termnated. The determnaton of THQP s emprcally obtaned n [20]. In the proposed method 76%-96%of computaton for searchng unnecessary reference frames can be avoded. Also, smlar to ntra-predcton we use an early termnaton technque based on early detecton of zero blocks. 5. Expermental Results Mult- Reference Algorthm Early Termnaton RDO RDO RDO YES RDO M1 Pan s Method RDO RDO YES M2 Improved Pan s RDO RDO YES method M3 Proposed Alg. RDO RDO YES M4 Proposed Alg. Proposed Alg.(splt/ merge) M5 Proposed Alg. Proposed Alg.(splt/ merge) RDO Proposed Alg. YES YES Our proposed algorthm was mplemented nto JM7.1, provded by JVT accordng to the test condtons specfed n VCEG-N81 document as lsted n Table 3 [21]. Experments were carred out on the recommended sequences wth varous quantzaton parameters for IPPP type. For IPPP experments, the total number of frames s 300 for each sequence, and the perod of I-frame s 100. The used test platform s Pentum IV-2.8 GHz wth 256 Mbytes RAM. We compared the performance of our proposed algorthm (fast moton estmaton + fast ntra + fast nter + selectve frame) wth other avalable approaches. The used fast moton estmaton algorthm s from JVT-F017, the UMHexagonS algorthm (combned wth the center based fractonal-pel search (CBFPS) and early termnaton technque) was ntegrated wthn verson 7.1 of the JVT software [15]. To show the mpact of dfferent parts of the algorthm, these parts are added at dfferent steps and the results are analyzed. Thus, the experments were ordered n sx states as lsted n Table 4. Table 3: Experment condtons. GOP IIIII or IPPP Codec JM 7.1 MV Search Range ± 16 Quantzaton 10, 16, 24,28,36,40 Parameter Number of References 5 Common Codng Opton Hadamard Transform, CABAC, RDO s enabled Format CIF and QCIF Number of Frames 300 Table 4. Dfferent methods used n the experments Comparsons wth the case of exhaustve search (RDO) were performed wth respect to the change of average PSNR ( ΔPSNR), the change of average data bts ( Δ Bt), and the change of average encodng tme ( ΔTme), respectvely. The PSNR s derved from 2 PSNR = 255 (7) 10 log 10 MSE Therefore, n the rest of ths paper we use the overall PSNR value of all three components Y, U and V usng Equ. (11). Δ tme be defned as: T prop T ref Δ Tme % = 100. (8) T Also, the btrate ncrease s defned as: Δ Btrate ref btrate prop btrate ref % = 100. (9) btrate A group of experments were carred out on dfferent sequences and the results are shown below. The encodng btrates, the PSNR values, and the tme savng factor (as compared wth the H.264 RDO method) for 4 test sequences wth dfferent quantzaton parameters are shown n Tables 5~7. Generally speakng, as can be seen from ths tables, we have saved 40-50% of the total encodng tme at the expense of only 0.1~1.5% rate ncrease n average and dstorton n average for these test sequences. Fgures 8, 9, and 10 show the examples of RD and the complexty curves of sequences Akyo (class A), Foreman (Class B), and Stefan (Class C) for IPPP sequences. From these fgures, one can see that the proposed fast decson scheme gves almost dentcal ratedstorton performance whle provdng a speed-up factor (rato of encodng tme usng the RDO technque and the proposed scheme) of 3-6. In ths fgure, the RDO, Pan s method, mproved Pan s method, and 3 forms of fast proposed methods (Fast ntra + Fast nter + Fast mult reference frames) are compared (see Table 4). Fgures 11, 12, and 13 gve examples of the classfcaton results and the fnal mode decson for Football, Stefan and walkng person sequences. In these fgures, the red, yellow, and green colors show the skp, ntra- mode, and nter-mode, respectvely. In part (c) of these fgures, the black blocks are the macroblocks wth nter-predcton and part (d) shows the moton vector and ntra-predcton mode for each block. ref

8 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May Concluson In ths paper, an effcent ntra- and nter-mode decson algorthm for H.264/AVC standard was proposed. In the proposed algorthm, we decreased the encodng tme by reducng the number of canddate modes. In order to acheve a better performance, some new deas (such as smlarty of references pxel, adjacent MB propertes, and statstcal propertes of varable block szes) wth some strength ponts to mprove Pan s algorthm were combned. Also, a fast ME algorthm was used to reduce the encodng cost. In order to evaluate the mpact of dfferent parts of the algorthm, they were added step by step and the related expermental results were shown untl the fnal algorthm was constructed. The expermental results show that the proposed algorthm has reduced the number of RDO calculatons wth respect to the orgnal and mproved algorthms wth neglgble loss n reconstructed vdeo qualty and a neglgble btrate ncrease. As the expermental results show, the proposed algorthm can be used for challengng work of ntra-predcton mode decson n the H.264/AVC vdeo encoders wth low computatonal cost. Acknowlegment Ths work was n part supported by a grant from ITRC. References [1] F. Pan, X. Ln, S. Rahardja, K. P. Lm, Z. G. L, D. Wu, and S. Wu, Fast Mode Decson Algorthm for Intrapredcton n H.264/AVC Vdeo Codng, IEEE Trans. On crcuts and systems for vdeo Tech., Vol. 15, NO. 7, pp , July [2] T. Wegand, G. J. Sullvan, G. Bjontegaard, and A. Luthra, Overvew of the H.264/AVC Vdeo Codng Standard, IEEE Trans. On crcuts and systems for vdeo technology, Vol. 13, no. 13, no. 7, pp , [3] ece.ut.ac.r/classpages/multmeda/h264.ppt [4] Envvo [5] Changsung Km, Qng L, C. C. Jay Kuo, Fast Intra- Predcton Model Selecton for H.264 Codec, SPIE Internatonal Symposum ITCOM 2003, Orlando, Florda, Sept. 7-11, [6] F. Pan, X. Ln, et al., Fast Mode Decson for Intra- Predcton, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6, JVT 7th Meetng, Pattaya II, Thaland, March [7] F. Pan, X. Ln, S. Rahardja, K. P. Lm, and Z. G. L, A Drectonal Feld Based Fast Intra-Mode Decson Algorthm for H.264 Vdeo Codng, IEEE Inter. Conf. on Multmeda and Expo, vol. 2, pp , June [8] J. Km and J. Jeong, Fast Intra-Mode Decson n H.264 Vdeo Codng Usng Smple Drectonal Masks, Proc. of SPIE Vol. 5960, pp , [9] R. Garg, M. Jndal, M. chauhan, Statstcs Based Fast Intra-Mode Detecton, Proc. of SPIE Vol. 5960, pp , [10] B. Jeon and J.lee, Fast Mode Decson for H.264, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6, JVT 10 th Meetng, Wakoloa, Hawa, December [11] C. Yang, L. PO, W. Lam, A Fast H.264 Intra- Predcton Algorthm Usng Macroblock Propertes, ICIP, pp , [12] B. Meng, O. C. AU, C. Wong, H. Lam, Effcent Intra-Predcton Algorthm n H.264, IEEE, pp , [13] C. Tseng, H. Wang, J. Yang, Improved and Fast Algorthms for Intra-4 4 Mode Decson n H.264/AVC, IEEE, pp , [14] C. Hsu, M. Ho, J. Hong, An Effcent Algorthm for Intra-predcton n H.264, IEEE, pp , [15] Jont Vdeo Team (JVT), reference software JM7.1, [16] M. Jafar, S. Kasae, An Effcent Intra-Predcton Mode Decson Algorthm for H.263 to H.264 Transcodng, 10 th IEEE nternatonal conference on computer systems and applcatons, page , march [17] M. Jafar, s. Kasae, Prortsaton of data parttoned MPEG-4 vdeo over GPRS/EGPRS moble networks, Asan nternet engneerng conference (AINTEC), Taland, pp , December [18] I. Rchardson, H.264/MPEG-4 part 10 whte paper, avalable: [19] C. Km, H.-H. Shh, and C.-C. J. Kuo, Feature-based ntra-predcton mode decson for h.264, n Proc. IEEE Internatonal Conference on Image Processng, [20] Yu-wen Huang, Bng-Yu Hseh, Shao-Y Chen, Shyh-yh Ma, and Lang-Gee Chen, Analyss complexty reducton of multple reference frames moton estmaton n H.264/AVC, February [21] G. Sullvan, Recommended smulaton common condtons for H.26L codng effcency experments on low resoluton progressve scan source materal, presented at the 14th VCEG-N81 Meetng, Santa Barbara, CA, Sep [22] Jang Gang-y, L Sh-png, Yu Me, L Fu-cu, An effcent fast mode selecton for ntra-predcton, IEEE Int. Workshop VLSI Desgn & vdeo Tech., Chna, pp , May 28-30, 2005.

9 138 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May 2008 [23] Yn, P., Touraps, H.C., toups, A.M., and boyce, J.: Fast mode decson and moton estmaton for JVT/H.264. Proc. Int. Conf. on Image Processng, Barcelona, Span, 2003, Vol.3, pp [24] Lm, K.P., Wu, S.,Wu, D.j.Rahardja, S., Ln, X., Pan, f., and L, Z.G.: Fast nter mode selecton. 9 th Jont vdeo Team(JVT) Mtg, San Dego, CA, USA, September [25] H. Km, Y. Altunbasak, Fast Mode Decson for Inter Predcton n H.264, ICIP 2004, Sngapore, [26] Y.H. Km, J.W. Yoo, S.W. Lee, J. Shn, Adaptve mode decson for H.264 encoder, n Electronc Letters, Vol.40 No.19, September, [27] A. C. Yu, G. Martn, Advanced block sze selecton algorthm for nter frame codng n H.264/MPEG-4 AVC, n ICIP 2004, Sngapore, [28] Q. Da, D. Zhu, R. Dng, Fast mode decson for nter predcton n H.264, n ICIP 2004, Sngapore, Table 5. Expermental results for IPPP type sequences, dstorton comparson Δ PSNR ( db ) Foreman News Contane r Slent M M M M M M M M M M M M M M M M M M M M Table 6: Expermental results for IPPP type sequences, rate comparson. Forema n News Contan er Slent Δ Bt M M M M M M M M M M M M M M M M M M M M % complexty comparson. Forma n News Conta ner Slent Δ Tme % M M M M M M M M M M M M M M M M M M M M Fg. 8 Akyo sequence (IPPP seq.). Computatonal complexty. R-D performance. Table 7. Expermental results for IPPP type sequences,

10 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May Fg. 9 Foreman sequence (IPPP seq.). Computatonal complexty. R-D performance (d) Fg. 12 Samples of Stefan sequence. MB dvson. MB dvson, wthout background. (c) Classfcaton result of ntra- and nter-predcton. (d) Moton vector, and ntra-predcton modes. Fg. 10 Foreman sequence (IPPP seq.). Computatonal complexty. R-D performance. (c) (d) Fg. 13 Samples of walkng person. MB dvson. MB dvson, wthout background. (c) Classfcaton result of ntra- and nter-predcton. (d) Moton vector, and ntrapredcton modes. (d) Fg. 11 Samples of Football sequence. MB dvson. MB dvson, wthout background. (c) Classfcaton result of ntra- and nter-predcton. (d) Moton vector, and ntra-predcton modes. M. Jafar receved the B.S. n electronc engneerng and M.S. degrees n communcaton systems from Shraz unversty of technology n 1992 and 1996, respectvely. Durng he stayed n sgnal processng research laboratory, mnstry of telecommuncatons of Iran. He receved Ph. D degree n communcaton systems n scence and research branch of Islamc Azad Unversty n January He s currently head of electrcal engneerng department of Islamc Azad unversty, Kerman Branch. Hs man research nterests are vdeo compresson, mage and vdeo processng, pattern recognton and predcton. Shohreh Kasae receved B.S. n electronc engneerng ( ), M.S. degrees n mage processng( ), Ph.D n mage processng( ), n Isfahan unversty of technology, unversty of Ryukyus, Oknawa, Japan and Queensland unversty

11 140 IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.8 No.5, May 2008 of technology Australa. Durng 1999-present, she s as assocate professor of Sharf unversty of technology. Her man research nterests are Image and vdeo processng wth prmary emphass on object based vdeo compresson, vdeo survellance, content based mage retreval, vdeo restoraton, moton estmaton, fngerprnt authentcaton/ dentfcaton, vrtual studos, object trackng, color mage processng, hyperspectral change detecton, multdmensonal sgnal modelng and predcton. Also, multscale analyss wth applcaton to vdeo compresson, mage enhancement, pattern recognton, moton trackng, texture segmentaton and classfcaton, dgtal vdeo watermarkng, and mage mosacng.