Improved H.264 Rate Control by Enhanced MAD-Based Frame Complexity Prediction

Transcription

1 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 Improved H.64 Rate Control by Enhanced -Based Frame Complexty Predcton Xaoquan Y and Nam Lng * Department of Computer Engneerng, Santa Clara Unversty, Santa Clara, CA USA Abstract Ths paper presents a revsed rate control scheme based on an mproved frame complexty measure. Rate control adopted by both MPEG-4 VM18 and H.64/AVC use a quadratc rate-dstorton (R-D) model that determnes quantzaton parameters (QPs). Classcal quadratc R-D model s sutable for MPEG-4 but t performs poorly for H.64/AVC because one of the mportant parameters, mean absolute dfference (), s predcted through a lnear model, whereas the used n MPEG-4 VM18 s the actual. Inaccurately predcted results n wrong QP and consequently degrades rate dstorton optmzaton (RDO) performance n H.64. To overcome the lmtaton of the exstng rate control schemes, we ntroduce an enhanced lnear model for predctng, utlzng some knowledge of current frame complexty. Moreover, we propose a more accurate frame complexty measure, namely, normalzed, to replace the current use of parameter. Normalzed has a stronger correlaton wth optmally allocated bts than that of the predcted. To mnmze vdeo qualty varatons, we also propose a novel long-term QP lmter (LTQPL). Fnally, a dynamc bt allocaton scheme among basc unts s mplemented. Extensve smulaton results show that our method, wth nexpensve computatonal complexty added, mproves the average peak sgnal-to-nose rato (PSNR) and reduces vdeo qualty varatons consderably. Keywords: H.64/AVC; Rate control; Quadratc rate-dstorton model; Frame complexty predcton; Vdeo codng 1. Introducton Recently, the ITU-T/ISO/IEC Jont Vdeo Team (JVT) establshed the H.64/AVC standard [1] to enable one to compress vdeo wth much hgher bt rate reducton when compared to the earler MPEG-4 standard. H.64 enables vdeo to be transmtted through low bandwdth channel and be receved wth a reasonable qualty. Applcatons that beneft from H.64 nclude wreless vdeo, Internet vdeo streamng, and many others. Partcularly, H.64/AVC offers hgh compresson effcency wth reasonable complexty and error reslency, whch makes H.64/AVC a prme canddate for wreless envronment []. Rate control performs an mportant role n H.64 and has been actvely studed. Many exstng vdeo encodng rate control schemes (e.g., [3]-[6]), are based on a classcal quadratc rate-dstorton (R-D) model. Lee et al. [3] proposed a scalable rate control scheme that can be smultaneously appled at dfferent codng contexts, such as at the frame level, object level, or macroblock (MB) level. The mean absolute dfference () parameter and non-texture overhead have been ntroduced nto the orgnal quadratc R-D model to accurately estmate the target bt rate wth scalablty. Vetro et al. [4] made a non-trval extenson from sngle vdeo object (SVO) to multple vdeo objects (MVOs). The new rate control, proposed by L et al. [5] and Sullvan et al. [7], whch has been adopted as an nformatve part by * Correspondng author. Present address: Department of Computer Engneerng, Santa Clara Unversty, 500 El Camno Real, Santa Clara, CA 95053, USA. Tel: , Fax: Emal: nlng@scu.edu. Ths work was presented n part at the 004 IEEE Workshop on Sgnal Processng Systems (SPS), Austn, Texas, USA. 1

2 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 JVT for H.64/AVC recently, employs a lnear model for predcton to solve rate control and rate dstorton optmzaton (RDO) recursve problem. However, peak sgnal-to-nose rato (PSNR) varance s qute hgh ( > 5.0 db) at low bt rates for non-typcal sequences (NonTS) that mx low and hgh complexty frames. Bascally, we classfy testng vdeo sequences nto two categores: 1) typcal sequences that are sngle vdeo sequences such as Carphone and Suze ; and ) non-typcal sequences (NonTS) that are combnatons of two or more vdeos wth scene changes, such as Mssam- Carphone (formed by concatenatng two vdeos Mssam and Carphone ). Test sequence classes are defned n MPEG- 4 Verfcaton Model (VM) 18 [8]. Among those aforementoned lteratures, the quadratc R-D model as a key part of the rate control remans the same. Xe and Zeng [9] suggested a smlar quadratc model n whch only one term s used. He et al. [10] proposed a novel lnear rate model by ntroducng a new complexty measure, denoted by ρ, whch s the percentage of zeros among quantzed transform coeffcents. More recently, He and Chen [11] showed that the lnear rate model could be appled to JVT codng. Snce the percentage of zeros s avalable only after actual encodng, mult-loop encodng s unavodable to collect global encodng complexty statstcs. Ths makes real-tme rate control n JVT very hard to be realzed for the already computatonally burdensome H.64. We gve a bref revew of rate control that uses the quadratc R-D model followed by the motvaton of our work. Typcal rate control conssts of three major stages: pre-encodng, encodng, and post-encodng. It can be summarzed n fve steps: ntalzaton, target bt-rate calculaton, quantzaton level calculaton, updatng of model parameters, and post frame-skp control. The man task of rate control algorthm s to provde nputs, such as quantzaton parameter (QP) values, to the encodng engne. Rate control for JVT s more dffcult than that for MPEG-4. Ths s because for H.64, QP s used for both rate control and mode selecton/rate dstorton optmzaton (RDO) [5] [7]. That s to say, the needed by the rate control algorthm s avalable only after the RDO has used a QP value to generate t. H.64 has ntroduced many new features such as varable block szes and varous INTER/INTRA/SKIP modes. Mode decson s based on a RDO framework [1]. The heart of RDO process s a Lagrangan mode decson for an MB S k by mnmzng the cost J MODE J MODE ( S, I QP, λ ) = D ( S, I QP) + λ R ( S, I QP) (1) k k MODE where λ MODE s the Lagrange multpler, D REC represents the total dstorton, R REC represents the rate, QP s the quantzaton parameter value, wth a partcular combnaton of codng mode I k. If the sum of absolute dfferences (SAD) s used n computng, λ MODE s expermentally decded by [1] REC k k MODE ( QP 1) / 3 λ MODE = () Rate control s conducted va controllng QP and therefore va adjustng the Lagrange parameter. To get around the rate control and RDO nterplay problem, L et al. [5] suggested an estmate for based upon complexty of pror pctures, namely, a lnear model for predcton. The lnear model assumes that the complexty surrogate vares gradually from pcture to pcture. However, for non-statonary vdeo (e.g. a scene change occurs), nformaton collected from prevous frames s no longer useful and the lnear model fals to predct a correct. In that case, decoded vdeo qualty degrades sharply, such as that shown n Fgure 4(c) (dotted lne), when scene change happens at frame 50, PSNR drops sharply from 41 to 4 db. Furthermore, naccurate results n QP and frame wndow sze (for updatng the coeffcents n quadratc R-D model) mscalculaton. Consequently, t affects RDO performance snce QP s used for RDO n H.64 as n Eq. (). Both Xu and He [13] as well as Jang, Y, and Lng [14] proposed some remedes by usng rato for H.64 to reduce vdeo degradaton va non-unform target bts estmaton accordng to frame complexty. However, the rato used to adjust target bts estmaton s obtaned through a lnear predcton n JVT [5] that may stll be naccurate for non-typcal sequences. Extendng our earler work [15], n ths paper, we enhance the frame complexty measure by mprovng the value of predcted used n the quadratc R-D model. Ths paper s structured as follows. The next secton revews the quadratc R-D model and formulates the problem. Secton 3 offers detals of our proposed algorthm. In Secton 4 we propose a novel long-term QP lmter (LTQPL) to reduce vdeo qualty fluctuaton. Secton 5 descrbes a dynamc basc unt bt allocaton optmzaton scheme. Our extensve expermental results are provded n Secton 6. Secton 7 draws conclusons. REC k k. R-D Model and Target Bt Count Quadratc R-D relatonshp s commonly used for sngle vdeo object and multple vdeo objects rate control. A typcal form of the quadratc R-D model s proposed by Lee et al. [3] as

3 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 X 1 X T texture, = ( + ) (3) Q Q where T texture, denotes the encodng bt count for the texture of current frame, s used as an ndcaton of the encodng complexty assocated wth the texture, Q s the QP for the current frame, and X 1 and X are the frst- and second-order model parameters. Note that the used n MPEG-4 VM18 s the actual. In H.64/AVC, one of the mportant technques to solve the nterplay problem between rate control and RDO s a lnear model for predcton [5]. The lnear model predcts the of current frame P by that of a prevous frame -1 (or predcts the of current macroblock n the current frame by that of the MB n the co-located poston n the prevous frame), as P a * a = (4) where a 1 and a are two coeffcents of the predcton model. Hence the quadratc model Eq. (3) actually becomes X 1 X T texture, = P ( + ). (5) Q Q Lnear predcton works fne for many typcal sequences, but performs poorly for non-typcal sequences. When a scene change happens, the basc unt n the current frame s not correlated wth that n the prevous frame. Ths would result n ncorrect QP and wndow sze calculaton as they are essentally based on the. Accordng to the rate control scheme n JVT [5], a basc unt can be an MB, a slce, a feld, or a frame. In ths paper, wthout the loss of generalty, we dscuss two types of basc unts, frame layer and MB layer. In [5], the fnal target bts allocated to a frame s gven as a weghted combnaton of T r and T buf : texture r T = β * T + (1 β ) * T (6) where the target bts T r s obtaned from the dstrbuton of remanng bts among remanng frames and T buf s the target bts obtaned from the actual buffer occupancy, target buffer level, frame rate, and channel bandwdth. β s a weghtng factor and ts typcal value s 0.5. In our prevous work [14], we proposed a method to reduce average PSNR varance usng a non-unform target bts estmaton technque based on rato ( ) and s defned as the rato of the predcted of current frame to the average of all prevously encoded P frames n a group of pctures (GOP). It can be easly calculated from the followng equaton buf P = 1 1 ( l ) 1 l (7) where P Predcted of current frame ; l Actual of prevously encoded P frame l; - 1 The number of prevously encoded P frames n the GOP. An rato s used as a measure of frame complexty. The dstrbuton of the bt count T r s scaled by a functon of rato ( ) va an adaptve formula T R r = SF (8) N r * where R r s the remanng bts, N r s the number of remanng frames, and SF s a scale factor based on the th frame, and s emprcally gven as r 3

4 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 SF = f f 0.9 < f 1.05 < f 1.3 < f 0.9 > (9) We have mproved the boundary values from our earler work [14]. Although the above adjustment releves PSNR surges and sharp drops, there are lmtatons: 1) The rato used to adjust target bts estmaton s obtaned through the lnear predcton used n JVT that s naccurate durng non-typcal sequence scene change. ) Target bts estmaton adjustment may be lmted due to buffer constrants. 3) It only ams at reducng PSNR varatons. The average PSNR s hardly mproved. 4) It lacks basc unt (or MB) layer bt optmzaton as the complextes for dfferent basc unts or MBs could be qute dfferent even wthn a frame. To solve those ssues, frst, we ntroduce an enhanced lnear model for predcton utlzng current frame knowledge wthout actually encodng t. Based on the more accurately obtaned, we then focus on mprovng the rate control algorthm by replacng the predcted wth a more accurate frame complexty measure, namely, a normalzed. The revsed quadratc R-D model outperforms that n the exstng JVT rate control scheme wth hgher average PSNRs and much lower PSNR varatons through the whole vdeo sequence. Ths s due to the fact that normalzed has a stronger correlaton wth optmal bt rates than that of a predcted, resultng n a more accurate R-D model. We also mplement a basc unt level bt allocaton optmzaton scheme n ths paper. The detaled algorthm s dscussed n Sectons 3 and Enhanced and Normalzed for Rate Control of the resdual component can be a good ndcaton of encodng complexty [3]-[5]. To a certan degree, t makes the R-D model scalable for varous bt rates, spatal, and temporal resolutons. However, the must be accurate to take ts advantage. To solve the problem of naccurate predcted parameter used n the JVT quadratc model, we frst ntroduce an enhanced lnear predcton method for usng the complexty of current frame. Based on the more accurate obtaned, we then propose a new frame complexty parameter, NORM to replace the predcted n Eq. (5). Our quadratc R-D model should be hghly scalable to hybrd vdeo sequences Enhanced Lnear Model for Predctng To provde accurate especally n the case of scene change or hgh moton, should be reevaluated after encodng the current pcture, but that would requre us to encode the pcture agan after the QP s decded, whch s undesrable n real-tme low delay envronments. Currently, rate control n JVT [5] uses a smple lnear model as shown n Eq. (4) to predct current frame. When a scene change occurs, msmatch can be huge. To solve ths problem, n the prevous pcture post-encodng stage, we read n the current orgnal pcture and compare t wth the prevous reconstructed pcture. Therefore we have some knowledge of the complexty of current frame. The PSNR of a pcture, PSNR(org, rec -1 ) s calculated based on the prevous reconstructed pcture rec -1 and the current orgnal pcture org assumng that pcture -1 s used to reconstruct pcture f we need to drop pcture. We defne PSNR dfference D(, -1) as the dfference between the PSNRs of current and prevous pctures. We then have a PSNR dfference rato D () between current and prevous pcture as D(, 1) D =. (10) D( 1, ) Usng D () and D(, -1), along the exstng lnear model n JVT, we found, va expermental results, that the followng relatonshp s effcent for current pcture enhanced predcted, EP : 4

5 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 P * D EP = P * D P *[ D.8.6] f f f D 1.5 D D < 1.5 < 3.0 >= 3.0 (11) where P s the predcted obtaned through Eq. (4). The above technque enhances predcted especally for scene change or hgh moton frames. A more accurate can therefore be predcted. Fgure 1 depcts the results of actual comparng wth the predcted usng the method n JVT G01 [5] and the enhanced predcted n ours. It s clear that the for scene change frames 50 and 70 are much more accurate when our method s used. Note that the actual s avalable only after encodng the frame. For statonary vdeo, the lnear predcton algorthm n JVT [5] produces farly accurate s. In those cases, D generally has a value close to 1.0. By Eq. (11), EP would reman about the same value as P. We tested our enhanced predcted usng varous vdeo sequences to show that Eq. (11) s effectve for mprovng accuracy. Eq. (7) s now updated wth P replaced by EP. Fgure 1. Actual vs enhanced predcted usng our method vs predcted usng JVT G01 [5]. 3.. Normalzed as a Measure of Frame Complexty The predcted absolute value of can be further enhanced n the quadratc R-D model, by a relatve measure of we propose, namely a normalzed ( NORM ), as a complexty measure of a frame. NORM s the rato of the enhanced predcted of current frame, EP, to the average of all prevously encoded P frames n a group of pctures (GOP). It can be easly calculated from the followng equaton: NORM = EP l l. (1) Two concatenated vdeo sequences Suze-Trevor and Mssam-Carphone were used as examples to llustrate that NORM s a better parameter suted for the quadratc R-D model. Frst we encoded both sequences wth a fxed QP wthout rate control. It generated optmal number of bts for each frame to mantan a constant vdeo qualty. Then we encoded the sequences wth the rate control of JVT G01 [5] and recorded the predcted. Fnally we encoded the sequences wth our rate control algorthm and recorded the NORM. The correlaton coeffcents are much greater when usng NORM than those usng predcted, as shown n Table 1, for varous bt rates. It can be seen that NORM 5

6 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 provdes stronger correlaton wth the optmal bts than that of the predcted n JVT G01 [5]. Further, we constran our NORM to be n between 0.5 and.0, except for scene change frames through clppng: NORM NORM = MAX( = MIN( NORM NORM, 0.5). (13),.0) The reason by mposng a lower bound s to prevent a frame from usng too lttle bts, avodng buffer underflow. On the other hand, mposng an upper bound s to prevent a frame from consumng too many bts, avodng buffer overflow. Fgure shows the correlaton results for vdeo Mssam-Carphone. Fgure. Correlatons between optmal bts and predcted usng JVT G01 [5] and NORM usng our method. Table 1 Correlaton coeffcent comparson between optmally allocated bts and predcted usng JVT G01 [5], as well as NORM usng our method, for varous bt rates (n kbps). Bt rate Suze- NORM Trevor Pred Bt rate Mssam- NORM Carphone Pred Revsed Quadratc R-D Model wth Normalzed * Rbas-Corbera and Le [16] nvestgated the optmum target bt rate for the th frame n a GOP. The texture target bt T for frame, after some straghtforward manpulatons, can be obtaned by S T T * = (14) Sˆ M 6

7 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 where S s the energy for the th frame, Ŝ s the average of all the frame energes, and T denotes the total number of bts * avalable for the remanng M frames. From Eq. (14), the optmum target bt rate T has a lnear relatonshp wth S / Sˆ. If * we replace S wth, then T can be lnearly scaled by / (Average -1 ), whch s the NORM from Eq. (1). From the results of Sectons and Eq. (1), we revse the quadratc R-D model, replacng the predcted wth normalzed n the quadratc R-D model (3), as follows: X 1 X T texture, = NORM ( + ). (15) Q Q Note that T texture, was also adjusted accordng Eqs. (6) (9) wth EP. Here, we further mprove n our R-D model usng NORM nstead of a smple. In our new model, model parameters X 1 and X are obtaned through lnear regresson method, same as those n [3]-[5]. The new quadratc model can be adapted to both frame level and MB level. In summary, the enhanced lnear predcton method mproves the accuracy of. The proposed revsed quadratc R-D model results n more accurate quantzaton parameters and wndow sze selectons, and hence mproves RDO performance. 4. Vdeo Qualty Smoothng va Long-Term QP Lmter For real-tme vdeo, a constant presentaton of qualty throughout the whole vdeo sequence s hghly desrable and s also challengng to acheve. In H.64 and MPEG-4, to prevent vdeo qualty from large changes, a rate lmter s appled whch typcally lmts the changes n quantzaton parameter (QP) to no more than ± between neghborng frames. Ths QP lmter may fal to prevent several contnuous changes that can lead to a large drft n vsual qualty after 10 or more frames. To further mnmze perceptble varatons n vdeo qualty and guarantee stablty, besdes the ± between neghborng frames, we propose a novel long-term QP lmter (LTQPL). Let the target QP be QP T whch s the average QP for a range of pctures. We can then lmt the QP of each frame n the next range of pctures to vary wthn a relatvely small range [such as (QP T - 5, QP T + 5) ]. The range of pctures can be determned by a sldng wndow mechansm. We choose a wndow sze of 0 n our experments. To demonstrate the effectveness of our LTQPL method, we plot the QPs of two non-typcal vdeo sequences of Suze-Trevor and Salesman-News n Fgure 3. We can see that the quantzaton parameter for each frame and each basc unt (or MB) s well controlled wth the proposed LTQPL method. Note that n H.64 codng, the maxmum quantzaton parameter s 51. Ths mples that n JVT, some frames or basc unts can be quantzed by extremely large QPs. As a result, the vdeo qualty over the whole sequence has a relatvely large varaton. By applyng our extra smoothng technque, the PSNR standard devaton s reduced sgnfcantly, and ths s shown wth further smulaton results n Secton 6. Fgure 3. Comparson of the quantzaton parameters n each frame. The test vdeo sequences are Suze-Trevor encoded at 7 kbps, wth ntal QP of 6, frame layer rate control s used; and Salesman-News encoded at 68 kbps, wth ntal QP of 8, basc 7

8 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 unt (MB) layer rate control s used. Note: the frame QP s obtaned by averagng the QPs of all MBs n a frame when MB layer rate control s used. 5. Extenson to Basc Unt Bt Allocaton Basc unt or macroblock (MB)-level rate control generally provdes strcter buffer regulatons and fner bt-rate control than those of frame-level rate control. Even wthn the same frame, codng complextes for dfferent MBs can be qute dfferent. The for MBs may be as hgh as 6.0 or as low as.0 wthn a frame. It s necessary that we optmze bt allocaton accordng to ther relatve complextes. Smlar to the frame level scheme descrbed earler, an adaptve target bts estmaton scheme among basc unts wthn a frame s mplemented. MB target bt estmaton scheme conssts of two steps: 1) frame-level target bt estmaton and ) MB target bt estmaton. In the frst step, target bts are allocated to frames accordng to relatve frame complexty, usng the EP-based method as descrbed n earler secton. In the second step, we use a basc unt (MB) rato ( BU- ), as a new complexty measure for MB n a frame. BU- s the rato of predcted of the current jth basc unt, P BU,j to the average of all prevously encoded basc unts n a frame. It s calculated as the followng P BU ( j) = j 1 1 j 1 l = 1 BU, j BU, l. (16) The dstrbuton of the basc unt bt count T BU,r s scaled by a functon of BU- va an adaptve formula R T * BU, r BU, r = SF (17) BU N BU, r where R BU,r s the remanng bts for the frame, N BU,r s the number of remanng basc unts for the frame, and SF BU s a basc unt scale factor based on the jth BU-, and s emprcally gven as SF BU = BU ( j) f BU f 0.9 < f BU ( j) 0.9 ( j) 1.8. (18) BU ( j) > 1.8 The general dea s to allocate more bts to complex basc unts (MBs) and lesser bts to smple basc unts (MBs). Fnally, QP for MBs can be calculated usng our revsed quadratc R-D model as n Eq. (15). Consderaton of header bts and QP lmtaton reman unchanged from earler method [5]. The computatonal complexty for MB layer s low. Only a few arthmetc operatons are requred for each MB n addton to the conventonal rate control scheme. The effectveness of our basc unt level bt allocaton and rate control s verfed by extensve experments whch are presented n the next secton. 6. Expermental Results Our methods have been mplemented and compared wth that of the JVT reference software [5] [17], whch hereafter we refer to as JVT G01. Numerous experments have been conducted to evaluate the performance of our mproved ratecontrol algorthm. All test sequences are n QCIF 4::0 formats. We encode the frst frame as I and subsequent frames as P frames. The search range was ±3 and one reference frame was used. 8

9 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 (c) (d) Fgure 4. PSNR results for sequences Hall Montor encoded at 3 kbps, 30 fps, Comb6, whch s concatenated by the frst 50 frames of sx vdeos, encoded at 80 kbps, 30 fps, (c) Suze-Trevor encoded at 7 kbps, 10 fps, and (d) Mssam-Carphone encoded at 44 kbps, 10 fps. Frame layer rate control s used Frame Layer Rate Control Results Fgure 4 depcts the PSNR results for four dfferent test sequences ncludng both typcal sequence and NonTS. Take, for nstance, n Fgure 4(c), due to the scene change, after the 50th frame, the PSNR of Suze-Trevor usng the rate control scheme n JVT drops to 4.50 db because of the naccurate predcton whereas t drops only to db usng our proposed scheme, achevng a sgnfcant dfference of db. Another example s Fgure 4 where the results for JVT G01 show four sharp PSNR drops due to four of the scene changes, our method produces sgnfcant mprovements n these areas. PSNR gans are reflected n the mprovement n the subjectve qualty of the reconstructed frames, especally after a scene change, as shown n Fgure 5 for Mssam-Carphone and Suze-Trevor. Fgure 6 shows the rate-dstorton plots for two selected test sequences for dfferent bt rates. Buffer fullness status s reflected n Fgure 7, whch shows that our algorthm mantans a steader buffer level than that for JVT G01. Fgure 8 shows the actual bts and target bts msmatches. The msmatch s calculated va subtractng target bts from actual bts then dvded by the actual bts for each frame. Wth the proposed algorthm, the actual output bt rate s well matched to the target bt rate. As tabulated n Table, our method reduces the standard devatons σ of PSNR sgnfcantly for all sequences. The percentage of reducton s up to 63%. 9

10 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 Fgure 5. Reconstructed frame: 70th frame n Suze-Trevor encoded at 7 kbps; and 67th frame n Mssam-Carphone encoded at 44 kbps, when the JVT G01 rate control s used (Left) and when our proposed rate control s used (Rght). Fgure 6. Average PSNR results for Suze-Trevor and Mssam-Carphone encoded at varous bt rates. 10

11 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 Fgure 7. Buffer fullness results for vdeos Hall Montor encoded at 3 kbps and Suze-Trevor encoded at 7 kbps. Fgure 8. Actual bts and target bts msmatch results for vdeos Suze-Trevor encoded at 7 kbps and Mssam-Carphone encoded at 44 kbps. 6.. Macroblock Level Rate Control Results Fgure 9 depcts the PSNR results for a hgh moton vdeo Carphone and a non-typcal sequence Salesman-News whch s concatenated by two sequences Salesman and News. The fluctuaton of vdeo qualty usng our proposed MB level rate control algorthm s much smaller than that of usng JVT G01. Buffer fullness status s reflected n Fgure 10 for the two vdeo sequences Carphone and Salesman-News respectvely. The actual buffer fullness usng our proposed algorthm s kept lower and closer to the target than that of usng JVT G01, showng the robustness and stablty of our basc unt (MB) level rate control scheme. The results show that our basc unt (MB) level rate control can generate smoother vdeo qualty and strcter buffer regulatons. 11

12 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 Fgure 9 PSNR results for sequences Carphone encoded at 3 kbps, 10 fps, Salesman-News, whch s concatenated by two vdeos of Salesman and News, encoded at 68 kbps, 30 fps. Basc unt (or MB) level rate control s used. Fgure 10 Buffer fullness results for vdeos Carphone encoded at 3 kbps, 10 fps; and Salesman-News encoded at 68 kbps, 30 fps. Basc unt (MB) level rate control s used Computatonal Complexty Analyss and Comparson From the descrpton of proposed algorthms, the computatonal complexty ntroduced n addton to the exstng JVT G01 [5] bt allocaton and rate control s farly low. To calculate normalzed, only a lmted number of arthmetc operatons are requred for both frame layer and MB layer. The long-term QP lmter (LTQPL) requres a few arthmetc operatons as well. To compare the complexty of our new algorthm wth the JVT G01 [5] rate control, we used a PC wth Pentum-IV CPU,.8 GHz, and 56 MB of RAM. Two test sequences Suze-Trevor and Hall Montor were encoded usng frame layer rate control and two test sequences Carphone and Sales-News were encoded usng MB layer rate control. Each of them was encoded ten tmes. The average encodng tme ncrease s 0.% for Suze-Trevor, 0.53% for Hall Montor, 0.18% for Carphone, and 1.37% for Salesman-News. The overall computatonal complexty s ncreased on average by about 0.36% for frame layer rate control and by about 0.78% for MB layer rate control. 1

13 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May Concludng Remarks Our paper addresses an mportant ssue of mprovng and mantanng vsual qualty when streamng vdeos over low bt rate channels. Our proposed approach ntroduces an enhanced lnear predcton model for predctng, utlzng the complexty knowledge of current frame. Based on the obtaned, a more accurate and sutable parameter n the form of normalzed s proposed n the quadratc R-D model. We have also added a long-term QP lmter (LTQPL) to reduce vdeo qualty fluctuaton. The method can be extended to the basc unt level. Our extensve expermental results show that our proposed approach outperforms the recent adaptve rate control algorthm n JVT wth sgnfcant average PSNR gans and sgnfcant PSNR fluctuaton reductons. Our mprovements conform to the ncreased flexblty of H.64/AVC whch requres ntellgent coder control strateges for real-tme wreless/internet streamng and conversaton applcatons. Table Comparsons of average PSNR (Y-component only) values, PSNR standard devatons (σ), and bt rates of sequences Sequence Suze- Trevor Mssam- Carphone Target Bt Rate (kbps) JVT G01 Average PSNR (db) Bt Rate (kbps) σ of PSNR Ours Gan JVT G01 Ours JVT G01 Ours Reducton % % % % % % % % % % Acknowledgment The authors would lke to thank the revewers for ther helpful comments and constructve suggestons. References [1] T. Wegand, G.J. Sullvan, and A. Luthra, Draft ITU-T Recommendaton H.64 and Fnal Draft Internatonal Standard AVC, JVT of ISO/IEC JTC1/SC9/WG11 and ITU-T SG16/Q.6, Doc. JVT-G050r1, Geneva, Swtzerland, May, 003. [] T. Stockhammer, M. Hannuksela, and T. Wegand, H.64/AVC n wreless envronments, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 13, pp , June 003. [3] H.-J. Lee, T. Chang, and Y.-Q. Zhang, Scalable rate control for MPEG-4 vdeo, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 8, pp , Sep [4] A. Vetro, H. Sun and Y. Wang, MPEG-4 rate control for multple vdeo objects, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 9, pp , Feb

14 Journal of Vsual Communcaton and Image Representaton (Specal Issue on Emergng H.64/AVC Vdeo Codng Standard), Elsever Scence, May 005 [5] Z. G. L, F. Pan, K. P. Lm, G. N. Feng, X. Ln, S. Rahardja, Adaptve basc unt layer rate control for JVT, JVT-G01-r1, 7th Meetng, Pattaya II, Thaland, Mar [6] Z. G. L, C. Zhu, N. Lng, X. K. Yang, G. N. Feng, F. Pan, and W. S. Q, A unfed archtecture for real-tme vdeo-codng systems, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 13, pp , June 003. [7] G. J. Sullvan, T. Wegand, and K. P. Lm, Jont model reference encodng methods and decodng concealment methods; Secton.6: Rate Control, JVT-I049, San Dego, Sep [8] Vdeo Group, MPEG-4 Vdeo Verfcaton Model Verson 18.0, n Codng of Movng Pctures and Assocated Audo MPEG 001/N3908, Psa, Italy, Jan [9] B. Xe and W. Zeng, A sequence-based rate control framework for constant qualty vdeo, IEEE Internatonal Conference on Image Processng, vol. 1, pp , New York, USA, Sep. 00. [10] Z. He, Y. Km, and S. K. Mtra, Low-delay rate control and smoothng for vdeo codng va ρ-doman source modelng, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 11, pp , Aug [11] Z. He and T. Chen, Lnear rate control for JVT vdeo codng, Internatonal Conference on Informaton Technology: Research and Educaton, Newark, NJ, USA, Aug [1] T. Wegand, H. Schwarz, A. Joch, F. Kossentn, and G. J. Sullvan Rate-constraned coder control and comparson of vdeo codng standards, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 13, pp , July 003. [13] J. Xu and Y. He, A novel rate control for H.64, Proceedngs of IEEE Internatonal Symposum on Crcuts and Systems, vol. III, pp , Vancouver, Canada, May 004. [14] M. Jang, X. Y, and N. Lng, Improved frame-layer rate control for H.64 usng rato, Proceedngs of IEEE Internatonal Symposum on Crcuts and Systems, vol. III, pp , Vancouver, Canada, May 004. [15] X. Y and N. Lng, Rate control usng enhanced frame complexty measure for H.64 vdeo, IEEE Workshop on Sgnal Processng Systems (SPS), pp , Austn, Texas, USA, Oct [16] J. Rbas-Corbera and S.-M. Le, A frame-layer bt allocaton for H.63+, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 10, pp , Oct [17] ftp://ftp.mtc-fles.org/jvt-experts/003_03_pattaya/jvt-g01r1_software.zp [18] T. Wegand, G. J. Sullvan, G. Bjontegaard, and A. Luthra, Overvew of the H.64/AVC vdeo codng Standard, IEEE Trans. Crcuts and Syst. for Vdeo Technol., vol. 13, pp , July