ACCURATE BIT ALLOCATION AND RATE CONTROL FOR DCT DOMAIN VIDEO TRANSCODING

Transcription

1 ACCUATE BIT ALLOCATION AND ATE CONTOL FO DCT DOMAIN VIDEO TANSCODING Zhjun Le, Ncolas D. Georganas Multmeda Communcatons esearch Laboratory Unversty of Ottawa, Ottawa, Canada {lezj, mcrlab.uottawa.ca Abstract For vdeo transmsson over heterogeneous networks and devces, the comressed vdeo stream has to be converted nto a lower transmsson bt rate or a small satal resoluton. The oeraton of convertng a vdeo n comressed format nto another vdeo also n comressed format s called Vdeo Transcodng. The secfc transcodng ssue mentoned here s referred to as bt rate adataton or rate shang. In ths aer, we resent a novel bt allocaton and rate control algorthm for DCT doman vdeo transcodng. Imlementaton results show that the roosed algorthm can rovde accurate bt allocaton and better vdeo qualty. Comared to TMN-8, the roosed scheme can generate a more stable vdeo stream and mantan an even buffer occuancy level. Ths rate control scheme can be used to rovde flexble vdeo bt rate adataton and stable transmsson of vdeo streams over heterogeneous networks. Keywords: Vdeo Transcodng, Bt Allocaton, ate Control.. INTODUCTION Wth varous networked vdeo alcatons emerge, there s a bg demand for adatng the bt rate of recoded vdeo streams. The heterogenety of the communcaton networks, user devces, and user references of qualty demand the converson of comressed vdeo stream nto another vdeo stream wth dfferent bt rate or format. The oeraton of convertng a vdeo n comressed format nto another vdeo also n comressed format s called Vdeo Transcodng, whch s an essental technque for vdeo communcatons over heterogeneous networks and dfferent clent devces. Varous vdeo transcoder archtectures for bt rate adataton have been ntroduced n the lterature [-5]. These transcoders have dfferent comlexty and erformance. Ths work focuses on the DCT doman transcoder, whch has much reduced comlexty as comared to the xel doman transcoder. Snce most vdeo standards use varable length entroy codng of quantzed transform coeffcents and syntax nformaton, resultng n a varable outut bt rate, some form of bt-rate control s requred to control the outut of vdeo transcoders for meetng the channel bandwdth or acket sze constrants. Generc rate control belongs to the budgetconstraned bt allocaton roblem. Methods based on ate-dstorton Theory have been consdered n the lterature to solve ths roblem [6-8]. These methods reanalyze the -D characterstcs of frames before alyng a bt allocaton strategy. A oular aroach for reducng the comlexty of -D technques s to rely on a bt allocaton model, (Q), whch defnes the relatonsh between the number of roduced bts and the quantzaton ste sze. Therefore, t s able to redct the number of comressed bts when a certan quantzaton ste sze s n use before the real quantzaton and VLC are actually aled to. Varous bt allocaton models have been roosed n the lterature based on Informaton Theory, such as [9- ]. Some other models derve -Q relatonsh of a coder based on the observed data [3,4]. In some of these models, the quantzaton ste sze of an mage block s selected accordng to ts actvty, whch s measured by varance of all DCT coeffcents, sum of all the DCT coeffcents absolute values wth or wthout DC coeffcent, etc. The theoretcal bt allocaton models have two assumtons. Frst, the DCT coeffcents are uncorrelated and have a Lalacan or Gaussan dstrbuton. Second, the codng bt rate s aroxmately equal to the entroy. However, the frst assumton s roceedngs of the IEEE Canadan Conference on Electrcal & Comuter Engneerng -78-xxxx-x//$ IEEE - -

2 nvald for DCT coeffcents n the moton comensated dfference frame. Closed-form exressons of entroy and dstorton for arbtrary robablty dstrbutons are generally unavalable. In transform codng of mage or vdeos, esecally at very low bt rate, there s a large msmatch between the theoretcal entroy and the actual codng bt rate. Addtonally, comlex formulas make the otmzaton comutatonally exensve. However, the bt allocaton and rate control n vdeo transcodng s dfferent from that n vdeo codng, snce the bt usage nformaton obtaned after decodng the ncomng vdeo can be reused to hel the bt allocaton for encodng the outut vdeo. In ths work, through our extensve exerments, we have found that there s an aroxmate lnear relatonsh between the number of VLC code words n a frame and the number of roduced bts for encodng those coeffcents. By usng ths relatonsh, we formulate a bt allocaton model to estmate the roduced bts for encodng a frame and roose a ractcal self-adatve rate control scheme for DCT doman vdeo transcodng. Ths scheme can adatvely allocate bts to each macroblock n a frame and make the total bts of a frame meet the target bts budget. The organzaton of ths aer s as follows. In Secton, we ntroduce and formulate the lnear bt allocaton model. In Secton 3, we ntroduce the roosed rate control algorthm. Secton 4 resents our mlementaton, exermental results, and comarson between the roosed algorthm and the TMN-8 rate control scheme. Secton 5 s the concluson.. LINEA BIT ODUCTION MODEL The vdeo encoder and decoder consdered n ths aer are based on the ITU-T H.63+ standard. In [5] a theoretcal -Q model was roosed and the rate control algorthm based on ths model was fnally adoted as the TMN-8 rate control for H.63+. Smlar to the other models based on nformaton theory, ths model assumes that the DCT coeffcents of the moton-comensated dfference frame are aroxmately uncorrelated and Lalacan dstrbuted. Actually, there s usually some correlaton left among the DCT coeffcents. Because of the lmted accuracy of ts rate model, the rate control algorthm suffers from relatvely large control error, esecally at lower bt rate. Through our extensve ex erments, we found out that the relatonsh between the roduced bts count and the number of VLC code words n a frame or a macroblock can be aroxmated by a lnear functon. In our exerments, we encode several test sequences wth a seres of quantzaton arameters. Let Bˆ be the number of roduced bts for encodng code words of every frame, and C the number of the code words, whch s equal to the quantzed non-zero AC coeffcents n every frame. In Fgure, we lot ( B ˆ, C ) for several test cases. It can be seen Bˆ s aroxmately a lnear functon of C. We also notce that, at the lower bt rate (larger Q), the aroxmate lnear relatonsh stll holds Foreman (Q=) Carhone (Q=) Foreman (Q=3) Carhone (Q=3) Fgure : the relatonsh between Bˆ and C, wth dfferent test sequences and quantzaton arameters. In ths fgure, the x-axs s the number of code words (x 3 ), the y-axs s the number of bts (x 3 ) used to encode these code words Ths lnear relatonsh can be examned by how the DCT coeffcents n a macroblock or a frame are quantzed and entroy coded. In the H.63+ standard, after DCT transform, Every INTA block ncludes DC and 63 AC coeffcents and every INTE block only ncludes 64 AC coeffcents. The DCT coeffcents are further quantzed to acheve more comresson. Most quantzaton schemes are based on Unform Threshold Quantzaton (UTQ) [6] llustrated as follows:, f x C( x) = UTQ [ Q,, x] =, Q f x +, Q f x ; x > + ; x < ; where C(x) s the quantzed coeffcents, Q s the quantzaton ste sze, x s the DCT coeffcent,? s the UTQ dead zone threshold. The quantzaton scheme emloyed by H.63+ s smlar to UTQ, and s gven by: x ound ( ), 8 C[ x, Q] = UTQ (Q,Q, x), UTQ (Q,.5Q, x), x s a DC coeffcent xs an AC coeffcent n an INTA _ MB; x s a coeffcent n aninte ; _ MB; The quantzed AC coeffcents n INTE and INTA mode macroblocks wll be varable length coded accordng to the VLC table. In H.63+, a 3-dmensonal varable-length codng table s used to encode (LAST, UN, LEVEL) of each code word. If we know the robablty dstrbuton of every code word and the bt count for encodng ths code word, we can calculate the average bt count of the table,.e., () () - -

3 α = N = L where s the robablty of the _th code word, N s the total number of code words n the VLC table, and L s the bt count for encodng the _th code word. At the frame level, f we defne a as the average bts count for encodng a frame, we can get the total bts count for encodng a frame s B = α C( Q) + β (4) where C(Q) s the total number of code words n a frame, and ß s the total bts used for encodng overhead nformaton, whch ncludes the DC coeffcents, Moton Vectors, and some other syntax nformaton. Accordng to Eq. (), for a secfc quantzaton arameter Q, the number of code words n a MB s: where C C (Q) =, ( k, n, Q) =, K 63 k = n = x C (k,n,q) > ; x ; (3) (5), x s the n_th AC coeffcent n the k_th block of a MB, K (=6) s number of blocks n a macroblock. Fgure 3 shows the model arameters, a and ß, of one test case. We can see that, the model arameter a for dfferent streams s aroxmately the same. On the other hand, the model arameter ß s changed drastcally wthn a stream. However, when the bt rate s lower (Q=), the model arameter ß of dfferent vdeo streams has the smlar rofle Alha of Foreman (QCIF, 99 frames) Q= Q=5 Q= Q=5 Q= Q=5 Q= Beta of Foreman (QCIF, 99 frames) Q= Q=5 Q= Q=5 Q= Q=5 Q= Fgure 3. a and ß of Foreman test case. The x-axs s the number of frame, and the y-axs s the value of a and ß (x 3 ). Based on these results, the roduced bts for encodng a frame can be estmated from the number of code words n ths frame. For the same vdeo, the model arameters from the ncomng vdeo stream can be reused for the transcoder bt allocaton and rate control. The detaled scheme s descrbed n the followng secton. 3. ADATIVE BIT ATE CONTOL SCHEME Based on analyss n the above secton, we can reuse the model arameters, a and ß, from the ncomng vdeo stream to control the outut bt rate of the vdeo transcoder. In the DCT doman transcoder, after decodng an ncomng vdeo frame, we know the number of bts that are used for encodng the whole frame, frame header nformaton, code words, moton vectors, and other syntax nformaton. In ths work, what we are nterested n s the number of code words, C, and the number of bts for encodng these coeffcents, Bˆ. Then we can calculate the model arameters, a and ß, of an ncomng frame as Bˆ B β α = = C C Durng the encodng hase, arameter a from the ncomng vdeo stream can be reused as a reference to estmate the roduced bts count of every frame. In order to estmate the roduced bts count by usng Eq. (4), we have to know the number of code words for every ossble quantzaton arameter, {C (Q ), ==N-, =Q =3}, where s the macroblock ndex, Q s the quantzaton arameter, and N s the number of macroblocks wthn a frame. Accordng to Eq. (5), after transcodng and obtanng a new set of DCT coeffcents, we can analyze the DCT coeffcents and get the table of C (Q ). If we use the same frame layer rate control scheme as TMN-8, frame bt target B s determned by the target btrate (), the target frame rate (F), and the buffer fullness B=/F-? [7], and B s adatvely allocated nto each macroblock. From the ncomng vdeo stream, we can get the average a as n Eq. (6). Here, we frst assume that we can know the total bts count, ß, for encodng the overhead nformaton of the outut frame. Actually, ths nformaton can only be obtaned after encodng the whole frame. In a later art, we wll ntroduce a heurstc method for estmatng ß. We defne Bˆ as the bt target for encodng the total code words n a frame,.e. Bˆ = B Bheader β N where B header s the number of bts for encodng the cture header, whch s known before encodng a frame, β s the average bt number for encodng the overhead (6) (7) - 3 -

4 nformaton n a macroblock and N s the total number of macroblocks n a frame. Then, ntally, we can estmate the total number of the code words n the outut frame by: C ( Q) = Bˆ / α (8) By searchng the table {C (Q )}, we can get a reference quantzaton arameter Q ref as Q = argmnc( Q) C( Q) ref and the reference code words number of the outut vdeo frame C(Q ref ) as C( Q ) = N ref = C ( Q ref ) (9) () Q ref s used as the reference quantzaton arameter for the frame and s also used for encodng the frst macroblock. After encodng the _th macroblock, we can know the roduced bts number, Bˆ, for encodng the code words, C, n ths macroblock. Then we can calculate the bt budget for encodng the remanng macroblocks as ˆ ˆ B B Bˆ, N, Bˆ = = ˆ () + L = B If we defne Bˆ as the total roduced bts for encodng the code words n the revous macroblocks, and C as the number of code words n the revous macroblocks, we can udate the model arameter, a +, for the remanng macroblocks as ˆ α + = B / C, (), ˆ where B = Bˆ + Bˆ, Bˆ =, C = C + C, C =, α = α = L N, Then the estmated number of code words n the remanng macroblocks, C +, can be calculated as ˆ C + = B+ / α + (3) Usng a smlar method as n Eq. (9) and Eq. (), we can calculate Q + for encodng the (+)_th macroblock as Q + = arg mn C( Qr ) C+, r = +, L N (4) After selectng the Q +, the (+)_th macroblock wll be quantzed and entroy coded. In ths scheme, the quantzaton arameter Q s only determned by the bt allocaton. However, n order to kee a smooth vdeo qualty, the H.63+ standard requres that the dfference between quantzaton arameters for two successve macroblocks not be greater than. Therefore, f the dfference between the quantzaton arameter obtaned from Eq. (4) and the revous quantzaton arameter s greater than, the new quantzaton arameter has to be adjusted. However, ths constrant can also hel us save some comutatonal cost. After encodng a macroblock wth quantzaton arameter Q, the quantzaton arameter for encodng the next macroblock could only be n {Q -, Q -, Q, Q + Q +}, so, n Eq. (4), we don t have to search the number of the non-zero coeffcents for the other quantzaton arameters. In the above analyss, we assume that, before encodng a frame, we already know the roduced bts, ß, for encodng the overhead nformaton, whch ncludes frame header, moton vectors, DC coeffcents, and other macroblock-layer syntax nformaton. However, ß can only be known after encodng the whole frame. For examle, n the H.63+ basc oton, f the quantzaton arameter of a macroblock s dfferent from that of the revous macroblock, there s a 5-bt enalty for changng the quantzaton arameter [5]. In ths case, before gettng the quantzaton arameter, Q, for the macroblock, we cannot know the bt count for encodng the overhead nformaton. At a hgh bt rate (smaller Q), the amount of ß s only a small art of the total bt budget, but at a lower bt rate (larger Q), ß s a sgnfcant art of total bt budget. Therefore, estmaton of ß has a sgnfcant effect on the bt allocaton. In ths work, we use a heurstc method to estmate the model arameter ß. Based on the analyss n the revous secton, we know that, at the lower rate, ß of the same frame n dfferent vdeo streams has a smlar value. Therefore, we can reuse the ß nformaton from the ncomng vdeo stream. At the hgher rate, we can reassgn an average value as an ntal ß. Before encodng a frame, we can know the total bts number for encodng the cture header, so we only consder ß as the bts number for encodng the overhead nformaton of the macroblocks n a frame. Then, before encodng a frame, we can calculate the average ß of every macroblock as β = β / N (5) After encodng the _th macroblock, we can get the actual bts number of ß. We defne β as the total bts number for encodng the overhead nformaton n the revous macroblocks,.e. β β + β = L N, β (6) =, = Then we can calculate the average ß for the revous macroblocks as β = β / and calculate the average ß for the remanng (N-) macroblocks as Fnally, β = β ( / N) + β( N )/ N (7) (8) β s used to estmate the total bts number for encodng the overhead nformaton n the remanng (N-) macroblocks as β = β ( N ) (9) In ths algorthm, the model arameters, a and ß, are udated after encodng every macroblock. The only extra comlexty ntroduced s the calculaton of the code word table, {C (Q ), ==N-, =Q =3}, by reanalyzng the DCT coeffcents. As ndcated n Eq. (5), - 4 -

5 we only need to comare the absolute value of DCT coeffcents, x, to the quantzaton dead zone,?, but don t have to really quantze the coeffcents. Moreover, due to the monotoncally decreasng roerty of quantzaton, the re-analyss can be otmzed n the real mlementaton. 4. IMLEMENTATION AND EXEIMENTAL ESULTS The roosed rate control scheme has been mlemented n a H.63+ vdeo transcoder based on [3]. Comarsons between results acheved by our rate control method and by the TMN-8 rate control method are dscussed n ths secton. For our exerments, several test sequences n QCIF format are used. The test sequences are frst encoded wth unfed quantzaton arameter. Every test sequence s encoded wth only I frame followed by frames wthout any rate control scheme. The resultng H.63+ test sequences are sent nto the vdeo transcoder, whch uses TMN-8 and the roosed rate control schemes. Frst, we want to examne the bt allocaton based on the roosed model and the model of the TMN-8. Fgure 4 shows the bt allocaton result of (Foreman, QCIF, K) test case. Wth the roosed algorthm, the number of bts roduced by each frame s well matched to the target bt rate Foreman (QCIF, K) roosed 5 5 Fgure 4, bt allocaton result at K wth the roosed scheme and TMN-8. In ths fgure, the x-axs s the number of frames, and the y-axs s the number of roduced bts (x 3 ). In Fgure 5, we lot the number of bts n the buffer for each transcoded frame. From Fgure 5, we can see that the roosed rate control algorthm can kee a stable buffer occuancy level. If we assume the maxmum buffer delay s one frame. When the buffer occuancy s larger than the sze of one frame (/F), the followng frame wll be sked. In the (Salesman, K) test case, when usng the TMN-8 rate control scheme, there are two frames are sked at frame 69 and frame 78. However, wth the roosed rate control, no frame s sked at these two onts. Because the actual bt count wth dfferent rate control schemes s dfferent, drectly comarng the TMN_8 SN doesn t ndcate the qualty dfference of dfferent rate control schemes. For a far comarson, we defne a relatve qualty measure of every frame as ( SN / B), where B s the actual bt count. In Fgure 6, we lot the relatve qualty of dfferent test cases. From ths fgure, we can see that, at dfferent test sequences and bt rate, the roosed rate control scheme has a better relatve qualty Salesman, K Fgure 5. Comarson of the number of bts n the buffer. In ths fgure, the x-axs s the number of frames, and the y-axs s the number of bts n the buffer (x 3 ). The dash lne s the u lmt (/F) of skng frame TMN-8 Foreman,K roosed 5 5 Fgure 6. Comarson of the relatve qualty measure. In ths fgure, the x-axs s the number of frames, and the y-axs s the relatve qualty measure. Table 4 summarzes the erformance of the roosed rate control scheme and that of the TMN-8. To measure rate control erformance, we defne the normalzed devaton as Actual bt counts Bt budget. From % Bt budget Table 4, we can see that for all test cases, the roosed algorthm can rovde accurate bt allocaton and mroved vdeo qualty. In all test cases, the acheved bt rate of the roosed algorthm s lower than that of the TMN-8, but the corresondng SN value of the roosed algorthm s hgher than that of the TMN-8. The average gan of SN s about.-.5 db. At low bt rate, the average normalzed devaton of the roosed algorthm s much smaller than that of the TMN-8. TMN-8 Frame skng 5. CONCLUSIONS In ths aer, we roosed a novel bt rate control scheme wth self-adataton ablty for fast DCT doman vdeo transcodng. Dfferent from the current exstng rate control schemes, the roosed scheme s based on a roosed - 5 -

6 lnear bt allocaton model, whch only requres the number of code words n a frame. The exermental results show that ths scheme can accurately allocate the bt budget to every macroblock n a frame. Comared to TMN-8, the roosed scheme can allocate the bt budget more accurately to the macroblocks n a frame and generate a more stable vdeo stream. In addton, at the same bt rate target, vdeo streams roduced by the roosed algorthm have better qualty than that roduced by the TMN-8. The roosed rate control scheme can be used n vdeo transcoders to generate dfferent bt-rate vdeo streams for transmsson over heterogeneous networks. Table 4. esult of SN, Bt ate, and Normalzed Devaton Carhone Foreman Target bt rate (Kbs) SN (db) TMN-8 Acheved Bt ate (Kbs) Normalzed Devaton (%) SN (db) roosed Acheved Bt ate (Kbs) Normalzed Devaton (%) Acknowldgement Ths research was suorted by the Ontaro esearch and Develoment Challenge Fund. eferences [] J. Youn, and M.-T. Sun, Vdeo Transcodng wth H.63 Bt-Streams, Journal of Vsual Communcaton and Image eresentaton, Vol.,. [] H. Sun, W. Kwok, and J. W. Zdosk, Archtecture for MEG Comressed Btstream Scalng, IEEE Trans.on CSVT, Vol. 6, Ar [3] Q.-F. Zhu, L. Keofsky, and M. B. Garrson, Low-Delay, Low-Comlexty ate educton and Contnuous resence for Multont Vdeoconferencng, IEEE Trans. on CSVT, Vol. 9, June 999. [4]. Assunção, and M.Ghanbr, ost-rocessng of MEG- Coded Vdeo for Transmsson at Lower Bt ates, roceedngs of ICASS 96, May 996. [5] S.-F. Chang and D.G. Messerschmtt, A New Aroach to Decodng and Comostng Moton Comensated DCT- Based Images, roceedng of ICASS 93, Ar [6] K. amchandran, A. Ortega, and M. Vetterl, Bt Allocaton for Deendent Quantzaton Wth Alcaton to Multresolutons and MEG Vdeo Coders, IEEE Trans. on Image rocessng, Vol., Set [7] I.-M. ao, and M.-T. Sun, Encodng DCT Coeffcents Based on ate-dstorton Measurement, Journal of Vsual Communcaton and Image eresentaton, Vol.,, [8] Yan Yang and Shela S. Hemam, Generalzed ate- Dstorton Otmzaton for Moton-Comensated Vdeo Coders, IEEE Trans. on CSVT, Vol., Set.. [9] H.-M. Hang, and J.-J. Chen, Source Model for Transform Vdeo Coder and Its Alcaton ---- art I: Fundamental Theory, IEEE Trans. on CSVT, Vol. 7, Ar [] W. Dng, and B. Lu, ate Control of MEG Vdeo Codng and ecordng by ate-quantzaton Modelng, IEEE Trans. on CSVT, Vol. 6, Feb [] B. Tao, B. W. Dcknson, and H. A. eterson, Adatve Model-Drven Bt Allocaton for MEG Vdeo Codng, IEEE Trans. on CSVT, Vol., Feb.. [] T. Chang, and Y.-Q. Zhang, A New ate Control Scheme Usng Quadratc ate Dstorton Model, IEEE Trans. on CSVT, Vol. 7, Feb [3] A.ur and. Aravnd, Moton-comensated Vdeo Codng wth Adatve ercetual Quantzaton, IEEE Trans. on CSVT, Vol., Dec. 99. [4] J.-B. Cheng and H.-M. Hang, Adatve ecewse Lnear Bts Estmaton Model for MEG Based Vdeo Codng, Journal of Vsual Communcaton and Image eresentaton, Vol. 8, Mar

7 [5] J. bas-corbera, and S. Le, ate Control n DCT Vdeo Codng for Low-Delay Communcatons, IEEE Trans. on CSVT, Vol.9, Feb [6] M. Ghanbar, Vdeo Codng: An Introducton to Standard Codecs, Insttuton of Electrcal Engneerng ress, London, 999. [7] J.. Corbera and S. Le, ate Control for Low_Delay Vdeo Communcatons, ITU-T Standardzaton Sector, Document Q5-A-, ortland, 4-7 June