Hri Vieo Coing Principle of Hri Vieo Coing Moion-Compene Preicion Bi Allocion Moion Moel Moion Eimion Efficienc of Hri Vieo Coing Thom Wiegn: Digil Imge Communicion Hri Vieo Coing
I h een cuomr in he p o rnmi ucceive complee imge of he rnmie picure. [...] In ccornce wih hi invenion hi ifficul i voie rnmiing onl he ifference eween ucceive imge of he ojec. Thom Wiegn: Digil Imge Communicion Hri Vieo Coing
Hri Vieo Encoer [ ] u[ ] - Decoer Coer Conrol Inr-Frme DCT Coer Inr-Frme Decoer Conrol D DCT Coefficien u' [ ] Inr/Iner 0 ˆ [ ] Moion- Compene Preicor Moion Eimor ' [ ] Moion D Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 3
Hri Vieo Decoer Conrol D DCT Coefficien Decoer Inr-Frme Decoer u' [ ] Inr/Iner 0 Moion- Compene Preicor ' [ ] Moion D ˆ [ ] Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 4
Moion-Compene Preicion Moving ojec Sionr ckgroun Previou frme Δ Curren frme Diplce ojec ime Preicion for he luminnce ignl [] wihin he moving ojec: ˆ [ ] ( Δ from: Giro Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 5
Moion-Compene Preicion: Emple Frme [-] (previou Frme [] (curren Priion of frme ino lock (chemic Frme wih iplcemen vecor Difference eween moioncompene preicion n curren frme u[] Reference lock in frme Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 6
Re-conrine Moion Eimion Bi-re R Moion vecor re R m D Preicion error re R u Opimum re-off: D R D m R u Diplcemen error vrince Diplcemen error vrince cn e influence vi Block-ize qunizion of moion prmeer Choice of moion moel from: Giro Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 7
Re-Conrine Coer Conrol Efficienc incree vi ing coing moe Wh pr of he imge houl e coe uing wh meho? Prolem cn e poe : minimize iorion D ujec o re conrin R c min { D }.. R<Rc Coer Conrol uing Lgrngin opimizion: Solve n unconrine minimizion prolem λ min { D R } [Evere III 963] [Shohm Gerho 988] [Chou Lookugh Gr 989] Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 8
Lgrngin Opimizion in Vieo Coing A numer of inercion re ofen neglece Temporl epenenc ue o DPCM loop Spil epenenc of coing eciion Coniionl enrop coing Re-Conrine Moion Eimion [Sullivn Bker 99]: min {D m λ R m } Diorion fer moion compenion Lgrnge prmeer Numer of i for moion vecor Re-Conrine Moe Deciion [Wiegn e l. 996]: min {D λ R} Diorion fer reconrucion Lgrnge prmeer Numer of i for coing moe Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 9
Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 0 Cmer Moel Cmer Moel Nurl cmer-view cene re projece from he 3-D worl ino he -D imge plne A cmer moel cn e ue o ecrie projecion len n mpling z Imge plne 3D Poiion Imge poiion Focl poin F z 3-D poiion Imge poiion Imge plne Perpecive projecion moel Orhogrphic projecion moel Cn e ue when F«Z for ll coniere poin Z Y X P Z Y X P F p F p Y Z F X Z F Y X
Moion Moel Moion in 3-D pce correpon o iplcemen in he imge plne Moion compenion in he imge plne i conuce o provie preicion ignl for efficien vieo compreion Efficien moion-compene preicion ofen ue ie informion o rnmi he iplcemen Diplcemen mu e efficienl repreene for vieo compreion Moion moel rele 3-D moion o iplcemen uming reonle rericion of he moion n ojec in he 3-D worl Moion Moel f ( f ( : locion in previou imge : locion in curren imge : vecor of moion coefficien : iplcemen Thom Wiegn: Digil Imge Communicion Hri Vieo Coing
Mhemicl moel: Rericion: Perpecive Moion Moel 3 3 c c c c Roion n cling of rigi o in 3-D pce u no rnlion Trnlion roion n cling of plnr pch in 3-D Avnge: correpon o perpecive projecion moel Divnge: hperolic moion funcion Thom Wiegn: Digil Imge Communicion Hri Vieo Coing
Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 3 Trnlionl moion moel 4-Prmeer moion moel: rnlion zoom (ioropic Scling roion in imge plne Affine moion moel: Prolic moion moel Cn lo e viewe Tlor epnion of perpecive moion moel Orhogrphic Moion Moel Orhogrphic Moion Moel 3 3 3 3 6 5 4 3 6 5 4 3
Impc of he Affine Prmeer 50 rnlion 00 50 0-50 -00-50 0 50 00 50-50 -00-50 50 50 00 50 0-50 -00-50 0 50 00 50-50 cling 00 50 0-50 -00-50 0 50 00 50-50 -00-00 -50-50 50 3 heering 00 50 0-50 -00-50 0 50 00 50-50 -00-50 Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 4
Impc of he Prolic Prmeer 50 4 00 50 0-50 -00-50 0 50 00 50-50 -00-50 50 00 50 0-50 -00-50 0 50 00 50-50 -00 5 50 00 50 0-50 -00-50 0 50 00 50-50 -00-50 -50 50 6 00 50 0-50 -00-50 0 50 00 50-50 -00-50 Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 5
Impc of he Perpecive Prmeer c 50 00 50 0-50 -00-50 0 50 00 50-50 -00-50 50 00 50 0-50 -00-50 0 50 00 50-50 -00-50 c 50 00 50 0-50 -00-50 0 50 00 50-50 -00-50 Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 6
Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 7 Differenil Moion Eimion Differenil Moion Eimion Aume mll iplcemen : Aperure prolem: everl oervion require Inccure for iplcemen > 0.5 pel muligri meho ierion Minimize Diplce frme ifference Horizonl n vericl grien of imge ignl S u Δ Δ Δ ( ( ( ( ˆ( ( ( ( min u B B
Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 8 Grien Grien-Be Affine Refinemen Be Affine Refinemen Diplcemen vecor fiel i repreene Cominion 3 3 iel em of liner equion: Sem cn e olve uing e.g. peuo-invere minimizing ( ( ( ( ( 3 3 u Δ B B u ( rg min 3 3 3 3 u
Principle of Blockmching Suivie ever imge ino qure lock Fin one iplcemen vecor for ech lock Wihin erch rnge fin e mch h minimize n error meure. erch rnge in previou frme S k- Block of curen frme S k Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 9
Blockmching: : Error Meure Men qure error (um of qure error B B SSE( [ ( ( Δ] Sum of olue ifference SAD( B B ( ( Δ Approimel me performnce SAD le comple for ome rchiecure Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 0
Blockmching: : Serch Sregie I Full erch All poile iplcemen wihin he erch rnge re compre. Compuionll epenive Highl regulr prllelizle Thom Wiegn: Digil Imge Communicion Hri Vieo Coing
Block mching: Serch Sregie III Compuionl Complei Block comprion D logrihmic full erch 8 69 69 Emple: m. horizonl vericl iplcemen 6 ineger-pel ccurc - for pecil vecor (6 - wor ce from: Giro Thom Wiegn: Digil Imge Communicion Hri Vieo Coing
Block Comprion Spee-Up Tringle inequliie for SAD n SSE S S ( S S S lock lock S k k S k k lock N k lock ( S k k S k lock numer of erm in um Sreg:. Compue pril um for lock in curren n previou frme. Compre lock e on pril um 3. Omi full lock comprion if pril um inice wore error meure hn previou e reul Performnce: > 0 pee up of full erch lock mching repore emploing. Sum over 66 lock. Row wie lock projecion 3. Column wie lock projecion (Lin Ti IEEE Tr. Commun. M 97 Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 3 k N lock S lock k S k lock S k
Su-pel Trnlionl Moion Moion vecor re ofen no rerice o onl poin ino he ineger-pel gri of he reference frme Tpicl u-pel ccurcie: hlf-pel n qurer-pel Su-pel poiion re ofen eime refinemen Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 4
Hior of Moion Compenion Inrfrme coing: onl pil correlion eploie Complei DCT [Ahme Nrjn Ro 974] JPEG [99] incree Coniionl replenihmen H.0 [984] (DPCM clr qunizion Frme ifference coing H.0 Verion [988] Moion compenion: ineger-pel ccure iplcemen H.6 [99] Hlf-pel ccure moion compenion MPEG- [993] MPEG-/H.6 [994] Vrile lock-ize (66 & 88 moion compenion H.63 [996] MPEG-4 [999] Vrile lock-ize (66 44 n muli-frme moion compenion H.6L [00] Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 5
PSNR [B] 35 40 38 36 34 3 30 8 Mileone in Vieo Coing Vrile lock ize (66 44 qurer-pel muli-frme moion compenion (H.6L 00 Vrile lock ize (66 88 (H.63 996 qurer-pel moion compenion (MPEG-4 998 Bi-re Reucion: 75% Ineger-pel moion compenion (H.6 99 Inrfrme DCT coing (JPEG 990 Hlf-pel moion compenion (MPEG- 993 MPEG- 994 Coniionl Replenihmen (H.0 0 00 00 300 Foremn 0 Hz QCIF 00 frme Re [ki/] Frme Difference coing (H.0 988 Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 6
SNR [B] 40 38 36 34 3 30 8 Mileone in Vieo Coing Vrile lock ize (66 44 qurer-pel muli-frme moion compenion (H.6L 00 Viul Comprion Vrile lock ize (66 88 (H.63 996 qurer-pel moion compenion (MPEG-4 998 Ineger-pel moion compenion (H.6 99 Inrfrme DCT coing (JPEG 990 Hlf-pel moion compenion (MPEG- 993 MPEG- 994 Coniionl Replenihmen (H.0 0 00 00 300 Frme Difference coing (H.0 988 Re [ki/] Foremn 0 Hz QCIF 00 frme Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 7
Foremn @ 00 ki/ 0 Hz Inrfrme DCT coing No moion compenion (JPEG 990 Vrile lock ize (66 44 qurer-pel muli-frme moion compenion (H.6L 00 Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 8
Summr Vieo coing hri of moion compenion n preicion reiul coing Lgrngin i-llocion rule pecif conn lope llocion o moion coefficien n preicion error Moion moel cn repreen vriou kin of moion In prcice: ffine or 8-prmeer moel for cmer moion rnlionl moel for mll lock Differenil meho clcule iplcemen from pil n emporl ifference in he imge ignl Block mching compue error meure for cnie iplcemen n fin e mch Spee up lock mching erch meho n clever pplicion of ringle inequli Hri vieo coing h een ricll improve enhnce moion compenion cpiliie Thom Wiegn: Digil Imge Communicion Hri Vieo Coing 9