A NOVEL RATE CONTROL SCHEME FOR PANORAMIC ...

6
$ 129(/ 5$7( &21752/ 6&+(0( )25 3$125$0,& 9,'(2 &2',1* Yufan Liu , Mai Xu †∗ , Chen Li , Shengxi Li , and Zulin Wang †‡ 6FKRRO RI (OHFWURQLF DQG ,QIRUPDWLRQ (QJLQHHULQJ %HLKDQJ 8QLYHUVLW\ &KLQD &ROODERUDWLYH ,QQRYDWLRQ &HQWHU RI *HRVSDWLDO 7HFKQRORJ\ :XKDQ &KLQD $%675$&7 7KH SRSXODULW\ RI PXOWLYLHZ SDQRUDPLF YLGHRV KDV EHHQ FRQ VLGHUDEO\ LQFUHDVHG IRU SURGXFLQJ 9LUWXDO 5HDOLW\ 95 FRQ WHQW GXH WR LWV LPPHUVLYH YLVXDO H[SHULHQFH :H DUJXH LQ WKLV SDSHU WKDW 3615 LV OHVV HIIHFWLYH LQ DVVHVVLQJ YLVXDO TXDOL W\ RI FRPSUHVVHG SDQRUDPLF YLGHRV WKDQ 6SKHUHEDVHG 3615 63165 LQ ZKLFK VSKHUHWRSODLQ PDSSLQJ RI SDQRUDPLF YLGHRV LV FRQVLGHUHG 7KXV WKH FRQYHQWLRQDO UDWH FRQWURO 5 & VFKHPHV RI 'LPHQVLRQDO ' YLGHR FRGLQJ ZKLFK RSWL PL]H RQ 3615 DUH QRW VXLWDEOH IRU SDQRUDPLF YLGHR FRGLQJ 7R RSWLPL]H 63615 ZH SURSRVH LQ WKLV SDSHU D QRYHO 5& VFKHPH IRU SDQRUDPLF YLGHR FRGLQJ 6SHFL¿FDOO\ ZH GHYHO RS DQ 63615 RSWLPL]DWLRQ IRUPXODWLRQ ZLWK FRQVWUDLQW RQ ELWUDWH 7KHQ D VROXWLRQ LV SURYLGHG WR WKH GHYHORSHG IRUPX ODWLRQ VXFK WKDW ELWV FDQ EH DOORFDWHG WR HDFK FRGLQJ EORFN IRU DFKLHYLQJ RSWLPDO 63615 LQ SDQRUDPLF YLGHR FRGLQJ )LQDO O\ WKH H[SHULPHQW UHVXOWV YDOLGDWH WKH HIIHFWLYHQHVV RI WKH SUR SRVHG 5& VFKHPH LQ LPSURYLQJ 63615 RI SDQRUDPLF YLGHR FRGLQJ Index Terms² 9LUWXDO UHDOLW\ SDQRUDPLF YLGHR UDWH FRQ WURO ,1752'8&7,21 3DQRUDPLF YLGHRV DV D IRUP RI 9LUWXDO 5HDOLW\ 95 FRQWHQW RIIHU GHJUHH YLHZLQJ GLUHFWLRQ ZKHQ SOD\LQJ YLGHRV 5H FHQWO\ SDQRUDPLF YLGHR KDV EHHQ LQFUHDVLQJO\ SRSXODU GXH WR LWV LPPHUVLYH YLVXDO H[SHULHQFH 2Q WKH RWKHU KDQG KLJK UHVROXWLRQ . RU EH\RQG LV UHTXLUHG LQ SDQRUDPLF YLGHR IRU SOHDVDQW YLVXDO H[SHULHQFH VXFK WKDW WUHPHQGRXV FRPPXQL FDWLRQ EDQGZLGWK LV FRQVXPHG E\ SDQRUDPLF YLGHR 7KLV PD\ FDXVH EDQGZLGWKKXQJU\ LVVXH LQ FRPPXQLFDWLRQV 7KHUH IRUH WKHUH LV D GHVLUH GHPDQG IRU LPSURYLQJ FRPSUHVVLRQ HI ¿FLHQF\ RI SDQRUDPLF YLGHR 'XULQJ WKH SDVW GHFDGH VHYHUDO DSSURDFKHV >±@ KDYH EHHQ SURSRVHG WR HQKDQFH FRPSUHVVLRQ HI¿FLHQF\ RI SDQRUDPLF YLGHR )RU H[DPSOH LQ WKH HDUO\ WLPH XVHUGULYHQ LQWHUDFWLYH FRPSUHVVLRQ >±@ ZDV SURSRVHG IRU SDQRUDPLF YLGHR FRGLQJ ,Q SDUWLFXODU WKH SDQRUDPLF YLGHR LV FRP SUHVVHG LQWR PXOWLSOH WLOHEDVHG VWUHDPV DQG WKHQ D VSHFL¿F &RUUHVSRQGLQJ DXWKRU 0DL ;X PDL[X#EXDDHGXFQ 7KLV ZRUN ZDV VXSSRUWHG E\ WKH 16)& SURMHFWV XQGHU *UDQWV DQG DQG )RN <LQJ7RQJ HGXFDWLRQ IRXQGDWLRQ XQGHU JUDQW VHW RI VWUHDPV LV WUDQVPLWWHG GHFRGHG DQG UHQGHUHG DFFRUGLQJ WR WKH XVHU¶V SUHVHQW YLHZSRLQW DW WKH GHFRGHU VLGH $OWKRXJK WKHUH H[LVW D FRXSOH RI DGYDQFHG XVHUGULYHQ DSSURDFKHV LQ UHFHQW \HDUV > @ XVHUGULYHQ DSSURDFKHV DUH QRW PDWXUD WLYH IRU SUDFWLFDO DSSOLFDWLRQV ,W LV EHFDXVH WKHUH H[LVWV ODUJH WUDQVPLVVLRQ GHOD\ RQ WKH IHHGEDFN RI WKH XVHU¶V SUHVHQW YLHZ SRLQW 5HFHQWO\ HQFRGHU RSWLPL]DWLRQ DSSURDFKHV > @ KDYH EHHQ SURSRVHG IRU SDQRUDPLF YLGHR FRGLQJ )RU LQ VWDQFH =KHQJ et al. >@ SURSRVHG D QRYHO PRWLRQ FRPSHQVDW HG SUHGLFWLRQ DSSURDFK WR LPSURYH WKH TXDOLW\ RI FRPSUHVVHG SDQRUDPLF YLGHR %HVLGHV <X et al. >@ SUHVHQWHG DQ + FRPSODLQW FRGLQJ RSWLPL]DWLRQ DSSURDFK ZKLFK DGDSWLYHO\ DGMXVWV GLIIHUHQW VL]HV DQG ELWUDWHV RI YDULRXV WLOHV WR HQKDQFH WKH RYHUDOO TXDOLW\ RI SDQRUDPLF YLGHR ,QVWHDG RI RSWLPL] LQJ WKH HQFRGHU VRPH SUHSURFHVVLQJ DSSURDFKHV >±@ KDYH EHHQ SURSRVHG IRU SDQRUDPLF YLGHR FRGLQJ ,Q >@ %XGDJDYL et al. SURSRVHG WR XWLOL]H UHJLRQ DGDSWLYH VPRRWKLQJ EHIRUH HQFRGLQJ SDQRUDPLF YLGHRV VR WKDW PRUH ELWV FDQ EH VDYHG LQ WKH UHJLRQV FORVH WR SROHV ZKLFK DUH UDUH WR EH YLHZHG 6LPL ODUO\ <RXYDODUL >@ SURSRVHG D UHJLRQDO GRZQVDPSOLQJ SUH SURFHVVLQJ DSSURDFK WR GHFUHDVH WKH H[WUD ELWUDWH FDXVHG E\ RYHUVWUHWFKHG SRODU DUHDV +RZHYHU WKH SUHSURFHVVLQJ DS SURDFKHV FDQQRW FRQWURO ELWUDWH IRU SDQRUDPLF YLGHR FRGLQJ DQG WKXV FRPSUHVVHG SDQRUDPLF YLGHRV FDQ KDUGO\ UHVXPH R ULJLQDO TXDOLW\ DW VXI¿FLHQW ELWUDWHV ,Q IDFW UDWH FRQWURO 5 & FDQ EH DSSOLHG WR RSWLPL]H YLVXDO TXDOLW\ RI FRPSUHVVHG SDQRUDPLF YLGHR DW D JLYHQ WDUJHW ELWUDWH 7KH H[LVWLQJ 'LPHQVLRQDO ' YLGHR FRGLQJ VWDQGDUGV VXFK DV WKH ODWHVW +LJK (I¿FLHQW 9LGHR &RGLQJ +(9& >@ KDYH EHHQ XVHG IRU SDQRUDPLF YLGHR FRGLQJ DQG WKHLU 5 & VFKHPH PD\ EH DSSOLHG DV ZHOO 0RVW UHFHQWO\ WKH Rλ VFKHPH >@ KDV EHHQ SURSRVHG IRU WKH ODWHVW +(9& DV WKH VWDWHRIWKHDUW 5& VFKHPH 7KH Rλ VFKHPH SHUIRUPV ZHOO LQ ERWK UDWH GLVWRUWLRQ 5' RSWLPL]DWLRQ DQG FRQWURO DFFXUD F\ 8QIRUWXQDWHO\ WKH 5' RSWLPL]DWLRQ RI WKH Rλ VFKHPH PDLQO\ IRFXVHV RQ LPSURYLQJ 3615 IRU ' YLGHR FRGLQJ ZKLOH 6SKHUHEDVHG 3615 63615 >@ LV PRUH HIIHFWLYH LQ PHDVXULQJ YLVXDO TXDOLW\ IRU SDQRUDPLF YLGHR FRGLQJ 7KXV LW LV QHFHVVDU\ WR FRQWURO ELWUDWH RI SDQRUDPLF YLGHR FRGLQJ ZLWK RSWLPL]DWLRQ RQ 63615 7R WKH EHVW RI RXU NQRZOHGJH WKHUH LV QR 5& ZRUN RQ RSWLPL]LQJ 63615 IRU SDQRUDPLF YLGHR FRGLQJ ,Q WKLV SDSHU ZH SURSRVH DQ 5& VFKHPH IRU SDQRUDP Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) 2017 10-14 July 2017 978-1-5090-6067-2/17/$31.00 ©2017 IEEE ICME 2017 691

Transcript of A NOVEL RATE CONTROL SCHEME FOR PANORAMIC ...

Yufan Liu†, Mai Xu†∗, Chen Li†, Shengxi Li†, and Zulin Wang†‡

Index Terms

et al.

et al.

et al.

R λ

R λ

R λ

Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) 2017 10-14 July 2017

978-1-5090-6067-2/17/$31.00 ©2017 IEEE ICME 2017

691

R λ

{gn}Nn=1 N

gn(xn, yn)

(xn, yn)

=

∑N

n=1(S(xn, yn) − S′(xn, yn))

2

N,

= 10 log102552

,

S(xn, yn) S′(xn, yn) n

(xn, yn)Lanczos

S(xn, yn) S′(xn, yn)

{(xn, yn)}Nn=1

Oculus Rift DK2

LiveViewRift

(100 − )

692

Fengjing 1 Tiyu 1 Yanchanghui 2

Dianying Hangpai 1 Hangpai 2

AerialCity DrivingInCountry

dm =∑

n∈Cm

(S(xn, yn) − S′(xn, yn))

2,

Cm m

R

min(M∑

m=1

dm) s.t.M∑

m=1

rm = R.

rm m M

λ

min{rm}M

m=1

J =

M∑

m=1

(dm + λrm).

J

J

∂J

∂rm=∂(∑M

m=1(dm + λrm))

∂rm

=∂dm

∂rm+ λ

= 0.

dm rm dm rm

=R

f ×W ×H,

f W H

dm = cm · (rm)−km ,

cm km

−∂dm

∂rm= cm · km · r(−km−1)

m.

rm = (cmkm

λ)(

1km+1

).

M∑

m=1

rm = R.

M∑

m=1

rm =

M∑

m=1

(cm · kmλ

)(1

km+1) = R.

693

rm

R λ

encoder lowdelay P main.cfg

5.34%

Dianying

694

Fengjing 1 Tiyu 1 Yanchanghui 2 Dianying

Hangpai 1 Hangpai 2 AerialCity DrivingInCountry

Fengjing 1 Tiyu 1 Yanchanghui 2

Dianying Hangpai 1 Hangpai 2

AerialCity DrivingInCountry

Fengjing 1 Tiyu 1 Yanchanghui 2 Dianying Hangpai 1 Hangpai 2 AerialCity DrivingInCountry

Tiyu 1

100%

695

DianyingDianying

λ

5.34%

IEEE Trans-actions on Circuits and Systems for Video Technology (TCSVT)

Proc.Panoramic Photogrammetry Workshop (PPW)

IEEE 13th International Symposium on Con-sumer Electronics

IEEE Transactions on Multimedia

Proceedings of the ACM on Multi-media Conference (ACM MM)

IEEEInternational Conference on Multimedia and Expo (ICME)

Multimedia Content Analysis and Mining

IEEE International Conference on Im-age Processing (ICIP)

Image Processing (ICIP), 2016 IEEE International Conference on

Proceedings of the 3rd International Workshop on Immersive MediaExperiences

Picture Coding Symposium

IEEESignal Processing Magazine

IEEE Transactions onImage Processing (TIP)

Mixed andAugmented Reality (ISMAR), 2015 IEEE International Symposium on

Input Document to JVET

Output Document of Standard for Immer-sive Visual Content Coding in IEEE 1857

IEEE Transactions on Circuitsand Systems for Video Technology (TCSVT)

696