(19) United States (12) Patent Application Publication (10) Pub. No.: US 2015/0121416 A1 MOON et al.
US 2015O121416A1
(43) Pub. Date: Apr. 30, 2015
(54) VIDEO DISPLAY DEVICE, TERMINAL DEVICE, AND METHOD THEREOF
Publication Classification
(51) Int. Cl. (71) Applicant: LG ELECTRONICS INC., Seoul (KR) H4N2L/8 (2006.01)
H4N2L/24 (2006.01) (72) Inventors: Kyoungsoo MOON, Seoul (KR); Jaehee H4N2L/2478 (2006.01)
CHUNG, Seoul (KR): Hyeonjae LEE, H4N2L/643 (2006.01) Seoul (KR); Jinpil KIM, Seoul (KR): H4N2L/2475 (2006.01) Aettie J, Seoul (KR); Kyungho KIM, H4N2L/2126 (2006.01) Seoul (KR); Sanghyun KIM, Seoul H4N2L/214 (2006.01) (KR) (52) U.S. Cl.
CPC ....... H04N 2 1/8173 (2013.01); H04N 2 1/4263 (21) Appl. No.: 14/553,547
(22) Filed: Nov. 25, 2014
Related U.S. Application Data (63) Continuation of application No. 13/567,689, filed on
Aug. 6, 2012. (60) Provisional application No. 61/549,236, filed on Oct.
20, 2011, provisional application No. 61/543.288, A video display apparatus pairs with a first terminal device. filed on Oct. 4, 2011, provisional application No. The video display apparatus receives an audio-visual content, 61/525,835, filed on Aug. 21, 2011.
(2013.01); H04N 2 1/4104 (2013.01); H04N 2 I/441 (2013.01); H04N 2 1/64322 (2013.01);
H04N 2 1/4753 (2013.01); H04N 2 1/478 (2013.01)
(57) ABSTRACT
and displays the audio-visual content. The video display apparatus acquires enhanced service information on an avail
(30) Foreign Application Priority Data able enhanced service for the audio-visual content, and noti fies an occurrence of the available enhanced service to the first
Jun. 14, 2012 (KR) ........................ 10-2012-0063969 terminal device.
10
CONTENT PROYONG SERVER 20
50 -
50
CONTENT RECOGNIZING
ENHANCED SERVICE PROVIDING SERVER
MULTCHANNEL WIDEO DISTRIBUTING SERVER
SERVICE PROVIDING SERVER
ENHANCED SERVICE INFOR MATION PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
70 400
TERMINAL DEVICE
WIDEO DISPLAY DEVICE
BROADCAST RECEIVING DEVICE
Patent Application Publication
FIG.1 10
CONTENT PROVONG SERVER
CONTENT RECOGNIZING SERVICE PROVIDING SERVER
ENHANCED SERVICE INFOR MATION PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
50-N-
50
MULTCHANNEL WIDEO DSTRIBUTING SERVER
Apr. 30, 2015 Sheet 1 of 96 US 2015/O121416 A1
70 4. OO
TERYNAL DEVICE
100
WICEO DISPLAY DEVICE
BROADCAST RECE WING DEVICE
60
Patent Application Publication Apr. 30, 2015 Sheet 2 of 96 US 2015/O121416 A1
FIG.2 10
CONTENT PROVIDING SERVER
WATERMARK 70 SERVER
400
TERMINAL DEVICE
1OO
MAGE DISPLAY DEVICE
ENHANCED SERVICE INFOR MATION PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
50
50
MULTCHANNEL WIDEO DISTRIBUTING SERVER
BROADCAST RECEIVING DEVICE
Patent Application Publication Apr. 30, 2015 Sheet 5 of 96 US 2015/O121416 A1
FIG.5
10
CONTENT PROVIDING SERVER
FNGERPRINT SERVER 400
TERMINAL DEVICE
VIDEO DISPLAY DEVICE
ENHANCED SERVICE INFOR MATON PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
50
ENHANCED SERVICE 50 PROVIDING SERVER
MULTCHANNEL WIDEO DISTRIBUTING SERVER
BROADCAST RECEIVING DEVICE
60
US 2015/O121416 A1 Apr. 30, 2015 Sheet 6 of 96 Patent Application Publication
BOMBS (BINH B'GOES |-?---------------
US 2015/O121416 A1
| | |
r– – – – – – – – – – –|
0
Apr. 30, 2015 Sheet 7 of 96
L'OIH
Patent Application Publication
Patent Application Publication Apr. 30, 2015 Sheet 8 of 96 US 2015/O121416 A1
FIG.8
10
CONTENT PROVIDING SERVER
WATERMARK SERVER
FNGERPRINT SERVER
ENHANCED SERVICE INFOR MATION PROVIDING SERVER
ENHANCED SERVICE PROVIDING SERVER
40
R
O
NA CE
TERMINAL DEY
WOEO DISPLAY DEVICE
22
40
ENHANCED SERVICE 50 PROVIDING SERVER
MULTCHANNEL WIDEO DISTRIBUTING SERVER
BROADCAST RECEWING DEVICE
60
US 2015/O121416 A1 Apr. 30, 2015 Sheet 9 of 96 Patent Application Publication
00700||090907 22| 20||
Patent Application Publication Apr. 30, 2015 Sheet 11 of 96 US 2015/O121416 A1
FIG.11
era Go App store situences s Broadband NetWOrk
w 1.
ACR M/W (e.g. Browser) ACRApp ||F
ACR Native
136a
ACR plat for Orm API
Fl
133a
Patent Application Publication Apr. 30, 2015 Sheet 12 of 96 US 2015/O121416 A1
FIG.12
OBTAN CONTENT INFORMATION OF MAN AW CONTENT
OBTAIN ENHANCED SERVICE FILE
EXECUTE OBTANED ENHANCED SERVICE FILE IN BACKGROUND
DISPLAY COLOR BUTTON S4O7
PLAY ENHANCED SERVICE EXECUTED IN BACKGROUND
ON PLAY DEVICE
TERMINATE ENHANCED SERVICE S411
S403
S405
S409
Patent Application Publication Apr. 30, 2015 Sheet 13 of 96 US 2015/O121416 A1
FIG.13
1390
(e.g. BrOWSer) XML 1380
ACR Engine 136b it
ACR plat for Orm API 135b
East Operating System E. 131b.
133b
Patent Application Publication Apr. 30, 2015 Sheet 14 of 96
FIG.14
OBTAIN INFORMATION ON ACR ENGINE THAT ACR APPLICATION TSELF USES
START ACR ENGINE THAT ACR APPLICATION TSELF USES
OBTAN CONTENT INFORMATION BY ACR APPLICATION THROUGH ACR ENGINE
OBTANAVAILABLE ENHANCED SERVICE BY ACRAPPLICATION
DISPLAY COLOR BUTTON BY ACRAPPLICATION
PLAY ENHANCED SERVICE BY ACRAPPLICATION
TERMINATE ACRAPPLICATION BY ACR MODLEWARE
S501
S503
S505
S507
S509
S511
S513
US 2015/O121416 A1
Patent Application Publication Apr. 30, 2015 Sheet 15 of 96 US 2015/O121416 A1
FIG.15
Brave 9 GApp store Clernet w -- Broadband NetWOrk
ACR M/W ACR Application IF Script (e.g. Browser)
ACR Pat form AP
... . NetWork Or
Operating System H
133C
Patent Application Publication Apr. 30, 2015 Sheet 16 of 96 US 2015/O121416 A1
FIG.16
TRANSMIT QUERY INCLUDING SPECIFIC INFORMATION TO ACR ADJUSTING S601
SERVER BY ACR ENGINE
RECEIVE FINGERPRINT SERVER ADDRESS BY ACR ENGINE S603
BTAIN FNGERPRINT SERVER ADORES OBTANF NT SER S S605 BY ACRAPPLICATION
OBTAIN FEATURE INFORMATION BY ACRAPPLICATION S607
TRANSMT QUERY INCLUDING FEATURE INFORMATION BY ACR APPLICATION S609
RECEIVE CONTENT INFORMATION IN RESPONSE TO S611 QUERY BY ACRAPPLICATION
RECEIVE AVAILABLE ENHANCED SERVICE S613 BY ACRAPPLICATION
DISPLAY COLOR BUTTON BY ACR APPLICATION S615
PLAY ENHANCED SERVICE BY ACR APPLICATION S617
TERMINATE ACRAPPLICATION BY ACR S619 YDOLEWARE
Patent Application Publication Apr. 30, 2015 Sheet 17 of 96 US 2015/O121416 A1
FIG.17
GOOgle Nave?
s Broadband NetWOrk
ACP Native App
ACR Native
1360
ACR Pat form AP
Frt. irrel
133d
Patent Application Publication Apr. 30, 2015 Sheet 18 of 96 US 2015/O121416 A1
FIG.18
CONFIRN ACR ENGINE MATCHING TO MAN AW w CONTENT BY ACR ENGINE MANAGEMENT UNIT
SLEEP REMAINING ACR ENGINES EXCEPT FOR ACR ENGINE RECOGNIZING CONTENT SUCCESSFULLY BY ACR ENGINE MANAGEMENT UNIT
OBTAN CONTENT INFORMATION OF MAN AW CONTENT BY ACR ENGINE MANAGEMENT UNIT
CONFIRMANAIABLE APPLICATION BY ACR ENGINE MANAGEMENT UNIT
EXECUTE CONFIRMED APPLICATION IN BACKGROUND BY ACR ENGINE MANAGEMENT UNIT
DISPLAY COLOR BUTTON BY APPLICATION S707
PLAY ENHANCED SERVICE S709
CHANNELS CHANGED OR MAN AN CONTENT
S CHANGED?
YES CONFIRMACR ENGINE MATCHING TO APPLICATION
BY ACR ENGINE MANAGEMENT UNIT
CONNECT CONFIRMED ACR ENGINE WITH APPLICATION BY ACR ENGINE MANAGEMENT UNIT
USER INPUT FOR TERMINATING APPLICATION
S RECEIVED?
YES
TERMINATE APPLICATION BY APPLICATION MANAGEMENT UNIT
S701
S702
S703
S704
S705
S710
S711
S712
S713
S714
Patent Application Publication Apr. 30, 2015 Sheet 19 of 96 US 2015/O121416 A1
EXTRACT ACERTAIN SECTION OF S801 MAN AV CONTENT
OBTAN CONTENT INFORMATION OF MAN AW S803 CONTENT ON THE BASIS OF EXTRACTEO SECTION
PERFORMSYNCHRONIZATION S805
OBTAIN ENHANCED SERVICE START TIME OF ENHANCED SERVICE ON THE BASIS OF OBTANED
CONTENT INFORMATION
S807
START OBTANED ENHANCED SERVICE WHEN SYNCHRONIZED PLAY TIME OF MAN AW CONTENT
|S|DENTICAL TO START TIME OF ENHANCED SERVICE
S809
Patent Application Publication Apr. 30, 2015 Sheet 20 of 96 US 2015/0121416A1
FIG20
CONTENT FNGERPRINT SERVER
(SY AY SAPLE QUERY/REPLY FP
Network ||F
QUERY RESULT
{{MetadataX Content ID . Secret Garden Play time : 68 min BrOadcaster : SBS Start time : 11 OOOMS End time : 1450OMS
Time line
Patent Application Publication Apr. 30, 2015 Sheet 21 of 96 US 2015/0121416A1
FIG21
210 200 220
SUBSCREEN ON WHCH ENHANCED SERVICE S
DISPLAYEO
Patent Application Publication Apr. 30, 2015 Sheet 22 of 96 US 2015/O121416 A1
FIG.22
230 200
MY APPLICATIONS
Broadc Broadc BrOadC BrOadc COMMON aster A aster B aster C aster D App
Mage
SUBSCREEN ON WHICH ENHANCED SERVICES
DSPLAYED
SUBSCREEN ON WHICH MENUS OSPLAYED
240 (B)
Patent Application Publication Apr. 30, 2015 Sheet 23 of 96 US 2015/O121416 A1
330 Voting will start in N Minutes. GO to oria e 370 32O PROGRAMINFORSOCIAL RECOY CATALOG
Patent Application Publication Apr. 30, 2015 Sheet 24 of 96 US 2015/O121416 A1
FIG.24
310 300
PARTICIPATION ADDITIONAL INFORMATION AREA AOVERTISEENT DSCOVERY AREA
Voting will Start in N Minutes. GO to
330
Patent Application Publication Apr. 30, 2015 Sheet 25 of 96 US 2015/0121416A1
FIG.25
1N USER INPUT CORRESPONDING
< PICTION SETIO D-S901 MENU DISPLAY S RECEIVED?
DISPLAY APPLICATION SELECTION MENU S903
USER INPUT CORRESPONDING CON SELECTION OF WIDEO
< DISPLAY DEVICE D-so MANUFACTURER APPLICATION IS RECEIVED?
EXECUTE WIDEO DISPLAY DEVICE MANUFACTURER APPLICATION S907
Patent Application Publication Apr. 30, 2015 Sheet 26 of 96 US 2015/O121416 A1
FIG.26
310 300
4.
BroadC BroadC BrOadC COMMOn r 370
Patent Application Publication Apr. 30, 2015 Sheet 27 of 96 US 2015/O121416 A1
Do you think .... ?
CHARACER INFORMATION
330 Voting will Start in N Minutes, GO to 320 ADVERT SEVENT AREA
Patent Application Publication Apr. 30, 2015 Sheet 28 of 96 US 2015/O121416 A1
310 300 342 34
Do yOu think ... ? 360
Do you think ... ? YES O80% NO 20%
330 Voting will Start in N minutes. GO to 32O
SFMFNT O ADVERTISEMENT E. 37
Patent Application Publication Apr. 30, 2015 Sheet 29 of 96 US 2015/O121416 A1
370 330 WOting will Start in N Minutes. G0 to 320 PROGRAMINFORSOCIAL RECOMCATALOG
Patent Application Publication Apr. 30, 2015 Sheet 30 of 96 US 2015/O121416 A1
FIG.30
OBTAIN CONTENT INFORMATION S1001
SEARCH SUB MENUS SELECTED? S1002
YES
OBTAIN SEARCH IST ON THE BASIS OF CONTENT INFORMATION S1003
DISPLAY SEARCH LIST S1007
ONE OF SEARCH TEMS |S SELECTED?
S1009
YES
SEARCH INFORMATION ON SELECTED SEARCH TEM S1011
DISPLAY SEARCHED INFORMATION S 1013
Patent Application Publication Apr. 30, 2015 Sheet 31 of 96 US 2015/O121416A1
310 300 342 341
330 Voting will Start in N Minutes, G0 to RTSEMENT AREA 320--PROGRAN SOCIAL RECOM CATALOG orse
Patent Application Publication Apr. 30, 2015 Sheet 32 of 96 US 2015/0121416A1
31 O 300 342 341
BRTHDAY: ... FAYORATE FOOO. . .
330
320 Voting Will Start in N minutes. G0 to
Patent Application Publication Apr. 30, 2015 Sheet 33 of 96 US 2015/O121416 A1
FIG.33
S1 101
TWITTER SUB VENUS SELECTED? S1 102
YES
OBTAIN WITTER CHATTING LIST ON THE BASIS OF S1 103 CONTENT INFORMATION
DISPLAY CHATTING DSPLAY S1107
Patent Application Publication Apr. 30, 2015 Sheet 34 of 96 US 2015/0121416A1
31 O 300 343 342 341
ENTRE Topchel
Guy A. : What do you like most? . . . Miss A:... do love to have the . juicy. Stake,
Rich: Ren is the start of next TV show of this? ... . . . . Miss B: Not know-AOY, But you...'... Cantook Up Ytaia, Iovii, COM... Guy A. : Waitt'll at you know ...
Erise it 330
320
Patent Application Publication Apr. 30, 2015 Sheet 35 of 96 US 2015/O121416 A1
FIG.35
OBTAN CONTENT INFORMATION
NEWS SUB MENUS SELECTED? S1202
OBTAIN NEWS ON THE BASIS OF CONTENT INFORMATION
DISPLAY NEWS
S1203
S1207
Patent Application Publication Apr. 30, 2015 Sheet 36 of 96 US 2015/O121416 A1
310 300 343 342 341
Strong. Earthquake Hits Indonesia's Papua:
S0 of east for Washington?
330
320 Voting will start in N minutes. Go to
ADVERTISEVENT AREA 370
Patent Application Publication Apr. 30, 2015 Sheet 37 of 96 US 2015/O121416 A1
FIG.37
S1301
WEWER LIST SUB MENUS
SELECTED?
YES
OBTAIN INFORMATION ON FRIEND STAND S1303 WATCHING ON THE BASIS OF CONTENT INFORMATION
DSPLAY FRIEND LIST S1307
S1302
FRIEND IS SELECTED? S1309
YES S1311
SELECTED FRIEND NO WEWS MAN AW CONTENT
S1313 S1315 YES
OSPLAY CHATTING WINDOW FOR SELECTED TRANSMIT MESSAGE RECOMMENDING FRIEND ON ADDITIONAL INFORMATION AREA WATCHING OF MAN AW CONTENT
Patent Application Publication Apr. 30, 2015 Sheet 38 of 96 US 2015/O121416 A1
FIG.38
310 300 343 342 341
PICTURE NAME: B
PICTURE NAME: A (IN WATCHING)
PICTURE NAME: C
PICTURE NAME: ) (N WATCHING)
330 CTTACT 320 ACVERSEMENT AREA
Patent Application Publication Apr. 30, 2015 Sheet 39 of 96 US 2015/O121416 A1
SOCIAL NETWORK SERVICE PROVIDING SERVER
st S1403 CONTENT A D / /ONTENTA ID CONTENT A D
USer D FRIEND A D se FRIENDB ID CONTENT A D
Aisi SE. IMAGE DISPLAY DEVICE IMAGE DISPLAY DEVICE
(FRIENDA) (FRIENDB) WDEO DISPLAY DEVICE
100
Patent Application Publication Apr. 30, 2015 Sheet 40 of 96 US 2015/O121416 A1
FIG.40
S1511
75 77 FRIEND A D
SOCIAL NETWORK SERVICE PROVIDING SERVER ACR SERVER FRIEND BID
S1501
S1505 \ FRIEND A D
S1509 FRIEND BO CONTENT A D
USer D
VIDEO DISPLAY DEVICE S1507 S 1515
(FRIENDA) ) ot A D IDEO DISPLAY DEVICE CONTENT A D FRIEND A D
(FRIENDB) CONTENT A D
S1513 N CONTENT A D FRIEND BID
VIDEO DISPLAY DEVICE
100
Patent Application Publication Apr. 30, 2015 Sheet 41 of 96 US 2015/O121416 A1
FIG.41
S1601 OBTAN CONTENT INFORMATION
PREFERENCE DISPLAY SUB MENUS SELECTED? S1602
YES
OBTAN PREFERENCE DISPLAY LIST ON THE BASIS OF CONTENT INFORMATION
DSPLAY PREFERENCE OSPLAY LIST
S1603
S1607
REFERENCE DISPLAY S SELECTED S1609
S1611 POST PREFERENCE DISPLAY CONTENT ON SNS
Patent Application Publication Apr. 30, 2015 Sheet 42 of 96 US 2015/O121416 A1
310 300 342 341
SCENE 3: LKE
SCENE #2 KE. T.
KE.
IKE
320 INFOR ADVERT SEVENT AREA 330 Voting will Start in N minutes. GO to
Patent Application Publication Apr. 30, 2015 Sheet 43 of 96 US 2015/0121416A1
FIG.43
S1701 OBTAN CONTENT INFORMATION
SUB MENUS SELECTED? S1702
OBTAN DIGITAL MEDALIST ON THE BASIS OF CONTENT INFORMATION
DISPLAY DIGITAL MEDALIST
S1703
S1707
DIGITAL MEDIA IS SELECTED? S1709
S 1711 DOWNLOADD GTAL MEDIA
Patent Application Publication Apr. 30, 2015 Sheet 44 of 96 US 2015/O121416A1
31 O 300
EPISODE #2
EPISODE #3
PREVIEWEPISODE 4 Voting Will Start in N Minutes, GO to
Hora woul AVERTISEET AE 330
320 370
Patent Application Publication Apr. 30, 2015 Sheet 45 of 96 US 2015/O121416 A1
310 300 343 342 341
WOOH
WOOH2
WOOH3
WOOH4.
330 Voting Will Start in N Minutes. GO to 320
Patent Application Publication Apr. 30, 2015 Sheet 46 of 96 US 2015/O121416A1
310 300 342 34
CON APPLICATION It
CON APP. CATION F2
CON APPLICATION A3
APPLICATION #4
330 ETHACT WE 320 As TiSEENT AREA
Patent Application Publication Apr. 30, 2015 Sheet 47 of 96 US 2015/O121416A1
FIG.47
S1801
SUB MENUS SELECTED? S1802
YES
OBTAN CATALOG LIST ON THE BASIS OF CONTENT INFORMATION
DISPLAY CATALOG LIST
S1803
S1807
S1809
YES
DISPLAY PURCHASE PAGE S1811
Patent Application Publication Apr. 30, 2015 Sheet 48 of 96 US 2015/O121416 A1
310 3OO 342 341
K. Motipi PURCHASE REVIEWBOOKMARK
MAKER: . . . IMAGE | PRICE: .
DESCRIPTION: . PURCHASE REVIEWBOOKNARK
330 Voting Wil Start in N MinuteS. G0 to 320 ADVERTSEMENT AREA
Patent Application Publication Apr. 30, 2015 Sheet 49 of 96 US 2015/0121416A1
310 300 3
SINGER: . . . |MAGE PRICE . . .
DESCRIPTION: ... ACASE REVIEWBOOKARK SINGER: :
iMAGE PRCE: , , , DESCRIPTION: , ,
ERCHASE REVIEEOKAK 330 ADVERTISEMENT AREA
Patent Application Publication Apr. 30, 2015 Sheet 50 of 96 US 2015/O121416 A1
FIG.50
310 300 342 341
PRCE: , , IMAGE ESEPTION:
PURCHASE REVIEWSOKNAR PRCE: ...
IMAGE ESRIPTION: PURCHASE REVIEWEKNARK
Patent Application Publication Apr. 30, 2015 Sheet 51 of 96 US 2015/0121416A1
FIG.51
SET ENVIRONMENT SELECT DES RED SETING MODE
YOU MAY SE ACR (AUTOMATIC CONTENT RECOGNITION) FUNCTION
ACR MODE k common App. D
Patent Application Publication Apr. 30, 2015 Sheet 52 of 96 US 2015/O121416 A1
FIG.52
S1901 SC YES
EXTRACT A CERTAIN SECTION OF MAN AN CONTENT S1902
OBTAN CONTENT INFORMATION OF MAN AN CONTENT ON THE BASIS OF EXTRACTED SECTION S1903
PERFORMSYNCHRONIZATION S1905
OBTAIN ENHANCED SERVICE ON THE BASIS OF OBTANED CONTENT INFORMATION S1907
CONFIRMACR MODE S1909
PLAY ENHANCED SERVICE ACCORONG TO ACR MODE S1911
Patent Application Publication
500
USER
S2005
Apr. 30, 2015 Sheet 53 of 96
FIG.53 ACO
TERYNAL DEVICE
100
EXECUTE INTERWORKING APP S2OOO
SEARCH FOR DEVICE S2001
\ DISPLAY SEARCHED DEVICE S2003 INPUT SELECTION FOR YOEO DISPLAY DEVICE
DISPLAY AUTHEN CATION NUMBER INPUT WINDOW INPUT AUTHENTCATION NUMBER
INPUT SELECTION FOR USER TYPE INPUT TV PASSWORD
YES REQUEST AUTHENTICATION NUMBER
S2011 AUTHENTCATION NUMBER GENERATION RESULT
S2019
S2O2
S2O23
S2O31- DISPLAY AUTHENTICATION RESULT CAPABILITY INFORMATION S2O33
REQUEST AUTHENTCATION
AUTHENT CATION RESULT
US 2015/0121416 A1
WIDEO DISPLAY DEWICE
S2009
GENERATE AUTHENTCATION NUMBER
DISPLAY AUTHENTICATION NUMBER
S2015
PERFORM AUTHENTCATION
ISSUE SESSION VALUE
S2029
PARING PROCESS
Patent Application Publication Apr. 30, 2015 Sheet 55 of 96 US 2015/O121416 A1
FIG.55
INPUT AUTHENT CATION KEY
Enter pairing key
USER TYPE FAMILY v
Enter TV paSSWOrd
NPUT AUTHENT CATION KEY
Enter pairing key
USER TYPE GUEST V
Patent Application Publication Apr. 30, 2015 Sheet 56 of 96 US 2015/O121416 A1
FIG.56
TERNAL WDEO DISPLAY DEVICE DEVICE
PARING(S2302)
EXECUTE INTERWORKING APP
ACR CAPABILITY REQUEST(S23O3) ACR CAPABILITY RESPOND(S2305)
S2307
ACR ACR SUPPORTED? CONTROL
PROCESS YES ACR SETTING STATE REQUEST (S2309) ) ACR SETTING STATE RESPONO(S2311
S233
<d ACR ON REQUEST (S2315) NO YES
RECOGNIZE CONTENT
INTERWORKING BETWEEN DEVICES(S2318)
Patent Application Publication Apr. 30, 2015 Sheet 57 of 96 US 2015/O121416 A1
FIG.57
400 1OO
VIDEO DISPLAY DEVICE
ERMINAL DEVICE
EXECUTE ACR INTERWORKING APP
PARING(S2503) ACR CONTROL (S2505)
S2507 RECOGNIZE CONTENT
REQUEST EVENT REGISTRATION(S2509)
REGISTER EVENT
CONFIRM EVENT REGISTRATION (S251)
S2512
NOTIFY EVENT (S2513) INTERWORKING BETEEN DEVICES
S2514 STORE EVENT
PERFORM OPERATION ACCORONG TO EVENT S2515
INQUIRY(S2516)
|NQUERY RESPONSE(S2517)
EVENT REGISTRATION RELEASE REQUEST (S2518)
EVENT REGISTRATION RELEASE
CONFIRM EVENT REGISTRATION RELEASE (S2521)
Patent Application Publication Apr. 30, 2015 Sheet 58 of 96 US 2015/O121416 A1
500 4OO 1OO
ENHANCED TERMINAL VIDEO DISPLAY SERVICE PROVIDING DEVICE
SERVER DEVICE
EXECUTE ACR INTERWORKING APP-S270
S2703 PARING -F
S2705 ACR CONTROL
RECOGNIZE CONTENT
CONTENT RECOGNITION STATE REQUEST S279
CONTENT RECOGNITION STATE RESPONSE--S272 ENHANCED SERVICE occur.N: EVENT REGISTRATION REQUEST S2723
ENHANCED SERVICE OCCURRENCE EVENT REGISTRATION CONFIRMATION S2725
ENHANCED SERVICE OCCURRENCE EVENT S2726
DISPLAY THAT THERE IS AVAILABLE INTER ENHANCEO SERVICE WORKING
BEINEEN RECEIVE ENHANCEO SERVICE EXECUTION INPUT DEVICES
RECEIVE ENHANCED SERVICE TYPE SELECTION INPUT
ENHANCED SERVICE TRANSFER REQUEST S2733
CONTROL ENHANCEO S2737 SERVICE
ENHANCED SERVICE REQUEST ENHANCED SERVICE TRANSFER RESPONSE S2735.
ENHANCEO SERVICE S2739 APP
INTER- EXECUTE ENHANCED SERVICE-S274 WORKING ENHANCED SERVICE EVENT S2743
ENHANCED SERVICE EVENT RESPONSE S2745
Patent Application Publication Apr. 30, 2015 Sheet 60 of 96 US 2015/O121416 A1
FIG.60
THERE IS INFORMATION REGARDING PROGRAM N WATCHING
WANT TO WIEW2
Patent Application Publication Apr. 30, 2015 Sheet 61 of 96 US 2015/O121416 A1
FIG.61
THERE IS INFORMATION REGARDING PROGRAM N WATCHING
WHICH DEVICE YOU WANT TO SELECT?
Patent Application Publication Apr. 30, 2015 Sheet 62 of 96 US 2015/O121416 A1
FIG.62
s SEARCH RESULT (Oi OjdKf j| / Interesting She's beautiful
(Wpeop / GOOd
(Smeil / Very nice
Patent Application Publication Apr. 30, 2015 Sheet 63 of 96 US 2015/O121416 A1
Giojdkfj| / Interesting. She's beautiful
GWOeop / GOOd
(Smeil / Very nice
HMMMMMMMM. . . . . . . . . . . . . . .
ow ER TOY Cui o P) As D F GH J K Cl
Shifz (xc) v B NMD de D C. 123 C return
Patent Application Publication Apr. 30, 2015 Sheet 65 of 96 US 2015/O121416 A1
FIG.65
SONG 1
SONG 2
SONG 3
SONG 4
Patent Application Publication Apr. 30, 2015 Sheet 66 of 96 US 2015/O121416 A1
FIG.66
400 1OO
TERMINAL DEVICE
EXECUTE ENHANCED SERVICE
RECEIVE USER INPUT FOR ENHANCED SERVICE TERMINATION
CONTROLLER
CONTROL FUNCTION TRANSFER RELEASE REQUEST (S2751)
RELEASE CONTROL FUNCT ON TRANSFER
CONTROL FUNCTION TRANSFER RELEASE RESPONSE(S2755)
MAGE DISPLAY DEVICE
ENHANCED SERVICE TRANSFER RELEASE REQUEST (S2757)
RELEASE ENHANCED SERVICE TRANSFER
ENHANCED SERVICE TRANSFER RELEASE RESPONSE(S2761)
TERMINATE ENHANCED SERVICE
Patent Application Publication Apr. 30, 2015 Sheet 67 of 96 US 2015/O121416 A1
FIG.67 400 1OO
WCEO DISPLAY DEVICE
PARNG
STORE WIDEO DISPLAY STORE TERMINAL DEVICE INFORMATION DEVICE INFORMATION
PARNG RELEASE
TERMINAL DEVICE
S2805
S2807
DISPLAY AWALABLE ENHANCED SERVICE
RECEIVE INPUT FOR ENHANCED SERVICE
RECEIVE INPUT FOR ENHANCED SERVICE TYPE SELECTION
DISPLAY AWALABLE TERMINAL DEVICE LIST
RECEIVE INPUT FOR TERMINAL DEVICE SELECTION
ENHANCED SERVICE EXECUTION REQUEST (S2819)
EXECUTE ENHANCED SERVICE
PARING
ENHANCED SERVICE EXECUTION RESPONSE(S2825) S2823
CONTROL ENHANCED SERVICE
Patent Application Publication Apr. 30, 2015 Sheet 68 of 96 US 2015/O121416 A1
410 400 1OO
TERNINAL ERMINAL WDEO DISPLAY DEVICE DEVICE DEVICE
ENHANCED SERVICE REQUEST (CONTROLLER) S290
S2903 CONTROL ENHANCED SERVICE
ENHANCED SERVICE RESPONSE (OK) S2905 S2911 CONTROL FUNCTION TRANSFER CONFLCT EVENT REGISTRATION REQUEST S290/
h CONTROL FUNCTION TRANSFER CONFLC EVENT REGISTRATION CONFIRMATION S2909 ENHANCED SERVICE REQUEST (CONTROLLER) ENHANCED SERVICE RESPONSE (CONFLCT)
CONTROL FUNCTION TRANSFER CONFLCT EVENT -S2915
RECEIVE USER INPUT FOR ENHANCED SERVICE TERMINATION
CONTROL FUNCTION TRANSFER RELEASE REQUEST--S2019
CONTROL FUNCT ON TRANSFER REGISTRATION RELEASE
s CONTROL FUNCTION TRANSFER RELEASE RESPONSE-S2923 CONTROL FUNCTION TRANSFER AWALABILITY NOTIFICATION
ENHANCED SERVICE REQUEST (CONTROLLER)
| 292 S2927 S2929
ENHANCED SERVICE RESPONSE (OK) --S2931
CONTROL ENHANCED SERVICE
Patent Application Publication Apr. 30, 2015 Sheet 69 of 96 US 2015/O121416 A1
FIG.69 400 100
TERMINAL VIDEO DISPLAY DEVICE DEVICE
RECEIVE USER
ENHANCED SERVICE DISPLAY CONTROL REQUEST (S3102)
INPUT
CONTROL ENHANCED SERVICE DISPLAY
ENHANCED SERVICE DISPLAY CONTROL RESPONSE(S3105)
Patent Application Publication Apr. 30, 2015 Sheet 70 of 96 US 2015/O121416 A1
FIG.70
400 100
TERMINAL VIDEO DISPLAY DEVICE DEVICE
RECEIVE USER INPUT
MEDIA PLAYBACK CONTROL REQUEST (S3302)
CONTROL MEDIA PLAYBACK
MEDIA PLAYBACK CONTROL RESPONSE(S3305)
Patent Application Publication Apr. 30, 2015 Sheet 71 of 96 US 2015/0121416A1
FIG.71 400 100
TERMINAL VIDEO DISPLAY DEVICE DEVICE
RECEIVE USER INPUT
MEDIA PLAYBACK CONTROL REQUEST (S3502)
CONTROL MEDIA PLAYBACK
MEDIA PLAYBACK CONTROL RESPONSE(S3505)
Patent Application Publication Apr. 30, 2015 Sheet 72 of 96 US 2015/O121416 A1
FIG.72 400 100
WDEO DISPLAY DEVICE
TERMINAL DEVICE
EXECUTE ACR INTERWORKING APP
PARNG
ACR CONTROL
COMMON APPEXECUTION REQUEST
EXECUTE COMMON APP
S3711
S3703
S3705
S3707
COMMON APPEXECUTION RESPONSE INTERWORKING
COMMON APP TRANSFER REQUEST S3713 BETWEEN DEVICES
CONTROL COMMON APP
COMMON APP TRANSFER RESPONSE S3717
EXECUTE COMMON APP-S3719 COMMON APP TRANSFER RELEASE REQUEST S3721
S3723 CONTROL COMNON APP
COMMON APP TRANSFER RELEASE RESPONSE S3725
TERMINATE COMMON APP-S3727 COMMON APP TERMINATION REQUEST
S3731- TERMINATE COMMON APP COMMON APP TERMINATION RESPONSE S3733
Patent Application Publication Apr. 30, 2015 Sheet 73 of 96 US 2015/O121416 A1
VOL/CH - I Touchpad Button I ACR
Patent Application Publication Apr. 30, 2015 Sheet 74 of 96 US 2015/O121416 A1
FIG.74
ACR ADO Cat iOn ACR History
ACRONO Common App v. Ge (c) (6) (G) (e) (IVE AA
Patent Application Publication Apr. 30, 2015 Sheet 75 of 96 US 2015/O121416 A1
FIG.75
ACR AOO iCat iOn ACR Histor ACR))) Top Chef 2000:08:21 http:llac?.app. CO.K. http:liac?. Seccondapp, CO.Kf MACR Status MACR Event
Debug MOce
N WO/CH
CK (1) EB 11:09
Patent Application Publication Apr. 30, 2015 Sheet 76 of 96 US 2015/O121416 A1
FIG.76
ACR Application ACR HiSt Or
G (e) Debug Wode
Top Chef
Matchlist Matchlist v Check in
VOL/CH CK (1N BBE 11:09
Patent Application Publication Apr. 30, 2015 Sheet 77 of 96 US 2015/O121416A1
FIG.77
ACR ADO Cat iOn ACR HiSt Or G (6) O Debug Mode
JE / ) -2 \
F. VOL/CH
KK (4) EEB 11:09
Patent Application Publication Apr. 30, 2015 Sheet 78 of 96 US 2015/O121416 A1
FIG.78
ACR AOO iCat iOn a ACR HiSt Or G) (6) (C) TV O Debug Mode Guest001 : Nice Me : Interesting, KKK. Interesting, KKK.
Patent Application Publication Apr. 30, 2015 Sheet 79 of 96 US 2015/O121416 A1
FIG.79
ACR ADO Cat iOn ACR HiStor G) (6) Debug Mode
DOWNLOAD-SORTED BY DATE
(QTODAY O TEST DOWNLOAD
(2) LAST 7 DAYS (OLAST MONTH
SORTED BY SZE
CK 1) EB 11:09
Patent Application Publication Apr. 30, 2015 Sheet 80 of 96 US 2015/O121416 A1
FIG.80
ACR ADO iCat iOn . . . . . . ACR H Stor ACR Application HiStory ()
v. Application Title Content Title Launched time http://appur I. Com(C)(6) O Application Title Content Title Launched time http://appur .COM (D)(6) OApplication Title Content Title Launched time http://appur.com(C)(6)
Patent Application Publication Apr. 30, 2015 Sheet 81 of 96 US 2015/0121416A1
FIG.81
ACR AOD. iCat iOn ACR HiSt Or
WOL 11:09
Patent Application Publication Apr. 30, 2015 Sheet 82 of 96 US 2015/O121416 A1
FIG.82
ACR ADO Cat iOn ACR HiSt Or AOR NOT FOUND TV APPLICATION 2ND DEVICE APPLICATION MACR Status M Common App MACR Event http:// G) (e) TV M Debug Mode
Patent Application Publication Apr. 30, 2015 Sheet 83 of 96 US 2015/O121416 A1
FIG.83
RECEIVE SERVICE PROVIDER DSCOVERY INFORMATION THROUGH P BASED NETWORK
ACCESS SERVICE PROVIDER
RECE WE APPLICATION INFORMATION TABLE FROM PROVIDER
S3901
S3903
S3905
S3907
Patent Application Publication Apr. 30, 2015 Sheet 86 of 96 US 2015/O121416 A1
Application
O
ApplicationLOCat iOn
Patent Application Publication Apr. 30, 2015 Sheet 87 of 96 US 2015/O121416 A1
FIG.87
RECEIVE PROGRAM MAP TABLE
CONFIRM PACKET DENT FER OF TRANSPORT STREAM PACKET CARRYING APPLICATION
NFORMATION TABLE
S4101
S4103
RECEIVE TRANSPORT STREAM PACKET CARRYING APPLICATION INFORMATION TABLE ON THE BASS
OF CONFIRMED PACKET DENT FER
EXTRACT APPLICATION INFORMATION TABLE FROM RECEIVED TRANSPORT STREAM PACKET
S405
S4107
Patent Application Publication Apr. 30, 2015 Sheet 88 of 96 US 2015/O121416A1
FIG.88 400 100
WIDEO DISPLA DEVICE
TERMINAL DEVICE
EXECUTE INTERWORKING APP
PARNG
ACR CAPABILITY REQUEST
ACR CAPABILITY RESPONSE
S4303
S4305
S4307
S4309 ACR SUPPORTED?
NO
TURN ON ACR S4310
OBTAN PART OF MAN AW CONTENT S4311
CONTENT RECOGNITION
EXECUTE ENHANCED SERVICE
INTERWORKING OF ENHANCED SERVICE
INTERWORKING BETEEN DEVICES S4319
Patent Application Publication Apr. 30, 2015 Sheet 90 of 96 US 2015/0121416A1
FIG.90
CURRENTLY WATCHING PROGRAM S "INTERESTING DRAMA 01"
CONFIRM
Patent Application Publication Apr. 30, 2015 Sheet 91 of 96 US 2015/O121416 A1
FIG.91 400 100
WDEO DISPLAY DEVICE
TERMINAL DEVICE
DISPLAY U TO INQUIRE OF USER WHETHER TO PROVIDE ENHANCED SERVICE TO WIDEO DISPLAY DEVICE
ENHANCED SERVICE S4321
EXECUTE ENHANCED SERVICE
ENHANCED SERVICE EXECUTION RESPONSE S4325
ENHANCED SERVICE INTERWORKING
CONTROL ENHANCED SERVICE S4329 S4327
Patent Application Publication Apr. 30, 2015 Sheet 92 of 96 US 2015/O121416 A1
FIG.92
WAN TO WATCH WIDEO RELATED TO CURRENTLY WATCHING
"INTERESTING DRAMA 01" ON TV?
CONFIRM
Patent Application Publication Apr. 30, 2015 Sheet 93 of 96 US 2015/O121416 A1
DISPLAY ENHANCED INFORMATIO RELATED TO CURRENTLY
WATCHING CONTENT
Patent Application Publication Apr. 30, 2015 Sheet 96 of 96 US 2015/O121416 A1
FIG.96 400
490 POWER SUPPLY UNIT
WRELESS COMMUNICATION UNIT
A5 DISPLAY UNT
SOUND OUTPUTTING MODULE
HAPTIC UNIT
452
453
454 414 CONTROL UNIT OCA AREA
COMMUNICATIONMODULE
POSITION INFORMATION MODULE
415
420 48O
421 CAMERA
422 MIKE
USER INPUT UNIT
SENSING UNIT
48
MEMORY
470
430 MULTIMEDIA MODULE
440 460
INTERFACE UNIT
US 2015/O121416 A1
VIDEO DISPLAY DEVICE,TERMINAL DEVICE, AND METHOD THEREOF
CROSS-REFERENCE TO RELATED APPLICATIONS
0001. The present application is a continuation applica tion of U.S. application Ser. No. 13/567,689 filed Aug. 6, 2012 and claims benefit and priority under 35 U.S.C. 119 and 35 U.S.C. 365 to U.S. Patent Applications No. 61/525,835 (filed on Aug. 21, 2011), No. 61/543.288 (filed on Oct. 4, 2011), No. 61/549,236 (filed on Oct. 20, 2011), and Korean Patent Application No. 10-2012-0063969 (filed on Jun. 14, 2012), which are hereby incorporated by reference in its entirety.
BACKGROUND
0002 The present invention relates to a video display device, a terminal device, and method thereof. 0003. As digital broadcasting is paving the way for its extensive spread, a broadcasting station transmits both main audio-visual (AV) contents and enhanced service data, which may be linked to the main AV contents to provide information and services or may be separately provided for other pur poses. 0004. However, a video display device in each home may unlikely receive broadcast signals directly through air under a current broadcasting environment. Rather, most of cases, a Video display device in each home is connected to a broadcast receiving device to display uncompressed audio-visual con tents that the broadcast receiving device provides. 0005 Additionally, the broadcast receiving device receives contents from a server (called a Multichannel Video Programming Distributor (MVPD)). The MVPD receives a broadcast signal from a broadcasting station, extracts con tents from the received broadcast signal, converts the extracted content into signals having a proper format for transmission, and provides the converted signals to a broad cast receiving device. During these processes, the MVPD may exclude extracted enhanced service data or may add another enhanced service data, so that the broadcasting sta tion may serve to provide AV contents but may not provide enhanced services led by the broadcasting station. 0006 Since the broadcast receiving device extracts main AV data from signal received from the MVPD and provides only uncompressed audio-visual data to a video display device, only enhanced services provided from the broadcast receiving device not the video display device are available. Furthermore, the broadcasting station may not provide enhanced services led by itself. 0007 Additionally, even when the broadcasting station provides enhanced services through wireless channels, the enhanced services may not be provided enough due to band width limitation. 0008. Additionally, when an enhanced service led by a broadcasting station is provided, it is displayed on a portion of a display unit of a video display device. Therefore, it may cause inconvenience for a user to watch contents.
SUMMARY
0009 Embodiments provide an image display, a terminal device, and an operating method thereof, which provide an available enhanced service led by abroadcasting station with out causing inconvenience for a user to watch contents.
Apr. 30, 2015
0010. In one embodiment, an operating method of a video display apparatus comprises pairing with a first terminal device; receiving an audio-visual content; displaying the audio-visual content; acquiring enhanced service informa tion on an available enhanced service for the audio-visual content; and notifying an occurrence of the available enhanced service to the first terminal device.
0011. In further embodiment, a video display apparatus, comprises a display unit; a receiver for receiving an audio visual content; an enhanced service manager for acquiring enhanced service information on an available enhanced ser Vice for the audio-visual content; and a reproducing control ler for reproducing the audio-visual content through the dis play unit, wherein the enhanced service manager pairs with a first terminal device, and notifies an occurrence of the avail able enhanced service to the first terminal device.
0012. In further another embodiment, an operating method of a terminal device comprises pairing with a video display apparatus; receiving, from the video display appara tus, an occurrence of the available enhanced service for an audio-visual content reproduced by the video display appa ratus; and executing the available enhanced service. 0013. In further another embodiment, a terminal device comprises a display unit; a communication unit for commu nicating with a video display apparatus; controller for pairing with a video display apparatus through the communication unit, for receiving, from the video display apparatus, an occurrence of the available enhanced service for an audio visual content reproduced by the video display apparatus, and for displaying the available enhanced service on the display unit
0014. In further another embodiment, an operating method of a video display apparatus comprises receiving an audio-visual content; displaying the audio-visual content; acquiring enhanced service information on an available enhanced service for the audio-visual content; and requesting an execution of the available enhanced service to a selected terminal device, if a terminal device is selected and executing of the available enhanced service is selected.
0015 The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
0016 FIG. 1 is a block diagram illustrating the network topology according to the embodiment. 0017 FIG. 2 is a block diagram illustrating a watermark based network topology according to an embodiment. 0018 FIG. 3 is a ladder diagram illustrating a data flow in a watermark based network topology according to an embodiment.
0019 FIG. 4 is view illustrating a watermark based con tent recognition timing according to an embodiment. 0020 FIG. 5 is a block diagram illustrating a fingerprint based network topology according to an embodiment. 0021 FIG. 6 is a ladder diagram illustrating a data flow in a fingerprint based network topology according to an embodi ment.
0022 FIG. 7 is a view illustrating an XML schema dia gram of ACR-Result type containing a query result according to an embodiment.
US 2015/O121416 A1
0023 FIG. 8 is a block diagram illustrating a watermark and fingerprint based network topology according to an embodiment. 0024 FIG. 9 is a ladder diagram illustrating a data flow in a watermark and fingerprint based network topology accord ing to an embodiment. 0025 FIG. 10 is a block diagram illustrating the video display device according to the embodiment. 0026 FIG. 11 is a hierarchy diagram of an enhanced ser Vice management unit according to an embodiment. 0027 FIG. 12 is a flowchart illustrating an operation of an enhanced service management unit according to an embodi ment.
0028 FIG. 13 is a hierarchy diagram of an enhanced ser Vice management unit according to an embodiment. 0029 FIG. 14 is a flowchart illustrating an operation of an enhanced service management unit according to an embodi ment.
0030 FIG. 15 is a hierarchy diagram of an enhanced ser Vice management unit according to an embodiment. 0031 FIG. 16 is a flowchart illustrating an operation of an enhanced service management unit according to an embodi ment.
0032 FIG. 17 is a hierarchy diagram of an enhanced ser Vice management unit according to an embodiment. 0033 FIG. 18 is a flowchart illustrating an operation of an enhanced service management unit according to an embodi ment.
0034 FIG. 19 is a flowchart illustrating a method of syn chronizing a playback time of a main AV content with a playback time of an enhanced service according to an embodiment. 0035 FIG. 20 is a conceptual diagram illustrating a method of synchronizing a playback time of a main AV con tent with a playback time of an enhanced service according to an embodiment. 0036 FIG.21 is a screen when an auto playing application
is executed according to an embodiment. 0037 FIG. 22 is a screen when a content provider appli cation is executed according to an embodiment. 0038 FIG. 23 is a screen when a video display device manufacturer application having a resizing user interface is executed according to an embodiment. 0039 FIG. 24 is a screen when a video display device manufacturer application having an overlay user interface is executed according to an embodiment. 0040 FIG. 25 is a flowchart illustrating a method of executing a video display device manufacturer application according to an embodiment. 0041 FIG. 26 is a screen when an application selection menu is displayed according to an embodiment. 0.042 FIG. 27 is a screen when an all sub menu of a program main menu in a video display device manufacturer application is selected according to an embodiment. 0043 FIG.28 is a screen when a participation sub menu of a program main menu in a video display device manufacturer application is selected according to an embodiment. 0044 FIG. 29 is a screen when a discovery sub menu of a program main menu in a video display device manufacturer application is selected according to an embodiment. 0045 FIG. 30 is flowchart illustrating a searching method according to an embodiment.
Apr. 30, 2015
0046 FIG. 31 is a screen when a search sub menu of an information main menu in a video display device manufac turer application is selected according to an embodiment. 0047 FIG.32 is another screen when a search sub menu of an information main menu in a video display device manu facturer application is selected according to an embodiment. 0048 FIG. 33 is flowchart illustrating a chatting window displaying method according to an embodiment. 0049 FIG. 34 is a screen when a Twitter sub menu of an information main menu in a video display device manufac turer application is selected according to an embodiment. 0050 FIG. 35 is a flowchart illustrating a news obtaining method according to an embodiment. 0051 FIG. 36 is a screen when a news sub menu of an information main menu in a video display device manufac turer application is selected according to an embodiment. 0052 FIG. 37 is flowchart illustrating a viewer list dis playing method according to an embodiment. 0053 FIG.38 is a screen when a viewer list sub menu of a Social main menu in a video display device manufacturer application is selected according to an embodiment. 0054 FIG. 39 is a signal flow when a video display device obtains a friends list and watching information according to an embodiment. 0055 FIG. 40 is a signal flow when a video display device obtains a friends list and watching information according to another embodiment. 0056 FIG. 41 is flowchart illustrating a preference dis playing method according to an embodiment. 0057 FIG. 42 is a screen when a preference display sub menu of a social main menu in a video display device manu facturer application is selected according to an embodiment. 0.058 FIG. 43 is flowchart illustrating a digital media downloading method according to an embodiment. 0059 FIG. 44 is a screen when a previous episode sub menu of a recommendation main menu in a video display device manufacturer application is selected according to an embodiment.
0060 FIG. 45 is a screen when a VOD sub menu of a recommendation main menu in a video display device manu facturer application is selected according to an embodiment. 0061 FIG. 46 is a screen when an application sub menu of a recommendation main menu in a video display device manufacturer application is selected according to an embodi ment.
0062 FIG. 47 is flowchart illustrating a catalog displaying method according to an embodiment. 0063 FIG. 48 is a screen when a style sub menu of a recommendation main menu in a video display device manu facturer application is selected according to an embodiment. 0064 FIG. 49 is a screen when a music sub menu of a recommendation main menu in a video display device manu facturer application is selected according to an embodiment. 0065 FIG. 50 is a screen when another sub menu of a recommendation main menu in a video display device manu facturer application is selected according to an embodiment. 0.066 FIG. 51 is a view of a user interface for controlling an ACR function according to an embodiment. 0067 FIG. 52 is a flowchart illustrating a method of con trolling a video display device according to the embodiment. 0068 FIG. 53 is a ladder diagram illustrating a pairing procedure according to an embodiment.
US 2015/O121416 A1
0069 FIG. 54 is a front view of a terminal having an interworking application executed according to an embodi ment.
0070 FIG.55 is an authentication number input window according to an embodiment. 0071 FIG. 56 is a ladder diagram illustrating an ACR control method according to an embodiment. 0072 FIG. 57 is a ladder diagram illustrating an inter working process between devices according to an embodi ment.
0073 FIG. 58 is a ladder diagram illustrating an inter working process between devices according to another embodiment. 0074 FIG. 59 is a front view of a terminal device display ing an enhanced service notification UI according to an embodiment. 0075 FIG. 60 is a front view of a terminal device display ing an enhanced service notification UI according to another embodiment. 0076 FIG. 61 is a front view of a terminal device display ing an enhanced service notification UI according to another embodiment. 0077 FIG. 62 is a front view of a terminal device where an enhanced service is executed according to an embodiment. 0078 FIG. 63 is a view when an enhanced service is executed in a video display device and a terminal device according to another embodiment. 007.9 FIG. 64 is a front view of a terminal device where an enhanced service is executed according to another embodi ment.
0080 FIG.65 is a front view of a terminal device where an enhanced service is executed according to another embodi ment.
0081 FIG. 66 is a ladder diagram illustrating an inter working process between devices according to another embodiment. 0082 FIG. 67 is a ladder diagram illustrating an inter working process between devices according to another embodiment. 0083 FIG. 68 is a ladder diagram illustrating an inter working process between devices according to another embodiment. 0084 FIG. 69 is a ladder diagram illustrating an inter working process between devices according to another embodiment. 0085 FIG. 70 is a ladder diagram illustrating an inter working process between devices according to another embodiment. I0086 FIG. 71 is a ladder diagram illustrating an inter working process between devices according to another embodiment. 0087 FIG. 72 is a ladder diagram illustrating an inter working process between devices according to another embodiment.
0088 FIG.73 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0089 FIG.74 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0090 FIG.75 is a front view of a terminal device where an interworking application is executed according to another embodiment.
Apr. 30, 2015
0091 FIG. 76 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0092 FIG. 77 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0093 FIG. 78 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0094 FIG.79 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0095 FIG.80 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0096 FIG. 81 is a front view of a terminal device where an interworking application is executed according to another embodiment.
(0097 FIG. 82 is a top front view of a display unit in a terminal device where an interworking application is executed according to another embodiment. (0098 FIG. 83 is a flowchart illustrating a method of obtaining an AIT through an IP based network according to an embodiment.
(0099 FIG. 84 is view illustrating a structure of SD&S information according to an embodiment. 0100 FIG. 85 is view illustrating a structure of SD&S information according to another embodiment. 0101 FIG. 86 is view illustrating a structure of SD&S information according to an embodiment. 0102 FIG. 87 is a flowchart illustrating a method of obtaining information on an available enhanced service through a broadcast network according to an embodiment. 0103 FIG. 88 is a ladder diagram illustrating a method of providing an enhanced service according to another embodi ment.
0104 FIG. 89 is a front view of a terminal device where an interworking application is executed according to another embodiment.
0105 FIG. 90 is a front view of a terminal device execut ing the service for showing content information according to another embodiment.
0106 FIG. 91 is a ladder diagram illustrating an inter working method between devices according to another embodiment.
0107 FIG. 92 is a front view of a terminal device display ing UI that inquires of a user to determine whether to provide an enhanced service to a video display device according to an embodiment.
(0.108 FIG. 93 is a front view of a video display device receiving an enhanced service from a terminal device and executing it. 0109 FIG.94 is a block diagram illustrating a structure of a fingerprint based video display device according to another embodiment.
0110 FIG.95 is a block diagram illustrating a structure of a watermark based video display device according to another embodiment.
0111 FIG. 96 is a block diagram of a terminal device according to an embodiment.
Top Related