| HAO HATA KAMA MARIA DE LA CANTATA TA TTI

| HAO HATA KAMA MARIA DE LA CANTATA TA TTI US009807442B2

( 12 ) United States Patent Bhatia et al .

( 10 ) Patent No . : US 9 , 807 , 442 B2 ( 45 ) Date of Patent : * Oct . 31 , 2017

( 58 ) ( 54 ) MEDIA CONTENT SYNCHRONIZED ADVERTISING PLATFORM APPARATUSES AND SYSTEMS

Field of Classification Search CPC . . . . . HO4H 20 / 14 ; H04H 60 / 33 ; H04N 21 / 435 ;

H04N 7 / 17318 See application file for complete search history .

References Cited ( 71 ) Applicant : Symphony Advanced Media , Inc . , San

Francisco , CA ( US ) ( 56 ) U . S . PATENT DOCUMENTS ( 72 ) Inventors : Manish Bhatia , Cranbury , NJ ( US ) ;

Michael Scott Saxon , Westfield , NJ ( US ) ; Nigel Stephen Pratt , Los Altos , CA ( US ) ; Pyeush Gurha , Mountain View , CA ( US )

4 , 697 , 209 A 5 , 621 , 454 A

9 / 1987 Kiewit et al . 4 / 1997 Ellis et al .

( Continued )

FOREIGN PATENT DOCUMENTS ( 73 ) Assignee : Symphony Advanced Media , Inc . , San

Francisco , CA ( US ) EP 0412335 2 / 1991 2093706 8 / 2009

( Continued ) @ ( * ) Notice : Subject to any disclaimer , the term of this patent is extended or adjusted under 35 U . S . C . 154 ( b ) by 0 days . This patent is subject to a terminal dis claimer .

OTHER PUBLICATIONS

( 21 ) Appl . No . : 15 / 222 , 116 ( 22 ) Filed : Jul . 28 , 2016

“ ABC app eavesdrops on your TV to synchronize interactive content using Nielsen tech ” Sep . 18 , 2010 ( http : / / www . engadget . com 2010 / 09 / 18 / abc - app - eavesdrops - on - your - tv - to - synchronize interactive - content / ) .

( Continued )

( 65 ) Prior Publication Data US 2016 / 0337709 A1 Nov . 17 , 2016

Primary Examiner — Sumaiya A Chowdhury ( 74 ) Attorney , Agent , or Firm — Hanchuk Kheit LLP ; Walter G . Hanchuk

( 63 ) Related U . S . Application Data

Continuation of application No . 14 / 621 , 346 , filed on Feb . 12 , 2015 , now Pat . No . 9 , 432 , 713 , which is a

( Continued ) ( 51 ) Int . Cl .

HO4N 21 / 2668 ( 2011 . 01 ) H04H 20 / 14 ( 2008 . 01 )

( Continued ) ( 52 ) U . S . CI .

CPC . . . . . . . . . . H04N 21 / 2668 ( 2013 . 01 ) ; G06Q 30 / 02 ( 2013 . 01 ) ; G06Q 30 / 0201 ( 2013 . 01 ) ;

( Continued )

( 57 ) ABSTRACT The MEDIA CONTENT BASED ADVERTISING SUR VEY PLATFORM APPARATUSES AND SYSTEMS ( “ AD - SURVEY ” ) transforms user advertisement exposure data via AD - SURVEY components , into ad effects data including user responses to survey questions . A system is disclosed , comprising : a memory ; a processor disposed in communication with said memory , and configured to issue a plurality of processing instructions stored in the memory , wherein the processor issues instructions for obtaining TV program schedule listing data including a plurality of ad tags at a server ; providing the obtained TV program schedule listing data to a user mobile device , receiving a user media program selection message from the user mobile device ;

( Continued )

TV Network 270 RT TV

Program 2 37a

Audience Stat Report 245

/ RT TV Program 2375 7 STB 2012 RT TV

Program 2 37€

1 .

2 wwwwww * + - Ten TV Event 235a O WL TV Event

2350 4 AD SURVEY Server 220

RT Data Record Sleer 2 33 Client TV 2 33 TV Event 2355 Viewing

Status 2752

Friends Recom 277 Viewing

Status Database ZH

2751

Social Media 250 Communication Natwork 2 13

AD - SURVEY Example Data Flows : TV Remote Monitoring

US 9 , 807 , 442 B2

retrieving an ad tag associated with the user selected media program from the TV program schedule listing data ; extract ing key terms from the ad tags based by parsing ad contents ; querying a survey question list based on the extracted key terms ; generating and sending a survey question from the query to the user mobile device ; and obtaining a user reaction to the survey question . luulwa 15 Claims , 65 Drawing Sheets

Related U . S . Application Data continuation of application No . 13 / 341 , 076 , filed on Dec . 30 , 2011 , now Pat . No . 8 , 978 , 086 .

( 60 ) Provisional application No . 61 / 504 , 913 , filed on Jul . 6 , 2011 .

7 , 222 , 071 B2 5 / 2007 Neuhauser 7 , 316 , 025 B1 1 / 2008 Aijala 7 , 349 , 827 B1 3 / 2008 Heller 7 , 363 , 243 B2 4 / 2008 Arnett 7 , 408 , 460 B2 8 / 2008 Crystal 7 , 471 , 987 B2 12 / 2008 Crystal 7 , 483 , 835 B2 1 / 2009 Neuhauser 7 , 487 , 435 B2 2 / 2009 Aviv 7 , 523 , 085 B2 4 / 2009 Nigam 7 , 584 , 484 B2 9 / 2009 Headley 7 , 587 , 728 B2 9 / 2009 Wheeler 7 , 587 , 732 B2 9 / 2009 Wright 7 , 590 , 568 B2 9 / 2009 Blumenau 7 , 607 , 147 B1 10 / 2009 Lu 7 , 609 , 853 B2 10 / 2009 Ramaswamy 7 , 613 , 635 B2 11 / 2009 Blumenau 7 , 623 , 823 B2 . 11 / 2009 Zito 7 , 627 , 872 B2 12 / 2009 Hebeler 7 , 640 , 141 B2 12 / 2009 Kolessar 7 , 644 , 156 B2 1 / 2010 Blumenau 7 , 647 , 604 B2 1 / 2010 Ramaswamy 7 , 650 , 407 B2 1 / 2010 Blumenau 7 , 650 , 616 B2 1 / 2010 Lee 7 , 650 , 793 B2 1 / 2010 Jensen 7 , 653 , 724 B2 1 / 2010 Blumenau 7 , 712 , 114 B2 5 / 2010 Ramaswamy 7 , 716 , 326 B2 5 / 2010 Blumenau 7 , 720 , 963 B2 5 / 2010 Blumenau 7 , 720 , 964 B2 5 / 2010 Blumenau 7 , 739 , 705 B2 6 / 2010 Lee 7 , 752 , 042 B2 7 / 2010 Srinivasan 7 , 756 , 974 B2 7 / 2010 Blumenau 7 , 757 , 248 B2 7 / 2010 Harkness 7 , 793 , 316 B2 9 / 2010 Mears 7 , 793 , 318 B2 9 / 2010 Deng 7 , 818 , 769 B2 10 / 2010 Peacock et al . 7 , 844 , 484 B2 11 / 2010 Arnett 7 , 844 , 488 B2 11 / 2010 Merriman 7 , 873 , 988 B1 1 / 2011 Issa et al . 7 , 876 , 997 B2 1 / 2011 Yabuta 7 , 904 , 069 B2 3 / 2011 Rye et al . 7 , 992 , 171 B2 8 / 2011 Amidon et al . 8 , 006 , 259 B2 8 / 2011 Drake et al . 8 , 060 , 390 B1 11 / 2011 Overstreet 8 , 090 , 822 B2 1 / 2012 Lee 8 , 176 , 057 B2 5 / 2012 Kim et al . 8 , 196 , 162 B2 6 / 2012 Van de Klashorst 8 , 296 , 380 B1 10 / 2012 Kelly et al . 8 , 320 , 746 B2 11 / 2012 Barrett et al .

2001 / 0052133 Al 12 / 2001 Pack et al . 2002 / 0019768 AL 2 / 2002 Fredrickson et al . 2002 / 0107726 A1 8 / 2002 Torrance et al . 2002 / 0174425 A1 11 / 2002 Markel et al . 2003 / 0033197 Al 2 / 2003 Saga et al . 2003 / 0033600 A1 2 / 2003 Cliff et al . 2003 / 0040964 Al 2 / 2003 Lacek 2004 / 0078809 A1 4 / 2004 Drazin 2004 / 0205810 A1 10 / 2004 Matheny et al . 2004 / 0220858 AL 11 / 2004 Maggio 2004 / 0230989 AL 11 / 2004 Macey et al . 2004 / 0237114 AL 11 / 2004 Drazin 2005 / 0038814 AL 2 / 2005 Lyengar 2005 / 0144632 A1 * 6 / 2005 Mears . . . . . . . . . . . . . . . . . .

( 51 ) Int . Cl . H04H 60 / 33 ( 2008 . 01 ) H04N 21 / 435 ( 2011 . 01 ) H04N 21 / 258 ( 2011 . 01 ) H04N 21 / 422 ( 2011 . 01 ) H04N 21 / 442 ( 2011 . 01 ) H04N 21 / 437 ( 2011 . 01 ) H04N 21 / 482 ( 2011 . 01 ) G060 30 / 02 ( 2012 . 01 ) H04N 21 / 25 ( 2011 . 01 ) H04N 21 / 475 ( 2011 . 01 ) H04N 21 / 81 ( 2011 . 01 ) H04N 21 / 41 ( 2011 . 01 ) H04N 21 / 478 ( 2011 . 01 ) H04N 21 / 8545 ( 2011 . 01 ) H04N 21 / 858 ( 2011 . 01 )

( 52 ) U . S . CI . CPC . . . . . GO6Q 30 / 0241 ( 2013 . 01 ) ; G06Q 30 / 0267

( 2013 . 01 ) ; H04N 21 / 251 ( 2013 . 01 ) ; H04N 21 / 252 ( 2013 . 01 ) ; H04N 21 / 25808 ( 2013 . 01 ) ; H04N 21 / 25891 ( 2013 . 01 ) ; H04N 21 / 4126

( 2013 . 01 ) ; H04N 21 / 4222 ( 2013 . 01 ) ; H04N 21 / 42203 ( 2013 . 01 ) ; H04N 21 / 42221

( 2013 . 01 ) ; H04N 21 / 437 ( 2013 . 01 ) ; H04N 21 / 44204 ( 2013 . 01 ) ; HO4N 21 / 44218

( 2013 . 01 ) ; H04N 21 / 44222 ( 2013 . 01 ) ; H04N 21 / 4758 ( 2013 . 01 ) ; H04N 21 / 47815 ( 2013 . 01 ) ;

H04N 21 / 482 ( 2013 . 01 ) ; H04N 21 / 4821 ( 2013 . 01 ) ; H04N 21 / 812 ( 2013 . 01 ) ; H04N

21 / 8545 ( 2013 . 01 ) ; H04N 21 / 8586 ( 2013 . 01 )

NNN

( 56 ) References Cited H04H 60 / 31 725 / 15 U . S . PATENT DOCUMENTS

6 , 130 , 726 A 6 , 370 , 543 B2 6 , 567 , 984 B1 6 , 615 , 408 B1 6 , 637 , 029 B1 6 , 694 , 316 B1 6 , 862 , 741 B1 7 , 039 , 599 B2 7 , 085 , 682 B1 7 , 181 , 412 B1 7 , 185 , 065 B1 7 , 188 , 078 B2 7 , 188 , 079 B2 7 , 197 , 470 B1

10 / 2000 Darbee et al . 4 / 2002 Hoffert 5 / 2003 Allport 9 / 2003 Kaiser et al .

10 / 2003 Maissel et al . 2 / 2004 Langseth 3 / 2005 Grooters 5 / 2006 Merriman 8 / 2006 Heller 2 / 2007 Fulgoni 2 / 2007 Holtzman 3 / 2007 Arnett 3 / 2007 Arnett

. 3 / 2007 Arnett

2005 / 0149965 AL 2005 / 0172311 A1 2005 / 0235310 A 2005 / 0283395 A1 2006 / 0041480 A1 2006 / 0067320 A1 2006 / 0129457 Al 2006 / 0129458 A1 2006 / 0179450 A1 2006 / 0195866 A1 *

7 / 2005 Neogi 8 / 2005 Hjelt et al .

10 / 2005 Bies 12 / 2005 Lesandrini et al . 2 / 2006 Briggs 3 / 2006 Wahl et al . 6 / 2006 Nyhan et al . 6 / 2006 Maggio 8 / 2006 Trappeniers et al . 8 / 2006 Thukral . . . . . . . . . . . . G06Q 30 / 02

725 / 34 2006 / 0230415 Al 2006 / 0253874 A1 2006 / 0286530 Al 2007 / 0011040 AL

10 / 2006 Roeding 11 / 2006 Stark 12 / 2006 Forrest et al . 1 / 2007 Wright et al .

US 9 , 807 , 442 B2

( 56 ) References Cited U . S . PATENT DOCUMENTS

WO wo WO WO WO WO Wo wo wo WO WO WO WO WO WO WO WO WO WO

WO / 01 / 71620 WO / 2005 / 020578 WO / 2005 / 039080 WO / 2007 / 065069 WO / 2008 / 030358 WO / 2008 / 042923 WO / 2008 / 058193 WO / 2008 / 109743 WO / 2008 / 156912 WO / 2009 / 097569 WO / 2009 / 120220 WO / 2009 / 126152 WO / 2009 / 126169 WO / 2009 / 023842 WO / 2010 / 098890 WO / 2010 / 121178 WO / 2011 / 031973 WO / 2011 / 035298 WO / 2011 / 055365

9 / 2001 3 / 2005 4 / 2005 6 / 2007 3 / 2008 4 / 2008 5 / 2008 9 / 2008

12 / 2008 8 / 2009

10 / 2009 10 / 2009 10 / 2009 12 / 2009

9 / 2010 10 / 2010

3 / 2011 3 / 2011 5 / 2011

OTHER PUBLICATIONS

2007 / 0107008 Al 2007 / 0130580 A1 2007 / 0174295 Al 2007 / 0192166 A1 2007 / 0208619 Al 2007 / 0217769 Al 2008 / 0059285 A1 2008 / 0059299 Al 2008 / 0059300 Al 2008 / 0126198 Al 2008 / 0139182 Al 2008 / 0155637 A1 2008 / 0172504 A1 2008 / 0196060 A1 2008 / 0214157 AL 2008 / 0244097 Al 2008 / 0297371 Al 2008 / 0301737 AL 2009 / 0015433 AL 2009 / 0017805 Al 2009 / 0018893 Al 2009 / 0037279 Al 2009 / 0083134 Al 2009 / 0111320 A1 2009 / 0150217 Al 2009 / 0150919 Al 2009 / 0171767 Al 2009 / 0204476 A1 2009 / 0216621 AL 2009 / 0249388 AL 2009 / 0276805 AL 2009 / 0292587 AL 2009 / 0298535 Al 2010 / 0058377 All 2010 / 0070987 Al 2010 / 0088364 A1 2010 / 0089996 A1 2010 / 0100445 A1 2010 / 0131356 A1 2010 / 0153983 A1 2010 / 0162285 A1 2010 / 0229195 AL 2010 / 0257023 Al 2010 / 0324990 A1 2011 / 0004922 AL 2011 / 0066507 A1 2011 / 0072461 Al 2011 / 0083101 A1 2011 / 0087780 A1 2011 / 0106631 A1 2011 / 0162002 Al 2011 / 0219403 A1 2011 / 0238826 AL 2011 / 0239243 A1 *

5 / 2007 Dybus 6 / 2007 Covell et al . 7 / 2007 Abraham 8 / 2007 Van Luchene 9 / 2007 Branam et al . 9 / 2007 Super 3 / 2008 Hamoui 3 / 2008 Hamoui 3 / 2008 Hamoui 5 / 2008 Ullah 6 / 2008 Levy et al . 6 / 2008 Du Breuil 7 / 2008 Kimura et al . 8 / 2008 Varghese 9 / 2008 Ramer et al .

10 / 2008 Candelore et al . 12 / 2008 Ida 12 / 2008 Hjelmeland Almas et al .

1 / 2009 James et al . 1 / 2009 Sarukkai et al . 1 / 2009 Aviv 2 / 2009 Chockalingam et al . 3 / 2009 Burckart et al . 4 / 2009 Johansson et al . 6 / 2009 Luff 6 / 2009 Lee et al . 7 / 2009 Kolessar 8 / 2009 Abraham 8 / 2009 Anderson et al .

10 / 2009 Seidel et al . 11 / 2009 Andrews , II et al . 11 / 2009 Fitzgerald 12 / 2009 Klein et al . 3 / 2010 Grob et al . 3 / 2010 Amento et al . 4 / 2010 Carter et al . 4 / 2010 Koplar 4 / 2010 Flood 5 / 2010 Stevens et al . 6 / 2010 Philmon 6 / 2010 Cohen 9 / 2010 Kueh

10 / 2010 Kendall et al . 12 / 2010 Agarwal et al .

1 / 2011 Bono et al . 3 / 2011 Lyer 3 / 2011 Moon et al . 4 / 2011 Sharon et al . 4 / 2011 McCann 5 / 2011 Lieberman et al . 6 / 2011 Jones et al . 9 / 2011 Nesamoney et al . 9 / 2011 Carre et al . 9 / 2011 Dierks G060 30 / 02

725 / 14 2 / 2012 Recker et al . 2 / 2012 Schultz et al . 3 / 2012 Smith et al . 3 / 2012 Wen et al . 4 / 2012 Ramer 8 / 2012 Mesaros 9 / 2012 Sivan

10 / 2012 Kuhlke et al .

“ Identify That Song . . . or TV Program ” Apr . 20 , 2011 ( http : / / www . practicalistuff . com / 2011 / 04 / identify - that - songor - tv - program . html ) . Study : In - Game Video Advertising Trumps TV Advertising in Effectiveness Mar . 24 , 2009 ( http : / / techcrunch . com / 2009 / 03 / 24 / study - in - game - video - advertising - trumps - tv - advertising - in - effec tiveness / ) . “ TV remote control on your mobile phone , ” Jan . 5 , 2008 ( http : / / www . ilocus . com / 2008 / 01 / tv _ remote _ control _ on _ your _ mobi . html ) . “ TV Remote Controller ( ARM / XScale ) for Windows Mobile ” Sep . 25 , 2007 ( http : / / download . cnet . com / TV - Remote - Controller - ARM XScale / 3000 - 2064 _ 4 - 10075364 . html ) . “ Use your cellphone as a remote " Mar . 7 , 2007 ( http : / / www . msnbc . msn . com / id / 17506526 / ns / technology _ and _ science - wireless / t / use your - cell - phone - remote / ) . 13 Ways to Monitor Your Brand on Social Media : Do You Know What Is Being Said About Your Company Online ? Sep . 6 , 2009 ( http : / / www . jeffbullas . com / 2009 / 09 / 06 / 13 - ways - to - monitor - con versations - about - your - brand - on - social - media - do - you - know - what is - being - said - about - your - company - online / ) . A Brief Look At In - Game Advertising May 26 , 2010 ( http : / / socialtimes . com / a - brief - look - at - in - game - advertising _ b13631 ) . Apple Gets a Big Slice of Product - Placement Pie Apr . 16 , 2006 ( http : / / www . washingtonpost . com / wp - dyn / content / article / 2006 / 04 / 14 / AR2006041401670 . html ) . Connect with your friends around the shows you love Apr . 29 , 2011 ( http : / / www . intonow . com / ci ) . ICrossing launches social media monitoring app , Oct . 14 , 2009 ( http : / / www . mobilemarketer . com / cms / news / social - networks / 4401 . html ) . Product Placement Jun . 26 , 2011 , ( http : / / en . wikipedia . org / wiki / Product _ placement ) . Social Media Marketing Aug . 25 , 2009 ( http : / / www . advmediaproductions . com / newsletter / NL _ Social - Media - Cam paign - Monitoring . html ) . This New Dashboard Lets You Monitor Social Media Conversations About Your Brand Everywhere May 8 , 2011 ( http : / / www . businessinsider . com / synthesio - unity - 2011 - 5 ) . XBMC — The best iPhone / iPad app for sharing media content Jan . 22 , 2011 ( http : / / surfpk . com / xbmc - best - iphone - app - for - sharing - me dia - content - jailbreak - app / - 0346 ) . International Search Report and Written Opinion of PCT / IL2010 / 000918 dated Mar . 15 , 2011 .

2012 / 0026726 A1 2012 / 0047529 Al 2012 / 0066331 A1 2012 / 0079518 A1 2012 / 0089996 A1 2012 / 0197705 Al 2012 / 0225645 AL 2012 / 0272287 Al

FOREIGN PATENT DOCUMENTS EP WO W0

2204766 WO / 98 / 58334 WO / 00 / 08802

7 / 2010 12 / 1998 2 / 2000 * cited by examiner

FIGURE 1A

atent

YYYYYYYYYY

UKKU ame

Targeted Mobile Ad 1 04a

14442420 4444444444

WORLINK

AD - SURVEY Platform 105 - - - - -

-

-

-

-

-

-

-

-

-

-

-

Oct . 31 , 2017

- 444

-

— -

-

-

-

23 : , : : .

Where should place my eds ?

9

:

Mobile , Social and TV Media 101

-

Planning 056

- -

37999999

-

-

ht

-

1 , 47

-

-

-

HHHH

Sheet 1 of 65

-

-

TV Ad 1 046 - - - - - - - - -

- -

-

-

- -

-

- -

- -

- - - - - - - - - - - - - - - - -

WWWWWWWWWWWWWWWWWW WWWWWWWWWWWWWWWWWWW WWWWWWWWW

Didmy ads meet my goals ( Sales / brand ???????

Online Internet 102

Performance 2066 Delivery ORUM

Were my ads 1 viewed by thel audienceli { 3 } { 2 } } } } |

22 : 22

-

.

.

- . - 51

.

. . . .

- -

WWW .

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

: . . .

.

.

OS . . IT ' i ' , . . . . . . .

.

.

AD - SURVEY Analytics Example

US 9 , 807 , 442 B2

FIGURE 18

U . S . Patent atent

AD - SURVEY Plug - in Accessory 1 20

Oct . 31 , 2017

CH 1 : ABC Family " Pretty Little Liars "

Lz

CH 2 : CBS “ The Big Bang Theory " 1 06 CH 3 : ESPN For

1 Rehash

BSO

Sheet 2 of 65

Home TV 130

Personal Mobile Device 1 15

John Smith is watching # The Big Bang Theory on # CBS .

135

US 9 , 807 , 442 B2

AD - SURVEY Example : " iPhone " TV Remote

FIGURE 1C

U . S . Paten

Audience 1 18

atent Oct . 31 , 2017

There are two persons watching CBS at John Smith ' s terminal at 23 : 11

138

w

MR - Platform Monitoring Photo / Video 1 35

Sheet 3 of 65

Ao Capturing Audience Atmospherics Data 1 25

Personal Mobile Device 1 15

iPhoto 00034566 . JPG TimeStamp : 23 : 11 : 56 09 - 09 - 1999

Faces : 2

US 9 , 807 , 442 B2

AD - SURVEY Example : Audience Activity Monitoring

FIGURE 1D

Pop - up Questionnaire 1 23

atent

Avis

Advertisement 1 45

If you are going to buy a car , you would choose : A . BMW B . Audi C . Porsche I D . Other

Oct . 31 , 2017

Home TV 1 30

Sheet 4 of 65

Dominiostricht dorpposicios pro VOD W : : 7 Personal Mobile Device 1 15

The advertisement on CBS is effective ! 155

Merchant 1 50 O Audi

US 9 , 807 , 442 B2

AD - SURVEY Example : Intelligent Mobile Questionnaire

FIGURE 1E

1 85

U . S . Patent

Ganda ???????? ???? ?????? ???? ???? . . . ?? . . . .

?????? ? ????????

????

Will With

What do people think about our show " Big Bang Theory " ? 1 80

Media Analytics Report of " Big Bang Theory " : Data Source : Twitter , Facebook

# The Big Bang is so great . Love the # Caltech geeks . Expecting the new

season . " 09 / 17 / XXXX 19 : 35 : 56

Oct . 31 , 2017

Lisa Smith

CBSO

190

Merchant 150

The Big Bang Theory

We are LOVING Sheldon on the Bs . com homepage today ! gre you ready for some

para - nerdal activity ? Tune in tonight at 8 / 701

CHS IV Network Primežime , Daytime , Late Night and

Qax Yerevisiae Sluws is . com

Watch Station online . Pod OS incima , dame , baix cort , ad cessiones , videos , 306 ato .

Sheet 5 of 65

connot Staro Yesterday at oopm

O $ 512 people like this .

1 862 1 862 -

1 86b

Vicen a 378 comments

49 Shoes

???????? ?? ????

???? 2 ???????? ??

-

US 9 , 807 , 442 B2

AD - SURVEY Example : Media Analytics

FIGURE 2A

U . S . Patent atent

TV Network 270

RT TV Program 2 37a

Audience Stat Report 2 45

RT TV Program 237b

Oct . 31 , 2017

STB 201

RT TV Program 2370

TV Event 235a

*

- -

one

TV Event 235c

IAD SURVEY Server 2 20

RT Data / Record

User 2 33a

Client TV 233b TV Event 2 35b

290a

Sheet 6 of 65

Viewing Status 2 75a

Friends Recom 277

Viewing Status 2755

Database 2 19

Social Media 250

Communication Network 2 13

US 9 , 807 , 442 B2

AD - SURVEY Example Data Flows : TV Remote Monitoring

FIGURE 2B

RT TV Program 237a

atent

TV Network 270

Audience Stat Report 245

RT TV Program 237b

Oct . 31 , 2017

. 12 . ODD * * *

AAD - SURVEY Server 2 20

RT Data Record

Heer 2 33a

Atmospherics of anderes 7

Device App Event 241

239

© 290a

Friends Recom 277

Sheet 7 of 65

Viewing Status 275a

Viewing Status 275b

Database 2V

Social Media 2 50


US 9 , 807 , 442 B2

AD - SURVEY Example Data Flows : Audience Monitoring

atent Oct . 31 , 2017 Sheet 8 of 65 US 9 , 807 , 442 B2

FIGURE 20 User 2 33a AD - SURVEY Server 2 20

Instantiate a AD - SURVEY Application on a Mobile Device ( 250 )

Submit a Session Request ( 253 )

Receive User Request and Determine Whether the User is Registered ( 255 )

. . . . . .

Registered ? ( 258 ) Submit Requested User Information to Register ( 260 )

Obtain and Store a User App / Session ID ( 265 )

Receive A List of TV Programs ( 270 )

Obtain Real - Time TV Program Listing ( 267 )

Submit a User Event ( 280 ) Monitor User Event / Messages ( 276 )

Determine Received Event Type ( 288 )

??? Device Status ( 2891 )

Non - live TV Consumption

( 289e )

Atmospherics ( 289a )

Live TV ( 289 )

Ad / Survey ( 2890 ) Social ( 2890 )

AD - SURVEY Example Logic Flow : User - Server Interaction


FIGURE 2D

289ble Parse Data Fields Received from the Remote App and Save it in a Raw Data Store 2001

Retrieve All Unprocessed Records Grouped Per User

2002 KO Raw Data Store 2 19a

For Every TWO Consecutive

Messages 2005

Calculate And Append An Elapsed Time Between the Two Consecutive Messages 2008

Elapse > Surfing Threshold ? 2009 > N Filter These Records as Indications

of Channel Surfing 2013

Elapse > Capping Threshold ? 2015 N

Retrieve TV Program Information on User Selected Channel 2018

Cap : by STB / TV Off . Event , TV Program Apply Capping Edit Rule to End Time , Cutoff Compute Watching Time 2025 Thresholds per Heuristics 2023

Generate a User Channel Selection Log File 2020

Audience Analytics 2 19b

Feed to TV Viewing Data for the Retrieved TV Program 2030

AD - SURVEY Example Logic Flow : Mobile Meter

FIGURE 2E 2891 289a

Receive Atmospherics Data at 289a

Decode the Atmospherics Data and Obtain Atmospherics Artifacts ( 296a )

Capture Active Applications Running on the Device ( 299a )

atent

GPS

- Visual

Determine Atmospheric Artifact Type ( 296 )

Audio

Oct . 31 , 2017

Human

Determine Graphics Content ( 297 )

Determine Audio Content ( 298 )

Media

Lighting Sensing

TV Screer

User

Voice Recognition ( e . g . , matches

characters ) 298a . 1

Ambient Noise

Determine User Activity When Viewing ( 2996 )

Determine User Environment When Viewing ( 2990 )

Facial Recognition to Determine Audience Presence ( e . g . , Number of Audience ) ( 297a )

Determine Environment of Audience ( Whether They are Watching ) ( 298b )

Non Voice ?

Sheet 10 of 65

N

? Tag Each Voice Source ( 298a . 3 )

Determine User Location Whether He is located with the TV ) ( 2990 )

Determine Whether the Screen Matches User Channel Selection ( 2976 )

Determine whether the Media Audio Matches User Channel Selection ( 2980 )

Extract Key Terms to Determine whether Matches TV Content ( 2988 . 4 )

Generate Audience Attendance Estimate and Refine TV User Viewing Data Generated at 293 ( 2100 )

AD - SURVEY Example Logic Flow : Atmospherics Analytics

US 9 , 807 , 442 B2


FIGURE 2F User 2 33a AD - SURVEY Platform 2 20

Capture Device App Status , Image , Audio , GPS

Coordinates , etc . ( 2105 ) Receive Message from User

( 2115 )

Extract identifying information ( 2106 ) Determine Message Type ( 2118 ) HOH External Event ? ( 2108 ) Atmospherics

Data ? ( 2120 )

Launch the Event to Generate Embedded Signals ( 2110 ) Decode and Analyze the

Atmospherics Data ( 2123 )

Aggregate and Package the Atmospherics Data ( 2113 )

Program Correct ( 2125 )

Determine User Attendance Status ( e . g . , how many users are

presented ) ( 2128 )

Generate Viewing Data ( 2130 )

AD - SURVEY Example Logic Flow Audience Monitoring


FIGURE 2G User 2 33a AD - SURVEY Platform 2 20

Submit a Non - Live Media Registration Request ( 2150 )

Submit User External P , Internal IP ( e . g . , TIVO ) ; Medial

Access Key ( 2165 ) Register the User ( 2156 )

Receive Non - Live Media Schedule ( 2158 )

2 89e Check Program Table to Determine What ' s the Program on Non - Live

Media ( 2160 )

Decode and Analyze the Atmospherics Data ( 2123 )

Log the User Channel Selection with a Timestamp ( 2125 )

Generate TV Viewing Data for the Selected TV Program ( 2127 )

2100

AD - SURVEY Example Logic Flow Audience Monitoring


FIGURE 2H wys AD - SURVEY Platform 2 20

( 29 ) 2 990 Determine an

Address Type ( 2163 ) he Calculate Atmospherics

Score ( 2200 ) Assign a Weight based on the Address Type

from a Atmospherics GPS Weight Table ( 2164 )

> Threshold ? ( 2205 )

Perform Device App Analytics at 299a

Determine an Active App Type ( 2165 )

Calculate Updated Atmospherics Score ( 2170 ) 1

Assign a Weight based on the Active App Type ( 2168 )

K o > Threshold ? ( 2175 ) Perform Audio Analytics at 298 sem Determine an Audio

Analytics indication ( 2178 ) ersten Calculate Updated

Atmospherics Score ( 2185 ) Assign a Weight based on the

Audio Analytics Indication ( 2180 )

Threshold ? ( 2190 )

- N Perform Graphical Analytics at 297a / b

Determine a Visual Analytics Indication ( 2192 )

Calculate Updated Atmospherics Score ( 2193 )

Assign a Weight based on the Visual Analytics Indication ( 2195 )

> Threshold ? ( 2196 ) - N

Conclude the User is Watching and Quit Scoring

( 2305 ) 2100

Conclude the User is not Watching and Quit Scoring

( 2300 ) AD - SURVEY Example Logic Flow : Atmospherics Scoring

U . S . Patent Oct . 31 , 2017 Sheet 14 of 65 US 9 , 807 , 442 B2

FIGURE 21 User ( Device ) 2 33a

User Plug - in AD - SURVEY Infrared Accessory 2500

Instantiate a AD - SURVEY Client Component on a User

Device ( 2505 )

plug - in Availabler ( 2610 )

Initiate an Automatic Scan on Communication Slack for TV / DVD Address

( 2515 )

Obtain User Submitted TV / DVD Parameters ( 2525 ) Available ? ( 2520 )

Adjust the Scanning Mode based on User Submitted Parameters

2526 )

Obtain a Physical Address and Determine a Type ( 2523 )

Log ihe Determined Facility Type with the Physical Address ( 2528 )

Transmit a Channel Selection Indication to the Logged Address

via the Infrared Plug - in ( 2535 )

Channel Submission ?

( 2530 )

AD - SURVEY Example Logic Flow : Atmospherics Scoring

FIGURE 3A

RT TV Program 237a

atent

TV Network 2 70

Ad Tags 343a

Popup Question / Product Placement Ad 338b

Oct . 31 , 2017

Popup Question / Ad 3 38a

LLLLLL LL TITULI TI

AAD - SURVEY Server 220 RT Data Record 2 900

Weer 2 33a

Purchase Request 3 65a

Friends ' Purchase 349

= Response 339

A

Sheet 15 of 65

Purchase Info 2 38

Purchase Request 365b

Database 21

Purchase Info 3 65b

Merchant 2 80

Social Media 250


US 9 , 807 , 442 B2

AD - SURVEY Example Data Flows : Product Placement / Questionnaire


User 2 33a AD - SURVEY Server 2 20

Submit Channel Selection 3052

Check Program Table to Determine What ' s the Program on Air 3 06

Retrieve Program Table to obtain Ad Tagged in the Selected Program

307

Parse Commercial Ad Information or Selected Channel 3 08a

- - - - Submit Mobile Ads |

Exposure Data ; Application Status ; Social Content 3 05b )

Parse Ad Program Information on Received Mobile Data 3 086

Query on a Questionnaire DB based on Parsed Ad Information 3 10

Receive a Pop - Up Survey Question 313

Generate a Pop - up Question ( with Question ID ) to the User 3 12

Submit 3 Response 315

AD - SURVEY Example Logic Flow Intelligent Ad / Survey Generation


FIGURE 3C User 2 33a AD - SURVEY Server 2 20

Submit Channel Selection 305

Check Program Table to Determine What ' s the Program on Air 3 06

Retrieve Program Table to Obtain Embedded Ads in the Selected

Program 307 Receive an Ad / Survey Question Related to User Viewing History

331 -

- Query on What Ads are Available 328 -

-

- Receive a Textual Pop - Up Ad at Synchronized Time 340

IN 1 - - < Synchronized Ad

Tag ? 3 30

Receive an Interactive Ad at Synchronized Time

342 Static Extract Sync Ad Tags Ahead of Time

333

Ad Tag Type ? 335

Interactive

Submit a Response 315

Generate Video Frames Containing Product Placement Tags , and Retrieve

Interactive Ad Templates 3 38

2 890 AD - SURVEY Example Logic Flow : Embedded Ad


FIGURE 3D AD - SURVEY Server 2 20

2890 Receive User Response to Ad / Questionnaire

at 2890

Question Response ? ( 3 624

Determine Question Classification 365

Determine Product Placement Ad Delivery indication 3 76 -

-

- -

-

- Z

-

-

Extract Questionnaire D 366 -

-

- Purchase Request ? 3 78

-

-

-

-

Store Questionnaire Results 3 68 -

- Provide a Merchant URL to the User 377a

Determine Rewards for User 370

Log the User Activity Associated with the Ad 377b

Send User Rewards 372

Run Analytics to Determine Ad Effects 3 75

AD - SURVEY Example Logic Flow : Event Analysis


FIGURE 3E AD - SURVEY Server 2 20

- - 375 Retrieve an Advertisement and Determine

Ad Classification 3 80 -

-

-

- Query for Stored Question Results on Ad Classification 3 81

-

-

-

-

Survey Available ? 382 82N

Query for Stored User Activity ( e . g . , Purchasing Request , etc . )

based on Ad ID 382 Periodic Update 392

Query the Survey Responses for Mentions of the Product 3 83 Activities

Available ? 3 87

For Each Retrieved Response

Determine a Type of the Activity 388

Determine a Relevance of the Question and Question Results

384 Determine a Weight Value for the Activity 3 89

Determine a Weight Value for the Response 385 Generate an Ad Effects Sub

Score from User Activities 3 866 - Generate an Ad Effects Sub

Score from Survey 3 86a -

- - - - - - - - - Generate an Indication of Ad Effects 3 90

AD - SURVEY Example Logic Flow . Ad Delivery Analysis


FIGURE 4A * *

. - - . - - .

. - . - . - . - . - . - . . - . - . . . .

: :

:

: :

: : : : : :

:

:

: :

: : : : . . . "

Communications Module Communications 1 / 0

432 youtube 430

User Questionnaire Component 4 30 Real Time TV

Remote Control Component 4 15

Mobile - Remote Controller 4 05

Content Based Embedded Advertising

Component 4 20 User Interface 4 10 wmmmmmmmmmmmmmmmmmmmmmmmmmmmmm Media Content

Questionnaire Synchronization Component 4 35 Sensor ( Video )

GPS ) Module 420

Social Media Connection

Component 4 45 w

Media Listen - In Component 440 w

Sensor I / O 4 25

AD - SURVEY Component 401 : 54 :

w

AD - SURVEY Example Client Mobile App Block View

FIGURE 4B

TV Remote Control Component 4 15

Channel Selection Event

-

Chanel Selection Event

U . S . Patent

Query 4 17

4 16

Media Program DB 4 19

atent

Program Information 4 18

Ad Tags 4 19b

Media Program 4 19a

User Questionnaire Component 4 30

Ad Synchronized Question 4 21

Oct . 31 , 2017

Ad Synchronization Component 4 55

Add Questionnaire Synchronization Component 4 35

Response Events 4 22

Ad Effect Engine Component 4 60 Media Viewing Stat Aanalysis Component 4 65 AD - SURVEY Server 4 50

Atmospherics Data 4 23

Sheet 21 of 65

Media Listen - In Component 4 40

- - Optionally Sent to Server

Re Redirect 472 / API call 471

Social Media Connection Component 4 45

Social Media Post 4 48

AD - SURVEY Client 4 01

Social Media DB 469

US 9 , 807 , 442 B2

AD - SURVEY Platform Overview 450

FIGURE 4C . . . . . . . . . . . . . . . . * . . . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . ! . ! . . . . . . 4 . 4 . 4 . 4 .

4 . 4 . 4 . 4 . 4 . 4 . 4 . 4 . 4 . 4 . 4 . . .

.

atent

PRA

JJJJJJJJ

FPJJJJJJJJJ

21277777rrrr

rrrrrrrrr

Table Top Device 473

TV Set 483

rred

YYY

Remote Control Mobile Application 4 71

???????????????? ??????

: : : : : : : : : : : : : :

Bluetooth Signal 4 76

cewwwrrrrrrrrrrrrrrrrrrrrrrrrrrrr

Oct . 31 , 2017

?

iiiiiii » » » » + + + +

Bluetooth Transmitter / Receiver 474

????

Infrared Signal 4 81

%

{ f frY Receiver 485

Wwwwwwwww with . . . . . . . . . .

?? ??

r errrrrrrrr

rrrrrrrrrrrrrrrrrrerwm

??

? ?? ? ? ! !

? ? ? ? ? ? ? ? ? ? ? ? ? ?????????urrrrrrrrrrrrrrrrrrrrrrrrrrrr??????????urrrr

Dimiuk

rossovA11112rererrrrrrrrrrrrrrrrrr

t? ??? ???

???????????????????????????????????????????????????????????????????????????????

. W

mummm till

+ + iiiiiiiiiiiii » » » » » + + O HOOP

?? ?? ??f

Set Top Box 486

?iit ! ( ? ? ?

Infrared Transmitter / Receiver 4 75

/ / / / / / ssrrrrrrrrrrr

?

my

Bluetooth Transmitter / Receiver 472

Infrared Receiver 487

Sheet 22 of 65

2 . 4 . 4 . 4 . 4 . 4 . 4

« « « « « « « « « « « «

17 : 22 : resSSIIIIIIIII /

rrrrrrrrrrrrr

Rechargeable Battery 478 JJJJJJJJJJJJJJJJJJJJJJJJJJFFFFFFFFFFFFFJ .

P1277721rrrrrrrrrrrr ? 2

Charging Input 4 80 IIFFFFFFFFFFFFFFFFFFFFFFIIIIII SIIIIIIFFFFFFFFFFFFFFFFFT

US 9 , 807 , 442 B2

AD - SURVEY Table Top Unit Infrastructure 4 70

12ESEPSIA . SK

FIGURE 5A

MOSTERS

RESTATA

U . S . Patent

Social Media Structured Data 5 25a

Social Media Unstructured Data 5 25b

STRA

Application ID ! User Token 5 18

487

Determine Data Update Needed

Oct . 31 , 2017

TIIUL

Data Request + Access Token 5 22

520 /

AD - SURVEY Server 220

User Auth Request 5 10

Social Media 250

User 233a watoto

Access Request 505

Xxx

User Authorization : Provide App 10 Permission 5 15

Sheet 23 of 65

4 .

w22 * * * * * * * * * * * * * * * * * * * * * * * * * * *

* * *

Web App Uber 511b

Mobile App UV 5112

US 9 , 807 , 442 B2


AD - SURVEY Example : Data Download From Social Media

FIGURE 5B

ZEEL

MR - Control Media Measurement Portal 5 70

atent

at

w

1

t

S

3 . 79

.

Analytics Platform 570a ( Structured data ) 571a

Single sign on Social Analytics Platform 570b * Neaupe ( Unstructured data ) 571b

WA

*

User Permissions Flow for Social Media Access ( 573 )

TV Mobile Web Veasurement Reports ( 573b )

Www

????? ! ????? Wuzzsurernent Reports ( 583a )

Taxonomy Management Console 15836

Oct . 31 , 2017

??????????????????????????????????????????????????????????????????????????

Facebook API ( 590a

???????????????????? ??????????

Sement

. . . . . .

0 77

. 77 . 1 .

0 . 0 . 0 . 08 . 2007

Media

HTTP

Twitter API $ 5805 )

X111111 *

???? . ???

?? .

Org user ???????? 15730

? ; , ?? ???? f

ry

wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwslet

wwwwww

Engine ( 5730

Text Analytics

Social

?????

Med

( 5830

Adapters ( 5830 )

AP 527 www .

northern Social Analytics Platform API ( 583€

( 578 )

Google + API ( 58001

X

Shop

?

Sheet 24 of 65

Single Source ETL Process 1573e

Other Providers G & G

he time

t

Warmesan

A

User cauth Tokens *

Sodal Graph Text Data ( 585b )

?????? 5C1 = Datamani - V . Mobile , Online & Social Data 1575a

Taxonomy Repository ( 5852

Wew

7

*

.

dition E

22 Horairúrr . . / . . . in /

US 9 , 807 , 442 B2

AD - SURVEY Example Infrastructure : Media Analytics


FIGURE 5C

2 890 .

* * * Per . .

User 3333 ?? ?????? ??ES .

$ 113 pare Management Website 511

5 32a 5 326

? ? sothosted ?

kter Authorized ?

WWW perown P 5 33a 5 336 kur ko

Redirect to Facebook Ox Twitter URL With IV Remote Coprication ta & Permission Attribute Request

gedirect to Facebook Twitter URL with SAM Application & Pemazion

Attribute Request remierer 534a 534b ??????????????????? Access

Approved by ver ? ?????

Approved by exer ? . . . . . . . . . . .

5 35a 5356 ??? ? Ves . . ime

r Save is response token in user repository that is provided by

Facebook or twitter

Save Auth response tokes in user E : * ?at is PV??? ?y

Face oak or twiter erererererereri . . .

. .

. 536a 5360

.

. . . . . . * * * * . * . . . . Www w COMME Using Moosa Aps

11 - = { g 5? ? Management Website

Generate Social Message View Social Updates via AD - SURVEY 537


FIGURE 5D ©

Receive Data Updates from Social Media Servers 540

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

atent

Determine Source of the Social Media Data ( e . g . Facebook ? Twitter ? Google + ? , etc . ) 5 43

Parsing the Structured Data to Extract Information 552

- -

Oct . 31 , 2017

- -

Apply Data Format Analytics Rules based on the Determined Data Source 5 45

Generate a Data Record includign User ID / Time Stampl Geo Source , etc . 555

— —

Sheet 26 of 65

<

Structured Data ? 538

Store the Generated Data Record in a Database

ecord in a Database L

557

rent de Benestar

Feed the Data to Taxonomy Engine For Data Tagging 500

e for Duos

* 550

US 9 , 807 , 442 B2


AD - SURVEY Example : Social Media Data

US 9 , 807 , 442 B2

w

WON

! ! . . . . . . . .

WWW WE REBUS

. . . . . . . . . !

. . . . . . . . . . . . .

: : : : : : : ? ? ? ? ?

VEL STAVO

OCH WU WCH

? ? ? ? ? ? ? ? ? ? ?

: : : : : : : : : WONDO COMO OK

OR W BO

KUWA

como

SOM DOO

WOW W

GUM O

930

Mao COMO

M

3

??

???? . . ?????? ? ?????

? . .

02W ? ?

XXXXXXX

? ? ?

Sheet 27 of 65

WISHES

,

? ? ? W

? ? ? ? ? ? ? ? ? ? ?

* * * * * * * * * *

mus 33

OSEBNOWKWL WHICH WASÓ GW w

Omdao

*

W

OB T

250

. . .

HR24 MWWW 299

* * * * * * * * * * * * * * *

Oct . 31 , 2017

OSTW OKO WORKEUW

. .

WOM H OSTUS ESSE UND 298

WR 80 O

ROOM OMWOKO *

W 2W

OR WHO WY

www

X

* * * * * * * * * * *

" X

WWW wa

08

UM REPEL XWWW

111111ssrrrrrrrr ' : ' 1 ' 111 ' tiriririri ' ' r ' 1111111ssrrrrrrr ' : ' 111 ' 11 ' i ' irri '

sii ' r ' r ' r ' 1 ' 111111 ' siririri ' ' . 1 ' 11 ' i ' icsi ' r ' rrrrrr ' r ' 1111111ss

ri ' i ' i ' r ' r ' ' ' 111ervicii ' ' ' r ' 111111 ' siririririr ' ' 111 ' 111 ' risis ' rrrrrrrr

rrrrrrrrrrrrrrrrrr

r rrrrrrrrrrrrr11111isverrrrrrrrrr ' r ' 111 ' 111 ' risi ' rrrrrrrrrrrrrrrrrrrrrrr ' r ' 1 ' 111111 ' sirisiri ' i ' i ' i ' r ' 11 ' ' 111 ' inciriririririri

s

atent

FIGURE SE URE

AD - SURVEY Example : Social Media Data

US 9 , 807 , 442 B2

WWWW : : : : : : : : : : : : : : : :

WWW

1 . . . . . . . . . . . 11111111111111111 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . . . . . . . . . . . . . . . . .

.

.

OR , F Pagesamom

L

WWWR 1000 EURO 2008

On wWww

*

08

O OVO

BER

WWW OU PERSON

*

rodua

. . . . . . .

Soma * O

E UR 3300 Com . . . . . .

: : : : : : : : : : : : : - - - - - - - - - - - - - -

. .

2000 ,

o

wa

WWW *

WHO MUISTO QO90 WE PRO290 WER WOCKHO WO 028 . . . . . . . . . . .

OWO OWROMA O WORKS

2000

Sheet 28 of 65

. .

: : : : ! ! ! ! ! ! ! ! ! ! ! ! ! ! !

S & W

19220221

30 wo W3101 WOWO ,

wowo

. . . . . . .

. .

? ! ! ! 244 ? 714W

0 : 00 STAR

.

. . . . . . . . . .

. . .

WEWE GOD

. . .

WWW WWW WWW

2 : 27 292222

.

W

10

Oct . 31 , 2017

2014

. . . . . .

Som mais W

WWKO

28

www : 0

WISHES

WWW WOWO Wowo WWWM250 NA OG 360

38CMW X WWWXW0

OX 30 Woo OR 1920 COM

SOM OM OSS SESVOX NEROBCHOD WHXSO O WWW

X4

96 2 W

0

time . W

111111111111 ? ? ? ? ? ?

: : . . . : : : : : : : :

atent

FIGURE 5F


FIGURE 6A User 2 33a AD - SURVEY Server 2 20

Submit a Request for Analytics Including a Product / Brand Name

ô 05

Generate a Ul Management Panel Including Analytics Options ( e . g . , Social Media Source ,

Time Range , Activity Type , etc . ) 6 10

Submit Configurations of Analytics Parameters 6 13

Form a Query based on Retrieved Social Media Data 6 15

Retums Query Results 6 20

Presentation Format Selected ? Display Raw Results 6 30

6 23

Select Output Visualization Format and Present the visualization of Query Results to

User 6 25

Display Visualized Results 633

AD - SURVEY Example : Logic Flow for Social Media Analytics


FIGURE 6B

Level 0 C All TV Shows 640

Level 1 Show Genre 641 )

Level 2 Show Names 6 42

Network 6 43a

Episodes 6 43b

Level 3 Cast Member 6 430 )

Character Name 6 43d

Show Names 6 43e )

Key Words text 6431

AD - SURVEY Example : Taxonomy Logic Tree For TV Shows


FIGURE 6C

Level 0 ( All Categories 6 45

{ c } { Categories 6 46

Level 2 Product Brand 647 )

Ad Content key Words 6 48a

Brand Name 648b

Level 3 Brand Name + Ad Content Key Words

6480

Product Mentions 6 48d

Product Name 6 48e

Key Words text 6 487

AD - SURVEY Example : Taxonomy Logic Tree for Brand Products

U . S . Paten atent Oct . 31 , 2017 Sheet 32 of 65 US 9 , 807 , 442 B2

FIGURE 6D

Start Determine " Low Level " Categories based on Structured Data Dictionaries 6 50

Extract Sample Text from Unstructured Data to Identify Commonly Used Key Words 6 53

Associate the Key Words With a Level for the Tagging 655

Identify Words with Close Spellings to the Key Words 6 57

Generate a Logic Rule with Logic Operator for Each Level 6 60

Store the Generated Logic Rule with the Taxonomy Tree 6 62

Continue

AD - SURVEY Example : Defining Taxonomy Rules


FIGURE 6E

552 Parse the Unstructured Data for and Delermine a Level One Category 6 80

Generate a First Tag Associated with the Unstructured Data with Brand Name by Queryin

on the Brand 682 Brand DB 6 84

Retrieve Taxonomy Logic Tree 6 86

For a Query on a Second Category , e . g . , Brand Name + Positive Mention 6 90

Found ? 6 92

Query on " Coke " + " Excellent " 6 93b

Generate a Sub Tag Associated with the Unstructured Data with Product Name 6 93a

Found ? 6 94

Generate a Sub Tag Associated with the Unstructured Data 6 95

Continue with Taxonomy Logic Tagging Fig . 6F

AD - SURVEY Example : Taxonomy Tagging Logic Flow ( 1 )


FIGURE OF

Centro Continue on from Fig . 6D a Query on " Coke " + Key word texts , e . g . , " too

much taste " 696 en a no Found ? 6 97 Fermenges

Generate a Sub Tag Associated with the Unstructured Data with Ad Attributes 6 989 mane olana unitate de una

Generate a Matching Score of the Applied Taxonomy Tree 6 99

> Threshold ? 6 100

Keep Current Tagging and Proceed with Another Taxonomy Tree 6 102

6 86

AD - SURVEY Example : Taxonomy Tagging Logic Flow ( 2 )


. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

. . . . . . " IIIIII .

???????????????? 730 ???????????????????????? facebook Facebook conversations AD - SURVEY Example Data Flow Interaction

?????????????????????????????????????????????????????????????????? TV 7 . 25 Viewing

AD - SURVEY Data Collection 7 05 ( Panelists 7 10 ) NY

Advertising 2015 Online

220

Mobile Device usage ?????????????????????????????????? FIGURE 7A

FIGURE 7B

7 95a

atent

TV Channel Changing

TV App

- 7 95b

* * * * *

7 91a

| Mobile Advertising

MR - PLATFORM App

Oct . 31 , 2017

wwwwwww

wwwwww

VY www

? . 44 ????????????? . . ???? . ?? . ? . . ???? . ? . ?????

1916

Mobile App Usage wwwww

750 750

end

Social Media Profile 7 78a

mencecerrarierer i

vorer27712700000007 6

4

444444444 . . .

w

Facebook / Twitter

AD - SURVEY DB 7190

Social Media Comments7 78b

Sheet 36 of 65

in rifinn

1147720

7 89b

intrinoporturi pentr

ter

-

-

-

mm

u mili178 . 11 . winnerPupphan

Sap003 BLLHUS

r itorio .

*

Exposure to tagged ad7 89a

*

7 90a

797

Advertising Exposure * Iphpeitsregd irrepetiap prillit ; * * * * * * *

Website visitation , 900

URL Tracking

US 9 , 807 , 442 B2

AD - SURVEY Example Data Flows

FIGURE 7C

7 88

atent U . S . Patent

Pro Seee

mitzettsliticii Social App Usage ( iPhone )

Mobile Advertising 1

- 7 19

r etplatitzatsititjirjiirisatilit ; fiti

P

Oct . 31 , 2017

AD - SURVEY App 7 . 95h

W MNNNNNN0768MMUNI16WwwwwwwwWwWWWWWWWWWWWWWWWWMriin BINH111100000M

Database il

W

TV Remote 795a

Social AppUsage GPS Location Surveys

Social App Usage TV usage from remote GPS Location Surveys

TV Viewing

Sheet 37 of 65

- Z 50

i

Facebook Twitter Social Media Engagement

Mobile search behavior Exposure to Social Media assets

US 9 , 807 , 442 B2

AD - SURVEY Example Mobile Ecosystem

FIGURE 7D

— 782 - 782e

- 7 82f 782f KWA

atent

7 82a

EXPRESOURCES

Client access via Website 7 84

Facebook Server

WWW

fifififiii ' fi

????????? . ?????? ? ?? . ???????????????????????????????????? :

19

omen

- 719

Database

Oct . 31 , 2017

:

API

W

interior paipinistra r

insson FSC - NO

illit

???????????????????????????????????????? ????? ??? ???????????????? 81 ?? 16 ?? 1 ????? # # # # ff ! "

Analytics Server 755

) ! 0netisripristiti

7 82b

: 10

Authorized Authorized by Panelist

Hirit ' .

- - -

Twitter Server

white

4

t

Authorized by Panelist

With

e

Untit

Sheet 38 of 65

:

-

WW

With Wh

Asalyties7830

20ta

Cloud Storage . 7 82d

- 7 82c

US 9 , 807 , 442 B2

AD - SURVEY Example Social Media Data Collection System

& Booty A &

U . S . Patent

Instantiate a AD - SURVEY Proxy Server to Monitor Data In / Out of the User Device 740

Serengeti

Receive Data at the Proxy Server 7 42

atent

Extract Information form the Received Data 7 45

Oct . 31 , 2017

N

Media Usage

URL ? 7 46

App Info ? 751

7 62

Survey / social 763

Determine whether the URL Comprises an Ad 7 48

Determine Device Application Inventory 7 53

Determine whether the Media Program has been Listened , Watched , Streamed 7 66

Sheet 39 of 65

Determine Classification of the Ad ( Media Genre , Source , Content ) 7 49

Obtain App Group Sharing Info 7 54

Process the Social Message to Extract User Impression Indications 7 65

Determine a Media Title 767

Obtain Identifying Information of the Ad 7 55

Media Analytics 7 68

AD - SURVEY Example Logic Flow : Cross Channel Ad Delivery Data

Collection

US 9 , 807 , 442 B2

AD - SURVEY Example Structure : Ad Recognition

US 9 , 807 , 442 B2

csirrrr

r ? FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF??????? ? ? ? ? ??????? ?

Sheet 40 of 65

wwwwwwwwww

PPP !

775 database against ad Compare

773 Transmission TCPIP

? ? ?i

772 digital file URLs and Sends

AM

??

?

?

27 ????

?

/ ??????

? ? ???

1111171111111 XXXYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY

?? ?????? ?????? ????

773 Transmission

?? ???? ???? ?? ???? ?? ?? ??????

Oct . 31 , 2017

776 Engine Classification ????????????????? ?

dido1

ww . m ww . un

W . mm

Wwwxxxww

software 771 Data collection

DLL JONAS

mmmmmmmmmmmmmmmmmm

1272222222222222222222222222222222222222222222222222222222222222222222222222222 222222222222222222wwwwwwwwwwwwwww

atent

FIGURE 7F


FIGURE 8A * * * * * * * *

WAS John Smith is watching Big Bang Theory on CBS . 8 20 What are your friends doing ? 8 05

Amy Smith likes Big Bang Theory . Jay Taylor recommends Preity Little Liars on ABC family . Anna Tuck participated in the AD - SURVEY survey on your favorite late night show .

Social 8 15 Prompts 8 20

Channel Program 8 10 ???????????????????????????????????????????????????????????????????????????????????? CBS ABC Family . . . ESPN

CBS ABC

Big Bang Theory 8 : 15pm - 9 : 30 pm The Pacific 9 : 30pm - 10 : 30 pm Pretty Little Liars 8 : 00pm - 9 : 00pm Desperate Housewives 9 : 20pm - 10 : 30pm . . .

NBA Finals Live 8 : 00pm - 11 : 00pm ESPN

CBS

Kameroon 8 50 Allowing Ambient Monitoring


FIGURE 8B

Wwwwwwwwwwww John Smith is watching Big Bang Theory on CBS . What are your friends doing ? 805

Amy Smith likes Big Bang Theory . Jay Tavlor recommends Pretty Little Liars on ABC family . Anna Tuck participated in the AD - SURVEY survey on your favorite late night show .

Social Prompts 8 20

11 Channel Prograr What is your favorite automobile commercial ? Tap to choose 720a A . Mercedes B . Honda C . Audi D . None of the above ?? ?????????????????????????????????????????????????????????????????????????????????????????????????????????? CBS

ABC Family . . . ESA ??

CBS ABC ESPN

Big Bang Theory 8 : 15pm - 9 : 30 pm The Pacific 9 : 30pm - 10 : 30 pm . . . Pretty Little Liars 8 : 00pm - 9 : 00pm Desperate Housewives 9 : 20pm - 10 : 30pm . . .

NBA Finals Live 8 : 00pm - 11 : 00pm ?????????????????????????????????????????????????????????? CBS CNN

wwwwwwwwwwwwwwwwwwww Allowing Ambient Monitoring


FIGURE 8C

ww . . . . . . . . . . . . . . .

. . . .

. . . . . .

John Smith is watching Big Bang Theory on CBS . What are your friends doing ? 8 05

You responded to the survey question " What is your favorite automobile commercial and like the Audi commercial . 8 30

- - - Amy Smith Likes your response . 8 31a - - - Anna Tuck commented on your status " I prefer the BMW commercial . " 531b

Jay Taylor participated in the survey . See her response , 8 310 Social Prompts 8 20 ??????????????????????????????????????????????????????

w wwwwwwwwwwwwwwwwwwwwwwwwwwwttttttttttttttttmann 11 Channel Program 8 10

. .

CBS ABC Family . . . E

CBS ABC ESPN CBS CNN CNN

Big Bang Theory 8 : 15pm - 9 : 30 pm The Pacific 9 : 30pm - 10 : 30 pm . . . . Pretty Little Liars 8 : 00pm - - - 9 : 00pm Desperate Housewives 9 : 20pm - 10 : 30pm .

NBA Finals Live 8 : 00pm - 11 : 00pm www S

.

ritiiviiviisssssss Allowing Ambient Monitoring W

NOOOOOOOOOOO000000000000000000000000000000OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO O XX X


FIGURE 8D * * * * * * * * * * * * * * * * * * * * * * * * * * * *

mm

John Smith is watching Big Bang Theory on CBS . What are your friends doing ? 8 05 ?????????????????????????? Kikwete Amy Smith likes Big Bang Theory . Jay Tavlor recommends Pretty Little Liars on ABC family . Anna Tuck participated in the AD - SURVEY survey on your favorite late night show . ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? Social 8 15 IT Prompts 8 20 wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww m 8 10 What do you want to say to your friends ? 8 15a John Smith likes the red Gucci bag in Big Bang Theory ( Tap to submit ) . 8 155

W

mm 8 150

* * * * * * * * * * *

* *

?????????????????????????????????????? QWERTY U TOP ASDFGHJKL © 2XCVBNM a . 2123 wetu


FIGURE DE

Wir

. . . . . . . .

. . . . . . . . Wwwwww John Smith is watching Big Bang Theory on CBS . What are your friends doing ? 805

' . ' . ' . ' . ' . ' . ' .

' . ' . '

. ' . ' . . ' . ' . ' . '

You said " I like the red Gucci bag in Big Bang Theory . " Learn more . . . 8 35 - - - Amy Smith Likes your post . 8 36a - - Anna Tuck commented on your status " is it the large red tote Sookie carried ? " 535b - - - Jay Taylor followed the link to learn more about the Gucci bag . 8 350

Social Prompts 8 20

It Channel Program 8 10

CBS ABC Family . . . ESPN

???????? . ???????????????????????????????????????????????????????????????????????????????????????????????????????????? ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? CBS ABC ESPN

Big Bang Theory 8 : 15pm - 9 : 30 pm The Pacific 9 : 30pm - 10 : 30 pm Pretiy Little Liars 8 : 00pm - 9 : 00pm Desperate Housewives 9 : 20pm - 10 : 30pm . . .

NBA Finals Live 8 : 00pm - 11 : 00pm CBS CNN

Allowing Ambient Monitoring


FIGURE 8F 111111111111 1111 : : : : : : : : : : : : : : : : : : : : : : : : : : : :

* * * * *

???????????????????????????????????????? ???? John Smith is watching Big Bang Theory on CBS . What are your friends doing ? 805

Amy Smith likes Big Bang Theory Jay Tavlor recommends Pretty Little Liars on ABC family . Anna Tuck participated in the AD - SURVEY survey on your favorite late night show ,

?????????????????????????? . .

Social Social Prompts 8 20

Channel Prograr Hey , are your interested in Sookie ' s White Framed sunglasses ? 8 20b Tap to check it out ! CBS

ABC Family . . . ESPN

CBS ABC

Big Bang Theory 8 : 15pm - 9 : 30 pm The Pacific 9 : 30pm - 10 : 30 pm . . . . Pretty Little Liars 8 : 00pm - 9 : 00pm Desperate Housewives 9 : 20pm - 10 : 30pm . . .

NBA Finals Live 8 : 00pm - 11 : 00pm ESPN CBS CNN CNN .

Allowing Ambient Monitoring


FIGURE 8G W

est

RB DRM SG 9 : 42 AM

XYZ - Designer Frenc Style Sunglasses Wome 8 603

MONNI . ANG 606 & N

DDD Red Polka Dots Bikini Halter Top

$ 59 . 99 uy Now !

Enter Your Own Rating : A * * * * * B . * * * * C . * * * D . * * ??? E? .

door

wwwwwwwwwwww

wenn man w wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww wwwwwwwwwww

90S

Next Page < < 8 52 < < Previous Page 8 51 See more interesting stuff in Big Bang Theory :

· View all ( 8 55a ) - View by Season Episode ( 855b )

- Please select : Season 3 - View by Character ( 8550 )

- Please select : Sookie • View by Item Category ( 855d )

- - Please select : Apparel , Accessories

Episode 8

?? ???????????????????????????????????????????????????????? I would like to search for : ( 858 )

FIGURE 8H

* *

* *

MMMMMMMMM * *

*

9 M WWW

atent

2

:

-

- -

-

- - -

* *

BRM

30

* MMMM * *

* * * * *

*

VVVUNUNLANANLANUL

*

*

Live News ( 862 )

All by Channel Line Up ( 860 )

Most viewed Now

CBS . 9 : 00 The Big Bang Theory 82 Users ( 860a ) > >

Oct . 31 , 2017

ABC . 8 : 30 Pretty Little Liars 23 Users ( 860b ) > >

CBS 9 : 00 The Big Bang Theory > >

Programs my friends watch ( 868 ) > >

Dance with Starts Entered the Top Programs List ( 861 )

Programs I am watching ( 866 ) > >

ABC , 8 : 30 Pretty Little Liars

Now 23 Users Watching ( 860b ) > >

?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

Sheet 48 of 65

Live ( 862 )

Guide ( 863 )

Remote ( 864 )

Settings ( 865 )

Live ( 862 )

Guide ( 863 )

Remote ( 864 )

Settings ( 865 )

22

:

22

. . . . Wwwwww

US 9 , 807 , 442 B2

???? 2

?????????????? ?????????????? ??????????

?????????????? ???????? ???????????????????????????????? AD - SURVEY Example Mobile Ul : Mobile TV Remote

AD - SURVEY Example Mobile Ul : Mobile TV Remote

US 9 , 807 , 442 B2

( 998 )

( 865 ) Settings ( 864 ) Remote ( 863 ) Guide ( 862 ) Live

Settings ( 864 ) Remote ( 863 ) Guide ( 862 ) Live

Ha XXX !

: : : . .

. : : :

:

:

Sheet 49 of 65

* *

*

*

7317314

W

( TLS )

BUNOH Disney

SM?U N

with

jitung

MD

CBS

NdS3

Oct . 31 , 2017

8700 8700

8700

870a 8700 WA

???

CBS

XO

( 873 )

wwwwwwwww

AW * * * * * *

WY

L

WAKYYYYYY

atent

X

* * * . X90X00 * 300X2000W

* *

* * * * XX

X XXXXXXX * * *

FIGURE 81

AD - SURVEY Example Mobile Ul : Mobile TV Remote

US 9 , 807 , 442 B2

( 698 )

( 865 ) Settings ( 864 ) Remote

Guide ( 862 ) Live

( 865 ) Settings ( 864 ) Remote ( 863 ) Guide ( 862 ) Live

PU

A SA mous

SCOWOO

. . " . " '

' ' ' ' ' ' ' ' ' ' ' ' ' '

' ' ' '

' ' ' ' ' ' ' ' ' '

' ' ' ' ' '

' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . ' . ' . . ' . ' , ' 1 ' , ' '

umum -

Boy 6 - Band

DVD ( 875e ) > >

Sheet 50 of 65

: ??? , ????? AFN Satellte cabw , Wanturvaton Smowa

TV ( 875d ) > >

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98104 4 ounibiN gy Amma $ e

A10

TIVO ( 875d ) > > Smart TV ( 875C ) > >

* * * * * * *

??????????????

Set - top Box ( 875b ) > >

20038

Oct . 31 , 2017

TV Provider ( 875a ) > >

WWW .

wwwwwwww

TV Providers ( 875a )

Settings

* *

. - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - .

36

DRM

W

28 %

W

atent

wwwwwwwwww

XV

* * * * *

*

* * * * * * * *

* * *

* * * * * *

X

XX . X .

* * * * * * * * *

.

*

* * *

X

. X

. XX

. X . XX

4

* * *

*

* *

*

*

FIGURE 8J

AD - SURVEY Example Mobile Ul : Social Profile

US 9 , 807 , 442 B2

. : . : .

: .

: . : : : .

: . : . : . : . : :

. : . : . : . 2988 . . . . . . . . . . . . . .

. . .

ritirir ;

688

MWWWWW 21 . 1 . . . . . .

.

. . . : : : : : : : : : :

. . . . . . . . .

: : : : : : : : : : : :

. . . . . ? : : : : : : : : : :

.

WWW

via AD - SURVEY 9 : 30 PM Today 9 : 00 PM - Theory .

@ The Big Bang @ CBS watching I am on channel

Bouw En 365

Sheet 51 of 65

268 8

S

.

LLLL

( 887

Check - in ( 886 )

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

e98 8

no proces

Discuss ( 888 ) > >

Today 9 : 00 PM - 9 : 30 PM CBS The Big Bang Theory

wwwwww 4 - 4 - 4 . 4 . 4 . 4 . 4 . 4 4 . 4 4 . 4 . . 4 . 4 . 4 . 4 . 4

" "

" " " " "

" " " " " " " " " " " " " " " " " " "

" "

Oct . 31 , 2017 Oct . 31 , 2017

Notisod Program I Am Watching ( 885 )

Program I Am Watching ( 885 )

* * * *

* *

* * *

* * *

: . - . - . - . - . - . - . - . - . - . - . - . - . - . . . . . . . ' . " , " . ' . ' . ' . ' . ' . - . - . - . - . - . - . - . - .

9 : 41 AM

XXXX

atent U . S . Patent

5

.

??

???? c

??????????????????????????????????????????????????????????????????????????????????

* * *

W

WW

* XXXXX *

* .

FIGURE 8K

AD - SURVEY Example Mobile UI : Social User Authorization

US 9 , 807 . 442 B2

s & & & &

h ???????

. ???? ????

??

.

??????????

;

??????

3

: : :

'

as

& 4 = ?•

EE " . " . " . " . " . " . " . " . " . " . " . " . . " . " . " . " . " . " " . " . .

" . " . " . " . . . " . " . . . " . . . " . " . " . . . " . " . . . " . . . " . " . . .

" . " . " . " . . . " . " . . . " . . . " . " . " . . . " . " . . . " . . . " . " . . .

' ' ' , ' , ' , ' , ' , ' , ' , ' , ' ' . " . . . . . . . . . 27 ' , " " " . . . . . . . . . . . . 6 " ' ' '

' ' . " . " , " , ' , ' , 5 ? 5 ? " . " . " . " , " , ' , ,

? 5 ? " . . . . . 5 ? 6

. . . ' , '

??

?? ??????

????

???????????? ???? ???? ?? "

30 ] G

"

???????? ????

" . " " " " " . " " " . " " " " " " " . " . . " " . " . . . " " " " "

Esf . . . . . . . . . . . 29935555

' , , " " , , " " , , " " , , " , " . " , " . " , " . " . " , " , " . " " " " " " " " " . " , " , " . " , " , " . .

. . . . . . . ' , , " " , , " " , , " " , , " " " " " " " " , " . " . " , " , " . " " " " " " " " " . " , " , " . " .

" . . . . . . . . . . . . . . . . . " . " . " . " , , ' , ' , ' ' ' , ' , ' ' , . "

28 , 318 ? ? 2 , 28 ( 2 )

4

?????? { ; 3 ? ?????? ??????

:

???? 3 ? . ????

???? ????????

" . . . . . . . . . . . . . " . " . " . " . . " "

. . 5 5 5 5 5 ' ' ' ' ' ' ' ' ' , . " . . . . . "

. " . . . " . " " . " . . " . " " . " . . " . . " . . . " . . . " .

- - - - - - - - - - - - - - - - - -

' , .

???? ?????????? . ?? ?? ?????? ????

" . . . . . . . . . . . .

" . . . . . . . . . . . ' ' '

. . . . . .

. . ' ' . .

. . ' ' . .

. . ' ' . .

. . ' ' . .

. . 5 6 . .

. . . . . . . . . . . . . . . . "

? 5 ? 4 ? 5 ? 4 ? 5 ? . . . . . . . . . . .

. .

" . . " " , ,

. . . . . . . . .

" 55 - ???? - 21

?? : : 3 ?? 4 : ?? . ???? :

?????? ?????? ?????? ????

. " , " , , " " , , " " , , " " , , " " , , " " .

" . . ' , ' , 15 5 ? 6 ? 5 . . " , ' , ' , ,

?

" , ,

??? . . ???? ! : ?????? " ?? ???? ???? : ?? 3 : : ???? 2 ?? ?????? ? ???? ???? ?? : : / ????????????

?? ?????? : : 2

: 4

???? ???? ???? ?? ?????? ? ???????? ?????????? ??

? , , " " , ,

" . .

* * * * * * * * * * * * * * * * *

Sheet 52 of 65

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * - - - - - - - - -

?? ??????????????

?? : 1 - / : ;

- -

- -

- -

- -

- -

- -

- - ???? ???? ???? ???? ???? ???? ???? ???? ???? - -

- -

- -

- -

- -

- -

- -

- -

???? ???? ???? ???? - -

?? |

: ???? ?????? ?????? ???? ???? ?????? ?????? ???? ” ?? ?????? ?? ???? 2 ) : ??

?? ???? : : ?????????? : ???? ???? ???? " ???? ?????? ???????????? . ???? ( ???? ) ? ???? ?? : ???? ?? ?? ?? *

rrrrrrrrrrrrrr???????????????????????????drrrrrrrrrrrrrrrr?????? ? ?? ?????? ? ???? i

?? .

832 - KB

?????? 332

f342 92 [ / vy

? * & & & & & & &

* * * * * * * * * * * * *

C {

*

. 2 ?? 7 ! "

???? ???? ???? ?? : ??? . ???? ?? : ?????????????????? : ?????? ???????????? " ????

?? ? 5 ? ???? ??????????

???????? : :

?? : - ?? ????

- ???? 2 : : ?? ???? ?? : : ???? : ???? : : : : ?? "

. ????

?????? ???? ???? : ????

RA2 ! 3 &

&

XX34 23 22 29 $

$ 3 &

X 24

25 23 :

*

= = = = = = & & &

? ???? ?? : ?? ????? : .

???? ???? :

*

* *

* * * * * * *

* *

Oct . 31 , 2017

You must log in first . ?? ????????????

) 1 ( AD - SURVEY is requesting permission to : ?????????????????????? ???????????????????

= = = = = = = = = = = = = = = 5555555258555555555555555555 % 92 % = " " "

w w = 4 +

??

alalg - -

A AT

28 :

88a - - 3 ???? ????

atent

?????????? + +

"

: : : : : : : : : : : : : : : ?????? . 44 + "

6994 * * * * * * * * * * * * * * , : : : : : : : : : : : : : : : : : : : * * * * * ; : ; ; ;

FCURE 2

FIGURE 9B

http : / / www . AD - SURVEY . com / user / management . html

Home > User Profile Settings > Social Media Access Control Settings

atent

Select Social Media ( 820 )

Login Confirmation 8 25

8 21a 8212

facebook

Oct . 31 , 2017

8 21b

Facebook requests your login details . Please enter your email and password :

wwwwwwwwwwwwwwwww

Social Analytics

Bring your friends and Info :

aceho

" ' . ' . ' . ' . . .

8210 : : . . .

. . . . : . ' : . . . , , ' , ' . .

Post stories to your Walk

. . .

wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

LLLLLLLLLLLLLLLLS

Sheet 53 of 65

. : Horvat

- " - - - - - 4 - - 4 - -

- - - - -

- -

- - - - - - - - -

- -

- - - - - - - - - - - - - - - -

- -

. .

Etial

wwwwwwwwwwwwwwwwwwwwwwwwwwww

tumblr .

8 210

PASSWORD :

. . . . . . . . . . . . .

na magigini , in

man na magagami

w

2

WW

WWWWW

Other :

Please specify : http : / /

Sign up Log Facebook

RECE

otvo

W

Send Request ( 825 ) Bare reset

Cancel ( 826 ) Caneca ( 120 )

US 9 , 807 , 442 B2

AD - SURVEY Example Web Ul : User Authorization

FIGURE 9C

http : / / www . AD - SURVEY . com / userimanagement . html

Home > User Profile Settings > Social Media Access Control Settings

atent

Select Social Media ( 820 )

Access Scope Configuration 8 27

821a

I allow AD - SURVEY lo view my : Everything m User Profiles 8 28 MUser Name D User Address 2 User Email DUser Phone Number DWork Education ODOB Pages O Groups ONetwork

See more «

Oct . 31 , 2017

821b 00 + 6216 8 210

Sheet 54 of 65

tumblr .

8210

Friends

829 Number of Friends ( 829a )

D Friend List

See more < <

Wall Posts + 8 30 My Post on My Wall ( 830a )

Sharing Liriks Photo Status 2 Messages My Post on My Friend ' s Wall

Only if the Friend Allows Access to his Wall ( 8300 )

D Linkes / Dislikes

Doniy if " liked " Item Owner Allows Access ( 830d )

o

Other :

Please specify : http : / /

Send Request ( 825 ) Send equest

Cancel ( 826 ) Cancel ( 820 )

See more «

US 9 , 807 , 442 B2

AD - SURVEY Example Web UI : User Authorization

FIGURE 9D

http : / / www . AD - SRVEY . com / client / edia analytics . htm

U . S . Patent

Home > Project > Media Analytics Reports ( 840 )

People 8 41

TV Shows 8 42 | Brands 843

Products 844

Media Source 844

850a

Amanda Morse Season 2 is definitely a huge thumb up ! !

27 minutes ago via Facebook iPhone App

What do people say about :

Please selecta TV neiwork : CBS ( 845a )

Julian Smith Penny sucks - she just can ' t read the lines !

27 minutes ago via Facebook iPad App

Please select a Genre :

Comedy ( 8450

Oct . 31 , 2017

Please select Show Name Big Bang Theory 6450 )

Lily Doyle I think Big Bang is the best production of CBS ever .

25 minutes ago via Facebook

OR , please specify : _

( 8450 )

Tom McDonald The theme song is pretty interesting - " the whole

universe started with a big bang . . . " Aha

25 minutes ago via Facebook Android App

See more «

Sheet 55 of 65

WHITE ?? ??? . ???????? ? ?? . ???????? . . ?? . ???????? . ?????????????? . . . ?????? ?????? ?? . ??????????

2

8500

I would like to view

Comments from : ( 848 ) Facebook Twitter @ Tumblr Google +

See more < <

Since : 09 / 01 / XXXX ( 849 ) 2 All comments texts ( 850 ) O Feedback Summary in ( 055 )

Plain Format ( 855a ) Chart / Plots ( 855b ) Table ( 8550 )

See more

" The Big Bang Theory is so awesome . Even I can understand string theory now ! Love Sheldon and Leonard . Expecting the new season . "

09 / 17 / XXXX 19 : 35 : 56

Lisa Smith 2222222222222222222222222224 US 9 , 807 , 442 B2

See more < <

AD - SURVEY Example Web UI : Media Analytics

FIGURE SE

http : / / www . AD - SURVEY . com / client / media _ analytics . html Home > Project > Media Analytics Reports ( 840 )

atent

People 8 41 | TV Shows 8 42

Brands 843

Products 844

Media Source 844

ga

What do people say about :

There are 20 , 898 posts from Facebook and 13 , 923 Tweets related to " Big Bang " since 909 / 01 / XXXX , including : 39 % positive , 38 % neutral ,

and 23 % negative .

See more «

Please select a TV network : CBS ( 845a )

Please select a Genre :

Drama ( 845b )

Oct . 31 , 2017

Audience Response

Please select Show Name Big Bang Theory ( 8450 )

860b

OR , please specify : _

( 8450 )

Positive Neutral Negative

I would like to view : Comments from : ( 848 ) Facebook

Sheet 56 of 65

Twitter

D Tumblr

B Google +

8600

See more

Since : 09 / 01 / XXXX ( 840 ) All comments texts ( 850 ) O Feedback Summary in ( 855 )

Plain Format ( 855a ) Chart / Piots ( 855b ) Table ( 8550 )

HBO True Blood Actors / Actresses Music

Positive 9 , 898 14 , 898 15 , 323 19 , 333

Neutral Negative 10 , 323 7 , 323 | 13 , 323 9 , 323 8 , 989 5 , 323 2 , 333 6 , 222

: 59 : 59 : 59 : 11

See more < <

US 9 , 807 , 442 B2

AD - SURVEY Example Web Wl : Media Analytics

AD - SURVEY Example Ul Flow : Media Analytics

Us 9 , 807 , 442 B2

10 450

P99 ,

10 45a

+ 38 44

10 15 : & & & & × } 44 ® .

0g . 44283 $ $ 24 % 363 ,

3 . 3434333 % 3 $ $ $ $ $ $ $ $ $ $ $ $

38 * { }

12 : 2a Report Ad Delivery

?e as atta :

"

"

Sheet 57 of 65

* 2255 * 2 * 22522722722 JAMJMJMX2 - JMJAJJAYAMA + JAYAMAYA

83100324 ? 1033330353

131131373337332333303303337333 ? 773311524133

$ $ 354338 382 33 33 % % %

{ $ 25a4 323583 % % # * - - - * - -

- * * * - - - * - - * 1 .

30 35

- 10 32

" APPAAPPALAPAM = = = = = = = = = = = = = = = = = = = = = = = = = = =

= = =

= = =

= = =

= = = =

= = = = = = = = = = = + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ + +

+ + + + + + +

+ + + + + + + + + + + +

??????????????•

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - .

# # # # # # # #

# # # # # # # # # # # # # #

# # # # # # # #

# # # # # # #

????

?? ????? ??? ????? ??? ??? ??????

• . • . • * . * . * . * . * . * .

{ { 323 ; 8 : 37 % 42 43

Oct . 31 , 2017

S + _ 40d a !

* . * . * . - * - - - - - - - - - - - -

Dashboard

# # # # #

# # # # #

# #

# # # #

# # # #

# # # #

# # # # # # #

rrrrrrrrrrrrrrrrrrrraz

p ! ]

8 0 .

|

•AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP #

# # # # # # # # # # # #

Welcome / 10gin Screen A

· 11 405

rrrrrrrrr r = r rr rrrrrrrrrrrrrrr :

1 , 2 , PP , PP , , PPP , PH , PP , P2

U . S . Patent

FIGURE 10A

a | L ] asnbar { { { ; 02

EY Exart

AL - S { {

US 9 , 807 . 442 B2

??

.

"

????

????

RE : P

26

09 /

act

su :

?? i l

92

??????

?????? ???????????? ???????????????????? ??????????????????

??????

????????

?????????????????????????? " ??????????

????

?? 2 % 21 ?? ? ??????

22

k25 ?????? 8 ???? ?? 5

. 3

F8

1 [ ] 3 )

14 2

?????? ?? ????

?? ???????? = = = = = =

??????

. ? . ? . ? . ) ????

???? ?????? ?? . ?? ?? ??

. . " . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 24

?????????? ?? ???? . ???? . ???? ?????? ???? ?????? * *

*

*

* * *

???? ?????????? ?? ??????

[ 2 - ????? pxc = ??????

.

: ???? ???? ???? ?? "

-

? ?????? ???? " : 5

: 22 "

???? ???? ?? : 2 ?? ?? " ?? ?? : ?????????? ???? ????????????

??

?? % 25 ????

· 12

- { { 25

Sheet 58 of 65

g } 25

???????? *

* * 2

i $ 4 ,

844f

???? 8FYx

FY ! A

$ f 8

* : . . . . . " " " . " " . . . " . . . . . . . " . " " . . . . . . : . . . " . . . . . . . " . " . " . . . . . " " . . " 1 . 9 . . . . . . , " . . . . . . . . . " 1 . 9 . . . . . . . . " . " " . " " . . . . . . . " " . 9 . . . . . . . . . . " . " . " . . . " . . . " . . . " . " : . . . . " . " . " . . . " . . . " . . . " . " . " . . . . . . " . " " . " . . . . . . " . . . " . " . " . . . . . . . " . . " 1 . 9 . " . . . . . " . * * . . . " . . . " . . . " . " . " . . . . . . " . " . . " . 5 " . " .

* " . '

" . . " 1 . 9 . . . . . . . " . " " . " " . . . . .

. . 9 . . . . . . . " . " . " " . " . " . . . . . . " . " " . 91 . . . : :

t

2

?? }

??????????? :

??????????? / ?? / ???????????????????????????????? 2

/

2 = ??????????????? . ? ) ???? ? " ???

: ?? # $ ; $

???? 8 ?????? " ???????? ??

r

???? /

???????????????????????????????? .

/ ????????? ?????????

? ? .

f a $ 12

" . " . . . . . . " " . " . " " . " " . " " , " . . . " " . " " . " " . " " . . " " . " " , " . . . " " " " . " " . " " . " " " . " " " " . " " . " " " . " " , " . . . " " " . r :

fj45 46 4

???????? ?? " ?? ????????

/

??????

?? ??????

0

0284 ????? 41 ???? " =

? r ?? /

0321

; & AA

Oct . 31 , 2017

?????? 4 / r & & &

326c63 &

jri ??

???? ? 95 . ????

4 : 43 ;

| ! ??

???????? ???????????? ?? ???? : A

•

/

200 [ 56 ] Z QL

?

" = = = = = = = = = = = = = = = = = = = = = = = = = =

{ } 2

. ?? J } $

? 688 336 3

???????????? Reports

akhmoard 2

4

422 21

U . S . Patent

3244323 :

38Xk4i ???? ??

& 2 } } } A & ; 24344 ?? ???????? & 42

: 2

:

33 :

213 ?

0249 283

282 ?? % 202848

F : : - - - - - - - - -

- - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - -

- - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - -

- - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - -

- - -

- - - - - - -

- - - -

" . " . " " " " " " " " " " " " " " " . " " . " . . . " " . " " " . " . . " . " . . " . " . . . " . " " . " . . " . . . " . " " . " . " . . . " " " " . " " . " . "

. - - : . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. - -

- - - - - - - - -

- - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - -

F : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

@ D

@

ri

3

ESTO AIRILLIPS Sale | Y

B EX A AAPS

. . .

.

. . . . .

. . . . . . . .

. . .

.

. . ' ' '

'

'

' '

' ' '

'

' .

. . . . . . .

. . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

?

?

:

: : : :

: :

* . ' ' ' . ' , ' . . ”

' ' ' ' ' '

:

' , ' ' " : "

"

: :

" : " : " " ' - ' * ' : : : : " . " . " . " . " . " ' . ' , ' ' , ' , ' ' , ' , ' , ' ' , ' , ' , ' ' , ' , ' ' ' ' ' : ' . * . * . * . * * . * . . * * : : : : : : :

US 9 , 807 , 442 B2

??? ??? ??? ??? ??? ?? ,

?? ?? ?

. .

. . . . . . . .

.

. rr . … .

- r .

. . . .

: :

. . .

.

. . . .

.

. … . … . . . . .

?

383

?

?

?? ????????????????????????????????

… … … … … . . .

??? 407

??

:

:

:

? :

: . : .

? ??? ??? ?? ?

??

200838 ???

? 33 ?

:

: : : : :

?

?

. . … . … . …

? ? , ??? , ? : ( ??? ??? , ?? , , ??

???? : ??

?? , ??? :

? ?????? ? ???? ,

??? ,

???? ??? ?????

?

?? ?? , ? ?? ? ?? ???? ????

???

??? : ????? ??????

? ” : ????? “

?? ”

?????

???? · ??? ~ ?? ?????

?? , ??? ?

??

???? ?????

????? : ?????? 83? , ???? ?

?

, ???????

?? ?????? ?? , ???

???

??? ????

?????

?? , ????

????

????? , ??? ? ?

??? ????

??

, ?? , ???

????

??? , ??? : ????

? , ???? ??

?? ? , ??

?3? ??? , ??? ” ???? “ ? ?? , ?? ?? “ ? ?? ?? ?????? : ?

????

??

?? ( ??

? ??

??????

?? , ?? ,

??? ?

?? ?

?

· ???

??

????? ????

? :

??

???????? , ??? ????? ?? , ????? , ?????

?????

?? , ?

???

3 ?

??? ”

? , ??

?? ??? ??

… … . . . . .

|

: : . . .

?? ??? ? ?? 1982 ???? ?? 2008 ??

?

?

?

. . . . . … . . .

. . . .

.

.

' ' ' ' . . ' : ' ,

" " , " ,

" s ' . . ' ,

. . . . . . . . . . .

.

.

. . .

Sheet 59 of 65

?

??

?? ?? ? ?

? ? ??

??

????

???

' ' ' '

' ' ' ' . .

" } ,

???????

?

“ ?

,

?

??

?

??88

. … . . . .

- - - - - - - - -

. . . . … .

.

?

,

?

? , ? , ??

. . .

. . .

. . … …

.

…

????? …

??? ?? , vier . . … . . . . . . . … . / v . tr . .

. . 1rrr .

? , , ?????????? ?? , ???

in . rr . . . .

.

… …

'

' ' ' ' ' ' '

. . .

.

. .

. - ri

… . . . . . . . . . … … . …

.

. * . * * . … … … … . ' ' '

' ' '

'

riti . . . . rrr .

.

. . .

. …

Oct . 31 , 2017

? , ?????????? ?? ?? ?? ??

201 -

? ,

FILE . . .

?

KNS )

atent

" " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " " . … .

Prevvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv : : : : : ??? , ??????

2013

ISLA Japala Jad isley OURITTY

FOLIEX AAAATS - TV

US 9 , 807 , 442 B2

???? ???

“ .

?

? ?? 1983

.

??

. . . . . .

?

: ?

“

* * * *

* *

?

? ·

? ?? ??? ??? ???? 8?? ? ?

?

?

. .

.

.

.

.

?

?? ? , ??? ?? ? ?? ???

. . . . .

?

?

?

.

. . " , " , " i ' - ' . ' : ' . . . . . . . . . . . . . .

. . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . … . .

.

. . . .

. . . . . . . . . . . . . . . . . . . .

.

…

…

: : : ' ' ' ' ' ” , ” ” , ” ” , ” ” , ” . . . . . . . . . . . . . . . . . . . . . . . . . . . .

* - - * - : : : : : :

??? ??? ??? | ? ?

?85 %

??

? ?? :

:

:

2012 - was anagenews ?? :

“ ???

?? “ ?

. . . . . . .

.

.

.

.

. .

?

??? ???? , ??? ???? ??? “ ??

??? , ??2?

?

?35 ? ?? , ?? ,

? : ?? ??? , ??? ? , ? ,

??

, ?? , ?? ?88 888 ?? , ???? ? , ?? ??? 30 ?

? ,

?? ??? ?

?? : 18 ?? ?SS ??? ?

?

??? ( 47?

??

????? ,

.

?

?? , ? ??

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . … . . . . . .

. . . . .

2018?

. . . . … . . . . . … …

. . . . . . . .

. . … . . . … . .

' '

' . . . . . . . . . . . .

* - * - * - * - * - * - -

. . . . . ' ' ' '

' ' '

' . ' . ' : ' i

s ' s r ' , -

“ . . . . . .

.

. . . . .

. . . . . . . . . . . . . . . . . . ”

•

r ' s c ' : ' . ' .

. . . … . .

… … … . . . . . . . .

8

? ???

:

:

:

" - " . . .

Sheet 60 of 65

?????

??? ?? ?? ??? ???

“

… .

. . . . . . . . . .

" . .

. . … . .

.

. …

. . . .

. . . . … .

Fr . . . . . . . . . . . . . . . .

. . .

. . . . . . . .

. .

… … . .

. . . . . . . . .

" " " " " . . . . . . . . . .

. … .

. . . .

" " " "

" " " " " , . .

.

. … . . . . . . . . . . . . . . . . … . . . .

. . . !

! . . . . . . . ' ' ' . . … .

?? ?? ??

??

?? ?? ,

?? ,

?

???

???

" . . . . . … rr .

' ' ' ' - fir ,

? ” ??

?

. . . . . . … . . .

. . . . . . . . . ”

.

.

.

. . . . . . . . . . . . . . . . . … .

.

… . … . . . . . . . . . .

. . . .

. . . . . . . . . .

. . .

. . . . . . . . . . . . . . . .

.

. . . . . . . . . . . . . . . . . . . . … . . … . . . . . . … . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . .

??

????

?

?

“

. … . .

. .

: : : . . . . .

.

??? ??

' ' . . .

.

“

… . … .

.

… .

?

??

?? ,

. .

. . . . . . .

. . . .

. . ”

? ?? ??

??

?? ?? ?? ?? , ????

?

?? ? ??? ? · ? ”

' ' ' ' ' '

:

?

?? ???

?

? . .

. … … . . …

. …

… .

- . . . .

. . . . . . . . . . . .

?

?

?

?

Oct . 31 , 2017

: : : : : :

:

:

: : :

: : : : : : : : : : :

:

10

, 0008

atent

2012 2323??33? , 38068288x , ?? ??? , ?

. . : : :

. .

. . . .

. . . . : :

. . . .

. . . . : :

. . . . : : :

. . . .

. .

. . . .

. . . . :

. .

?? ?

. . . .

. . :

. . . . :

. . . .

. .

. . . .

. . . . : : :

?

,

? 1983 ? ?

? ? : :

.

: :

. . . :

. . . :

. . . .

:

. . .

: :

. . .

. . . .

. . :

. . . :

. . .

. .

. . . : : :

. . .

. . . :

. . . : : : :

. . .

. . .

. . : : :

. . .

. . . :

. . : :

:

:

:

:

:

:

: : :

: : :

:

:

:

:

book BE

?

AD - SURVEY ExaInpe | Audience Report by Age 1015

.

.

…

.

US 9 , 807 , 442 B2

?? ,

. . . . . . . .

“ . . . . . . . .

.

.

.

. . . . . . . . .

“ . . . . . . . . . . . . . . . . . . .

. . . . . .

. . . .

?? ?? , ? ?

: : : : :

?? ,

,

?

?

??

* * . * * * ?

. . . . . . .

. . . . .

?? :

? ,

???

: . . : : : : : : : : : :

? ?

?? :

. .

* * t " : " . . .

.

. . .

. .

: : . .

. " " . " . . . . . .

: " . " . " . " . " . " . _ . ' ' ' ' ' ' ' ' ' ' . . . . . . . . . . . .

.

. . . . " " . " .

" . " . " .

??

??? ??

" "

" " " " " " " " " " " " " . " . " . " . "

?

?? ???

2008 · ? ??

:

??? ?

|

?? : ?? :

. . . … .

: : : :

??

?? ??

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

??

?

? ???

? ?? ?? ????

??

???? ?? ?? : : : :

. . . …

?? ???

.

?? ?

881 :

??

: : : : :

.

? …

PPPTPPP

?

NS?

??? ?? ???

?? ???

:

?

? ?

??

???

??

,

?

??

????????????????????????????????????

? : ???

.

. . .

. . .

.

lead . . .

' ' . . ' ,

' . . .

??

??

???

?? ” ? ???

??? ?? : ???

. . . . . . . . . . . . . . . . .

? ,

??

Sheet 61 of 65

?? , ? ???

?? , ??? ?

???

???

. . . . . . . . . . . .

?? ??

. . . . . . . . … . . . . . . . . .

…

.

. . . . . .

. . . . . .

. .

. .

. . . . . .

. .

. .

… … . . . .

… . . . . .

… .

. . . . . . . . . .

.

…

. .

.

?

“

???

??? ?? ??

???? ?? ?52 ??? ?? : ???

?

???

? ? ? “ ??

?? ?? ???

? , ??

? “ ? ??

. . . . . . .

??

? ?? , ?

?

?? ,

????? ???

“ ' ' ' ' '

' ' ' ' ' -

-

.

. . . … … . .

" " " " " " " " " " " ' ' ' '

,

?? :

“ ?

, ,

???????

-

- . .

?

???? : ?? ?? : ? :

?

?

” ? :

| ?? :

iiiii , , , , ,

???? . . . . … . . .

. . . . . .

. . . . . . .

.

. . .

- i ' , ' , ' ' ' ' ' ' . . . . . . . . .

. .

. . . . . . . . . . . . . . . .

.

. . . . . . . . . .

. .

. . . . . . . . . . . . . . … . . . . . . . . . . . . . . . .

???

Oct . 31 , 2017

?

10 . 1 % ?

Reports

Dashboard

atent

?????8883793 201280

- - - - -

. …

…

333?

?

|

AD - SLIRVEY Examples : Airience Reporthy Day of hreek 2015

US 9 , 807 , 442 B2

??? ?

??? ????? ? .

??

?????

?

?

FTTTTTTTTTTTTTTTT PPTTTTTTTTTTTTTTTTTTTTTTT

YYYYYYYYYYYYYYYYYYYFFFFFFFFFFFFFFFFFFFFFFFF

??? ??? ?

?

“ ?? :

???

?

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' '

' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . .

?? ?

?

?

? ? ”

: ?

… … .

??

?

??

??

| . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . .

. . . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' '

' ' ' ' ' ' ' ' . . . . . . . . .

. . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

? ? ? ?

??

???

?

?? ??

???

?? ??

?

?

??? ??????

Sheet 62 of 65

? ? ?

?

??

??

?

|

;

?

?

?

?

? rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrry rrrrrrrrrrrrrrrrrrrrrrrrr

??? ?

?? :

Hulu O Neflix

?

??

?

F ' . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . . . . . . . . . . .

.

… . … . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' '

? ?

???????? ?????????? ?? ?????? : ????

. . . . . . . . . . . . . . . . . . ' ' ' ' . . . . . . .

2 /

? :

??????????

an116?

?

?

?? ?? ?

? ??

32

" . " . " . " . " . " . " . " . " . " . " . " . " . " . _ . "

…

… … . … . … ”

???????????? ?????? . ??????

?? ??

' . . ' . . ' , . . ' . ' ' . ' . ' ,

. . . . . _ . " . " . " . " . " . " . " .

" . " . " . " . " . " r . . . . . . . . . . . . . . . . .

? ? . . . . ' . ' . . ' . ' . ' . ' . . ' . ' . ' . ' . . ' . ' . . ' . '

Pandora M Twitter a Facebook

. . . " . _ . " . _ . " . . . . . . " . " . " . " . . . "

. . . . . . . . . . " ,

" . " . " . " . " . " . " . " . " . " . " . " . " . " . " . "

|

??

?

Oct . 31 , 2017

?

?

ddy

Reports

Dashboard

atent

. . . . . . . . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . - . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

- - - -

- - - - - -

- - - - * - * - * - * - * - * - * - * - * - * - * * * * - * - * - * - * - * * - * - * * * - * * - * * argetPartPart Part Part Part Part Parrary ret freet of ret after of rest of arrier of arry of are true arrier at argetPers arre arre arre arr = = r = = r = red > =

=

= = =

=

=

=

= =

= =

“ . . . . . . . . . . ” ?

/ / r = referefere effere - -

- - - * - * - * - * rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr

. . . .

.

. .

. . . . . . . . . . . . . . .

.

. . " . " . " . " " "

" "

" " . " . " . " .

“

??????? “

?

.

. .

. . . .

… . .

- - - - - - - - - -

- - - - -

-

- - - - -

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . - . - . - . - . -

.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

F . . . . . . . . . . . . . . . . . . . . . . . .

MR - Contoi Media Analytics

.

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

. .

:

:

:

:

:

:

:

:

:

:

:

:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

.

.

* * .

F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CLR ”

?????????????????????????????????????????????? ???????????????????????????????????????????????????????????????????????????????????????? 888888888889 ???????????????????????????????????????????????????????

?????????????????????????????????????????????????????????????????????????????????????????????? . ????????????????????

sipic gd inded ause IT ULELEX AJAHTS - 17 2 :

23 : 22

???

??

??

? ??

?

US 9 , 807 , 442 B2

' .

' ' ' ' ' ' ' ' ' ' ' ' ' .

. . . . . . . . . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . . . . . . . . . " . " . . . . . . . . . . . . . . . . . . . . . . . . . . … . . . . . . . . . . . . . . . .

- . . . . ' , ' . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . ' , ' . ' ' ' ' .

' ' ' ' ' . .

. . . " " " " . " . " . " . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

' . ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . . . . . . . . . . . . . . . . . . . . ' '

' ' ' . . . . . . . . ”

| : ? ??

. . , , , , , ' ' ' ' ' . : . . . . .

rilllllllllll ,

:

.

.

?

?? ??

. . .

:

. . . . . .

. .

. .

' ' ' ' ' ' ' ' ' " " , " , " , .

.

. .

. : * : " : " : " : " : ' : ' : ' : '

.

… .

…

…

…

… … …

.

. . . . .

. . . . * : ' ' ' ' ' ' ' ' ' ' ' ' ' ' . . . : : : : : : : : " " " " " " " " " "

. …

?? ?

??

… . . . .

. . . . . . . . . . . .

. .

: : : : : : . . . .

. . . . . .

|

?? ?? , 2016?06020 2012

? .

.

: ?

FirlFirl ' s

tr . ririririririrt

. . . .

· ??

. . . .

: : :

?? ?

:

. .

?

?

ni ,

?? ?

S

. .

:

. - -

?

?

:

?? ?? ? ??

?

?

?? ??

?

? ?

?? ??

?? ?

?

S

: :

.

?????

??

?

?? ?

. .

:

?? ?

. . . . . . . . . .

. .

:

?

?

??

Sheet 63 of 65

.

: :

?? ?? ? ??

* - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * . r ' . ' - ' . * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * . . . . . . . . . . * . * . * . * . t ' s cret ' s ' s ret ' s r ' s ' s ' } - "

r "

r " :

" ,

"

"

' - ' . . . . . . . . . . . ale " : " e " : " e " : " " , " p " : " " , " - " , " - " : " . " . " .

" - - - - - - *

… … .

. .

:

· ·

· · · · · · · · · · · · · · · · · · . … . … . … . . . . . … . … . …

. . . . . . . . - - - - - - - * - * - - - - - - * - * - * - * - * - "

r ' . . ' . . * . . * . . * . * :

: ' ; ' ' = = " - " : " " , " - " , " - " , " - " , " - "

' . . . . * . . . * . * . * . * . * - * - * - * - * - * - * - ' - ' s ' = ' ' ' ' ' ' , ' - ' , ' , " - " , " - " ,

?? ??

??

??

:

?

{ }

}

* - * - * - * ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . …

. .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. …

. . . . . . . .

. . . . . . . . . . . . . . . . .

- . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

:

?

??

??

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

:

??? ?? ;

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . … . . . .

. . . . .

. . . . . . .

. . . . . . . .

. . . ' ' ' ' ' ' . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . .

.

. . . . . . .

.

?

.

:

. .

?

S : N . W : * * k : www

. .

:

' . ' . ' . ' . ' . . . . . . . . . . . . . . . .

. . . . . . . . '

' . . ' . . . . . . . . . . .

' . ' , ' , ' , ' , ' ' ' ' ' ' ' ' ' ' ' ' ' ' .

. . . . ' ' ' ' . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . .

. .

" . . . . .

. . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

??

?? . … . . . . . . .

. . .

???

? ??

: ? :

. . . . . . . . . . . . . . . .

?

??

STEP170? 210yuez S1314 02

… .

. . . . .

??

Fre are

??? ?? ?? ???

. . .

Oct . 31 , 2017

*

¥ :

? ???

??

???? ?3X83

atent

???????????????

3 23288

88332385327 2838383 8

: 38

-

: :

-

: :

-

: : : :

-

. . . . . . . . . .

:

:

:

:

: :

:

:

: :

:

: : : :

: :

: : :

:

:

:

:

:

:

: :

:

:

: :

: : :

:

: : : : :

:

:

sor

orang

US 9 , 807 , 442 B2

4 .

WWW 999999999999 XXXXXXX KKKKKKKKKKKKKKK

800L pueoqysed In edwexg ABAUNS OY

ped ! Sue ) 210601

LIM

??? ! ? 44oqu?????? ?LH lew9 snbbg lepunah

OC OT

0310

?????? ???? weds yoogen szonpold Sdou

OG OL

ejollow

VOZUSA

2015ea1 ! 4M

elokol

aeg MOYS ADS COD

6VOL

al2009

128121

SOBE BEST

eqiysol

OmQuay

Samo u ovom

87 OL

29 OL

algowi

00019 df

1980el

SEASON 3D

W OL

sinuog umuna

syunqueis

S OL

OWOH

MW8

MORES CON

9V OL

addy Spueng op dong To

Choo

T709

BE

SH

sorozat

DOL

LI

mama preko msson

$ VOL

SZI OOT % SC % OZ

293233SC

ZOL ZHOI —

% 5

pate

% OC

5323uantes

1D OL

Venous nous

MASOL 1999

Sheet 64 of 65

WWW Mimmi

wwwwwwwwwwwwwwwwwwww

ery

OOOOOOOOOOOOOOOO & S OL

WA QUIN 3103

Will

3

Wwwwww

w www

wwwwwwwww toooooooooooo w

XXXXXX :

:

XXXXX : :

:

???? ??????????????????????????

50

* mmmmmmmmmm

G

ooooooooo BRANT

what 2 :

????

???? ??

???? ?? ?? ?????????? ?????? ?? ?? ??

Organo ?????? . ???? ???? ???? / 5

2 : 37 : : 13

:

:

????????????????????????????????????????????????????????????????????????????

Oct . 31 , 2017

OL : 13 : 73 : 27 : 31331332 3717 ? wowwwwwwwwwwwww Wi

% S

2 3

:

?

?

?

?????? ?????? ???? ??

M

MHHHHH

RE

AW

WA

WWW

octu s

ult

WIN

O

preoqused

atent

mun

^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ XXXX

XXXXXXXXXXXXXXXXXXXXXXX7777777777777777777777777777777777 777

7 77777777777777777

7777777777777 7777777777777777777

ur soigneus esawu usefuosovo - W

M

: :

3 : 23 : 25 :

: 2013

1 : 32 :

13 : 33 132 1333333333333333333333333333319 " 20 : 23 : 222 * 22372354241631332333333333333333333333333333333

HOL 38091


FIGURE 11 Computer Systemization 11024

000000000 1175 Clock 11 30 00000000 oooooooo

ser Input Devices 1111

Peripheral Device ( s ) V 11 12

CPU pery 11 03 Cell

000000008 GPS , etc 11 . 1

System Bus | 1104

Crypto Processor Interface 1127 Input Output

Interface ( 10 ) 1108 Interface Bus

11 07 Network Interface

11 10 Storage Interface

1109

- Crypto Device 1128 Power

11 86 - -

+ L . . Communications Network 11 13 RAM ROM Crvoto 0 : : : D

D

D

D Client ( s ) 11336

User ( s ) 1133a

O

. : B D

- CPO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - .

-

-

. -

. -

- .

. -

. -

- .

-

- . Storage Device 11 14

-

-

-

-

-

-

-

-

-

-

-

. -

-

. -

-

-

. -

. - AD - SURVEY component 1130 .

-

-

-

-

-

-

-

-

-

- OLEDODDORDEID . -

-

-

-

-

-

-

-

- O

-

-

-

-

-

-

-

-

-

P -

-

-

- POU -

-

-

-

- P

-

AD - SURVEY Database Media Analytics 11 47 11 19 Social Media Connection 1146 Audience Stat Analysis 11 45 Atmospherics Analysis 11 44 User Accounts Real ime TV Media Program Ad Survey Synchronization 1143 11 19a 11419b | | 19c

Real Time TV 1142 Survey | Atmospherics Ad 11 19e 11 19d 11 19 User Device Reports Social Content

11 190 1 16 11 191 Taxonomy | User Token Analytics Crypio Sr?r 11 20 Mail Client 11 22 11 191 11 19k 11 191 Mail Server 1121 Web Browser 11 18

Info . Server 11 16 User Interface 1117 Operating System ( OS ) 11 15

-

- U

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

PELLE - - - - - - - - - - - - - - - - - - -

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

- Memory 11 29 -

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . * * * * * AD - SURVEY Controller 1101

US 9 , 807 , 442 B2

5

MEDIA CONTENT SYNCHRONIZED The entire contents of the aforementioned applications are ADVERTISING PLATFORM APPARATUSES herein expressly incorporated by reference .

AND SYSTEMS FIELD

RELATED APPLICATION The present innovations are directed generally to media

This application is a continuation of and claim priority control , and more particularly , to MEDIA CONTENT under 35 U . S . C . $ 120 from U . S . application Ser . No . 14 / 621 , BASED ADVERTISING SURVEY PLATFORM APPARA 346 titled “ Media Content Synchronized Advertising Plat - TUSES AND SYSTEMS . form Apparatuses and Methods , ” filed on Feb . 12 , 2015 which is a continuation of and claims priority under 35 BACKGROUND U . S . C . $ 120 from U . S . application Ser . No . 13 / 341 , 076 A home TV user may view TV programs from a plurality titled “ Media Content Based Advertising Survey Platform of channels . The user may operate a handheld remote TV Methods , " filed on Dec . 30 , 2011 , which in turn claims 15 controller sold with the TV set to select TV channels . For priority under 35 U . S . C . $ 119 for U . S . provisional patent example , the user may push buttons on the remote controller application Ser . No . 61 / 504 , 913 , filed Jul . 6 , 2011 , entitled to switch channels , turn up / down audio volume , power “ Mobile Remote Media Control Platform Apparatuses , on / off the TV . Merchants advertise their products to attract methods and systems . ” consumers . Thus the TV audiences may interact with the TV

This application for letters patent disclosure document 20 and select a desired channel without physically touching it describes inventive aspects directed at various novel inno via operating the remote TV controller . vations ( hereinafter “ disclosure ” ) and contains material that is subject to copyright , mask work , and / or other intellectual BRIEF DESCRIPTION OF THE DRAWINGS property protection . The respective owners of such intellec tual property have no objection to the facsimile reproduction 25 The accompanying appendices and / or drawings illustrate of the disclosure by anyone as it appears in published Patent various non - limiting , example , innovative aspects in accor Office file / records , but otherwise reserve all rights . dance with the present descriptions :

The instant application is related to PCT international FIGS . 1A - 1E show block diagrams illustrating example application no . PCT / IL2010 / 000918 , publication no . embodiments of AD - SURVEY ; WO / 2011 / 055365 , filed Nov . 7 , 2010 , entitled “ System And 30 i 30 FIGS . 2A - 2B provide data flow diagrams illustrating data Method For Mobile Computing Transmission On A Network flows of TV remote monitoring within embodiments of

AD - SURVEY ; Of Data Associated With A Television Display . ” The instant application is further related to U . S . applica FIGS . 2C - 21 provide logic flow diagrams illustrating logic

flows of TV remote monitoring within embodiments of tion Ser . No . 13 / 341 , 400 , filed Dec . 30 , 2011 , entitled ( 35 AD - SURVEY ; “ Mobile Remote Media Control Platform Methods ” ; U . S . FIG . 3A provides a data flow diagram illustrating data application Ser . No . 13 / 341 , 857 , filed Dec . 30 , 2011 , entitled flows of media content based advertising within embodi “ Mobile Remote Media Control Platform Apparatuses and ments of AD - SURVEY : Methods ” ; U . S . application Ser . No . 13 / 340 , 933 , filed Dec . FIGS . 3B - 3E provide logic flow diagrams illustrating 30 , 2011 , entitled “ Audience Atmospherics Monitoring Plat - 40 media content based advertising within embodiments of form Methods ” ; U . S . application Ser . No . 13 / 340 , 953 , filed AD - SURVEY ; Dec . 30 , 2011 , entitled “ Audience Atmospherics Monitoring FIG . 4A provides a block diagram illustrating a client Platform Apparatuses and Systems ” ; U . S . application Ser . mobile component within embodiments of AD - SURVEY ; No . 13 / 341 , 036 , filed Dec . 30 , 2011 , entitled “ Media Con FIG . 4B provides a combined data flow and logic flow tent Based Advertising Survey Platform Methods ” ; U . S . 45 diagram illustrating client - server interaction within embodi application Ser . No . 13 / 341 , 076 , filed Dec . 30 , 2011 , entitled ments of AD - SURVEY ; “ Media Content Based Advertising Survey Platform Appa FIG . 4C provides a block diagram illustrating AD - SUR ratuses and a Systems ” ; U . S . application Ser . No . 13 / 341 , VEY table top unit infrastructure within embodiments of 096 , filed Dec . 30 , 2011 , entitled “ Media Content Synchro AD - SURVEY ; nized Advertising Platform Methods " . U . S . application Ser 50 FIG . 5A provides a data flow diagram illustrating data No . 13 / 341 , 118 , filed Dec . 30 , 2011 , entitled “ Media Con downloading from social media within embodiments of

AD - SURVEY ; tent Synchronized Advertising Platform Apparatuses and FIG . 5B provides a block diagram illustrating infrastruc Systems ” ; U . S . application Ser . No . 13 / 341 , 137 , filed Dec . ture of a media measurement portal within embodiments of 30 , 2011 , entitled “ Social Content Monitoring Platforms “ 55 AD - SURVEY ; Methods ” ; U . S . application Ser . No . 13 / 341 , 165 , filed Dec . FIGS . 5C - 5D provide logic flow diagrams illustrating 30 , 2011 , entitled “ Social Content Monitoring Platform obtaining social media content within embodiments of AD Apparatuses and Systems " ; U . S . application Ser No . 13 / 341 , SURVEY : 183 , filed Dec . 30 , 2011 , entitled “ User Impression Media FIGS . 5E - 5F provide example data records illustrating Analytics Platform Methods ” ; U . S . application Ser . No . 60 data structures of social media data within embodiments of 13 / 341 , 199 , filed Dec . 30 , 2011 , entitled “ User Impression AD - SURVEY : Media Analytics Platform Apparatuses and Systems " ; U . S . FIGS . 6A - 6F provide logic flow diagrams illustrating application Ser . No . 13 / 341 , 217 , filed Dec . 30 , 2011 , entitled social media content taxonomy within embodiments of “ Mobile Content Tracking Platform Methods ” ; and U . S . AD - SURVEY ; application Ser . No . 13 / 341 , 254 , filed Dec . 30 , 2011 , entitled 65 FIGS . 7A - 7F provide example data flow and logic flow “ Mobile Content Tracking Platform Apparatuses and Sys - diagrams illustrating cross - channel data collection of media

analytics within embodiments of AD - SURVEY ; tems . ”

US 9 , 807 , 442 B2

FIGS . 8A - 8K provide screen shots illustrating user inter - program table to obtain information with regard to the faces of a mobile client component within embodiments of advertisement streamed during the intervals of the TV play AD - SURVEY ; “ The Big Bang Theory ” and / or the product placement adver FIGS . 9A - 9E provide example screen shots illustrating tisements tagged in the TV play “ The Big Bang Theory ” on

user interfaces of media analytics within embodiments of 5 “ CBS . " AD - SURVEY ; In another implementation , the AD - SURVEY may parse FIGS . 10A - 10H provide example block diagrams and commercial ad information retrieved at and generate prompt

exemplary screen shots illustrating cross - channel media questions , surveys , and / or the like 408 . For example , if the analytics within embodiments of AD - SURVEY ; and AD - SURVEY determines the user is supposed to watch a

FIG . 11 shows a block diagram illustrating embodiments 10 series of “ Audi ” commercial during the show “ The Big of a AD - SURVEY controller ; Bang Theory , " the AD - SURVEY may prompt a survey

The leading number of each reference number within the including questions with regard to automobile purchasing . drawings indicates the figure in which that reference number In another implementation , the AD - SURVEY may keep a is introduced and / or detailed . As such , a detailed discussion record of advertisements that has played on channels the of reference number 101 would be found and / or introduced 15 user has recently selected ( e . g . , for a period of past 2 weeks , in FIG . 1 . Reference number 201 is introduced in FIG . 2 , etc . etc . ) , and generate prompt questions based on such adver

tisements . In one implementation , the user may submit DETAILED DESCRIPTION responses to such questions .

In a further implementation , when a user is provided a The MEDIA CONTENT BASED ADVERTISING SUR - 20 question with regard to an embedded advertisement in the

VEY PLATFORM APPARATUSES AND SYSTEMS pro - TV show ( e . g . , " are you interested in the red bag the vides a client - server interactive platform whereby a user character is carrying ? ” etc . ) , the user may submit a request may operate a general purpose personal mobile device ( e . g . , to learn more and / or purchase the product . In that case , the a smart phone , etc . ) to receive a list of TV programs AD - SURVEY may provide a merchant URL to the user schedules and submit a selection of TV channel via the 25 and / or redirect the user to the merchant site . personal mobile device . In one implementation , the user In one implementation , AD - SURVEY may monitor may operate the personal mobile device as a TV remote whether the user is “ actually ” attending and watching the controller . In one implementation , the AD - SURVEY may selected TV channel . For example , the user ' s mobile device receive the user ' s selection of a channel and determine what may capture , aggregate and packetize atmospherics data media contents the user has elected to watch . In one imple - 30 ( e . g . , taking photos of the user , recoding audio clips , obtain mentation , the user ' s channel selection and viewing status ing GPS information , etc . ) and submit to the AD - SURVEY , may be populated to a social media platform , and the which may in turn decode the atmospherics data to analyze AD - SURVEY may obtain user response with regard to a TV ad effect and audience perception , as further illustrated in program from the social media to perform analytics for TV FIGS . 2E - 2G . program feedback review . 35

For example , in one implementation , a user may plug a Ad - Survey AD - SURVEY accessory ( e . g . , 120 in FIG . 1 ) into his mobile device , such as a smartphone ( e . g . , an Apple iPhone , Black FIG . 1A shows a block diagram flow chart illustrating Berry , Google Android , Palm , HTC Evo , Samsung Galaxy , work flows of AD - SURVEY within embodiments of the etc . ) , laptop , personal digital assistant ( PDA ) , tablet com - 40 AD - SURVEY . Within embodiments , the AD - SURVEY plat puter ( e . g . , Apple iTouch , iPad , etc . ) , and / or the like , to form 105 may facilitate clients , such as merchants , brand facilitate communication between the mobile device and a name manufacturers , media producers , and / or the like to home TV set . In an alternative implementation , the AD - plan 105a advertisement campaign program , track adver SURVEY accessory may be a standalone table top unit tisement targets to determine ad delivery 105C ( e . g . , whether which may not need to be attached to a user mobile device . 45 the ads are viewed by the audience , etc . ) and performance For example , the table top unit may communicate with a 105b ( e . g . , whether the advertisements meet a sales / brand desktop computer , a laptop computer , a cell phone or mobile image goal , etc . ) . device and / or the like via wired or wireless connection ( e . g . Within implementations , the AD - SURVEY platform 105 Bluetooth , WiFi , etc . ) . In further implementations , the table may interact with user mobile devices , e . g . , PDAs , smart top unit may monitor audience activities as further illus - 50 phones , etc . , for targeted mobile advertisement delivery 104 . trated in FIGS . 1C , 2E - 2H . Further implementations of the For example , in one implementation , AD - SURVEY plat table top unit are illustrated in FIG . 4C . Within implemen - form may obtain a TV viewing status information from a tations , a AD - SURVEY server may obtain real - time TV user ' s mobile device , and determine the TV program content program listing , including the TV program schedule , adver - the user is / has been watching based on TV schedules , as tisement schedule , and / or the like , from a TV network . The 55 further discussed in FIG . 1B . When AD - SURVEY deter user may then obtain the list of TV program schedules from mines the user selected channel contains a TV ad of “ Geico " the AD - SURVEY , e . g . , as shown at 115 in FIG . 1 . The user 104b , AD - SURVEY may deliver promotions , rewards , cou may then submit a channel selection , e . g . , tap on the touch pons , questionnaires , etc . , related to “ Geico " as a targeted ad screen of the mobile device as shown at 105 in FIG . 1 . In one 104a to the user mobile device . implementation , upon receiving the user channel selection , 60 In one implementation , AD - SURVEY platform 105 may the AD - SURVEY server may retrieve data record from a obtain data related to user interactive activities with regard media content database and check program table to deter - to mobile ads , TV viewing , Internet 102 ( including online mine what ' s the TV program on air . For example , if the user browsing , purchasing , etc . ) , social media , and / or the like to selects the channel “ CBS , " the AD - SURVEY may ascertain analyze ad effects , TV rating so that to determine delivery “ CBS ” has “ The Big Bang Theory ” on air based on the 65 105c and performance 105b of an advertisement campaign . timestamp when the user submits the selection . In one Within implementations , the ad campaign planning 105a , ad implementation , the AD - SURVEY may further retrieve a performance 105b and ad delivery 105c , may be separately

US 9 , 807 , 442 B2 executed by the AD - SURVEY platform 105 for each media clip of the audience scene . For another example , AD type , e . g . , TV , print , Internet , social media , etc . SURVEY may prompt the user to indicate how many

FIG . 1B shows a block diagram illustrating a user engag - audience are present to watch the on - going TV program . For ing a personal mobile device as a TV remote control within another example , AD - SURVEY may monitor whether the embodiments of AD - SURVEY . In one embodiment , a user 5 user is engaging in other application activities on the per may download a AD - SURVEY client component for instan - sonal mobile device 115 , e . g . , email ( s ) , Facebook , browser tiation on his general purpose personal mobile device 115 . activities , gaming applications , etc . For another example , For example , the user may obtain a AD - SURVEY applica - AD - SURVEY may include GPS information of the personal tion from the iTunes Store and download it to his Apple? mobile device 115 into the atmospherics data . iPhone , iTouch , iPad , and / or the like . In one implementation , 10 In one implementation , AD - SURVEY may be configured the user may plug - in a AD - SURVEY accessory 120 to the to automatically create photo / video / audio captures . In mobile device 115 so that the mobile device may commu - another implementation , AD - SURVEY may prompt the user nicate with a home TV set 130 as a remote controller . to manually enter desired information , such as how many

In one implementation , upon instantiating the down - individuals are present to watch the TV program , and / or loaded AD - SURVEY client component , the user may 15 request the user position the built - in camera of the mobile receive a schedule listing of TV programs , and may select a device to snap photos / video clips . In one implementation , channel that the user is interested . For example , the user may AD - SURVEY may provide promotion incentives for the tap on the listed item to select " CH2 : CBS The Big Bang user to cooperate with such requests , e . g . , lottery opportu Theory 105 . ” In one implementation , the AD - SURVEY nities , etc . plug - in accessory 120 may receive the indication of user 20 In one implementation , AD - SURVEY may analyze the channel selection and transmit such indication to the home obtained audience atmospherics data 125 . For example , as TV set 130 , and the TV 130 may switch to the channel “ CBS shown in FIG . 1B , AD - SURVEY may identify the number 110 ” accordingly . of audience 138 at an associated timestamp from an audi

In a further implementation , the user ' s selection of chan - ence scene photo / video 135 via face recognition software , nel “ CBS 105 ” may be transmitted to a AD - SURVEY , which 25 e . g . , Apple iPhoto face recognition , etc . may in turn automatically populate a message on social FIG . 1C shows an example of intelligent mobile ques media , e . g . , a Facebook status update showing the user “ is tionnaire within implementations of the AD - SURVEY . In watching The Big Bang Theory on CBS 135 . ” In an alter one implementation , AD - SURVEY may send synchronized native implementation , the user may elect to manually enter questionnaire to a user ' s personal mobile device 115 based and populate the social media feeds via the AD - SURVEY 30 on the media program on - air at the user ' s selected TV client component instantiated on his personal mobile device . channel . For example , if the user ' s selected TV channel 130

For another example , the social message , e . g . , a Tweet includes an advertisement 145 of Audi automobiles , AD message , may contextually tag the text on key terms to SURVEY may generate a question 123 to inquire the user ' s indicate what the user is watching , e . g . , the Tweet hashtags desired automobile brand . Such inquiry results may be “ # The Big Bang Theory , ” “ # CBS , ” etc . In one implemen - 35 collected by AD - SURVEY and fed to the advertising brand tation , the hashtags may link to profile information of a TV merchant 150 ( e . g . , Audi , etc . ) to determine performance of show , including its scheduled airing time , crew information , the advertisement 145 . For example , if the user selects description , and / or the like . In further implementations , the “ Audi ” when inquired about his desired automobile brand Tweet hashtags may be employed for social content data after watching the Audi advertisement channel , it may show taxonomy engine , as further illustrated in FIGS . 6B - 6E . 40 effectiveness of the advertisement 155 over the TV channel .

FIG . 1B shows a block diagram illustrating an example In further implementations , AD - SURVEY may include embodiment of audience attendance monitoring within product placement advertisements , and / or the like in the embodiments of the AD - SURVEY . Within implementations , pop - up questionnaire 123 , as further discussed in FIGS . 2 AD - SURVEY may monitor the audience attendance of and 7B . selected TV program . For example , upon a user selecting a 45 FIG . 1D shows an example of media analytics within TV channel at his personal mobile device 115 ( e . g . , by embodiments of the AD - SURVEY . In one implementation , tapping on the channel listing as shown at 105 in FIG . 1A ) , a AD - SURVEY client may desire to know public opinions of the user may or may not be present watching the TV their products , e . g . , brand name products , TV programs , program broadcasting on the channel , e . g . , the user may step and / or the like . For example , as shown in FIG . 1D , the away , may engage in other activities distracting him from 50 producer “ CBS , " e . g . , merchant 150 , may want to know the TV , may invite one or more other audience watching the audience ' s reaction about the production “ The Big Bang selected TV channel , etc . In such scenarios , AD - SURVEY Theory ” 180 . In one implementation , the AD - SURVEY may may constantly , periodically and / or intermittently “ monitor ” collect data from social media platforms , such as , but not the audience status to obtain knowledge of the audience limited to Twitter , Facebook , and / or the like , user comments viewing data . 55 and activities with regard to the show “ The Big Bang

As shown in one example in FIG . 1B , the personal mobile Theory . ” For example , AD - SURVEY may obtain Tweets device 115 may automatically snap a photo and / or a video about users ' comments on “ The Big Bang Theory ” 185 . For clip of the audience scene including the audience 118 another example , AD - SURVEY may obtain Facebook user watching the TV program 110 . For example , the AD comments , activities ( e . g . , user “ like ” of the “ The Big Bang SURVEY may be configured to snap a photo or video clip 60 Theory ” page , news feed , etc . ) from Facebook . Further periodically ( e . g . , every 10 minutes , etc . ) . The AD - SUR - implementations of obtaining data from social media plat VEY may then packetize the obtained photo / video clip as forms are discussed in FIGS . 5A - 5F . audience atmospherics data 125 for audience attendance In one implementation , AD - SURVEY may generate a analysis . media analytics report 190 based on the obtained social

In further implementations , AD - SURVEY may include 65 media user comments to reflect audience reaction to the various data into the atmospherics data packets . For show “ The Big Bang Theory . ” For example , in one imple example , AD - SURVEY may " listen ” and record an audio mentation , the report may comprise statistical data with

US 9 , 807 , 442 B2 regard to audience age , demographics , occupation , etc . - continued Further examples of media analytics report are discussed in FIG . 8C . < Weekday > Thursday < / Weekday >

FIG . 2A shows a block diagram illustrating data flows < Frequency > Weekly < / Frequency >

between NR - Control server and affiliated entities within 5 < / AiringTime > various embodiments of the AD - SURVEY . Within various < Key Words >

< KeyWordl > Geeks < / Keyword1 > embodiments , one or more users 233a , AD - SURVEY server < Key Word2 > Physicists < / Keyword2 > 220 , social media 250 , TV network 270 , AD - SURVEY database ( s ) 219 , merchant 280 , and / or social media 250 are < / Key Words > shown to interact via various communication network 213 . 10 < Characters >

< Characterl > Sheldon Cooper < / Character1 > In one embodiment , the AD - SURVEY may receive a list < Character2 > Penny < / Character2 > of real time TV program 237a . For example , in one imple mentation , the TV program 237 data may comprise infor < / Characters >

mation such as channel information , media program infor - 15 . mation of each channel , program schedule information , < / TVProgram > and / or the like . For example , the TV network 270 may provide a ( Secure ) Hypertext Transfer Protocol In one embodiment , the user 233a may operate a client ( " HTTP ( S ) ” ) PUT message including the TV schedule data mobile device , which may receive a list of real time TV 237a in the form of data formatted according to the exten - 20 programs 237b . In one embodiment . upon reviewing the sible Markup Language ( “ XML " ) . Below is an example received channel schedule 237b , the user may submit a HTTP ( S ) PUT message including an XML - formatted TV channel selection by tapping on a selected channel ( e . g . , see schedule for the AD - SURVEY server : 105 in FIG . 1B ) . In one implementation , the user mobile

device may be enabled with an infrared remote control PUT / TVschedule . php HTTP / 1 . 1 component ( e . g . , a plug - in accessory 120 in FIG . 1B ) , and Host : www . TV . com may send the channel submission indication 235a to a client Content - Type : Application / XML TV 233b ( e . g . , a home TV set ) via an infrared communica Content - Length : 718 tion channel , which may result in a channel switch on TV . < ? XML version = “ 1 . 0 " encoding = " UTF - 8 " ? > < TVSchedule > 20 In another embodiment , when the user selects the channel

< Date > 09 - 09 - 2000 < / Date > from his mobile device ( e . g . , an Apple iPhone , etc . ) , the < Channel1 > channel selection message 235b may be transmitted to the < ChannelID > CH001 < / ChannelID >

< ChannelName > CBS < / ChannelName > AD - SURVEY server 220 in real time . < Programl > In another implementation , the AD - SURVEY server 220 < StartTime > 9 : 00 : 00 < / StartTime >

< MediaID > 1234456 < / MediaId > 35 may provide the TV program schedule data to a TV set - top < EndTime > 9 : 45 : 00 < / EndTime > box ( STB ) 201 , e . g . , via a cable network , wherein the STB

< / Programl > may receive user TV event messages 235a and forward such < Program2 > . . . < / Program2 > information 235c to the AD - SURVEY server 220 . In another < / Channel 1 > implementation , the STB 201 may directly communicate < Channel2 > . . . < / Channel2 > with a AD - SURVEY infrared component ( e . g . , 120 in FIG .

< / TVSchedule > 1B ) , a table unit , and / or the like . In one implementation , the TV event 235a / b / c may comprise a variety of TV events , such as , but not limited to TV on / off , STB on / off , channel

The media program may further comprise information is switch , and / or the like . In further implementations , when the such as media air time , media length , advertisement tag user has registered non - live media facility ( e . g . , DVR , DVD , timestamp , ad name , ad product information , and / or the like . TiVo , etc . ) with the AD - SURVEY , the TV event 235a / b / c The media program may further comprise a sub - table com may comprise DVD player on / off , TiVo on / off , TiVo channel prising embedded advertisement tags ( e . g . , see 343a in FIG . change , and / or the like . In one implementation , to operate 3A ) . 50 and exchange data with the STB , the presentation layer on In further implementations , the TV schedule 237 may the user mobile device may adopt development tools , such comprise sub - tables including information with regard to the as but not limited to Android , iOS app development tools , media programs . For example , an exemplary XML - format and / or the like . In one implementation , the TV set - up top ted TV program table takes a form similar to the following : box may similarly employ a presentation layer development

55 tool compatible with that of the user mobile device , and may < TVProgram > additionally employ HTML5 and web 2 . 0 presentation lay

< ID > 00001 < / ID > ers . < Name > The BigBang Theory < / Name > < Genre > Comedy < / Genre > In one implementation , a TV channel selection event 235b < Networks > CBS < / Networks > 60 may be logged by the AD - SURVEY server 220 and stored < Actors > as a real time data record in a AD - SURVEY database 219 . < Actor1 > Jim Parsons < / Actor1 > For example , in one implementation , the user device may < Actors > provide a HTTPS POST message including the TV channel < FirstRun > 08 - 2008 < / FirstRun > selection message 235b in the form of data formatted < AiringTime >

< Start Time > 9 : 00 : 00 < / StartTime > 65 according to the XML . Below is an example HTTP ( S ) POST < EndTime > 9 : 45 : 00 < EndTime > message including an XML - formatted user trigger for the

AD - SURVEY server :

10

INT 15

US 9 , 807 , 442 B2 10

TABLE 2 - continued PUT / CHsubmission . php HTTP / 1 . 1 Host : 255 . 000 . 00 . 1 Event Data Parameters Content - Type : Application / XML Content - Length : 701 Parameter Type Details < ? XML version = “ 1 . 0 " encoding = “ UTF - 8 " ? > < CHSelection Event > SourceID INT An ID in a media service

< EventID > 1111 < EventID > program that identifies a < EventType > CH Switch < / EventType > channel < Time > 19 : 00 : 00 < / Time > Time - zone INT The offset from GMT < Date > 09 - 09 - 2000 < Date > tmsID String Unique identifier of a < UserID > JDoe < / USerID > program ( an episode or a < UserName > John Doe < / UserName > program with a single < DeviceID > JD0001 < / DeviceID > occurrence ) . Using the tms < IP > 255 . 000 . 00 . 1 < / IP > data , these can be mapped < HardwareID > 00001 < / HardwareID > also to the show / series < AppID > JDDOEMOBILE 00001 < / App ID > is Social A boolean that is used to < MAC > 00 : 00 : 00 : 00 : 00 : 00 < MAC filter social event data < Channel > from being shared with < ChannelName > CBS < / ChannelName > friends ( e . g . , see 275a ) < Category > Comedy < / Category > < Program > The Big Bang Theory > < / Program > < Episode > 4 - 1 < / Episode > 20 In one embodiment , upon submitting a channel selection , < StartTime > 19 : 00 : 00 < / StartTime > < EndTime > 19 : 29 : 45 < / EndTime > the user 233a may populate social media feeds of his < Duration > 29 ' 45 " < / Duration > viewing status 275a to the social media network 250 , < / Channel >

wherein the user ' s friends may view his status , knowing < / CHSelection Event > 25 what the user is watching , electing to “ like , " " dislike , "

and / or comment on his status , and / or the like . For example , In further implementation , the user ' s mobile device may in one implementation , a AD - SURVEY client app may

send event messages . Such event messages may include comprise a social feature ( e . g . , see 715 in FIG . 7A ) so that channel selection message 235b , user checkin / checkout 30 a user may input a social media message ( e . g . , 715b / c in action ( e . g . , user signing in / out to a AD - SURVEY mobile / 30 web - based client portal , etc . ) , and / or the like . In one imple FIG . 7A ) . In another implementation , the AD - SURVEY may mentation , the events may be sent to the AD - SURVEY automatically generate a social message based on a pre server 220 via Https Post web based API , which may populated message format ( e . g . , a Tweet format with comprise a type identifier and a set of parameters of the 25 hashtags , etc . ) send a message indicating the user ' s viewing event data , e . g . , channel selection , user response , etc . In one status 275b to the social media 250 , which may automati implementation , AD - SURVEY server 220 may save such event data at CSV format . For example , the following Tables cally populate a social media status update . In such cases , I and II provide an exemplary data structure of an event the social media platform 250 may request user authoriza message : tion for AD - SURVEY server 220 to access , e . g . , download

social data or send social data to social media , etc . The TABLE 1 AD - SURVEY may obtain a user token for authorization to

Watching Event Data Structure access the user ' s social media profile , content , and / or the User Action Event Type Parameters 45 like . The user authorization may be further discussed in

FIGS . 5A - 5C . AD - SURVEY App Started EventID , UserID Pulse Event ( keep alive EventID , userID , zipcode , For example , in one implementation , the AD - SURVEY ping , sent every 10 min ) SourceID , time - zone , may provide a HTTPS POST message including a social isSocial , tmsID , timestamp Change change Same as Type 2 50 message 275b in the form of data formatted according to the Check in Same as Type 2 XML . Below is an example HTTP ( S ) POST message Check out Same as Type 2 including an XML - formatted user viewing status 275a / b for

the AD - SURVEY server :

FEW NA 50

TABLE 2 55

Event Data Parameters

Parameter Parameter Ivpe Type Details

EventID INT 60

PUT / UserSocial . php HTTP / 1 . 1 Host : www . AD - SURVEY . com Destination : www . Facebook . com Content - Type : Application / XML Content - Length : 701 < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > < SocialMessage >

< MsgID > Facebook1111 < MsgID > < Time > 19 : 00 : 00 < / Time > < Date > 09 - 09 - 2000 < Date > < UserID > JDoe < / USerID > < UserName > John Doe < / UserName > < User TokenID > 12324 < / User TokenID > < AppID > MR0001 < / AppId >

UserID

Unique identifier that is generated for each event that is added to the DB Unique identifier for a AD SURVEY user . The client app may send the device UDID . Zipcode as entered by the user at the client app

String

Zipcode INT 65

US 9 , 807 , 442 B2 li 12

- continued - continued

. . .

10 . . .

30

< Action > < AtmosID > AT0003 < / AtmosID > < Action Type > Status Update < / Action Type > < Atmos Type > Listen - in < / Atmos Type > < Content > “ is watching The Big Bang Theory on CBS ” < / Content > < Time > 19 : 00 : 00 < / Time > < Rating > N / A < / Rating > < Date > 09 - 09 - 2000 < / Date >

< UserID > JDoe < / USerID > < / Action > < UserName > John Doe < / UserName > < Tags > < DeviceID > JD0001 < / DeviceID >

< Tagl > Big Bang Theory < / Tagl > < IP > 255 . 000 . 00 . 1 < / IP > < Tag2 > CBS < / Tag2 > < HardwareID > 00001 < / HardwareID >

< AppID > JDDOEMOBILE 00001 < AppID > < / Tags > < MAC > 00 : 00 : 00 : 00 : 00 : 00 < / MAC >

< Datal > < / SocialMessage > < Type > audio < / Type >

< FileFormat MP3 < / FileFormat > 15 < FileName > MyRecord < / FileName >

In the above example , the AD - SURVEY server 220 may < Length > 10s < / Length > automatically populate a social message to the user ' s Face < Size > 567 KB < / Size >

. . . book page , showing a status update “ John Doe is watching < / Datal > The Big Bang Theory on CBS . ” < Data2 >

In a further implementation , the user 233a may receive 20 < Type > Image < / Type > < Format > JPEG < / Format > friends ' recommendations 277 of TV programs from the < FileName > MyPic < / FileName >

social media 250 ( e . g . , see 705 in FIG . 7A ) . For example , the < Exif >

user may view a scroll - down list of friends ' recommenda < Source > iPhone < / Source > < Timestamp > 19 : 00 : 00 < Timestamp > tions from a user interface via the user ' s mobile device . < size > 1024 x 768 < / size >

In further embodiments , AD - SURVEY may load data 278 25 < focus > 600 / 100 < / focus > from the social media platform 250 , e . g . , user profile infor mation , user comments / activity data related to an advertise < / Exif >

ment / a TV program , and / or the like . Further implementa < Data2 > tions and example data structures of the social media data < Data3 > 278 are discussed in FIG . 5F . < Type > GPS < / Type >

FIG . 2B shows a block diagram illustrating data flows < Content > 45 Garden Street < / GPS >

between AD - SURVEY entities within alternative embodi < / Data3 > ments of the AD - SURVEY . Within implementations , user < Data4 > mobile device 233a may perform “ ambient listening ” on the < Type > Device App < / Type > user . For example , when a user has submitted a channel 35 < Timestamp > 19 : 00 : 00 < / Timestamp >

< Appl > selection , it is not guaranteed whether the user is indeed < Name > Angry Bird < / Name > watching the TV program selected , e . g . , the user may walk < Type > Gaming < / Type > away , engaging in other activities such as playing video < Status > Active < / Status > games , chatting with others , etc . As such , the user ' s mobile < / Appl > device 233a may “ listen - in ” the user ' s behavior , e . g . , by 40 < / Data4 > capturing live video of the surroundings of the user , by recording audio clips , by capturing a picture , and / or the like , < / Atmospherics > to capture information as to whether and / or what the user is watching the TV . In one implementation , such “ listen - in " The user device may perform “ ambient listening ” and data ( e . g . , video clips , audio clips , pictures , device applica - 45 generate atmospherics data package constantly , intermit tion status , GPS coordinates , etc . ) may be aggregated and tently , and / or periodically ( e . g . , every hour , etc . ) to “ listen packetized as atmospherics data 239 and transmitted to the AD - SURVEY server 220 , which may analyze such atmo in ” user ' s watching status , e . g . , whether the user is paying spherics data to determine whether the user is watching , and attention to the selected TV program . In the above example , integrate the atmospherics analytics in an audience statistics 50 the es 50 the generated atmospherics data package may comprise a report 245 . In one implementation , the AD - SURVEY may variety of data segments , such as a “ MyRecord . mp3 ” audio provide the audience statistics report 245 to the TV network clip which may indicate whether the played audio matches 270 for TV program audience feedbacks . a selected channel program , whether the user is chatting with For another example , in one implementation , the user friends ( e . g . , see 298 at FIG . 2D ) ; a “ MyPic . JPEG ” photo

device 233a may generate an atmospherics data package as 55 taken by an iPhone camera , which may indicate whether the a HTTPS POST message in the form of data formatted user is present with the TV set , and / or one or more indi according to the XML . Below is an example HTTP ( S ) POST viduals are present ( e . g . , see 297 at FIG . 2D ) ; GPS infor message including an XML - formatted atmospherics data mation of the user mobile device which may indicate 239 to provide to the AD - SURVEY server 220 : whether the user is present at his residential address where

60 the home TV is located ( e . g . , see 299c at FIG . 2D ) ; user mobile device application status information including an

PUT / Listen - in . php HTTP / 1 . 1 actively running gaming application ( e . g . , “ Angry Bird ” ) , Host : 255 . 000 . 00 . 1 which may indicate the user is playing the video game Content - Type : Application / XML instead of paying attention to the TV program ( e . g . , see 299a Content - Length : 701 < ? XML version = “ 1 . 0 " encoding = " UTF - 8 " ? > 65 at FIG . 2D ) . In one implementation , the atmospherics data < Atmospherics > 239a and / or analytics may be incorporated into a data record

290a and stored in the database 219 .

14 US 9 , 807 , 442 B2

13 In another implementation , the user mobile device may also obtain a physical address , a hardware ID of the user

send a device application event 241 indicating user device mobile device , etc . , for registration purposes . application status to the AD - SURVEY server 220 . For I n one implementation , upon registration , the AD - SUR example , the device application event 241 may comprise an VEY server 220 may obtain and store a user application ID , actively engaged application information on the device , e . g . , 5 and / or a session ID 265 to start the AD - SURVEY service application ID , application name , application category , push session . In one implementation , the AD - SURVEY server email heart beat pulse , etc , which may suggest audience 220 may obtain a real - time TV program listing 267 , e . g . , activities while watching TV , e . g . , video gaming , texting , from a TV network , a TV broadcasting service , etc . In one calling , checking email , browsing , playing music , editing , implementation , the AD - SURVEY server 220 may obtain photos , and / or the like . For example , in one implementation , the TV schedule updates on a periodical basis ( e . g . , daily , the user device 233a may generate a mobile application etc . ) , and store the TV schedule information at a TV sched event as a HTTPS POST message in the form of data ule database . In one implementation , the user 233a may formatted according to the XML . Below is an example receive a list of TV programs 270 via the AD - SURVEY HTTP ( S ) POST message including an XML - formatted is application , e . g . , see 710 in FIG . 7A . device application status 241 to provide to the AD - SURVEY In one implementation , the user mobile device may send server 220 : a user event message 280 to the AD - SURVEY server 220 ,

and the AD - SURVEY server may monitor user event mes sages 276 from the user device . As discussed in FIGS . PUT / Mobile App . php HTTP / 1 . 1

Host : 255 . 000 . 00 . 1 20 2A - 2B and will be further discussed in FIGS . 3A and 5A , the Content - Type : Application / XML received user event at 276 may comprise a variety of Content - Length : 701 events / messages , such as , but not limited to user TV event < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > ( e . g . , 235b in FIG . 2A ) , user device application event ( e . g . , < Mobile App >

< Time > 19 : 00 : 56 < / Time > 241 in FIG . 2B ) , atmospherics data ( e . g . , 239 in FIG . 2B ) , < Date > 09 - 09 - 2000 < / Date > social content ( e . g . , 525a / b in FIG . 5A ) , response to ads / < UserID > JDoe < / USerID > surveys ( e . g . , 339 in FIG . 3A ) , and / or the like . In one < UserName > John Doe < / UserName > implementation , upon receiving the message , the AD - SUR < DeviceID > JD0001 < / DeviceID > < IP > 255 . 000 . 00 . 1 < / IP > VEY may determine a type of the message 288 . In one < HardwareID > 00001 < / HardwareID > implementation , the user event message may comprise user < AppID > JDDOEMOBILE 00001 < / AppId > 30 channel submission event 289b , check - in / out messages , < MAC > 00 : 00 : 00 : 00 : 00 : 00 < / MAC > and / or the like , e . g . , see 235a in FIG . 2A . For another < EventType > gaming < / EventType > < Appl > example , the user event message may comprise atmospher

< Name > Angry Bird < / Name > ics data 289a ( as further illustrated in FIG . 2E ) , ad / survey < Type > Gaming < / Type > response 289c ( as further illustrated in FIGS . 3B - 3C ) , social < Status > Active < / Status > 35 data 289d ( as further illustrated in FIG . 5B ) , non - live TV < StartTime > 19 : 00 : 45 < / StartTime > < EndTime > 19 : 00 : 56 < / EndTime > consumption message 289e ( as illustrated in FIG . 26 . For < Duration > o ' 11 " < / Duration > example , in one implementation , the AD - SURVEY may

determine the message types 289a - e based on the data < / Appl > structure , e . g . , a field value of a field “ message type ” as 40 shown in Table 2 , message type code in the header infor

In one implementation , the device application status 241 mation of a received data packet , and / or the like . may comprise a list of application names that have been FIG . 2D provides a logic flow diagram illustrating TV open and active for a minimum amount of time ( e . g . , 10 channel submission ( e . g . , 289ble ) message ( for live TV , seconds , etc . ) . In another implementation , the device appli - and / or non - live TV program such as DVD playing , Tivo , cation status 241 may be periodically harvested to the 45 media on demand , and / or the like ) processing within imple AD - SURVEY serer . mentations of the AD - SURVEY . In one implementation ,

FIG . 2C provides a logic diagram illustrating user - server upon receiving a channel TV event message at 289ble , the interactions within embodiments of the AD - SURVEY . In AD - SURVEY server may parse the received data message one embodiment , a user may instantiate a AD - SURVEY 2001 and save it in a raw data message database , e . g . , as application component on a mobile device 250 . The user 50 XML - formatted records in 219a . For example , a variety of may also plug a AD - SURVEY accessory ( e . g . , 120 in FIG . exemplary XML - formatted TV event message data struc 1 ) into his mobile device to facilitate communication tures ( e . g . , TV on / off , STB on / off , DVD on / off , channel between the mobile device and a home TV set . change , etc . ) are discussed at 235a in FIG . 2A .

In one implementation , the user may submit a session Within implementations , the AD - SURVEY may deter request 253 to the AD - SURVEY server 220 via the mobile 55 mine whether each received TV event message indicates a application , e . g . , as shown in Table 1 , the session request stable TV channel program selection , or a channel surfing . 253 may comprise a user ID and an event / session ID to In one implementation , the AD - SURVEY retrieve a list of indicate the AD - SURVEY application has stared . Upon unprocessed TV message records ( e . g . , grouped per user receiving a user request , the AD - SURVEY server 220 may profile ) 2002 from the raw data store 219a . For every two determine whether the user has registered with AD - SUR - 60 consecutively received messages 2005 , the AD - SURVEY VEY 255 . For example , in one implementation , the user may may calculate the elapsed time in - between 2008 , and deter be a new user to the AD - SURVEY application service , who mine whether the elapse time is greater than a pre - deter may just download and install the mobile application but has mined surfing threshold ( e . g . , 5 seconds , 10 seconds , etc . ) . not registered . If the user is not registered 258 , the user may For example , when the elapsed time is shorter than the be requested to submit registration information , such as user 65 surfing threshold , indicating the user may be frequently name , phone number , email address , residential address , switching channels to browse the program , the AD - SUR and / or the like for registration 260 . The AD - SURVEY may VEY may not consider the channel selection message as

15 US 9 , 807 , 442 B2

16 effective TV viewing , and may filter such message records geography , and / or the like . In one implementation , the from TV viewing analysis 2013 . AD - SURVEY may generate an audience summary via a

In another implementation , when the elapsed time is dashboard , e . g . , see FIG . 8H . greater than the surfing threshold 2009 , suggesting the user FIG . 2E provides a logic flow diagram illustrating atmo may at least spend an amount of time staying on the selected 5 spherics data ( e . g . , 289a ) analytics within implementations channel , the AD - SURVEY may further determine whether of the AD - SURVEY . In one implementation , upon receiving the elapsed time is greater than a capping threshold 2015 . an atmospherics data message 289a , the AD - SURVEY For example , when the elapsed time is too long , greater than server may decode the atmospherics data and obtain atmo

the capping threshold ( e . g . , 2 hours , 3 hours , etc . ) , it may spherics artifacts 296a , e . g . , a photo , an audio clip , and / or suggest a user may just let the TV on without watching . In 10 the like . In one implementation , for each decoded artifact ,

the AD - SURVEY may determine the artifact type 296b , e . g . , such cases , AD - SURVEY may apply capping edit rules to based on the file name , file extension , etc . compute a “ real ” watching time 2025 . For example , if the In another implementation , the AD - SURVEY may incor elapsed time between a first TV channel switch and a second porate the received device application status ( e . g . , 2890 for channel switch is 5 hours , the AD - SURVEY may not log 5 1 8 15 user activity analytics . For example , the AD - SURVEY may hours as the watching time for the first selected channel , as capture active application s running on the device 299a from the capping threshold is 2 hours . The AD - SURVEY may in the received device application data , and determine user turn determine the watching time of the first selected chan activities when viewing 299b , e . g . , emailing , browsing nel as capped by a STB / TV off event ( e . g . , when a STB / TV Internet content , texting , video gaming , and / or the like . Such off event is received during the elapsed time , the watching 20 indicated user activity data may be incorporated into audi time may not exceed the timestamp of the STB / TV off ence attendance estimation 2100 . event ) , TV program end time ( e . g . , when the playing TV In one implementation , the artifact may comprise a visual program on the first selected channel ends during the elapsed data file 297 , such as a video file ( e . g . , " wmv , " " mp4 , ” “ avi , ” time , the watching time is calculated as the time elapse “ rm , ” etc . ) , an image file ( e . g . , " JPEG , " " bmp , " " tiff , ” “ gif , " between the first TV channel event and the TV program end 25 etc . ) , and / or the like . In one implementation , the AD time ) , and / or the like , e . g . , at 2023 . In further implementa SURVEY may determine graphical content 297 of the visual tions , the AD - SURVEY may apply cutoff thresholds based file . For example , the AD - SURVEY may perform image on historical heuristics via statistical analysis 2023 . For analysis to determine whether the photo image , and / or video example , the AD - SURVEY may determine the watching frames comprise a scene of audience , and / or a TV screen . In time based on individual habits , e . g . , a user has been en 30 one implementation , a mobile application at the user mobile observed to stay on the same channel for at most 1 hour , etc . device , e . g . , iPhone , etc . , may perform face recognition at a

photo taken at an iPhone , and integrate such data in the For another example , the cut - off threshold may be analyzed atmospherics data package , e . g . , an iPhoto including two by channel , e . g . , 1 hour on CBS , 2 hours on ABC family , faces , etc . In another implementation , the AD - SURVEY and / or the like . 35 server may perform facial recognition to determine audience In one implementation , when the elapsed time does not presence 297a . In other implementations , the AD - SURVEY exceed a capping threshold at 2015 , and / or when the watch server may perform image analysis to determine user activi ing time has been re - calculated based on capping rules at ties in the photo image , e . g . , reading a book , doing house 2025 , the AD - SURVEY may retrieve TV program informa work , and / or the like . In further implementations , a AD tion on the user selected channel 2018 , and generate a user 40 SURVEY panelist may review the photo image and channel selection log file 2020 . For example , the log file may determine audience status . comprise fields such as user ID , channel ID , channel selec - In another implementation , the AD - SURVEY may deter tion time , user watching time , channel program name , mine whether a TV screen image matches the TV program channel program ID , and / or the like . Such generated log files associated with the user channel selection 297b , e . g . , the may be fed to a user TV viewing data engine 2030 for 45 user may switch to watch recorded program ( e . g . , Tivo , audience analytics 219b . For example , the audience analyt - DVD player , etc . ) instead of live TV program on the ics database 219b may be utilized to analyze TV viewing channel , and in such cases , the AD - SURVEY may not rates of a TV program , product / brand impression of adver - receive an indication of such change . For example , in one tised products during the user watching time , and / or the like . implementation , AD - SURVEY may perform image analysis Exemplary audience analytics reports are discussed at FIG . 50 to determine whether the received image photo ( and / or a 10A - 10H . video frame grab ) contains a TV screen shot 297b , e . g . , by

the AD - SURVEY server 220 may retrieve TV program detecting edges of a rectangular shaped object on the image , information to determine the TV program played on the user etc . For another example , the AD - SURVEY may store a selected channel 292 . For example , the AD - SURVEY may plurality of sample screen shots from the TV program played query on a TV program table ( e . g . , obtained at 267 at FIG . 55 at the user submitted channel , and may compare the received 2C ) based on an instant timestamp and the user selected image photo with each of the stored sample screen shots . In channel . The AD - SURVEY may log the user channel selec further implementations , a AD - SURVEY panelist may assist tion with the timestamp 292 in a database , e . g . , at 290a in in reviewing and determining whether the user TV screen FIG . 2A matches the played TV program . In one implementation , if

In one implementation , AD - SURVEY may generate TV 60 the AD - SURVEY determines the user is absent from a TV viewing data for the retrieve TV program 293 . In one set , or engaging in other activities from the graphic analysis , implementation , AD - SURVEY may associate the user selec - AD - SURVEY may generate negative heuristics with regard tion to the TV viewing rate of the retrieved TV program , and to TV viewing data of the TV program on air . may refine the TV viewing rate with atmospherics analytics In another implementation , if the received atmospherics at 306 . For example , in one implementation , the AD - SUR - 65 data comprises an audio artifact ( e . g . , with a file extension VEY may monitor groups of audience ' s channel selection , of " way , " " mp3 , " " ape , " " m4a , ” etc . ) , the AD - SURVEY wherein the audience groups may be defined based on age , may perform audio analysis to determine a content of the

18 US 9 , 807 , 442 B2

17 audio 298 . For example , the AD - SURVEY may analyze the manner as illustrated at 297 , 298 in FIG . 2E , but without the frequency range of the audio content to determine the sound user ' s channel selection indication . source , e . g . , whether it is human voice , ambient noise , media In one implementation , the AD - SURVEY may perform an broadcasting , and / or the like . audio / video recognition procedure to identify a TV program ,

In one implementation , if the audio content comprises 5 e . g . , via digital signatures embedded in the program , and / or human voice ( e . g . , within the frequency range 60 – 7000 Hz ) , the like . In another implementation , the AD - SURVEY may the AD - SURVEY may determine whether the human voice extract key terms from the audio / video captures , and form a is from the audience or broadcasting media . For example , query on a database of TV programs to find a match . For the AD - SURVEY may perform voice recognition to deter example , in one implementation , the AD - SURVEY may mine whether the human voice matches with any of the 10 extract textual terms from the obtained audio media program characters in the TV program on air on the user submitted excerpts , such as “ big bang , " " quantum physics , ” “ Shel channel , e . g . , at 298a . 1 . If not , the AD - SURVEY may don , ” etc . , and may form a query in a database of TV determine such human voices may indicate audience pres ence 298a , e . g . , whether more than one user is present with programs which may return a result indicating the obtained the TV set . is audio media program excerpt may be related to the TV show

In further implementations , if the audio file comprises “ Big Bang Theory . ” human voice , the AD - SURVEY may extract verbal content In an alternative implementation , rather than uploading from the audio file to determine whether an audience con - sampling and / or the entirety of the captured audio / video versation , or a human conversation from media playing , is median content , the AD - SURVEY may analyze the recorded related to the TV program on air on the user submitted 20 audio / video content to generate a unique signature and / or a channel 298a . 2 . For example , the AD - SURVEY may adopt unique hash , which may be used for further matching . For speech recognition software ( e . g . , Nuance , IBM WebSphere example , the unique signature / hash may take a form similar Voice , etc . ) to extract key terms from the conversation , and to a sequence of 0 - 1 representation of a sampling of the compare whether the key terms are related to key words recorded media content . In one implementation , the signa stored with the TV program in the database . For example , if 25 ture / hash generation may be performed at a user device ( e . g . , AD - SURVEY extracts key terms " quantum mechanics , " the user ' s mobile phone , the table top unit , etc . ) , which may " physics , " “ big bang , ” etc . , from the human conversation in upload the generated signature / hash sequence to the AD the received atmospherics audio artifact , and the user sub SURVEY server . In another implementation , the user device mitted channel CBS is playing " The Big Bang Theory , " may upload media content sampling , clips or the captured AD - SURVEY may determine the audience is watching the 30 entirety to the AD - SURVEY server , which may then per show on air . In such cases , the AD - SURVEY may not need form the signature generation . Within implementations , soft to distinguish whether the human conversation in the audio ware tools / components such as , but not limited to i - brainz , file is from the audience or TV , but focus on mining the and / or the like may be adopted to generate audio signaturel content of the conversation . hash .

In another implementation , if the AD - SURVEY deter - 35 In further implementations , such “ listen - in ” activities may mines the audio artifact comprises ambient noise , AD be performed on a standalone table unit , which may com SURVEY may determine an environment of the audience municate with a user computer via a wireless network , 298b . For example , if the background is overly noisy , the and / or transmit the “ listen - in ” results to the AD - SURVEY user may not be watching the TV program . server .

In another implementation , if the AD - SURVEY deter - 40 In further implementations , lighting sensor data 299c may mines the audio artifact comprises media sound ( e . g . , music , indicate the lighting condition of the user environment to etc . ) , the AD - SURVEY may determine whether the audio determine the viewing status . GPS information contained in media content matches the TV program on air on the user the atmospherics data may indicate whether the user is submitted channel 298C via a media recognition software located with the home TV 299d , e . g . , by comparing the ( e . g . , a Shazam alike music recognition kit , etc . ) . For 45 instant GPS location with a registered user residential example , if the user selected channel CBS has “ The Big address , etc . Bang Theory ” scheduled at the moment , but the AD - SUR In one implementation , AD - SURVEY may analyze the VEY determines a Lady Gaga song in the received audio variety of atmospherics data to determine whether the user atmospherics , this may indicate the user is not watching the should be accorded as a “ viewer ” of the real - time TV TV program . In one implementation , if the AD - SURVEY 50 program on air . In one implementation , as shown in FIG . 2H , determines the user is distracted from the TV program based the AD - SURVEY server may adopt a procedure to generate on the audio content , AD - SURVEY may generate negative a progressive weighted sum of atmospherics scores to deter heuristics with regard to TV viewing data of the TV program mine whether the user is “ watching ” or “ not watching . " on air . When the weighted sum of different atmospherics scores

In an alternative implementation , the AD - SURVEY client 55 exceeds a predetermined threshold , the user may be consid component , which may be instantiated on a user mobile ered “ not watching , ” and the AD - SURVEY may not need to device ( e . g . , a downloadable application such as , but not proceed with further atmospherics analysis to improve effi limited to an iPhone application , an Android application , ciency of the atmospherics analytics . etc . ) and / or a table top standalone unit , may obtain atmo - For example , at 299d in FIG . 2H ( e . g . , comparing spherics data to determine the program the user is watching 60 received user GPS information with the user ' s registered without user indication of the channel . For example , the residential address in FIG . 2D ) , the AD - SURVEY may AD - SURVEY component may obtain audio recording , determine an address type 2163 , and may assign a weight to video recording , signatures image captures , and / or the like the GPS atmospherics factor based on the address type from of the audience watching environment , and submit the an atmospherics GPS weight table 2164 . For example , the obtained data to AD - SURVEY server . In one implementa - 65 atmospherics GPS weight table may assign 0 . 1 to a residen tion , the AD - SURVEY server may analyze the obtained data tial address , 0 . 2 to a Starbucks store , 1 . 0 to an outdoor to determine what the audience is watching in a similar address ( e . g . , a national park , etc . ) , and / or the like . In one

US 9 , 807 , 442 B2 19 20

TABLE 5 example , the AD - SURVEY may assign a GPS factor weigh ing score similar to the following table :

Audio Heuristics Weighing Scores TABLE 3 Audio Heuristics Type Weighing Score

5 GPS Information Weighing Scores Ambient Noise

Level Loud Medium Minor

0 . 5 0 . 2 0 . 02 GPS Address Type Weighing Score

0 . 1 Human Voice 0 . 5

Residential address Commercial Coffee Shop , Address Restaurants

Hotel 10

Key Terms Matches TV program Key Term not Match

0 . 1 0 . 3 . . . . . .

O

3 - . .

Outdoor Address

Shopping Site Highway National Parks

Media Sound 0 . 8

Matches TV program Not match

15 0 . 3

. . . . . .

35

. . .

: 5230 0 . 5

40 New Email 0 . 4

0 . 3

In one implementation , the AD - SURVEY may calculate In one implementation , the AD - SURVEY may update the the atmospherics score 2200 , which may be the assigned GPS weight at this stage , and determine whether the score 20 au 20 atmospherics score 2185 by adding the audio scores to

determine whether it exceeds the threshold 2190 . Otherwise , is greater than a predetermined threshold ( e . g . , 1 , etc . ) 2205 . If so , the AD - SURVEY may conclude the user is not AD - SURVEY may proceed to perform graphical analytics at watching the TV and quit scoring 2300 . Otherwise , the 297alb given such visual data is available . In one imple AD - SURVEY may proceed with analyzing device app ana mentation , the AD - SURVEY may retrieve and / or determine

lytics data at 299a , and repeat the process of updating 25 25 visual analytics indications 2192 , e . g . , user activity , user atmospherics score to determine whether a threshold has presence , number of individuals , TV screenshot , etc . In one been met to suggest the user is not watching . example , the AD - SURVEY may assign a weight based on

At 2165 , the AD - SURVEY may determine an active visual analytics indications 2195 similar to the following table : application type 2165 , and assign a weight based on the

application type 2168 . For example , an active gaming appli - 30 TABLE 6 cation may be accorded 0 . 7 ; an active email application may

be accorded 0 . 5 ; an active internet browser may be accorded Visual Heuristics Weighing Scores 0 . 5 , and / or the like :

Visual Heuristics Type Weighing Score TABLE 4 Audience None 0 . 5

Presence More than 1 0 ( count 2 viewers ) Device Application Activity Weighing Scores Audience Reading etc . 0 . 4 Active Application Type Weighing Score Activity Housework 0 . 3

Working Gaming Application 0 . 7 TV Screen Matches the Email Shot TV program Application Window Does not Inbox Match

Internet Opening New 0 . 4 Browser Link

Scrolling In one implementation , if the updated score 2193 does not Down exceed the threshold , the AD - SURVEY may conclude the

P2P Chatting 0 . 4 user is watching the TV program 2305 , and feed such Messenger indication to 2100 for studying TV viewing rates . Otherwise ,

50 the AD - SURVEY may conclude the user is not watching Software 2300 , and may not count the user as a “ viewer ” of the TV Office Text Editor 0 . 2 program . Application In one implementation , the AD - SURVEY may process the

decoded atmospherics data based on a progressive mecha 55 nism , to reduce processing complexity . For example , as

In one implementation , the AD - SURVEY may calculate shown in FIG . 21 , the AD - SURVEY may start with a less an updated atmospherics score 2170 and determine whether complicated analysis of GPS information , device application it exceeds the threshold 2175 . If it has not exceeded the status , and progressively proceed with visual data process threshold , the AD - SURVEY may proceed with audio ana ing with a higher complexity . lytics results from 298 , provided such audio analytics is 60 FIG . 2F shows a logic diagram illustrating user - server available from the atmospherics data . In one implementa interactions of listen - in ” within alternative embodiments of tion , the AD - SURVEY may retrieve and / or determine audio the AD - SURVEY . In one embodiment , the AD - SURVEY analytics indications 2178 , e . g . , ambient noise level , media may instantiate " listen - in ” when a user has submitted a music which does not match the TV program on air , human channel selection to determine whether the user is watching voice chatting on irrelevant topics , etc . In one example , the 65 the selected channel . In another implementation , the AD AD - SURVEY may assign a weight based on audio analytics SURVEY may periodically check the listen - in data ( e . g . , indications 2180 similar to the following table : every 30 minutes , etc . ) .

. . . · · · . . .

45 0 . 4

. . . . . .

Dialing 0 . 5

. . .

: : :

US 9 , 807 , 442 B2 21 22

se

Within embodiments , the user ' s mobile device may cap - request 2150 to the AD - SURVEY , e . g . , the user may register ture image , audio data , video data , GPS coordinates and / or TiVo , etc . so that AD - SURVEY may fold the viewing data the like 2105 . In one implementation , the AD - SURVEY of non - live media into TV viewing data analytics . In another client component may automatically configure the user implementation , the user may select a channel that com device to capture atmospherics data , e . g . , obtaining GPS 5 prises on - demand video service . In one implementation , coordinates , capturing audio data , capturing device applica - upon sending a request , the user device may submit the user tion status data , etc . In another implementation , the AD - external IP ( e . g . , the IP address of the user mobile device , SURVEY client component may prompt a request for the etc . ) , internal IP address ( e . g . , the IP address of the Tivo user to hold up the mobile device to position the camera for facility , a set - top box , etc . ) to the AD - SURVEY for regis image / video capturing of the TV screen , the audience scene , 10 tration 2155 . The user device may further provide a media and / or the like . access key to AD - SURVEY as authorization credentials for

In one implementation , the AD - SURVEY may extract remote monitoring of the user ' s non - live media facility . In identifying information from the captured monitoring data further implementations , the user may configure parameters 2106 , such as a hardware ID , MAC address , and / or the like . of the facility ( e . g . , a set - top box , etc ) as shown in FIGS . The AD - SURVEY may determine whether there is any 15 71 - J . external event 2108 , e . g . , the user is sending a response to In one implementation , AD - SURVEY may register the survey question , submitting a channel selection , etc . If there user ' s non - live media facility 2156 , and may establish a is such external event 2108 , the AD - SURVEY may launch secure communication channel with the non - live media the event and embed the atmospherics information into the facility . In one implementation , the AD - SURVEY may user responses 2110 for transmission . For example , in one 20 receive a non - live media schedule 2158 , which may be implementation , the embedded atmospherics data may have automatically downloaded from the user ' s non - live media the same timestamp as the original user response data ( e . g . , media on - demand , Internet TV streaming service such payload . as ABC episodes , Hulu . com , etc . ) , and / or provided by the

In another implementation , the AD - SURVEY may aggre user ( e . g . , a list of recorded programs for replay ) . gate , and packetize different atmospherics data 2113 in a 25 In one implementation , upon receiving an indication of compliant data format for transmission to the AD - SURVEY non - live media selection 289e , the AD - SURVEY platform server ( e . g . , see 239 in FIG . 2B ) . may check the program table to determine the TV program

In one implementation , upon receiving a message from on the non - live media 2160 , and log the user channel the user device 2115 , the AD - SURVEY server may deter - selection of an associated TV program with a timestamp mine the message type 2118 . In one implementation , if the 30 2125 . The AD - SURVEY may also obtain and analyze atmo message is an atmospherics data packet 2120 , the AD - spherics data 2123 , e . g . , in a similar manner as discussed in SURVEY server may decode and analyze the atmospherics FIG . 2D , to generate TV viewing data for the selected TV data packet 2123 to extract information as to the user ' s program 2127 . viewing status . For example , the AD - SURVEY server may FIG . 21 provides a logic flow diagram illustrating user perform an optical character recognition ( OCR ) procedure 35 mobile device as a TV remote ( e . g . , 235a in FIG . 2A ) within on a photographic frame extracted from the received atmo - implementations of the AD - SURVEY . In one implementa spherics data to determine whether the TV program played tion , a user may plug a AD - SURVEY infrared accessory on TV matches the program schedule associated with the ( e . g . , 120 in FIG . 1B ) into a user mobile device 2500 . The user ' s selected channel . , e . g . , whether the program is “ cor - user mobile device may be a general purpose personal rect ” 2125 . For example , if the user has submitted a selection 40 device ( e . g . , as opposed to a dedicated TV remote facility , of channel “ CBS , ” the AD - SURVEY may query on a etc . ) , such as , but not limited to an Apple iPhone , iTouch , program table to determine that “ The Big Bang Theory " iPod , iPad , BlackBerry , Palm , HTC Evo , Google Android , shall be on air at the timestamp when the atmospherics data Samsung Galaxy , and / or the like . The user may instantiate a is received . The AD - SURVEY may then ascertain whether AD - SURVEY client component on the user device 2505 the received photo of the user ' s TV set indicates the show on 45 ( e . g . , see FIGS . 7A - 7L , etc . ) , wherein the AD - SURVEY TV is “ The Big Bang Theory . ” client component may determine whether an infrared plug - in

In a further implementation , the AD - SURVEY may deter - is available 2510 . If so , the user device may initiate an mine whether the user is watching 2128 . For example , the automatic scan on its communication stack for TVs / DVDs AD - SURVEY may perform OCR on the received graphic 2 515 . If the query returns a TV / DVD is available 2520 , the data ( e . g . , photos , video clips , etc . ) to determine whether the 50 user device may obtain a physical address of the scanned user is present in front of the TV . For another example , the facility , and determine a type of the facility 2523 , wherein AD - SURVEY may determine how many users are watching such information may be logged into a communication stack the TV program by being present . For another example , the 2528 . In another implementation , if no results come out of AD - SURVEY may determine whether the user is present in the automatic scan at 2520 , the user device may obtain user front of his home TV by analyzing the received GPS 55 submitted TV / DVD parameters 2525 ( e . g . , TV / DVD brand , coordinates , e . g . , when the user ' s GPS coordinates reflects type , etc . ) , and adjust the scanning mode based on the user he has migrated from his home address to a second address , submitted parameters 2526 . it may indicate the user is no longer watching the TV Upon establishing communication with a TV / DVD set , program after submitting channel selection . the user device may monitor on user ' s channel submission

In a further implementation , the AD - SURVEY may gen - 60 2530 . When a channel selection is obtained , the user device erate viewing data 2130 to determine audience rating of a may transmit a channel selection indication to the logged TV program , wherein analysis of the atmospherics data may TV / DVD address via the infrared plug - in accessory 2535 . In contribute to the viewing statistics . this manner , the user may operate a general purpose mobile

FIG . 2G provides a logic flow diagram illustrating non - device as a TV / DVD remote . live media consumption message ( e . g . , 289e ) analytics 65 FIG . 3A shows a block diagram illustrating data flows within implementations of the AD - SURVEY . In one imple between AD - SURVEY entities within alternative embodi mentation , a user may submit a non - live media registration ments of the AD - SURVEY . Continuing on from FIGS . 2A

23 US 9 , 807 , 442 B2

24 and 2B , in one embodiment , the AD - SURVEY may receive - continued a list of real time TV program 237a from the TV network 270 , and a list of advertisement tags 243a associated with < / AdRedirect >

< AdInteractive > Screen Shot . gif < / AdInteractive > the TV programs 237a . For example , for each TV media program , which may comprise both segments of TV pro - 5 < AdTag2 >

grams ( e . g . , a TV play ) and a plurality of advertisements < / Media > ( e . g . , interleaved during the TV play broadcasting ) . For

another example , the advertisement tags 243a may label embedded advertisement in a scene of the TV play , e . g . , a . The above XML example shows a media program “ The pair of sunglasses carried by a character in the TV show 10 Big Bang Theory season 3 , episode 2 ” which is scheduled to “ The Big Bang Theory ” may comprise a product placement be on air on CBS at 9 pm on Sep . 9 , 2000 . The example advertisement tag . In a further implementation , for product media program comprise an ad tag which may be a regular placements , the ad tags 343a may comprise video frame advertisement ( e . g . , non - product placement or embedded in grabs with embedded graphic indications of the placed 15 the scene ) of Audi automobile , and another ad tag which products in the scene , e . g . , see FIG . 7G . may be an embedded product placement , e . g . , a “ pair of

For example , in one implementation , the TV media pro - XYZ - designer sunglasses ” as shown in a scene during the gram table 237a ( e . g . , see also 237a in FIGS . 2A - 2B ) may TV program ( e . g . , see FIG . 7G ) . In a further implementation , comprise a sub - table comprising embedded advertisement the AD - SURVEY may redirect the user to a URL tags . For example , in one implementation , an exemplary 20 20 " www . buybags . com / XYZ / spring2000 ” if the user clicks to

learn more about the product . In one implementation , the XML record of a media program data structure with ad tags AD - SURVEY may generate synchronized advertisement to 343a may take a form similar to the following : a user based on the ad tag by providing a pop - up static ad 338b , e . g . , " Do you like Penny ' s sunglasses ? " ( e . g . , see

PUT / AdTag . php HTTP / 1 . 1 25 720b in FIG . 7F ) . In another implementation , the AD Host : www . TV . com SURVEY may generate an interactive ad 338b including a Content - Type : Application / XML tagged screenshot of the TV program containing the placed Content - Length : 718 product ( e . g . , see 750c in FIG . 7G ) . < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > In another embodiment , the AD - SURVEY server 220 may < Media >

< MedialD > 123456789 < / MediaID > 30 generate questions 338b synchronized and / or related to the < MediaName > The Big Bang Theory < / MediaName > TV program ads 343a , which may be devised by the < Content > AD - SURVEY based on the media content the user has

< Season > 3 < / Season > viewed , the advertisement the user has viewed , and / or the < Episode > 2 < / Season > like . For example , if the channel the user has been watching < MediaLength > 68 ' 34 " < / MediaLength > 35 recently has played advertisement of “ Audi , ” the AD - SUR < Keywords > Vampire , Blood , South , < / Keywords > VEY may prompt a question to the user such as “ which

< / Content > brand automobile would you prefer ? ” to determine the < MediaChannel > CBS < / MediaChannel > advertisement effects . In another example , the user 233a < MediaAirTime > 9 pm 09 / 09 / 2000 < / MediaAirTime > may receive real - time information synchronized with a < MediaSource > XXX Production < MediaSource > 40 product placement embedded in the TV program . For < MediaGenre > Comedy < / MediaGenre > example , when a user is watching a TV show , he may < AdTagl > receive a prompt question 338b related to an embedded

< AdID > M0008 < / AdID > advertisement in a scene of the TV show , e . g . , “ Do you want < AdType > Regular < / AdType > to learn more about Penny ' s sunglasses ? " In a further < AdStartTimeStamp > 20 ' 34 ” < / AdStartTimeStamp > 45 implementation , the AD - SURVEY may redirect the user to < AdEndTime > 22 ' 45 " < / AdEndTime > a URL “ www . buythings . com / XYZ - designer / spring2000 ” if < AdSponsor > Audi < / AdSponsor > the user clicks to learn more about the product . < AdProductInfo >

< ProductName > Audi R8 < / ProductName > For example , in one implementation , the AD - SURVEY server 220 may provide a HTTPS PUT message including

< / AdProductInfo > 50 the questionnaire 338b in the form of data formatted accord ing to the XML . Below is an example HTTP ( S ) PUT

< / AdTag1 > message including an XML - formatted questions 338b to < AdTag2 > provide to the user 233a : < ADID > M0009 < / AdID > < AdType > Product Placement < / AdType > < AdStartTimeStamp > 25 ' 54 " < / AdStartTimeStamp > 55 < AdEndTime > 28 ' 45 " < / AdEndTime > PUT / question . php HTTP / 1 . 1 < AdSponsor > XYZ - Designer < / AdSponsor > Host : www . TV . com < AdProductInfo > Content - Type : Application / XML

< ProductName > French Style Sunglasses Content - Length : 718 < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > < / ProductName > < Question >

< QuestionID > Q000123456789 < / QuestionID > < / AdProductInfo > < QuestionName > Automobile Inquiry < / QuestionName > < AdPrompt > < TemplateType > static < / TemplateType > “ Do you want to learn more about the red bag in < TemplateID > TD0001 < / TemplateID > the scene ? " < Question Description > < / AdPrompt > < body > < AdRedirect > “ What automobile brand would you like ? ” www . buythings . com / XYZ / spring2000 < / body >

11

65

25 US 9 , 807 , 442 B2

26 - continued participated in a survey . See her response ” ( e . g . , see 731c in

FIG . 7C ) . The user may also obtain his friends ' AD < Option1 > A . BMW < Option ] > SURVEY activities , including participation in surveys , pur < Ooption2 > B . Audi < / Option2 > chases 349 , etc . , from social media news feed .

< / QuestionDescription > FIG . 3B provides a logic flow diagram illustrating gen < Question Channel > CBS < / QuestionChannel > erating a survey question within embodiments of the AD Question MediaID > 123456789 < / Question MediaID > SURVEY . Within embodiments , upon user submitting a

< QuestionGenre > Product Ad < / QuestionGenre > channel selection 305 , the AD - SURVEY server may check QuestionPromptTime > 50 ' 56 " < / Question PromptTime > program table to determine what ' s on air on the selected < / Question > 10 channel 306 , and retrieve a program table to obtain ads

tagged in the selected TV program 307 . In one implementation , the AD - SURVEY may parse In one embodiment , upon receiving questions and / or ads commercial ad information on the selected channel 308a , to at the mobile device , the user may submit a response 339 to extract key terms . For example , in one implementation , the

the AD - SURVEY server , e . g . , an answer to the question , a 15 AD - SURVEY may retrieve the advertised product brand click on the provided ad URL , and / or the like . In another name , product name , category , etc . In one implementation , implementation , upon viewing an embedded advertisement the AD - SURVEY may query on a questionnaire database while watching a TV program , the user 233a may desire to based on the parsed ad key terms 310 . For example , for an learn more or purchase the product , and submit a request of “ Audi ” commercial , the AD - SURVEY may parse key terms purchase 365a ( e . g . , by clicking on “ Buy it Now ” 750c in 20 as “ Audi , " " car , " " automobile , " and select and generate FIG . 7G ) to the AD - SURVEY server 220 . The AD - SURVEY pop - up questions related to such key terms to the user 312 . 220 may forward the purchase request 365b to a merchant In another implementation , the AD - SURVEY may incor website 280 , and redirect the user to view the merchant site porate a variety of user media content exposure data to to obtain more information of the interested product . In one generate media content based survey questions 308b . For implementation , the AD - SURVEY may log the user ques - 25 example , AD - SURVEY may incorporate mobile ads expo tion responses , indication of interests , purchase transaction , sure data ( e . g . , user web visits , AD - SURVEY generated and / or the like at the database 219 to indicate ad effects . mobile ads , etc . ) , user application status ( e . g . , browsing

For example , in one implementation , the AD - SURVEY history , Internet gaming content , etc . ) , social content ( e . g . , server 220 may provide a HTTPS PUT message including social pages , social ads , friends ' recommendations , user ' the questionnaire response 339b , purchase request 365a in 30 30 likes , etc . ) 305b , and / or the like . In one implementation , the

AD - SURVEY may receive the various user content expo the form of data formatted according to the XML . Below is an example HTTP ( S ) PUT message including an XML sure data from a AD - SURVEY client component instantiated

on the user mobile device , e . g . , an iPhone app , etc . In formatted questions responses / purchasing request to provide to the database 219 : another implementation , the AD - SURVEY may receive

mobile data from a mobile meter , a proxy server , a TV metering system , and / or the like .

PUT / responses . php HTTP / 1 . 1 In one implementation , the AD - SURVEY may generate Host : www . AD - SURVEY . com synchronized pop - up survey questions to the user . For Content - Type : Application / XML example , in one implementation , the AD - SURVEY may Content - Length : 718 < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > 40 analyze the ad tags prior to the TV program on air , and < Response > prepare pop - up questions associated with each ad tag . The

< ResponseID > R000123456789 < / ResponseID > generated pop - up questions may be sent to the user accord < Timestamp > 19 : 00 : 00 < / Timestamp > ing to the timetable of the ad tags . In another implementa < Date > 09 - 09 - xxxx < / Date > < QuestionID > Q000123456789 < / QuestionID > tion , the AD - SURVEY may retrieve the user ' s viewing < QuestionName > Automobile Inquiry < / QuestionName > 45 history , e . g . , the TV programs the user has recently watched , < QuestionDescription > etc . , and determine the ads associated with the TV programs

< body > the user has watched to generate non - synchronized pop - up “ What automobile brand would you like ? ” survey questions . < / body > < Option1 > A . BMW < Option1 > Upon receiving the pop - up survey question 313 , the user < Ooption2 > B . Audi < / Option2 > 50 may elect to submit a response 315 , which may indicate ad

effects . In further implementations , the survey questions < / QuestionDescription > may be generated based on advertisement the user has < QuestionResponse > exposed to ( e . g . , via cross - channel ad measurement , as

< UserID > JS001 < UserID > further illustrated in FIG . 7E ) , social media contents , and / or < UserName > John Smith < / UserName 55 the like . < DeviceID > JSiPhone0002 < / DeviceID > FIG . 3C provides a logic flow diagram illustrating gen < Response > B < / Response > erating product placement ads within embodiments of the

< / QuestionResponse > AD - SURVEY . In one implementation , continuing on with retrieving ad tags in the TV program on air on the user < / Response > 60 selected channel 307 , the AD - SURVEY server may query on what ads are available 328 in an ad database . For example ,

In further implementations , the AD - SURVEY may popu the AD - SURVEY may obtain a list of ad tags and it late social media feeds of the user ' s questionnaire responses / associated types , whether a screen shot for product place purchase information 365b to a social media platform 250 . ment is available , etc . In one implementation , if there is a In another implementation , the user may share purchase 65 synchronized ad tag 330 , the AD - SURVEY may prepare information 238 to the social media 250 . For example , the synchronized ad generation prior to the timestamp of the ad user ' s Facebook news feeds may comprise a message " XXX tag 333 . In one implementation , AD - SURVEY may deter

US 9 , 807 , 442 B2 27 28

mine an ad type 335 , e . g . , whether it is static ad or interactive one implementation , if the user submits a purchase request ad . In one implementation , if it is a static ad , the AD - 378 , the AD - SURVEY may provide a merchant URL , and / or SURVEY may select a static ad template and populate the direct the user to a merchant page to proceed with purchase product information into the template , and provide a textual transaction 377a . The AD - SURVEY may log user activities pop - up ad at a synchronized time 340 to the user . In another 5 associated with the product placement advertisement 377b , implementation , if the ad tag indicates an interactive ad is e . g . , with an ad ID , etc . available , AD - SURVEY may retrieve a tagged screenshot In one implementation , the AD - SURVEY may aggregate associated with the interactive ad tag . For another imple - data analysis results from all different types of messages mentation , the AD - SURVEY may generate a video frame received from the user and run aggregate analytics 375 for grab comprising the product placement tags 338 via video 10 ad effects . FIG . 3E provides a logic flow diagram illustrating frame grabbing software ( e . g . , Windows Media Player , ad delivery / effects analysis within embodiments of the AD Quicktime Player , etc . ) . In one implementation , the AD SURVEY . For example , in one implementation , AD - SUR SURVEY may retrieve an interactive ad template and popu - VEY may obtain a correlation of an advertisement and user late the video frame into the template to provide the inter perception of the product , based on responses to the survey active ad ( e . g . , see FIG . 7G ) at a synchronized time 342 to 15 question ( e . g . , if a user selects “ Audi ” in an automobile the user . survey , after viewing an “ Audi ” advertisement , etc . ) . In

In one implementation , if the TV program at the user further implementation , AD - SURVEY may generate an ad submitted channel contains no synchronized ad tags 330 , the effect score for each advertisement . AD - SURVEY may elect not to send ads / questions to the In one implementation , AD - SURVEY may retrieve an user . In another implementation , the AD - SURVEY may 20 advertisement and determine an ad classification 380 , e . g . , a retrieve user ' s recent viewing history ( e . g . , the past week , category of the advertised product ( e . g . , apparel , accesso etc . ) , and generate a non - synchronized ad / survey question to ries , automobile , electronics , etc . ) . For example , for an the user based on the user ' s recently viewed TV programs . “ Audi ” advertisement , the AD - SURVEY may query for In further implementations , the AD - SURVEY may re - send stored questions ( e . g . , 368 in FIG . 3D ) results with the ad ads that were synchronized with one of user ' s recently 25 classification " automobile ” 381 . If such survey responses viewed TV programs to the user . are available 382 , the AD - SURVEY may query the retrieved

In further implementations , the synchronized product survey responses for mentions of the product 383 , e . g . , on a placement ads may be applied to in - game ads in a similar brand name “ Audi , ” on a make and model of the “ Audi ” manner . For example , the AD - SURVEY may determine a automobile , etc . For each response , the AD - SURVEY may user is engaging in a gaming application via the received 30 determine a relevance of the question and question results device application event ( e . g . , 241 in FIG . 2B ) , and deliver 382 . For example , the AD - SURVEY may retrieve the cor an interactive advertisement of related virtual goods , e . g . , responding question with the question results based on a gaming points , widget , etc . , to the user mobile device . question ID ( e . g . , stored at in FIG . 3E ) , and perform text

FIG . 3D shows a logic flow diagram illustrating ad analytics to determine a relevance level of the question to the synchronization question message processing ( e . g . , 2890 ) 35 brand name product “ Audi ” automobile ( e . g . , based on within embodiments of the AD - SURVEY . Within embodi - whether the question contains key terms such as " prefer ments , upon receiving a message from a user and determin ence , " " car purchase , " etc . ) . In one implementation , the ing the type of the message as an ad / question response 289c , AD - SURVEY may determine an ad effect weight value for the AD - SURVEY may determine whether the received mes the response 385 . For example , the AD - SURVEY may sage comprise a response to a survey question 362 . 40 perform text analytics of the question and questions results ,

In another implementation , if the received message com - and if the textual question / question results contain key terms prises responses to prompt questions 362 , the AD - SURVEY such as “ car purchase ” and “ Audi , ” such response may be may determine a classification of the question 365 , e . g . , a accorded with a high weight value . In one implementation , response to survey , a response to embedded advertisement , the weighting value determination at 385 may be based on and / or the like . In another implementation , the question 45 a pre - stored weight evaluation table , e . g . , 0 . 5 for submitting responses may be classified by the products , e . g . , automo - a response of “ Audi , " 0 . 8 for clicking on a merchant site , biles , apparels , electronics , and / or the like . In one imple - 50 . 0 for transacting a sale on an “ Audi ” automobile , " etc . , mentation , the AD - SURVEY may extract a questionnaire ID and calculate an impact score of the advertisement based on and / or a survey ID 366 from the received user response , and a group of users ( e . g . , see 10191 at FIG . 10 ) . The AD store the questionnaire results 368 associated matched with 50 SURVEY may generate an ad effect sub - score based on the questionnaire based on an ID query . analytics of user survey responses 386a , e . g . , taking a

In a further implementation , the user who responds to weighted sum , etc . questionnaires may be credited for a reward . For example , In another implementation , the AD - SURVEY may query after obtaining and storing questionnaire results , the AD - for stored user activities ( e . g . , stored at 377b in FIG . 3D ) in SURVEY may determine rewards for the user 370 , e . g . , five 55 response to an ad ( e . g . , purchasing request , click for more AD - SURVEY points for each question answered , etc . , and information , etc . ) base on an ad ID 382 . If such activities are credit the points to the user ' s AD - SURVEY account 372 . In available 387 , the AD - SURVEY may determine a type of the another implementations , the rewards may comprise virtual activity 388 , e . g . , clicks on the ad for more information , currency ( e . g . , store points , gaming points , etc . ) , coupons , clicks on a provided merchant URL , user rating of the discounts , and / or the like , sponsored by an advertising 60 advertised product , clicks on “ Buy It Now , " closing the ad merchant . without browsing , and / or the like . Based on the activity type ,

In another implementation , when the AD - SURVEY deter - the AD - SURVEY may associate a weight value for the mines the response does not comprise a response to a survey activity 389 , and generate an ad effect sub - score based on question at 362 , the AD - SURVEY may determine whether it analytics of user activities 386b , e . g . , taking a weighted sum , comprises an interactive activity indication 376 . For 65 etc . For example , a click on a merchant URL may be example , a user may submit a rating of the product with the accorded with a high positive weight value ; and a prompt interactive ad , click on the interactive ad , and / or the like . In window close of the ad may be accorded with a low or zero

US 9 , 807 , 442 B2 29 30

weight value , and / or the like . In one implementation , the Media Listen - In Component 440 may configure , aggregate AD - SURVEY may generate an indication of ad effects 390 and packetize atmospherics data captured by the sensor based on an integrated ad effects score ( e . g . , taking a sum of module component 420 in a data format suitable for data the subscores from 386a and 387b , etc . ) . transmission via the sensor I / O 425 . In a further implemen

In one implementation , AD - SURVEY may compare the 5 tation , the Media Listen - In Component 440 may process and ad effects score of the same ad on different media channel , analyze the obtained atmospherics data , e . g . , a photo cap to determine efficiency of the ad placement . For example , if tured by the mobile device , etc . , to identify whether the user the same “ Audi ” ad has a higher impact score on channel is watching , and / or how many individuals are watching from “ ESPN ” than “ Disney , " it may provide heuristics to the the photo , via image processing . For example , in one merchant that such advertisement is more efficient on 10 embodiment , the iPhone SDK toolkit and / or runtime librar “ ESPN . ” In further implementations , AD - SURVEY may ies may be installed and / or used to perform such image determine efficiency of the time and the TV program to place processing . the advertisement based on the ad effects score . In one embodiment , the AD - SURVEY Controller 405

FIG . 4A provides a block diagram illustrating a AD - may further be coupled to a communications module 430 , SURVEY client component 401 within embodiments of the 15 configured to interface with and / or process data transmission AD - SURVEY . Within embodiments , a AD - SURVEY com - from communications I / O components 435 . The communi ponent 401 may contain a number of sub - components and / or cations I / O components 435 may comprise components data stores . AAD - SURVEY client controller 405 may serve facilitating transmission of electronic communications via a a central role in some embodiments of AD - SURVEY opera variety of different communication protocols and / or formats tion , serving to orchestrate the reception , generation , and 20 as coordinated with and / or by the communications module distribution of data and / or instructions to , from and between 430 . Communication I / O components 440 may , for example , client mobile device ( s ) and / or the server via AD - SURVEY contain ports , slots , antennas , amplifiers , and / or the like to components and in some instances mediating communica - facilitate transmission of TV program listing information , tions with external entities and systems . user submission of channel selection , user responses to

In one embodiment , the AD - SURVEY controller 405 25 survey questions , and / or the like , via any of the aforemen and / or the different components may be instantiated on a tioned methods . Communication protocols and / or formats user mobile device , e . g . , an Apple iPhone , etc . In an alter for which the communications module 230 and / or commu native embodiment , the controller may be housed separately n ications 10 components 435 may be compatible may from other components and / or databases within the AD - include , but are not limited to , GSM , GPRS , W - CDMA , SURVEY system , while in another embodiment , some or all 30 CDMA , CDMA2000 , HSDPA , Ethernet , WiFi , Bluetooth , of the other modules and / or databases may be housed within USB , and / or the like . In various implementations , the com and / or configured as part of the AD - SURVEY controller . munication I / O 435 may , for example , serve to configure Further detail regarding implementations of AD - SURVEY data into application , transport , network , media access con controller operations , modules , and databases is provided trol , and / or physical layer formats in accordance with a below . 35 network transmission protocol , such as , but not limited to

In one embodiment , the AD - SURVEY controller 405 may FTP , TCP / IP , SMTP , Short Message Peer - to - peer ( SMPP ) be coupled to one or more interface components and / or and / or the like . The communications module 430 and com modules . In one embodiment , the AD - SURVEY Controller munications I / O 435 may further be configurable to imple may be coupled to a user interface ( UI ) 410 . The user ment and / or translate Wireless Application Protocol ( WAP ) , interface 410 may be configured to receive user inputs and 40 VoIP and / or the like data formats and / or protocols . The display application states and / or other outputs . The UI may , communications I / O 435 may further house one or more for example , allow a user to adjust AD - SURVEY system ports , jacks , antennas , and / or the like to facilitate wired settings , select communication methods and / or protocols , and / or wireless communications with and / or within the manually enter texts , engage mobile device application AD - SURVEY system . For example , the communication I / O features , and / or the like . In one implementation , the user 45 432 may be extended by a plug - in accessory as shown at 120 interface 410 may include , but not limited to devices such as , in FIG . 1 . keyboard ( s ) , mouse , stylus ( es ) , touch screen ( s ) , digital dis - Numerous data transfer protocols may also be employed play ( s ) , and / or the like . In another implementation , the user as AD - SURVEY connections , for example , TCP / IP and / or questionnaire component 430 may provide user survey higher protocols such as HTTP post , FTP put commands , questions and receive user responses via the user interface 50 and / or the like . In one implementation , the communications 410 . module 430 may comprise web server software equipped to

In one implementation , the AD - SURVEY Controller 405 configure application state data for publication on the World may further be coupled to a sensor module 420 , configured Wide Web . Published application state data may , in one to interface with and / or process signals from sensor input / implementation , be represented as an integrated video , ani output ( 1 / 0 ) components 425 . The sensor I / O components 55 mation , rich internet application , and / or the like configured 425 may be configured to obtain information of environ - in accordance with a multimedia plug - in such as Adobe mental conditions , and / or the like to generate atmospherics Flash . In another implementation , the communications mod data that may be received and / or processed by other AD - ule 430 may comprise remote access software , such as SURVEY components . A wide variety of different sensors Citrix , Virtual Network Computing ( VNC ) , and / or the like may be compatible with AD - SURVEY operation and may be 60 equipped to configure user application ( e . g . , a user mobile integrated with sensor I / O components 425 , such as but not device ) . In another implementation , the communications limited to a camera , an audio recorder , a GPS component , module 430 may transmit TV program listing information to and / or the like , configured to capture video clips / photos of the real time TV remote control component 415 , which may what is playing on the TV and / or whether the user is in turn receives user channel selection form the user inter watching the program , audio recording clips indicative of 65 face 410 . what is playing on the TV , GPS information indicative of the In further implementations , the AD - SURVEY 405 may be user ' s location , and / or the like . In one implementation , the configured to communicate with a content based embedded

US 9 , 807 , 442 B2 31

advertising component 420 , media content questionnaire implementation , the table top device 473 may comprise a synchronization component 435 , and / or the like ( as further rechargeable battery 478 for power supplies from charging discussed in FIGS . 3B and 3C ) . inputs 480 .

FIG . 4B provides a combined logic and data flow diagram The table top device 473 may facilitate mobile remote illustrating AD - SURVEY client and server interactions 5 control to operate in a similar manner as the AD - SURVEY within embodiment of the AD - SURVEY . In one embodi - accessory 1120 in FIG . 1B . Instead of being plugged into a ment , the TV remote control component 415 may submit a user mobile device , the table top device 473 may commu channel selection event 416 to the AD - SURVEY server 450 , nicate with a user device running the remote mobile appli which may in turn query a media program database 419 for cation 471 via wireless connections . For example , the table TV program information . For example , in one implementa - 10 top device may be operated for remote TV channel control , tion , the AD - SURVEY server 450 may form a first query collecting atmospherics data for audience monitoring , and / 417 on a media program table 419a based on the user ' s or the like . In further implementations , the table top device channel selection , to determine the TV program the user is may be positioned so that the table top device may capture watching and send such program information 418 back to images from the screen of the TV set . the user ; and a second query on an Ad Tags table 419b to 15 In one implementation , the table top device may be determine what advertisement and embedded product place configured to periodically monitor audio contents , video ment advertisement the user may received during the TV contents , etc . , in the atmosphere , with or without having user program streaming . The queried results , including the pro - input of a channel selection . For example , the table top gram information and ad information , may be fed into an Ad device may record an audio / video clip of media program synchronization component 455 at the AD - SURVEY server 20 being played , and send such audio / video clip ( or generate a 450 , which may generate ad synchronization popup ques - signature / hash based on sampling of the recorded audio / tions 421 based on the ad the user may be watching . video media content ) to the AD - SURVEY server , which may

In one implementation , the generated ad synchronization in turn determine what the user is watching . Further imple questions 421 may be received and provided to the user via mentations are discussed in FIG . 2E . a user interface generated by the user questionnaire compo - 25 FIGS . 5A - 5B provide combined data flow and logic flow nent 430 on the user ' s mobile device , which may in turn diagrams illustrating data downloads from social media provide user ' s response events 422 to an Ad effect engine platforms within implementations of the AD - SURVEY . component 460 at the MR - PLATOFRM server 450 to ana Within implementations , AD - SURVEY may obtain social lyze advertisement effects . media data to measure , and / or influence consumer consump

In one embodiment , the media listen - in component 440 30 tion of media and advertising . For example , AD - SURVEY may collect and aggregate atmospherics data 423 , e . g . , video may track social media discussion to obtain comments , recording clips , audio recording clips , photo streams , GPS mentions , responses related to an object under evaluation information , and / or the like , to a media viewing statistics ( e . g . , a brand name product , a TV show , etc . ) . In one analysis component 465 , which may analyze the media implementation , a user may befriend with panelists to viewing data , and determine the audience reception rate of 35 “ Friend " on Facebook , and / or allow a panelist on Twitter , a TV program and / or advertisement . wherein the panelist may be a AD - SURVEY personnel ,

In a further implementation , the social media connection and / or an avatar , etc . The AD - SURVEY may then obtain component 445 may generate and transmit social media post users ' conversations , wall posts , pages of brands / products 448 indicating the user ' s viewing status to a social media the user follows , products / content the user “ like ” , social database 469 , and may optionally send the social media post 40 advertising the user has been exposed to , clicks on pages , to the AD - SURVEY 450 as well . In one implementation , the etc . , to obtain social media exposure data . In one implemen AD - SURVEY server 450 may obtain the user ' s social media tation , the AD - SURVEY may incorporate comments of status updates information , including friends ' recommenda - representative panelists , identification of friends across tions , comments , and / or the like via an API call 471 to the panel , social activities profiling of panel , and / or the like . social media database 469 . In another implementation , the 45 In a further implementation , the AD - SURVEY may track AD - SURVEY server 450 may redirect a user to the social social media content ( e . g . , Facebook and Twitter , etc . ) of media website 472 from the user ' s instantiated AD - SUR - AD - SURVEY consumers , e . g . , users who has authorized VEY client component to engage in social media activities . AD - SURVEY to access their social media content . In one For example , the user may click on a social media link via implementation , the AD - SURVEY may link demographic , the AD - SURVEY client component user interface and be 50 behavioral , and attitudinal data from the user ' s profile infor redirected to the social media page . mation with social media behavior . The social media data

FIG . 4C provides a block diagram illustrating an example downloading may be obtained via API calls , as discussed in infrastructure of a AD - SURVEY table top unit component FIGS . 5A - 5B . within implementations of the AD - SURVEY . Within imple - In a further implementation , the AD - SURVEY may mentations , the table top device 473 may be a standalone 55 recruit consumers ( e . g . , Facebook , Twitter users who have device that may be placed on a desk , wherein the user may allow AD - SURVEY to access their social content ) as AD be requested to place it to face the screen of a TV set or the SURVEY panelists , e . g . , by providing incentive rewards to TV STB . In one implementation , the table top device 473 the users , etc . In one implementation , the AD - SURVEY may may communicate with a user mobile device , a laptop track how social media messages propagate throughout a computer , a desktop computer , and / or the like , via a Blu - 60 network of social media users ( e . g . , the recruited panelists , etooth transmitter / receiver interface 472 - 474 . In one imple - etc . ) , based on the profiles of the individuals . Such measures mentation via Bluetooth connection 476 . a user may down - of connectivity may be analyzed to measure propagation of load and install a remote control mobile application 471 . In marketing communications . another implementation , the table top device 473 may com - In another implementation , each panelist may be associ municate with a TV set 483 , a set top box 486 , DVR 65 ated with a social media specific profile so that their social equipment , and / or the like , via infrared transmitter / receiver media activities may be tracked to determine whether they interfaces 475 , 485 / 487 via infrared connection 481 . In one are influencers in certain categories , disseminators of infor

33

. . .

? ??

US 9 , 807 , 442 B2 34

mation , information consumers , and / or the like . For - continued example , in one implementation , a panelist may be labeled as a “ The Big Bang Theory Fan Wiki , ” so that users < UserProfile > interested in the show " The Big Bang Theory ” may follow < UserName > Yes < / UserName >

< UserAge > Yes < UserAge > the panelist to obtain information of the show via the 5 < UserDOB > Yes < / UserDOB > panelist ' s posts , comments , and / or the like , related to “ The < Work > Yes < / Work > Big Bang Theory . ” < Education > Yes < / Education >

In one implementation , individual social media profiles < Pages > Yes < Pages > < Network > Yes < / Network > may be incorporated to assess advertising targeting perfor

mance , enable advertisers to plan social media campaigns by 10 < / UserProfile > targeting product influencers , and / or the like . < Friends >

As shown in FIG . 5A , AD - SURVEY server 220 may send < Number > Yes < / Number > < Friends List > No < / FriendsList > an access request to a social media platform ( e . g . , Facebook ,

Twitter , Google + , etc . ) for access to a user ' s profile infor < / Friends > mation and social media content , e . g . , news feeds , posted 15 < Activities >

< Postson WallSelf > Yes < Postson WallSelf > photos , Tweets , comments , activities ( “ Likes , ” “ Dislikes , ” < Postson Frieds Wall > No < PostsonFriends Wall > etc . ) . < Others WallComments > No

In one implementation , upon receiving the access request < Others WallComments 505 , the social media 250 may generate and send a user authorization request 510 to the user 233a . For example , 20 < Activities >

Facebook and / or Twitter may send an email to the user 233a , < / scope > wherein the email may comprise an authorization link directing the user to a AD - SURVEY authorization page < / Authorization > ( e . g . , as included in the access request 505 ) . In one imple mentation , the user 233a may be access the included autho - 25 In the above example , the authorization message to Face rization link via a mobile application UI 511a ( e . g . , see FIG . book may comprise information as to the scope of informa 8A ) , and / or a web based application UI 511b ( e . g . , see FIG . tion access , e . g . , the user may permit AD - SURVEY to 8B ) . As shown in FIG . 8A , the user may click “ Allow " to access the user “ JDOE ’ s ” Facebook profile including his grant permission of AD - SURVEY to access the user ' s social media content . In further implementation , as shown at 827 30 name , age , date of birth , work an education information , in FIG . 8C , the user may configure a scope of information interested pages , network , and / or the like ; a number of the AD - SURVEY may be allowed to access . friends of “ JDOE , ” but may not access an exact friends list .

For example , in one implementation , the user ' s mobile The user may allow AD - SURVEY to obtain “ JDOE ' s ” posts application and / or a web - based application may generate a on his own wall , but may not permit access to his posts on ( Secure ) HTTPS PUT authorization message 515 including 35 his friends ' wall or friends ' comments on his wall , and / or the an application ID and permission indication for the social like . media platform in the form of data formatted according to In one implementation , the social media 250 may pass on the XML . Below is an example HTTP ( S ) PUT message the application ID from the user ' s mobile or web application including the XML - formatted access authorization message and generate a user token 518 to AD - SURVEY for confir provide to Facebook : 40 mation of access permission . In one implementation , the

AD - SURVEY may determine when data update is needed 520 , e . g . , the data update from social media may be per

PUT / AccessRequest . php HTTP / 1 . 1 formed on a periodic basis ( e . g . , daily , weekly , etc . ) . The Host : 172 . 16 . 244 . 1 AD - SURVEY server 220 may generate a data request 522 Destination : www . Faceobook . com 45 together with the received user authorization token ( e . g . , Content - Type : Application / XML Content - Length : 518 518 ) and transmit to the social media platform . < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > In one implementation , the data request 522 may be sent < Authorization > to the social media platform via a user oAuth protocol , and

< AuthID > 00001 < AuthID > comprise a AD - SURVEY application ID , and / or a user < Time > 0 : 00 : 00 > / Time > < Date > 1 - 12 - XXXX < / Date > 50 social media ID , and / or the like . For example , in one < UserID > JDOE < UserID > implementation , the AD - SURVEY server 220 may provide < AppID > Mobile0001 < / AppId > a ( Secure ) HTTPS PUT message including a data request < Permission > Yes < / Permission > 522 for Facebook in the form of data formatted according to < scope > / * Name of Permission elements like User

Age , Friends , Messages , Likes etc that AD - SURVEY application the XML . Below is an example HTTP ( S ) PUT message may have access to * / including the XML - formatted access request provide to

Facebook :

PUT / AccessRequest . php HTTP / 1 . 1 Host : www . AD - SURVEY . com Destination : www . Faceobook . com Content - Type : Application / XML Content - Length : 518 < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > < Request >

< RequestID > 000001 < / RequestID > < TimeStamp > 00 : 00 : 00 < / TimeStamp > < Destination > www . Facebook . com < / Destination >

US 9 , 807 , 442 B2 35 36

- continued < UserID > JDOE < / UserID > < ClientID > ccccccc < / ClientID > / * AD - SURVEY Mobile Remote Application

Id or web - based AD - SURVEY Panel management App Id obtained after registering the application with Facebook * /

< RedirectURL > www . AD - SURVEY . com / user ? 8989898988 . com < / RedirectURL > / * AD - SURVEY application URL where the user may be redirected after authorization * /

< Content > / * Name of Permission elements like User Age , Friends , Messages , Likes etc that AD - SURVEY application may have access to * /

< UserProfile > < UserName > Yes < / UserName > < UserAge > Yes < UserAge > < UserDOB > Yes < / UserDOB > < Work > Yes < / Work > < Education > Yes < / Education > < Pages > Yes < Pages > < Network > Yes < / Network >

< / UserProfile > < Friends >

< Number > Yes < / Number > < FriendsList > No < / FriendsList >

< / Friends > < Activities >

< Postson WallSelf > Yes < Postson WallSelf > < PostsonFriedsWall > No < PostsonFriends Wall > < Others WallComments > No < Others WallComments >

< / Activities >

< / Content >

< / Request >

In the above example , the data request generated by an education information , interested pages , network , and / or AD - SURVEY to Facebook may comprise a user ID “ JDOE ” the like . The AD - SURVEY may also request to obtain indicating the request is directed to Facebook information of as information of a number of friends of “ JDOE , ” but may not the user “ JDOE ” ; a client ID indicating the application ( e . g . , request to access an exact friends list . The AD - SURVEY the AD - SURVEY mobile application ID , etc . ) indicating a may further request to obtain “ JDOE ' s " posts on his own source of the request ; and a URL link which may be wall , but may not request to obtain his posts on his friends ' provided to the user for authorization , e . g . , a link that wall or friends ' comments on his wall , and / or the like . requests the user to click a " OK ” or “ Cancel ” button on the For another example , in one implementation , the AD page to authorize or deny AD - SURVEY to gain access to the SURVEY server 220 may provide a ( Secure ) HTTPS PUT user ' s Facebook content . The access request may further message including a data request 522 for Twitter server in comprise information as to the scope of information access the form of data formatted according to the XML . Below is e . g . , AD - SURVEY may request to access the user “ JDOE ' s " an example HTTP ( S ) PUT message including the XML Facebook profile including his name , age , date of birth , work formatted access request provide to Twitter :

PUT / AccessRequest . php HTTP / 1 . 1 Host : www . AD - SURVEY . com Destination : www . twitter . com Content - Type : Application / XML Content - Length : 518 < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > < Request >

< RequestID > 000002 < / RequestID > < TimeStamp > 00 : 00 : 01 < / TimeStamp > < Destination > www . Twitter . com < / Destination > < UserID > JDOE < / UserID > < oauth _ consumer _ key > / * The Consumer Key for AD - SURVEY Remote

Application and / or AD - SURVEY panel management obtained after registering the application with twitter * /

NNNNNNN < / oauth _ consumer _ key > < oauth _ signature _ method > / * The signature method that the consumer

used to sign the request * / email

< / oauth _ signature _ method > < oauth _ signature > / * The signature as defined in twitter signing

requests > electronic < / oauth _ signature >

US 9 , 807 , 442 B2 37

- continued 38

< oauth _ timestamp > / * Time stamp of authoroization * / 00 : 00 : 02

< / oauth timestamp > < oauth _ nonce > / * Unique string to identify each request * / . 8888ddddd < oauth nonce >

< / Request >

- - -

In the above example of data request to Twitter , the - continued request may comprise a user signature request . For example , a user may provide electronic signature by clicking on a link < StartTime > 2006 < / StartTime >

< EndTime > present < / EndTime > ( e . g . , at 515 , etc . ) noting “ I hereby provide my signature by 15 pressing this button to allow AD - SURVEY access my Twit < / Employers > ter content , ” etc . < College > Good University < / College >

In one implementation , upon verification of the data < Classyear > 2000 < / Class Year > request by Facebook / Twitter / other social media platform , < / WorkandEducation > AD - SURVEY server 220 may download social media struc - 20 tured data 525a and unstructured data 525b ( e . g . , see 571a - b < Interests >

in FIG . 5B ) for media analytics . < Interests1 > Graduate Program < / Interests1 > < Interests2 > Cool Cars < / Interests2 > FIG . 5B shows a block diagram illustrating an example < / Interests >

infrastructure map of AD - SURVEY media analytics within embodiments of the AD - SURVEY . In one implementation , 25 < Friends >

< Number > 82 < Number > AD - SURVEY may provide a media analytics and reporting < List > Lisa Smith ; . . . . < / List > platform portal 570 for analyzing TV , Internet , mobile and social media data upon client request , e . g . , analyzing public < / Friends > feedbacks and comments on a brand name products , TV

< User > shows , and / or the like . In one implementation , the media 30 measurement and analytics portal platform 570 may provide class visualization , self service administration with seamless In one implementation , the analytics platform for pro integration between the different architectural components , cessing structured data may store the structured data , such as e . g . , 570a and 570b , etc . TV channel selection , mobile / web content data , social net

In one embodiment , the AD - SURVEY media measure - 35 work user profile data , etc . in a database 5750 . ment portal 570 may load data from social networks via a via a For another example , unstructured data 571b may com HTTP network 578 via API calls , e . g . , Facebook APPI 580a , prise raw text downloaded from social media platform , e . g . , Twitter API 580b , Google + API 580c , and other social data friends ' comments from Facebook , original Tweets , etc . In providers 580d . The media measurement portal 570 may one implementation , AD - SURVEY social analytics platform process the loaded data within different analytics platforms 40 570a / b based on loaded data types , e . g . , structured data 571a 570b may perform data mining on unstructured data 571b to

measure user feedbacks of a brand name product , TV or unstructured data 571b . For example , in one implementation , structured data 571a progran program , etc . For example , a query may be performed on the

may be already stored in a structured format when loaded unstructured data 571b to determine how many mentions of from the data source , such as , but not limited to user TV 45 “ The Big Bang Theory ” are posted by Facebook users . channel selection indication with timestamp , web displaying For example , in one implementation , an exemplary XML content with timestamp , social media user profile informa - record of unstructured Facebook user message 571b down tion , a number of user ' s social connections ( time stamped ) , loaded from Facebook may take a form similar to the a posted photo on social media platform with timestamp , following : and / or the like . For example , an exemplary XML record of 50 structured Facebook user profile data 571a downloaded

PUT / usercomments . php HTTP / 1 . 1 from Facebook may take a form similar to the following : Host : www . AD - SURVEY . com Content - Type : Application / XML Content - Length : 718

PUT / userprofile . php HTTP / 1 . 1 55 < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > Host : www . AD - SURVEY . com < SocialMessage > Content - Type : Application / XML < UserID > JDoe < / UserID > Content - Length : 718 < UserFirstName > John < / UserFirstName > < ? XML version = " 1 . 0 " encoding = " UTF - 8 " ? > < UserMI > Null < / UserMI > < User > < UserLastName > Doe < / UserLastName >

< UserID > JDoe < / UserID > < MessageID > 00001 < MessageID > < UserFirstName > John < / / UserFirstName > < Time > 19 : 00 : 45 < / Time > < UserMI > Null < / UserMI > < Date > 09 - 09 - 2011 < / Date > < UserLastName > Doe < / UserLastName > < SocialType > Like < / SocialType > < UserProfile Photo > Me . JPG < / UserProfilePhoto > < Object > < WorkandEducation > < Type > Facebook Page < / Type >

< Employers1 > < Name > Big Bang Theory < / Name > < EmployerName > Data 65 < Category > TV < / Category > Inc . < EmpoyerName > < Genre > Comedy < / Genre >

39 US 9 , 807 , 442 B2

40 - continued 583d , social analytics platform API 583e , and / or the like .

The social analytics platform API 583e may exchange data via API calls 587 with the single source ETL process < / Object > component 573e , such as user authorization tokens , and / or < Activity Time > 14 : 33 : 56 < / Activity Time >

< Activity Date > 09 - 09 - 2011 < / Activity Date > 5 the like . In one implementation , the taxonomy management con

< / User > sole 583b may define taxonomy tags and taxonomy logic rules . For example , the taxonomy may be defined at three

For further implementations , the unstructured data 571b different levels : a standard taxonomy specific to an industry may comprise a social post , a social media check - in status , us 10 and business function as provided by the platform vendor

( e . g . , tagging a unstructured data record by industry , etc . ) ; social mentions , and / or the like . taxonomy defined by analyst and subject matter experts In one implementation , the AD - SURVEY structured data ( e . g . , AD - SURVEY analytics defined taxonomy rules , etc . ) ; analytics platform comprises a variety of processing com and / or taxonomy defined by clients , and / or the like . For ponents , such as but not limited to user permission compo 15 example , for a Tweet “ Good adaption of the Southern nent for social media access 573a , TV / mobile / web measure Vampire series . Love the CBS actors / actresses . Expecting ment reports component 573b , media analytics engine 573c , the new season ” ( e . g . , 185 in FIG . 1D ) , the text analytics organization / user account management component 573d , engine 583c may apply a taxonomy rule to tag it by a single source data extraction , transformation and load ( ETL ) hierarchy of tags “ TV show - > CBS - > The Big Bang component 573e , and / or the like . Within implementations , 20 Theory - > Positive Feedback . ” For another example , the cli the user permission component may present panel users with ent ( e . g . , CBS , etc . ) may desire to categorize the commented a set of user interface screens requesting them to provide target of the Tweet , and therefore the taxonomy rule may permission for AD - SURVEY to access their social content comprise an additional sub - category " actor / actresses , " and / For example , AD - SURVEY may provide incentives of pro - or the like . In one implementations , such taxonomy rules motional rewards , points , coupons and / or the like to users 25 may be stored in a repository 585a , and the original social during questionnaire distribution ( e . g . , AD - SURVEY text data and social graph , tagged social data may be stored mobile questionnaires 238a , etc ) to allow AD - SURVEY at 585b . Further implementations and applications of the access to their social content . taxonomy rules are discussed in FIGS . 5E - 5F .

In one implementation , if the user agrees to provide the In one implementation , the social media measurement access to their social content , the permission component 30 reports 583a may provide a user feedback measurement 573a may get the authorization token from the respective report to a user via UI integration and data visualization . social platforms ( Facebook , twitter etc ) as per the authori - Within implementations , the reporting portal 583a may zation protocol and persist the token in a repository 575b . leverage the web based visual components ( e . g . word cloud , The user permission component 573a may pass the user traffic light components etc ) provided by the social media token and other application authorization details to social 35 analytics platform 570 for analysis that deliver insights analytics platform so that the user ' s social content can be purely on social media data . In further implementations , for extracted at a scheduled frequency . For example , AD - SUR insights generated on combined data set from social and VEY may periodically , constantly and / or intermittently load other data sources ( e . g . , via data mesh - up 572 between the social content data from the social media platforms via API structured data analytics platform 570a and unstructured calls once authentication is established . Further implemen - 40 data analytics platform 570b , etc . ) the taxonomy manage tations of user permission flows are discussed in FIGS . ment console 583b may be integrated to an integrated 5A - 5B . reporting portal to ensure the clients have the self service

In one implementation , the media analytics engine 5730 capability of defining the taxonomy and building the reports . may analyze the loaded structured data 573c , e . g . , per user For example , single sign on and UI widget meshup 572 may profile , per media type , etc . In one implementation , the 45 be adopted between the two platforms 570a and 570b for organization / user management 573d may create and manage integration of the two . user accounts with the AD - SURVEY . In further implemen - In one implementation , the AD - SURVEY media measure tations , the TV / mobile / web measurement reports componentm ent portal 570 may combine structured social media data 573b may generate media measurement reports ( e . g . , includ - with other data sources to generate cross media insights . The ing audience ratings , ad effectiveness , etc . ) based on struc - 50 social media analytics platform 570b may have the API to tured media data , such as audience TV channel selections , extract data for predefined metrics and attributes ( e . g . , mobile / web browsing activities , etc . taxonomy logics , data tags , etc . ) . The metrics and attributes

In one implementation , the AD - SURVEY unstructured may be predefined in the social media analytics tool to data analytics platform 570b may have a rule to define the compute from the structured and unstructured content specific fields for which the content needs to be extracted for 55 extracted from the social media platforms . a given user ( e . g . user demographics , no of friends , no of In one implementation , the AD - SURVEY media measure messages in a given duration , actual text for message and ment portal 570 may use different platforms for social media comments etc ) . Based on the authentication token the social reporting and structured data reporting , and have a tight media analytics platform 570b may query the social plat - integration at the data and UI layers of both the platforms . form for the content . In one implementation , the social 60 Additional implementations of the AD - SURVEY media analytics platform 570b may schedule the query tasks and measurement portal 570 may comprise : out of the box persist structured and unstructured content that is extracted connectors for social media platforms like Facebook , Twit

In one implementation , the AD - SURVEY social analytics ter , Google + , etc ( e . g . , at 583e , etc . ) ; web based taxonomy platform 570b may comprise a variety of processing com - creation and management user interfaces ; industry specific ponents , such as but not limited to , social media measure - 65 prebuilt taxonomies ( e . g . , at 583b , etc . ) ; text analytics ment reports component 583a , taxonomy management con - engine with sentiment analysis with learning algorithms sole 583b , text analytics engine 583c , social media adapters ( e . g . , at 583c , etc . ) ; ability to define fields for which social

US 9 , 807 , 442 B2 41

content needs to be extracted ( e . g . , at 583c , etc . ) ; data API In one implementation , if the user approves the access calls to extract large data set in and out of the platform in real request then in the response Facebook or Twitter may time and / or in batch mode ( e . g . , at 583d , etc . ) ; data API calls provide an authentication token associated with permission for UI widget integration to create mesh - up ( e . g . , at 572 , to the data elements ( e . g . , 518 at FIG . 5A ) . In one imple etc . ) ; multiple tenant support to ensure taxonomies defined 5 mentation , the authentication tokens may be requested with at organization and department level ( e . g . , at 583b , etc . ) ; constant permission to allow user data access even when the single sign on support with active directory service inter user is neither logged into AD - SURVEY user portal ( e . g . , faces ( ADSI ) and other light weight directory access proto - the mobile app 511a or the web based app 511b ) nor any col ( LDAP ) providers ( e . g . , at 572 , 573a , etc . ) ; components social media platform . In alternative implementations , the to generate insights from large historical data ( e . g . , over two 10 authentication token may request timely update and re terabytes of data ) ; components to schedule data extraction authentication from user on a periodic basis , e . g . , weekly , jobs from social media sites ( e . g . , at 583e , etc . ) ; components monthly , etc . In further implementations , the user may to support to host the service on a cloud , and / or the like . revoke access of AD - SURVEY from to their social media

FIG . 5C provide a logic flow diagram of user authoriza account if they wish to opt out . tion for social media access and social message ( e . g . , 289d 15 If the user granted the permission , AD - SURVEY may at FIG . 2C ) processing within implementations of the AD - save the authorization response token provided by the social SURVEY . In one embodiment , upon receiving a social media platform in a user repository 535a / b . The authenti message from a user , the AD - SURVEY may determine cation token may be used in all subsequent requests to social whether the user , and / or the AD - SURVEY is authorized to networking sites to access user ' s data elements . In another send social messages to the user ' s social media platform . 20 implementation , if the user did not grant permission , the user

In one implementation , the user 233a may allow AD - may continue with the mobile application or the web based SURVEY platform to access to their social network . For panel management website 536a / b . example , in one implementation , the user 233a may sign up Upon establishing and / or confirm user authorization for for AD - SURVEY permission via a AD - SURVEY mobile social media access , AD - SURVEY server may generate application 511a from a mobile device ( e . g . , an Apple 25 social messages to populate to social media platforms via iPhone , an Android , etc . ) . For another example , the user AD - SURVEY 537 , e . g . , see 275b in FIG . 2A , 365b in FIG . 233a may visit a AD - SURVEY social data panel manage - 3A , etc . ment website 511b ( e . g . , as illustrated in one example in FIG . 5D provides a logic flow diagram illustrating data FIG . 8B ) . In each case , AD - SURVEY may provide a sign up download from social media within implementations of the link to a social media platform to the user . Upon user 30 AD - SURVEY . Continuing on with 9 at FIG . 5A , AD clicking the link , the AD - SURVEY may verify whether the SURVEY may receive data updates from the social media user previously authorized 532a / b AD - SURVEY to access servers 540 . The AD - SURVEY may determine a source of their Facebook or Twitter information on their behalf and the social media data 543 , e . g . , Facebook , Twitter , Google + , their previous authorization token ( e . g . , 522 in FIG . 5A ) is and / or the like , and apply data format analytics rules based not expired . 35 on the determined data source 545 .

In another implementation , the AD - SURVEY may receive For example , if the data record is obtained from Face user attempts to log in and determine whether the user is book , the data format analytics may determine whether it is authorized 532a / b to grant social media data access permis - structured user profile information , a user posted photo , sion for AD - SURVEY . For example , when a user has unstructured user posting on the wall , others ' comments , received an email from a social media platform ( e . g . , Face - 40 and / or the like . For another example , if the data record is book , Twitter , etc . ) comprising a AD - SURVEY link for obtained from Twitter , the AD - SURVEY may determine authentication and the user clicks the link to proceed to grant whether it comprises raw text of a Tweet , and / or the like . permissions for AD - SURVEY , the user may deemed autho - In one implementation , if the received data is structured rized to configure social media permissions via AD - SUR - 538 , AD - SURVEY may parsing the structured data to extract VEY . In another implementation , when AD - SURVEY deter - 45 information 552 . For example , a structured user profile data mines the user is not authorized , AD - SURVEY may redirect record may be parsed per data field , e . g . , user _ id , user _ the user to a social media page 533a / b with a AD - SURVEY name , user _ DOB , user _ interests , and / or the like . The AD application ID and a permission attribute request , e . g . , as SURVEY may generate a data record including user discussed at 515 in FIG . 5A . ID / timestamp / geo source , etc . 555 and store the data record

In one implementation , the AD - SURVEY may determine 50 for the structured data 557 at a database ( e . g . , see 575a in whether access is approved by the user , e . g . , the user may FIG . 5B ) . select “ Yes ” or “ Cancel ” to indicate permission decision for In another implementation , if the received data is unstruc AD - SURVEY to access the user ' s social media content tured 538 , e . g . , raw texts of Facebook comments , Tweets , when the social media platform sent an email notification etc . , AD - SURVEY may feed the data to a Taxonomy engine indicating the access request from AD - SURVEY , e . g . , at 515 55 for data tagging 560 , as further illustrated in FIGS . 6A - 6C . in FIG . 5A . FIGS . 5E - 5F provide example data structures of social For example , if the previous authorization is not valid or media data updates within embodiments of the AD - SUR

if it is a first time access then the AD - SURVEY application VEY . For example , the data segments in FIGS . 5E - 5F show may use oAuth protocol to request Facebook or Twitter to database structure of social data elements associated with a provide access to user profile and messages ( e . g . , sending an 60 user , wherein various data fields may be linked by a user ID , access request 505 as shown in FIG . 5A ) . In such cases , the e . g . , " USER _ TBL _ ID . " user may be redirected to Facebook or Twitter web site with Within implementations , a AD - SURVEY analytics plat request to log in and provide access parameters , e . g . , see 515 form ( e . g . , 570 at FIG . 5C ) may extract and use different sets in FIG . 5A , and 827 in FIG . 8C . Facebook or Twitter Web of data elements from a user social media profile . For site may present a login and permission screen with option 65 example , in one implementation , as shown in FIG . 5E - 5F , a to allow or deny requested access to AD - SURVEY ( e . g . , see data record of a Facebook user profile may comprise data 810 at FIG . 8A ) . fields such as user work information , user education infor

US 9 , 807 , 442 B2 43 44

mation , television viewing history , interested music , inter In one implementation , AD - SURVEY may obtain query ested books , data feeds , likes , status message , comments on results 620 , which may comprise a number of followers on status message , wall posted comments , check - in history , the social media platform , a series of raw text comments and / or the like . from the social media , and / or the like . The AD - SURVEY

In one implementation , data elements are categorized 5 may then determine a presentation format 623 to provide the based upon how frequently they are updated and AD results to the user . For example , when the user elects to SURVEY may accordingly determine the fetch and refresh choose " raw texts " ( e . g . , see 850 in FIG . 8D ) , the user may schedules . For example , the categorization may comprise view a list of raw text comments 630 ( e . g . see 850a / b in FIG . users static descriptors , such as user demographic attributes 8D ) . In another implementation , AD - SURVEY may select like date of birth , gender , etc . , which may be relatively 10 an output visualization format and present the visualization

of query results to the user 625 , and the user may view “ constant ” and “ static , and thus may be determined to visualized results ( e . g . , see 860a - c in FIG . 8E ) . scheduled to update every 6 months . For another example , FIGS . 6B - 6C provide logic flow diagrams illustrating the categorization may comprise dynamic descriptors , such example taxonomy tagging logics within embodiments of as number of friends , Likes , television viewing , books and 15 the AD - SURVEY . In one implementation , as discussed in other preferences , which may be updated by a Facebook user FIG . 5C , unstructured data ( e . g . , raw texts of Facebook more frequently , and may be scheduled for updates monthly . comments , Tweets , and / or the like ) may be parsed and For another example , the categorization may comprise text tagged with category tags by a taxonomy engine ( e . g . , see messages , status , comment , messages , posts , etc . , and may 583b - 383c in FIG . 5C ) . be scheduled to update daily and / or on demand . In one 20 In one implementation , AD - SURVEY may apply tax implementation , AD - SURVEY may specify the requested onomy model logics , which may comprise any of a first content in a data request accordingly , e . g . , for user descrip - logical section associated with the semantic categorization tors only , for dynamic descriptor only , for text messages of keywords and a second logical section associated with the only , and / or any combination of the above . ( e . g . , see the sentiment keywords . In one implementation , the taxonomy “ RequestedContent ” field in the example Facebook data 25 may assign weighted scores to the logical tagging in con request 522 in FIG . 5A ) . junction with the semantic text .

FIG . 6A provides a logic flow diagram illustrating social For example , in one implementation , the semantic cat media analytics within embodiments of the AD - SURVEY . In egorization logic may be hierarchical and specific to a one embodiment , a user may submit a request for media domain to maintain simplicity of in model management and analytics 605 , e . g . , for a brand name product , a TV program , 30 run time executions , e . g . a specific taxonomy model for TV and / or the like . For example , a TV production company may shows , a specific taxonomy model for advertisement and desire to know audience comments about their TV program brand mentions , a taxonomy model for any combination of ( e . g . , see FIG . 1D ) . The AD - SURVEY server 220 , upon the above , and / or the like . receiving the user request , may generate a media analytics FIG . 6B shows an example TV shows semantic catego management panel user interface screen including a list of 35 rization model within implementations of the AD - SURVEY . analytics options ( e . g . , see FIG . 8C ) 610 , and the user may In one implementation , unstructured data related to TV submit media analytics parameters 613 . For example , as shows will be tagged with “ TV shows ” 640 as a level 0 tag , shown in FIG . 8D , the AD - SURVEY management panel and progressively be tagged with level 1 , 2 , 5 , etc based on may allow a user to select social media source ( e . g . , Face genre 641 , show names 642 , show network 643a , show book , Twitter , Google + , and / or the like ) , a time range , 40 episodes 643b , show cast member name 6430 , show char feedback activity type , and / or the like . acter names 643d , show episode names 643e , other keyword

In one implementation , AD - SURVEY server 220 may texts in the descriptor 643f and / or the like . For example , the download social media data updates ( e . g . , as discussed in example Tweet 185 in FIG . ID , “ The Big Bang Theory is a FIGS . 5A - 5C ) , and form a query based on the retrieved good adaptation of the Southern Vampires series . Love the social media data 615 based on the user inquiry . For 45 CBS actors / actresses . Expecting the new season , ” may be example , when the producer CBS would like to know tagged as “ TV show ” - > “ Soap ” ( genre ) - > “ Comedy ” ( sub audience feedbacks of the show “ The Big Bang Theory ” genre ) - > “ The Big Bang Theory ” ( show name ) , “ CBS ” ( net ( e . g . , see FIG . 1D ) , AD - SURVEY may search for data work ) , “ new season ” ( episode ) , “ geeks ” ( key words in related to “ The Big Bang Theory ” on the retrieved updated descriptor ) , and / or the like . social media data . 50 FIG . 6C shows an example advertisement brand aware

For example , in one implementation , AD - SURVEY may ness model within implementations of the AD - SURVEY . In form a query on the structured data based on key word “ The one implementation , unstructured data related to an adver Big Bang Theory ” and / or any character names and / or actors / tised and / or brand product may be tagged with all catego actress names for the associated profiles , e . g . , a Facebook ries ” 645 as a level 0 tag , and progressively be tagged with pages , Twitter profiles , and obtain a number of followers 55 level 1 , 2 , 5 , etc based product categories 646 , product brand from the structured data . In another example , AD - SURVEY 647 , ad content key words 648a , brand name 648b , brand may form a query based on the key term “ The Big Bang name + ad content key words 648c , product mentions 648d , Theory ” and obtain raw text comments containing the key product name / type / model 648e , key word text in descriptor term . In one implementation , AD - SURVEY may perform a 648f , and / or the like . For example , in one implementation , a progressive search over the raw text ( e . g . , unstructured data , 60 Facebook wall post “ the new Audi R8 commercial is so etc . ) . For example , AD - SURVEY may search for “ The Big cool ! ” may be tagged as “ Consumer Products ” ( all catego Bang Theory , ” and then refine the search results by “ The Big ries ) - > “ Automobiles ” ( categories ) - > “ Audi ” ( brand ) - > “ R8 ” Bang Theory AND CBS , ” and refine the search results by ( product model ) , “ commercial ” ( ad content ) , “ so cool ” “ The Big Bang Theory AND CBS AND Show , " and / or the ( product mention / feedback ) , and / or the like . like . In one implementation , AD - SURVEY may search the 65 FIG . 6D provides an example logic flow diagram illus unstructured data based on data tags associated therewith , as trating taxonomy tree definition within embodiments of the further discussed in FIGS . 6B - 6C . AD - SURVEY . Within embodiments , the taxonomy logic

45 US 9 , 807 , 442 B2

46 rules may be pre - defined by AD - SURVEY panel experts Excellent 693b , and / or the like . Similarly , if that is found , using structured data dictionaries . In one implementation , AD - SURVEY may generate a sub tag “ Coke Excellent ” with the AD - SURVEY may determine the “ low level ” categories the unstructured data 695 . based on structured data dictionaries 650 . For example , in Continuing on with FIG . 6F , the taxonomy engine may one implementation , the TV show model may be designed 5 progressively query on “ Coke ” and key word texts , e . g . , " too based on a TV program guide to ensure the reporting label much taste , " 696 , etc . If such key words are found 697 , the

AD - SURVEY may generate s sub tag ( e . g . , level 2 ) with the and naming is followed . In this way , the taxonomy may accelerate and automate the model development process . unstructured data 698a . If not , the AD - SURVEY may move

on with the next taxonomy rule . The AD - SURVEY may extract sample text 653 for a TV show from social media by a text miner ( e . g . , the text + 10 Within implementations , during the taxonomy rule execu

tion process , each taxonomy rule may be executed at a leaf analytics engine 583c in FIG . 5C ) to identify most com node and then the next higher level may be computed . Upon monly used key words by the social media users . For finishing with a taxonomy tree , the AD - SURVEY may example , user comments posted on the Facebook page of generate a matching score of the applied taxonomy logics “ The Big Bang Theory ” ( e . g . , see 186b in FIG . 1D ) may be 15 ID ) may be 15 699 . For example , the score may be based on a similarity sampled to extract key words ; if words such as “ Werewolf " percentage of the key terms in each node of the taxonomy ( characters ) , “ Bontemp ” ( fictional town in the story ) , etc . , tree and the compared unstructured data segment . To main are frequently used in the comments , the taxonomy engine tain efficiency , taxonomy execution models may be logically may adopt these words for taxonomy tags , e . g . , the key word partitioned . For example , for a segment of raw text , the same text in descriptor ( see 643f in FIG . 6B ) . This process may 20 unstructured text segment may be executed through different enrich the dictionary and provide inputs to defining tax - models and scores generated may be merged and aggre onomy rules . The AD - SURVEY may be associated with the gated . When the similarity score is greater than a threshold identified key words with a level for the tagging 655 . For ( e . g . , 80 % , etc . ) 6100 , the current tagging generated from example , in one implementation , the keywords identified 682 - 698a may be saved 6102 . Otherwise , the AD - SURVEY from sampled social media raw texts ( e . g . , Tweets , Facebook 25 may proceed with a different taxonomy model , e . g . , at 686 posts , etc . ) may be associated with appropriate nodes , e . g . , in FIG . 6E . level 0 , 1 , 2 , 5 , etc . In one implementations , AD - SURVEY may apply one or

In a further implementation , AD - SURVEY may identify more taxonomy logics to an unstructured data segment , as keywords set for each TV show / topic including words the data segment ( e . g . , a Tweet , a Facebook post , etc . ) may spelled differently but meant to refer the same entity 657 to 667 to 30 be related to one or more brand products , etc .

In further implementations , the taxonomy tagging mecha expand the query scope , e . g . “ COCA - COLA , " " COCA COLA ' S , ” “ COKE , ” “ COKE ' S , ” “ COCA COLA , " nism may be associated with a weighting score at each

" node ” of the taxonomy hierarchy . For example , in the " COCACOLA , ” etc . Within implementations , a variety of above example for “ Coke , ” if the AD - SURVEY receives a logical combination of such key words may be coded as a zs coded as a 35 client request to analyze consumer impression about a new logical rule 660 with AND , OR , Not operator , e . g . product of Coca Cola on social media analytics , the AD SURVEY may perform taxonomy mining upon unstructured data from the social media ( e . g . , consumer comments ) . Each Coke ( Level 0 )

[ coke + good ] | [ coke + excellent ] ( Level 1 ) “ node ” may be progressively associated with a weight score [ coke + good ] | [ coke + awesome ] ( Level 1 ) 40 to determine consumer impression . For example , when the [ coke in advert + too much taste to call zero ] ( Level2 ) data comprises " Coke , ” a level 0 weight score may be

assigned ( e . g . , 0 . 1 , etc . ) ; when the data comprises “ Coke + good ” or “ Coke + excellent , ” a level 1 weight score may be

wherein the “ + ” is an AND operator and “ ” is an OR assigned ( e . g . , 0 . 5 , etc . ) ; but when the data comprises operator . The AD - SURVEY may establish a taxonomy tree 45 “ Coke + horrible ” or “ Coke " with other negatively indicative combining the rules and store the generated logic taxonomy adjectives , a negative level 1 weight score may be assigned tree in a taxonomy database ( e . g . , see 585a in FIG . 5C ) . ( e . g . , - 0 . 5 , etc . ) . In one implementation , the taxonomy

FIGS . 6E - 6F provides an example logic flow illustrating engine may calculate an overall score of an unstructured taxonomy logic rule application within embodiments of the data record when progressively querying upon taxonomy AD - SURVEY . In one embodiment , continuing on with 552 50 key terms , and generate statistical results of a group of in FIG . 5C , e . g . , upon receiving unstructured data for unstructured data to determine the consumer impression . For analytics and tagging , etc . , AD - SURVEY may parse the example , the AD - SURVEY may generate statistical report as unstructured data for a first level analytics , e . g . , whether it to how many consumers are positive , neutral , or negative is related to a TV show , and / or a brand product 680 , and may towards “ Coke , ” based on their calculated scores , as illus generate a first tag associated with the unstructured data with 55 trated in one example in FIGS . 6E - 6F . For another example , a brand name 682 by querying on each brand in the brand the AD - SURVEY may generate popular brands , TV shows database 684 . The AD - SURVEY may then retrieve a tax that are the mostly mentioned or positively commented from onomy logic tree for the brand “ Coke ” 686 , as shown in the social media users , e . g . , see FIG . 10H . In further implemen above example . tations , the AD - SURVEY may determine a social group to

In one implementation , AD - SURVEY may follow the 60 analyze their social content . For example , within the social taxonomy tree for “ Coke , " forming a query on a second group , the AD - SURVEY may determine user ' s influence category , e . g . , brand name and positive mentions 690 . In the over above example , the taxonomy tree may apply a query on over other members of the population , e . g . , whether a user “ Coke + Good ” etc . If such text is found , AD - SURVEY may is an “ influencer . ” For example , when a user posts comments generate a sub tag associated with the unstructured data with 65 to the CBS show “ The Big Bang Theory ” on Facebook , and “ Coke + good ” 693a . If not , the AD - SURVEY may apply the his Facebook friends have " liked " his comments and sub taxonomy for an alternative query key terms , e . g . , “ Coke + sequently watch the show , the user may be considered as a

47 US 9 , 807 , 442 B2

48 Facebook “ influencer . ” In one implementation , the notion of on usage as a bi - product of delivering content and adver influencer may be specific to a social media platform ; a tising , e . g . , via application session ID , cookie , etc . , to create person who is an influencer on one platform may not be an dataset that is used for tactical content and advertising influencer on another . decisions . For example , the MR - PLATFORM may set up a

In one implementation , the social influencer may be 5 group of users ( e . g . , 100 , 000 users , etc . ) to track their TV determined by social media indices . For example , the AD - viewing , Mobile usage , Online surfing history , advertising SURVEY may analyze prerequisites for consideration as a exposure , demographic information , product ownership info social media influencer , such as whether a user has an for auto , location , financial services , product usage infor account on a social web platform , whether the user has mation for CPG / Pharma , and / or the like to provide survey generated content on that platform within the past 30 days , 10 research for ad effectiveness . In one implementation , AD and / or the like . SURVEY may recruit users with incentive rewards ( e . g . , the

In one implementation , the AD - SURVEY may calculate participants may be required to allow AD - SURVEY to social media index of a user to determine an influencer . For access their social media content as illustrated in FIGS . each member that meets the prerequisites , a social media 5A - 5B ) , e . g . , Credit in iTunes or Google App Store account index is calculated using various measures . For example , the 15 ( e . g . , $ 10 / quarter = $ 40 a year ) , local coupons / offers based on AD - SURVEY may calculate a reach measure , e . g . , over the zip code , etc . past 30 days , the maximum size of the network , which may As shown in FIG . 7A , the AD - SURVEY may automati be measured through “ friends ” , “ followers ” , or other similar cally collect data 705 from various source , e . g . , online measures . For another example , the AD - SURVEY may advertising usage 715 , mobile device usage 720 , TV viewing calculate a frequency measure , e . g . , over the past 30 days , 20 data 725 , social media data ( e . g . , Facebook posts / conversa the total number of posts to the platform , which may be tions , etc . ) 730 via API calls ( e . g . , see FIGS . 5A - 5B ) . In measured through things like " status updates ” , “ tweets ” , or another implementation , the AD - SURVEY may employ " comments ” , depending on the relevant content generation panelists 710 to configure data downloads parameters , sys opportunities for the particular platform . For another tem maintenance , and / or the like . For example , the panelist example , the AD - SURVEY may calculate a resonance mea - 25 may comprise social media users that allow AD - SURVEY to sure , e . g . , over the past 30 days , the total number of access their social profile and content for analytics . responses to the individual ' s content . Depending on the FIGS . 7B - 7C provide example flow diagrams illustrating platform , these responses may take the form of “ retweets ” , AD - SURVEY cross - channel data collection in alternative comments on " posts " or " status updates ” , or direct messages embodiments of the AD - SURVEY . In one embodiment , a responses to the individual . The responses may or may not 30 variety of data may be obtained and stored in a AD need to come from individuals within the population being SURVEY database ( e . g . , see 219 at FIG . 2A ) . In one measured . implementation , as shown in FIG . 7B , AD - SURVEY may

In one implementation , the calculated measures are then obtain data from different channel for cross - channel media ranked against the same measure from other individuals in measurement . For example , the AD - SURVEY may obtain his or her demographic group . The demographic measures 35 TV channel changing ( e . g . , user submission of channel may include , but are not limited to , age , gender , race , selection ) 790 from a AD - SURVEY TV client application education , and income . The specific groupings used for age , 795a ( e . g . , see FIGS . 7A - 7G ) , mobile advertising 791a and education , and income can vary based on the population mobile application 79ab usage from a AD - SURVEY appli being analyzed . An individual is considered an influencer if cation 795b , social media profile 778a and social media he or she ranks in the top 20 % of at least two of the variables . 40 comments 778b from social media ( e . g . , Facebook Twitter ,

In further implementation , the influencer determination 750 ) , user exposure to tagged advertisement 789a from a may be refined by product category . Each post made by an client flash / HTTP cookies 789b ( e . g . , see 375 in FIG . 3D ) , individual may be classified as mentioning a product , or not advertising exposure 790a ( e . g . , whether a user clicks on an mentioning a product , based on text analysis against a ad ) and website visits 790b from URL tracking of a user standard taxonomy of products and brands . The volume of 45 browser 797 , and / or the like . In one implementation , the posts in each category can be tabulated , and an individual AD - SURVEY may incorporate the variety of data for a classified as an influencer in any product category which cross - channel study of user feedbacks of an advertisement , represents at least 20 % of his or her product - classified posts . a brand , a product , a TV show , and / or the like . In further implementation , a social influencer may be speci - In a further implementation , the AD - SURVEY may utilize fied and / or classified with regard to a TV show category , a 50 the AD - SURVEY client component installed at a user category of products , a category of brands , and / or the like . mobile device to capture TV viewing in the home , survey

FIG . 7A - 7E provide exemplary data diagrams and logic responses , and / or the like . In a further implementation , the flow diagrams illustrating cross - channel data collection and AD - SURVEY may adopt a mobile meter to measure mobile media measurement within implementations of the AD - usage . In another implementation , the AD - SURVEY may SURVEY . FIG . 7A provides a block diagram illustrating 55 provide a client component which may provide history AD - SURVEY data collection within embodiments of the information from a user ' s personal computer when user AD - SURVEY . Within embodiments , the AD - SURVEY may connects his mobile device to his computer to sync up so that adopt a variety of technologies including flash cookies , the AD - SURVEY may track online Internet usage of the mobile applications , browser plug - ins , and / or the like , to user , e . g . , browsing history , clicks on ads , etc . In a further capture media usage across different channels , e . g . , TV , 60 implementation , the AD - SURVEY may track TV ad expo mobile , internet , social media , and / or the like . In one imple sure from a variety of meter data , e . g . , TNS , M + , AceMetrix , mentation , the AD - SURVEY may analyze the media usage and / or the like . In further implementations , data may be data to study the advertisement effects of a brand product . collected passively via mobile phones in almost real time ,

In one implementation , AD - SURVEY may track audience and / or when phone is being charged . activities to content ( planning and competitive analysis ) and 65 Within implementations , TV distribution may be driven advertising ( post analysis , ad effectiveness ) . In one imple - by an over the air broadcast and a “ one - to many ” cable mentation , the AD - SURVEY may create digital footprints infrastructure . Unlike other media and industries , like the

US 9 , 807 , 442 B2 49 50

Internet and the CPG / retail environment , the TV infrastruc FIG . 7D provides a data flow diagram illustrating AD ture may not create footprints on usage with content ( or SURVEY data flow between entities within alternative product ) distribution . In such cases , the industry may have embodiments of the AD - SURVEY . Within embodiments , a a panel based research to understand the size and composi AD - SURVEY database 719 may obtain data from Facebook tion of TV audiences . All decisions related to programming , 5 server 782a and Twitter server 782b via API calls ( e . g . , see ad sales , and carriage deals between multiple system opera FIGS . 5A - 5C ) . In further implementations , the access may tor and cable networks may be based upon panel data , which be validated and / or authorized by AD - SURVEY panelists . In may enable research providers to realize outsized rewards further implementations , other media exposure 782e ( e . g . ,

see FIG . 7B - 9C ) , survey data 782f ( e . g . , from synchronized for providing insights . In one embodiment , the AD - SURVEY may obtain TV 10 v 10 questionnaire , or other questionnaires handled by panelists ,

etc . ) , purchasing data 782c , may be provided to the database . viewing data via Return Path Data ( RPD ) from Digital Set In a further implementation , the cross - channel data may be Top Boxes ( RDTB ) , which may facilitate analysis of TV stored with a cloud storage 782d . viewing to provide insights on viewing to small networks In one implementation , a client ( e . g . , a user , a merchant and small geographies , e . g . , local market measurement 15 for analytics report , etc . ) may access the AD - SURVEY reflecting a local TV station or cable MSO zone . For another analytics server 755 via a website 784 , which may in turn example , metered data from RPD may be applied to analyze operate with a text analytics platform 783c to analyze social a group of categorized audience ( e . g . , heavy Coke drinkers ) gorized audience ( e . g . , heavy Coke drinkers ) content , cross - channel data , and / or the like . with targeted ad content ( e . g . , a 30 second commercial on a FIG . 7E shows a logic flow diagram illustrating monitor niche , targeted network ) . 20 ing user device data transmission across different channels

In further implementations , the AD - SURVEY may study within embodiments of the AD - SURVEY . Within embodi ad effects by collecting data with regard to user purchasing ments , the AD - SURVEY may instantiate a proxy server to activities of the advertised products . For example , the AD monitor data in / out of the user mobile device 740 , wherein SURVEY may track user clicks on a “ Buy it Now ” button , the data may be transmitted to / from different channels . In e . g . , see 750c in FIG . 7G . In another implementation , the 25 one implementation , the proxy server may receive data 742 , AD - SURVEY may obtain data form retailers , manufactures , and extract information from the received data 745 to Internet players , and / or the like , wherein real store data , determine a data type and monitor the user device activities frequent shopper cards usage , and log files ( Internet shop which may indicate advertisement delivery / usage informa ping ) may be analyzed to study placement decisions , tion . For example , if the data comprises a website URL 746 , increase traffic and sales ( ad visits ) , use the granular traffic 30 the AD - SURVEY may determine whether the URL com ( click - stream ) data to design customized products / content / prises an advertisement component 748 . If yes , the AD advertising to shoppers / viewers , and real time analytics to SURVEY may determine characteristics of the advertise better manage ad campaigns , and / or the like . In one imple m ent 749 , e . g . , classification of the advertisement ( e . g . , mentation , a weighing scoring mechanism similar to that media genre , media source , content description , etc . ) . In illustrated in FIG . 21 may be employed , but expanded to a 35 further implementations , the AD - SURVEY may determine variety of cross channel tracking data , to analyze ad effects whether the advertisement running on the URL is an auto within a cross channel dataset . matic advertisement , or requires user ' s manual trigger ( e . g . ,

In further implementations , the MR - PLATFORM may user clicks for display ) . The AD - SURVEY may obtain access to non - live media , such as , Charter , TiVo , Rentrak , identifying information , e . g . , an Ad ID , 755 , to generate Internet TV ( e . g . , Google TV , Apple TV , and / or the like , and 40 prompt questions , e . g . , at 312 in FIG . 3B . incorporate viewing data for analysis ( e . g . , see FIG . 2H ) . In further implementations , the AD - SURVEY may iden

In further implementation , Internet measurement data tify an advertisement comprised in a URL link via adver ( e . g . , from ISP data , etc . ) may be collected , e . g . , HTTP tisement image recognition . For example , the AD - SURVEY cookies , click - stream data with demographic information , may identify graphical contents contained in a URL link and / or the like . 45 based on empirical pattern of web content format . Such

In further implementations , mobile usage may be tracked empirical patterns may be classified per URL link type ( e . g . , via specific applications ( e . g . , in a similar manner to log file a shopping site link such as Amazon . com may have an analysis ) , anonymous location based tracking of cell users , advertisement placed at the center frame of the web page ; a and / or the like . news link such as New York Times page may have an

FIG . 7C shows data collection via a proxy server within 50 advertisement placed at the side / bottom bar of the web page , implementations of the AD - SURVEY . For example , in one and / or the like ) . For another example , the AD - SURVEY implementation , a AD - SURVEY client application 795b may identify dynamic contents on the web page , such as , but operated on a user device may collect and send information not limited to flash contents , banners , applets , and / or the such as social application usage , GPS location , response to like , as displayed advertisements . surveys , etc . , to a database 719 . In further implementations , 55 Within implementations , upon obtaining an image capture social application usage and mobile advertising data may be of an advertisement , the AD - SURVEY may adopt software passed to a AD - SURVEY proxy server 788 ( e . g . , see FIG . packages to identify contents of the advertisement ( e . g . , a 7E ) , and / or a VPN server , which may forward it to the featured product name , a brand name , etc . ) so that it can be database 719 . In one implementation , TV viewing data may associated with a user ' s ad exposure . In one implementation , be provided to the database 719 via a TV remote application 60 the AD - SURVEY may generate a unique identifier indica 795a ( and / or a TV measurement network ) . In another imple - tive of visual characteristics of the captured ad graphical mentation , social media engagement data ( e . g . , user posts , contents ( e . g . , a matrix representation of sampling of the comments on the social media ) 750 may be sent to the captured ad image , etc . ) , and form a query on an ad database database as well . In further implementations , the AD - SUR - based on the unique identifier . In another implementation , VEY may obtain mobile search behavior of a user , exposure 65 the AD - SURVEY may adopt software packages similar to , to social media ( e . g . , user viewing friends ' recommenda - but not limited to Google image search , and / or the like . tions on social media , etc . ) , and / or the like . Further details of advertisement image match may be similar

b 1 : 120

US 9 , 807 , 442 B2

to that described in U . S . Pat . No . 7 , 565 , 139 , entitled “ Image which type of media to place an ad , developing media plans Based Search Engine for Mobile Phone with Camera , " with the optimal mix across media , determine the impact of which is herein expressly incorporated by reference . advertising on brand awareness , favorability measures ,

In one implementation , the AD - SURVEY may dissect an intent to buy and actual purchase across media ( ROI ) . In advertisement identification from the advertisement embed - 5 another implementation , the AD - SURVEY may provide ded in the web content ( e . g . , the URL link ) to determine a advertising measurement solutions using emerging sources product name , a brand name , and / or the like , which the user of data , e . g . , media consumption data ( TV , online , mobile , has been exposed to . In further implementations , the cap - social , etc . ) , shopper data for key categories , and / or the like . tured graphical advertisement contents may be tagged with In one implementation , the AD - SURVEY may provide metadata in compliance with formats associated with an 10 clients with insights on campaign effectiveness and recom advertisement , e . g . , exif data tags ( which may include mendations on optimal media allocation using survey ques unique advertising identifiers in the software tags , in the tionnaires ( e . g . , as further discussed in FIGS . 3A - 3D ) based inter - operability tags , in the extension name / ID , extension on statistical modeling and regression analysis . In further description tags , and / or the like ) . Other graphical metadata implementations , the impact of each media may be sepa formats may also be contemplated such as XMP , GIF , IPTC 15 rately analyzed to make recommendation on optimal spend information interchange model metadata format , and / or the and predict sales based upon survey responses . like . In further implementations , the AD - SURVEY may link

In further implementations , the AD - SURVEY may deter - different type of data for cross channel analysis . In one mine whether a user clicked a URL link comprising media implementation , the AD - SURVEY may create a unified program content , e . g . , a Youtube link , etc . The AD - SUR - 20 dataset that profiles audiences for TV and online media VEY may extract an identifier , e . g . , a web ID , etc . , to consumption , wherein each viewing source ( e . g . , individual determine the name of the media program . In another TV set , household TV set , etc . ) with viewing and ad expo implementation , the AD - SURVEY may obtain excerpts of sure information for TV and online is associated with a the media program , and determine a name of the media unique identifier . For example , AD - SURVEY may adopt program via an embedded digital signature . In further imple - 25 direct linkage by getting TV and online data for the same mentation , the AD - SURVEY may set time stamp on the household ( e . g . , via ISP , cable provider , etc . ) . For another user ' s click on the URL link comprising a media program to example , the AD - SURVEY may segment TV and online record how long the user has been exposed to the media data , and link using segments ( e . g . , segmented by program program . Further implementations of the ad identification category , zip code , air time , etc . ) . For another example , the are discussed in FIG . 7F . 30 AD - SURVEY may devise and distribute survey questions

In further implementations , the AD - SURVEY may track about TV viewing and linking with the respondents ' online a user ' s browsing history by monitoring a stream of “ clicks " surfing data . the user has submitted on his mobile device . For example , In one embodiment , the AD - SURVEY may obtain media the AD - SURVEY may monitor user ' s " clicks ” to determine consumption data from a variety of channels , such as , but a type of the click , e . g . , usage of media playing , visits of a 35 not limited to geographically dispersed TV viewing data sets different URL link , posting of social media contents , usage ( e . g . , CANOE ) , persons tracking with EPGs or embedded in of an application , and / or the like . Using an app . The AD TV software or cell phones , metering data from STB , SURVEY may then determine advertisement exposure asso - individual smartphone ( e . g . , Apple iPhone , etc . ) based track ciated with each “ click ” ( e . g . , via ad image identification ing ( e . g . , social content , persons ' watching activities , etc . ) , illustrated in FIG . 7F ) . 40 and / or the like .

In another implementation , the received data may com - FIG . 7F provides a block diagram illustrating example prise application information from the user device 751 . The infrastructure of advertisement recognition within imple proxy server may ascertain an application inventory list of mentations of the AD - SURVEY . Within implementations , the user device 753 , and / or application group sharing infor the AD - SURVEY may measure which advertisement audi mation 754 . For example , one or more users who are 45 ence has been exposed to by mining the monitored data Facebook friends , may form a group to share their interested in / out of the user mobile device , user computer , and / or the TV watch list and viewing status via AD - SURVEY , and such like ( see e . g . , 740 at FIG . 7E ) . For example , URL links and information may be captured by the proxy server . data file may be collected and sent 772 by a data collection

In another implementation , the received data may com - software 771 running at a client component instantiated on prise indication of media usage 762 , e . g . , channel selection , 50 a user mobile device . In another implementation , the data atmospherics data , etc . , the AD - SURVEY may determine collection may be performed at a proxy or VPN server . whether the TV program on the selected channel has been for digital advertising ( online and mobile ) , AD - SURVEY listened , watched , and / or streamed 766 . The AD - SURVEY may classify the advertisements via a hybrid manual / auto may determine a title of the media program 767 , and retrieve matic process . For example , AD - SURVEY may adopt a ad tags embedded in the media program from a media table 55 mobile or PC - based system ( using a software meter , VPN , to determine user ad exposure information , e . g . , as discussed and / or Proxy Server , among other technologies ) to pass in FIGS . 3B - 3C . In further implementations , the received along to a staging server the URL of the ad , along with the data may comprise survey responses / social messages sent actual file ( typically a . gif , jpg , or . png ) . For example , upon 763 to the AD - SURVEY , as further discussed in FIGS . receiving ad data including URLs and associated digital files 5A - 5E . 60 ( e . g . , media program excerpt files , etc . ) via a network

In one implementation , the AD - SURVEY may generate connection 773 , the AD - SURVEY server 774 may compare media analytics report 768 based on the obtained media data , the received advertisement data with ad profiles in a data including information as to user impressions to a brand base 775 . The filename , file size , and other data may be name product , TV shows , etc . , e . g . , as shown in FIGURES compared against an ad database , and if the ad has already io A - 10H . 65 been classified , then the new ad exposure event is transmit

In one implementation , the AD - SURVEY may provide ted to a classification engine 776 classified based on clas advertisers solutions to design advertising campaigns as to sification rules in the database . Classification rules may

53 US 9 , 807 , 442 B2

54 include brand and product mentions , as well as ad size and VEY . In one implementation , AD - SURVEY may provide a other descriptors . If a match is not found , then the ad is put static ( e . g . , textual ) ad 820b in a prompts drop - down menu in a queue for manual classification by a AD - SURVEY 820 , as shown in FIG . 8F . For example , when the user representative . For example , the AD - SURVEY representa selected channel “ CBS ” has “ The Big Bang Theory ” on air , tive may identify the object ( e . g . , a product , a brand name , 5 and the media content arrives at a timestamp wherein a pair etc . ) that is advertised within the ad . of " white framed sunglasses " is tagged in the scene , AD

In one implementation , the advertisement may be identi - SURVEY may generate a pop - up ad 820b and provide a link fied via graphical content match , as discussed in FIG . 7E . In for a user to tap on to check out for more details . an alternative implementation , the ad identification may be FIG . 8G shows an example interactive ad . For example , performed via an automated system , by which ads that do 10 the AD - SURVEY may generate an interactive ad including not match any items in the ad database may be examined a screen shot of the TV program containing the placed automatically by a computer program for clues as to the products . The featured products may be tagged ( e . g . , high proper classification . For example , when an ad that mentions lighted by white - line boxes ) in the screen image , e . g . , a pair Acura in the ad image , the AD - SURVEY may automatically of “ XYZ - designer French style sunglasses ” 850a , and classify the ad as an Acura advertisement . Within imple - 15 " DDD Red Polka Dots Bikini Halter Top ” 850b . A user may mentations , the AD - SURVEY may perform character rec - tap on a “ Buy it Now ” label 850c to check out more details , ognition procedures ( e . g . , optical character recognition , etc . ) and / or be directed to a merchant site to transact a sale . to obtain key terms from advertisement images for adver - In a further implementation , the interactive ad may com tising exposure identification . Such ad identification may be prise a user rating of the featured product showing beneath used for advertising effectiveness measurement . Numerous 20 the product . In one implementation , the user rating may be OCR engines may be adopted , such as GOCR , Java OCR , obtained from historical user rating data , social media rating , OmniPage , SimpleOCR , and / or the like . and / or the like . In one implementation , the user may enter

FIGS . 8A - 8E provide exemplary mobile screen shots his own rating by tapping on the box 850d . illustrating user interfaces within embodiments of the AD In a further implementation , the user may browse the SURVEY . As shown in FIG . 8A , the AD - SURVEY user 25 interactive ads ( e . g . , including screen shots from the TV interface may comprise a section for social media status , program comprising product placement tags , etc . ) by going e . g . , the Facebook user " John Smith is watching The Big to a previous page 851 , and / or a next page 852 . In further Bang Theory on CBS ” 820 . The user may also view a list of implementations , the user may elect to browse interactive his friends ' status 805 , e . g . , what the friends are watching . ads associated with the TV program in a variety ways . For

In one implementation , the user may view a list of channel 30 example , the user may elect to view a list of all ads 855a ; program 810 schedules , and may elect to tap on the screen may elect to view by season episode 855b ; may elect to view to choose one of the listed channels . In a further implemen - by character 8550 ( e . g . , products carried by , or associated tation , the user may select to allow ambient monitoring 850 , with the character names in the show ) ; may elect to view by so that the AD - SURVEY may " listen - in ” and submit atmo - item category 855d ( e . g . , apparel , accessories , furniture , hair spherics data to the AD - SURVEY server , as discussed at 241 35 products , etc . ) , and / or the like . In further implementations , in FIG . 2B . the user may initiate a search on desired products 858 . For

In further implementations , the user may click on the example , if the user is interested in a red hat the character “ Prompts ” 820 button and view a drop down list of survey “ Penny ” wore in one of the scenes , the user may form a questions , e . g . , as shown in FIG . 8B . The survey question query on the embedded ads table based on key terms may be generated based on the media content the user has 40 “ Penny , ” “ red , " " hat , ” etc . been watching , as discussed in FIG . 3B . Upon the user FIGS . 8H - 8L provide example mobile screen shots illus submitting a response , as shown in FIG . 8C , the user trating a AD - SURVEY TV remote client component within response may be populated as a social media message 830 . embodiments of the AD - SURVEY . In one implementation , In one implementation , the user ' s friends may view the as shown in FIG . 8H , a user may receive TV guide ( e . g . , user ' s activity and “ likes ” the user ' s response 831a , com - 45 863 ) as a list of channel program information 860 . The user mented on the response 831b , and / or follow the link of the may also receive live TV information updates in a pop - up survey and participate in the survey 831c . window 861 at a “ live ” section 862 . The user may further

In further implementations , as shown in FIG . 8D , the user configure parameters in a “ Remotes ” section 864 and “ Set may tap on the “ Social ” 815 button and launch a drop down tings ” 865 , as further illustrated in FIGS . 81 - 8L . panel for social media inputs 815a . In one implementation , 50 When the user tap on the “ live ” section 862 , the user may the AD - SURVEY may generate an automatic message tem - view a list of most viewed programs , e . g . , ranked by plate for the user based on the media ad tags , e . g . , the user registered AD - SURVEY users . The user may also see a list likes an embedded product placement 815b . For another of program that the user ' s social friends are watching 868 , example , the user may manually type texts to update his and a list of programs the user has selected 866 . social media status 815c . 55 FIG . 81 provides example screens illustrating the

In further implementations , as shown in FIG . 8E , if the “ Remotes ” 864 at FIG . 8H within implementations of the user elects to submit a message on social media indicating AD - SURVEY . For example , upon plugging an infrared he likes the embedded product placement 835 , the user ' s accessory ( e . g . , see 120 in FIG . 1B ) to a user mobile device , friends may be aware of the product . For example , the user ' s the user may turn the mobile device ( e . g . , an Apple iPhone , friends may like the post 835a , comment on the product 60 iPod , iTouch , iPad , BlackBerry , Google Android , Palm , etc . ) 835b . For another example , the AD - SURVEY may feed a into an infrared TV remote control . Upon tapping on link directing to a merchant site comprising the placed " remote ” 864 , the user may view a virtual TV remote panel product along with the social media message , and the user ' s which comprises control buttons for TV program informa friend may follow the link to learn more about the placed tion 870 from which the user may tap to select channels , product via the social media 835c . 65 adjust volume levels , etc . The AD - SURVEY may provide a

FIGS . 8F - 8G provide example mobile screens illustrating set - top box control panel to the user for the user to control synchronized ads within implementations of the AD - SUR - live / on - demand video playing , and / or the like . The user may

US 9 , 807 , 442 B2 55 56

also configure DVR control 870a , live TV 870b , on demand In another implementation , as shown in FIG . 9A , a user TV 870c , TiVo 870d , TV guide 870d and intelligent TV may configure social network connection settings under the ( e . g . , Apple TV 871 , Google TV 872 ) , and / or the like . For settings ( e . g . , see 865 at FIG . 8H ) of AD - SURVEY client another example , the user may view an array of fast keys for mobile application . In one implementation , the AD - SUR different channels 873 . 5 VEY may send an access request so that a user may view a FIGS . 81 - 8K provides example screens illustrating the prompt from his mobile device 911 , and may elect allow or

“ Setting ” 865 at FIG . 8H within implementations of the disallow the AD - SURVEY access attempt 902 . The user AD - SURVEY . In one implementation , the user may config may be directed to a Facebook login page 910 , wherein ure TV provider 875a , set - top box 875b , Smart TV 875c , Facebook may request user to provide login credentials to TiVo 875d , DVD 875e parameters for the mobile TV remote té 10 verify the permission authorization . For example , the user for AD - SURVEY . In one implementation , the user may enter may enter his email address 913 and password 915 to login

to Facebook . For another example , if the user does not have the zip code 876 to receive a list of TV providers available a Facebook account yet , upon tapping on “ Yes , ” the user in the area , and select his own provider . In another imple may be directed to create a new account . In another imple mentation , the user may elect to choose “ use TV ” only , or to V only , or to 15 mentation , if the user ' s mobile application ( e . g . , an iPhone set - up and test a set - top box for his television set . In further Facebook app , a mobile browser , etc . ) stores user previously implementations , the user may configure TV and DVD entered login credentials , the user mobile device may send parameters so that the infrared plug - in accessory may query the login credentials to the social media to proceed with an address of the TV and DVD set . In one implementation , access authorization , so that the user may not need to view the user may select a TV brand 880 from a brand list , and 20 the login page 910 to manually provide user credentials . then select a type of the TV . For example , the user may tap Upon providing Facebook login credentials and verifica on a few testing buttons under each type , e . g . , " Power , ” tion of Facebook , the user may receive a request for per “ Volume , ” “ Select , ” etc . , to test whether the remote control mission screen 911 to select to “ Allow ” 908 AD - SURVEY works for the TV . to access the user ' s Facebook profile . In further implemen

In another implementation , the user may configure DVDs 25 tations , the user may configure access parameters in a via the AD - SURVEY . The AD - SURVEY may initiate an similar manner as illustrated in FIGS . 9B - 8C . automatic scan upon user selection , and / or receive an indi - FIGS . 9B - 8C provide exemplary web - based user interface cation from the user of the DVD brand . Upon indicating the ( e . g . , 511b in FIGS . 5A - 5B ) illustrating user authorization of DVD brand , the user may test connection with the DVD set AD - SURVEY access to the user ' s social media content to select a DVD type . 30 within embodiments of the AD - SURVEY . For example , in

In another implementation , the user may enter a zip code one implementation , a user may access a web - based AD to configure the TV provider 875a , so that the AD - SURVEY SURVEY configuration page via an Internet browser ( e . g . , may provide a list of TV providers for the user to choose . Internet Explorer , Safari , Firefox , etc . ) to bridge his social

FIG . 8K provides an exemplary mobile screen shot illus media accounts with AD - SURVEY . In one implementation , trating social profile of a TV program within implementa - 35 the user may select a list of social media platforms 920 to tions of the AD - SURVEY . In one implementation , the user join AD - SURVEY , e . g . , the user may click on checkboxes to may select a TV program ( e . g . , “ Channel 5 ” 885 ) on a choose Facebook 921a , Twitter 921b , Google + 921c , Tumblr channel to view its social profile . In one implementation , the 921d and / or the like . In further implementations , the user user may tap on “ Check - in ” 886 to populate a Facebook may specify other social media platforms not listed by message with regard to the checked TV program , and / or 40 AD - SURVEY by typing a URL address , e . g . , “ FourSquare , ” “ Twitter ” icon 887 to share such information on Twitter . For etc . Upon selecting the social media , the user may click example , to “ Check - in , ” the user may select post onto his “ Send Request " 925 so that AD - SURVEY may send a Facebook wall by typing a message 886a , and / or share the connection request to the selected social media platforms , automatically generated message 886a by AD - SURVEY . In e . g . , Facebook 921a and Twitter 921b as shown in the another implementation , if the user chooses to “ Tweet , " the 45 example of FIG . 9B . In another implementation , the user user may view a list of " discussion ” 888 Tweets with regard may click “ Cancel ” 926 to abort the access control configu to the show " The Big Bang Theory , " and may generate his ration . own Tweet . In one implementation , upon submitting the request 925 ,

In one implementation , the user may view a social rating the social media platform may request login confirmation 889 of the TV program “ The Big Bang Theory ” under its 50 925 . For example , the user may be redirected to the social profile . For example , the rating 889 may be given by social media homepage to login . For another example , the user users of AD - SURVEY who has viewed the program , and the may be presented a pop - up window 928 for the social user may elects to submit his own rating 889a . In a further analytics to connect with Facebook , e . g . , the user may need implementation , the user may obtain a list of “ social watch to provide email 926 and password 927 to login to Face ing ” 890 to see a list of social users ( e . g . , social media users 55 book . who has allowed AD - SURVEY to access their social pro - As shown in FIG . 9C , upon providing user Facebook files , etc . ) who is watching the selected TV program . credentials , the user may configure access scope 927 for

FIG . 9A provides an exemplary mobile screen shot ( e . g . , Facebook content . For example , the user may elect to allow 511a in FIGS . 5A - 5B ) illustrating user authorization of AD - SURVEY to access his user profile , e . g . , the user may AD - SURVEY access to the user ' s Facebook content within 60 select among a list of checkboxes for user name , user embodiments of the AD - SURVEY . In one implementation , address , user email , user phone number , work information , a user may receive a message ( e . g . , a pop - up message education information , date of birth , pages , groups , net window , an email , etc . ) requesting a user to elect whether to works , and / or the like . For another example , the user may allow AD - SURVEY to access his Facebook content . For configure the AD - SURVEY may access his friends infor example , the user may tap on a link in the message , e . g . , 65 mation 929 , e . g . , the user may allow the AD - SURVEY to “ Yes ” or “ Not Now ” to grant or deny permission to his obtain a number friends 929a but may not allow the Facebook content . AD - SURVEY to access details of the friends list . For

US 9 , 807 , 442 B2 57 58

another example , the user may allow the AD - SURVEY to prior to the export of the physical information , and as such , access his wall posts 930 , e . g . , the user may allow the and data imported into the package will be limited to the AD - SURVEY to access his post on his own wall 930a selector ( e . g . , genre , name , etc . ) . The output from a standard ( including sharing links , posted photos , status update , mes statistical output from package SAS may be used in its sages , etc . ) . The user may conditionally allow AD - SURVEY 5 entirety and / or parsed for the dashboard report . For example , to access his post on his friend ' s wall when the friend allows the AD - SURVEY may download the statistical output in a access to his wall 930c . Similarly , the user may conditionally text format and present it in a dashboard user interface . In allow AD - SURVEY to access the user ' s likes / dislikes when another example , the SAS output may be reported in a the liked or disliked item owner allows AD - SURVEY access statistical format ( e . g . , comma delineated XML file ) , which 930d as well . 10 may enhance parsing for the dashboard report generation .

FIGS . 9D - 9F provide exemplary web - based user interface For example , the AD - SURVEY may provide a welcome ! illustrating media analytics within embodiments of the AD - login screen to a user 1005 , wherein the user may be a SURVEY . For example , as shown in FIG . 9D , a AD - AD - SURVEY client , such as a TV producer , an advertiser , SURVEY user / client ( e . g . , an advertising merchant , a TV a merchant , and / or the like , who may be interested to learn media producer , etc . ) who may desire to know audience 15 their TV audience statistics , brand / products impressions , feedbacks to an ad , TV program , and / or the like , may access and / or the like . a media analytics reports 940 module . In one implementa - Upon user login at 1005 , the AD - SURVEY may provide tion , the user may select tabs for different options to view a customized dashboard page 1008 , e . g . , as further illus analytics of a targeted object ( e . g . , a TV show , an adver - trated in one implementation at FIGS . 10B / 10H . The dash tisement , etc . ) . For example , a use may elect to view 20 board page may provide an overview of the media analytics comments categorized by people 941 , TV shows 942 , brand results , such as generated report types 1010 , available sta names 943 , products 944 , media sources 944 , and / or the tistics charts / plots , and / or the like . For example , the AD like . SURVEY may perform analytics to generate advertising

In one implementation , as shown in FIG . 9D , if the user reports 1012 ( e . g . , advertisement delivery reports 1012a , selects TV shows 942 , the user may provide information of 25 etc . ) , audience reports 1015 ( e . g . , an audience summary the TV show by selecting a TV network , ( e . g . , “ CBS ” at report 1015a , audience reports by gender reach 1015b , by 945a ) , genre ( e . g . , “ Comedy ” at 945b ) , show name ( e . g . , age reach 1015c , reports per day of week 1015d , reports per " The Big Bang Theory ” at 945c ) , from a drop down list . In hour during a day part 1015e , etc . ) . another implementation , the user may manually enter a FIG . 10B provides an exemplary screen shot for media name of the TV show 945d to query on a TV show database 30 analytics dashboard 1008 . In one implementation , a user at AD - SURVEY . may elect to configure media analytics parameters . For

In one implementation , the user may configure analytics example , the user may select an object of the media analyt parameters such as social content source 948 ( e . g . , check on ics , which may be a media platform , a TV channel , an “ Facebook ” and “ Twitter ” ) , time range 949 ( e . g . , between a application platform , and / or the like . The dashboard board specified time and present ) . The user may further select a 35 may provide a dropdown menu to allow the user to select presentation format , e . g . , whether to view raw textual com various applications and media platforms e . g . , the user may ments 950 , or a visualized summary 955 ( e . g . , plain format select view application usage of “ Foursquare ” 1018 . In other 955a , chart / plots 955b , or table 955c ) . For example , if raw implementations , the user may select targeted AD - SURVEY text 950 is selected , the user may view a list of Facebook Social users for the study , e . g . , by checking user gender comments 950a , and Tweets 950b related to the show “ The 40 1020 , age groups 1021 , user interface types ( e . g . , AD Big Bang Theory . ” SURVEY application , mobile , web , etc . ) , phone types 1023 ,

In another implementation , as shown in FIG . 9E , if the day part time range 1024 , user location 1025 , and / or the like . user elects to view analytics report in a visualized format In further implementations , the user configured social user 955 , the summary may be presented in a plain textual group may be saved as a group for further analysis . format , e . g . , 960a , showing statistical results of the positive , 45 In one implementation , the media analytics dashboard neutral and / or negative comments . For another example , page may provide a plot of total AD - SURVEY user sessions charts / plots 960b , and a table 960c summary may be pre - 1028 based on the user configured AD - SURVEY social user sented . In one implementation , the analytics summary may group . For example , the personal trending data statistics be broken down to different categories , e . g . , the comments 1028 may further comprise plots of number of users , number for each category “ CBS , ” “ The Big Bang Theory , " " Char - 50 of sessions , number of new users , median / mean session acters , ” “ Music " of the show , and / or the like . length , geographic regions of users , and / or the like . In

FIGS . 10A - 10H provides exemplary user interfaces of another implementation , AD - SURVEY may provide analy media analytics reports within embodiments of the AD sis of AD - SURVEY application usage statistics 1029 , such SURVEY . FIG . 10 A provides an exemplary AD - SURVEY as a total number of sessions , a total number of registered media analytics UI flow . Within implementations , upon 55 users , a median session length per user , and / or the like . In obtaining measurement data from a variety of data channels another implementation , the AD - SURVEY may provide a ( e . g . , social media , URL links , mobile metering , etc . ) , the summary table 1030 showing the study of user selected AD - SURVEY may provide web based media analytics plat - reporting application “ Foursquare , ” including audience form for a user to access via a Internet web browser . The active reach , total visits , visits per person , total web page recorded data analytics as shown in FIGS . 10A - 10F may be 60 views , page view per person , duration , time per person , web explored via a variety of data file formats including XML , user gender , age , and / or the like . For example , a report on ASC , and / or the like , through the database export mecha the number of users may summarize the number of users an nism . Within implementations , these exported files may be application ( e . g . , “ Foursquare " ) has had over period of imported to analytical tools , such as SAS , etc . , wherein tracking to indicate whether the number is growing or various statistical analysis may be engaged . In one imple - 65 declining . The report may also analyze user sessions in mentation , selections made from the dashboard , e . g . , a genre different scenarios ( e . g . , whether by a single user ) . For selection , 1041 of FIG . 10H , may be used as the query filter example , when a mobile site has been visited for 50 times ,

US 9 , 807 , 442 B2 59 60

but 30 of those times it was from the same user , versus view " Age Reach ” 1030c , for a data table showing appli another month the number visits from different users was 25 , cation report classified by age , including total audience , the number of users may not be considered as increased . unique audience , active reach ( % ) , total duration ( in min

For another example , a report on number of sessions may utes ) , time per person , time per person over a month period , include number of sessions by all users within the last 5 total page views , pages per person , total visits , visits per calendar month , which may be tracked on a monthly trend person , and / or the like broken up in different age groups . ing basis to indicate how often an application / site was seen FIG . 10G provides an exemplary audience report per day by all users . For another example , a report on number of new 1015e . As shown in FIG . 10G , the user may select to view users may show the number of new users for the last “ Day Part ” 1030e , for a data table showing application calendar month using the application or mobile site . For 10 report classified by hours of a day . For example , for this another example , the median / mean session length may indi report , for this report , data for each app / site may be broken cate the time spent on application / site . For another example , up by the hour of the day . For example , as shown in FIG . the geo graphic region of users may show a geographical 10G , the Youtube app usage data may be broken up by the distribution of users engaging or visiting the application hour of the day . advertisement 15 FIG . 10 H provides an exemplary analytics report within

FIG . 10C provides an exemplary audience summary an alternative implementation of the AD - SURVEY . In one report 1015a . As shown in FIG . 10C , the user may select to implementation , the AD - SURVEY may collect social data view " summary ” 1030a , for a data table showing : for each from social media platforms of different social groups ( e . g . , reporting application ( e . g . , Facebook , ESPN SportsCenter , 1066a , 1066b ) , e . g . , the social groups determined by user etc . ) , statistics data for Unique Audience , Percent Active 20 configured parameters as shown in FIG . 10B , to determine Reach , Time per person ( minutes ) , TPP / Month , Number of a variety of measures of social TV watching status , such as , visits , Number of pages viewed , Visits / person , Male / Female but not limited to profile measures 1051 , liking measures ( for unique and non - unique users / visitors ) , Age groups ( for 1052 , brand measures 1053 , TV shows watching measures unique and non - unique users / visitors ) . For example , the 1054 , and / or the like . In one implementation , the AD unique audience may be calculated as the total real number 25 SURVEY may define a social group based on age ranges , of users of an app / site without double counting users that gender , demographic information , geographical location , have reused the site / app more than once in a month , and / or income level , educational level , and / or the like . In another the like . The percent active reach of all smartphone users implementation , the user may configure parameters for TV may be calculated as the percentage of users that checked a shows , brands / products , social groups , and / or the like , e . g . , particular mobile website or application within the last 30 genre 1041 , actors 1042 , networks 1043 , first run time 1044 , calendar month , e . g . , when there are 100 smartphone users , airing time 1045 , brand 1046 , product type 1047 , date 1048 , 88 checked Facebook within the last calendar month , active geography 1049 , influencer s 1050 , age 1020 , gender 1021 , users equal 88 % . The time per person ( minutes ) may be and / or the like . calculated as the amount of time spent during a day on a For example , in one implementation , for a specified TV site / app by avg . active user , e . g . , if there was 88 active users 35 show ( e . g . , “ Big Bang Theory ” at “ CBS ” ) , the profile and they used a site / app for a total of 20 hours during the measure 1051 may provide social data metrics with regard day , then the time per person is equal to 22 . 72 minutes to social groups , such as percentage of influencers in a social ( 20 / 88 = 0 . 2272 ) . The TPP / month may be calculated as the group ( e . g . , social users whose watching recommendations time per person per month that is spent on the site . e . g . , if have been followed by other social users , etc . ) , percentage there are 70 users and in total and they spend 1000 hours on 40 of users who engage in watching the show , percentage of the site / app then the time per person per month would be users who is distracted from watching the show ( e . g . , via 11 . 11 hours ( 1000 / 90 ) . The number of visits may be calcu atmospherics analysis as illustrated in FIG . 2E ) , number of lated as the total number of times an app / site was visited friends who have engaged in the TV watching , number of during the last calendar month by all users . The number of posts per week , a number of brand pages the social group has pages viewed may be calculated as the total number pages 45 followed , number of photos the social group has posted , viewed on an app / site during the last calendar month by all and / or the like . In further implementations , the AD - SUR users . The visits / person may be calculated as the number of VEY may analyze the social media content to provide liking visits to app / site on a per active users basis . In one imple measures 1052 , such as the most positively commented 10 mentation , the above mentioned metrics may be assessed by brand pages , music pages , TV show pages , and / or the like ; male / female gender classification ( for unique and non - 50 brand measures 1053 , such as the top 10 brands , top 5 unique users / visitors ) , age groups ( for unique and non - products mentioned within a social group , and / or the like ; unique users / visitors ) , e . g . , age brackets may be 18 - 24 , the top TV shows watched 1054 , and / or the like . 25 - 34 , 35 - 44 , 45 - 54 , 55 + , etc .

FIG . 10D provides an exemplary audience report per AD - SURVEY Controller gender reach 1015b . As shown in FIG . 10C , the user may 55 select to view " Gender Reach ” 1030b , for a data table FIG . 11 shows a block diagram illustrating embodiments showing application report classified by gender , including of a AD - SURVEY controller . In this embodiment , the AD unique audience active reach ( % ) , total duration ( in min - SURVEY controller 1101 may serve to aggregate , process , utes ) , time per person , time per person for the month , total store , search , serve , identify , instruct , generate , match , and / web page views , pages per person , total visits , visits per 60 or facilitate interactions with a computer through social person , and / or the like . For example , the audience report by network and electronic commerce technologies , and / or other gender may include data for each app / site broken up by related data . gender . For example , Facebook may provide male and Typically , users , which may be people and / or other sys female break of user base , about time men spend on the app , tems , may engage information technology systems ( e . g . , number of pages , etc . 65 computers ) to facilitate information processing . In turn ,

FIG . 10F provides an exemplary audience report per age computers employ processors to process information ; such reach 10150 . As shown in FIG . 10C , the user may select to processors 1103 may be referred to as central processing

nu

US 9 , 807 , 442 B2 62

units ( CPU ) . One form of processor is referred to as a " processor ( s ) ” ( these terms are used interchangeable microprocessor . CPUs use communicative circuits to carry throughout the disclosure unless noted to the contrary ) ) and pass encoded ( e . g . , binary ) signals acting as instructions 1103 , a memory 1129 ( e . g . , a read only memory ( ROM ) to bring about various operations . These instructions may be 1106 , a random access memory ( RAM ) 1105 , etc . ) , and / or operational and / or data instructions containing and / or refer - 5 an interface bus 1107 , and most frequently , although not encing other instructions and data in various processor necessarily , are all interconnected and / or communicating accessible and operable areas of memory 1129 ( e . g . , regis through a system bus 1104 on one or more ( mother ) board ( s ) ters , cache memory , random access memory , etc . ) . Such 1102 having conductive and / or otherwise transportive cir communicative instructions may be stored and / or transmit cuit pathways through which instructions ( e . g . , binary ted in batches ( e . g . , batches of instructions ) as programs 10 encoded signals ) may travel to effectuate communications , and / or data components to facilitate desired operations . These stored instruction codes , e . g . , programs , may engage operations , storage , etc . The computer systemization may be

connected to a power source 1186 ; e . g . , optionally the power the CPU circuit components and other motherboard and / or source may be internal . Optionally , a cryptographic proces system components to perform desired operations . One type of program is a computer operating system , which , may be sor 1126 and / or transceivers ( e . g . , ICs ) 1174 may be con executed by CPU on a computer ; the operating system 15 nected to the system bus . In another embodiment , the enables and facilitates users to access and operate computer cryptographic processor and / or transceivers may be con information technology and resources . Some resources that nected as either internal and / or external peripheral devices may be employed in information technology systems 1112 via the interface bus I / O . In turn , the transceivers may include : input and output mechanisms through which data be connected to antenna ( s ) 1175 , thereby effectuating wire may pass into and out of a computer ; memory storage into 20 less transmission and reception of various communication which data may be saved ; and processors by which infor and / or sensor protocols ; for example the antenna ( s ) may mation may be processed . These information technology connect to : a Texas Instruments WiLink WL1283 transceiver systems may be used to collect data for later retrieval , chip ( e . g . , providing 802 . 11n , Bluetooth 50 . 0 , FM , global analysis , and manipulation , which may be facilitated positioning system ( GPS ) ( thereby allowing AD - SURVEY through a database program . These information technology 25 controller to determine its location ) ) ; Broadcom BCM4329 systems provide interfaces that allow users to access and FKUBG transceiver chip ( e FKUBG transceiver chip ( e . g . , providing 802 . 11n , Blu operate various system components . etooth 2 . 1 + EDR , FM , etc . ) ; a Broadcom BCM4750IUB8 In one embodiment , the AD - SURVEY controller 1101 receiver chip ( e . g . , GPS ) ; an Infineon Technologies X - Gold may be connected to and / or communicate with entities such 618 - PMB9800 ( e . g . , providing 2G / 3G HSDPA / HSUPA as , but not limited to : one or more users from user input 30 communications ) ; and / or the like . The system clock typi devices 1111 ; peripheral devices 1112 ; an optional crypto - * graphic processor device 1128 ; and / or a communications cally has a crystal oscillator and generates a base signal network 1113 . through the computer systemization ' s circuit pathways . The

Networks are commonly thought to comprise the inter clock is typically coupled to the system bus and various connection and interoperation of clients , servers , and inter clock multipliers that may increase or decrease the base mediary nodes in a graph topology . It should be noted that 35 operating frequency for other components interconnected in the term “ server ” as used throughout this application refers the computer systemization . The clock and various compo generally to a computer , other device , program , or combi - nents in a computer systemization drive signals embodying nation thereof that processes and responds to the requests of information throughout the system . Such transmission and remote users across a communications network . Servers reception of instructions embodying information throughout serve their information to requesting “ clients . ” The term 40 a computer systemization may be commonly referred to as " client ” as used herein refers generally to a computer , communications . These communicative instructions may program , other device , user and / or combination thereof that further be transmitted , received , and the cause of return is capable of processing and making requests and obtaining and / or reply communications beyond the instant computer and processing any responses from servers across a com systemization to : communications networks , input devices , munications network . A computer , other device , program , or 45 other computer systemizations , peripheral devices , and / or combination thereof that facilitates , processes information the like . It should be understood that in alternative embodi and requests , and / or furthers the passage of information ments , any of the above components may be connected from a source user to a destination user is commonly directly to one another , connected to the CPU , and / or referred to as a “ node . ” Networks are generally thought to facilitate the transfer of information from source points to organized in numerous variations employed as exemplified destinations . A node specifically tasked with furthering the e 50 by various computer systems . passage of information from a source to a destination is The CPU comprises at least one high - speed data proces commonly called a " router . ” There are many forms of sor adequate to execute program components for executing networks such as Local Area Networks ( LAN ) , Pico net - user and / or system - generated requests . Often , the processors works , Wide Area Networks ( WANs ) , Wireless Networks themselves may incorporate various specialized processing ( WLANs ) , etc . For example , the Internet is generally 55 units , such as , but not limited to : integrated system ( bus ) accepted as being an interconnection of a multitude of controllers , memory management control units , floating networks whereby remote clients and servers may access point units , and even specialized processing sub - units like and interoperate with one another . graphics processing units , digital signal processing units ,

The AD - SURVEY controller 1101 may be based on and / or the like . Additionally , processors may include inter computer systems that may comprise , but are not limited to , 60 nal fast access addressable memory , and be capable of components such as : a computer systemization 1102 con mapping and addressing memory 1129 beyond the processor nected to memory 1129 . itself ; internal memory may include , but is not limited to :

fast registers , various levels of cache memory ( e . g . , level 1 , Computer Systemization 2 , 5 , etc . ) , RAM , etc . The processor may access this memory

65 through the use of a memory address space that is accessible A computer systemization 1102 may comprise a clock via instruction address , which the processor can construct

1130 , central processing unit ( “ CPU ( s ) ” and / or and decode allowing it to access a circuit path to a specific

63 US 9 , 807 , 442 B2

64 memory address space having a memory state . The CPU Power Source may be a microprocessor such as : AMD ' s Athlon , Duron and / or Opteron ; ARM ' s application , embedded and secure The power source 1186 may be of any standard form for processors ; IBM and / or Motorola ' s DragonBall and Pow - powering small electronic circuit board devices such as the erPC ; IBM ' s and Sony ' s Cell processor ; Intel ' s Celeron , 5 following power cells : alkaline , lithium hydride , lithium ion , Core ( 2 ) Duo , Itanium , Pentium , Xeon , and / or XScale ; lithium polymer , nickel cadmium , solar cells , and / or the like . and / or the like processor ( s ) . The CPU interacts with memory Other types of AC or DC power sources may be used as well . through instruction passing through conductive and / or trans - In the case of solar cells , in one embodiment , the case portive conduits ( e . g . , ( printed ) electronic and / or optic cir - provides an aperture through which the solar cell may cuits ) to execute stored instructions ( i . e . , program code ) 10 capture photonic energy . The power cell 1186 is connected according to conventional data processing techniques . Such to at least one of the interconnected subsequent components instruction passing facilitates communication within the of the AD - SURVEY thereby providing an electric current to AD - SURVEY controller and beyond through various inter - all subsequent components . In one example , the power faces . Should processing requirements dictate a greater source 1186 is connected to the system bus component 1104 . amount speed and / or capacity , distributed processors ( e . g . , 15 In an alternative embodiment , an outside power source 1186 Distributed AD - SURVEY ) , mainframe , multi - core , parallel , is provided through a connection across the I / O 1108 inter and / or super - computer architectures may similarly be face . For example , a USB and / or IEEE 1394 connection employed . Alternatively , should deployment requirements carries both data and power across the connection and is dictate greater portability , smaller Personal Digital Assis - therefore a suitable source of power . tants ( PDAs ) may be employed .

Depending on the particular implementation , features of Interface Adapters the AD - SURVEY may be achieved by implementing a microcontroller such as CAST ' s R8051XC2 microcon Interface bus ( ses ) 1107 may accept , connect , and / or com troller : Intel ' s MCS 51 ( i . e . , 8051 microcontroller ) ; and / or municate to a number of interface adapters , conventionally the like . Also , to implement certain features of the AD - 25 although not necessarily in the form of adapter cards , such SURVEY , some feature implementations may rely on as but not limited to : input output interfaces ( 1 / 0 ) 1108 , embedded components , such as : Application - Specific Inte - storage interfaces 1109 , network interfaces 1110 , and / or the grated Circuit ( “ ASIC ” ) , Digital Signal Processing ( “ DSP ” ) , like . Optionally , cryptographic processor interfaces 1127 Field Programmable Gate Array ( “ FPGA " ) , and / or the like similarly may be connected to the interface bus . The inter embedded technology . For example , any of the AD - SUR - 30 face bus provides for the communications of interface VEY component collection ( distributed or otherwise ) and / or adapters with one another as well as with other components features may be implemented via the microprocessor and / or of the computer systemization . Interface adapters are via embedded components ; e . g . , via ASIC , coprocessor , adapted for a compatible interface bus . Interface adapters DSP , FPGA , and / or the like . Alternately , some implemen - conventionally connect to the interface bus via a slot archi tations of the AD - SURVEY may be implemented with 35 tecture . Conventional slot architectures may be employed , embedded components that are configured and used to such as , but not limited to : Accelerated Graphics Port achieve a variety of features or signal processing . ( AGP ) , Card Bus , ( Extended ) Industry Standard Architec

Depending on the particular implementation , the embed - ture ( ( E ) ISA ) , Micro Channel Architecture ( MCA ) , NuBus , ded components may include software solutions , hardware Peripheral Component Interconnect ( Extended ) ( PCI ( X ) ) , solutions , and / or some combination of both hardware / soft - 40 PCI Express , Personal Computer Memory Card Interna ware solutions . For example , AD - SURVEY features dis - tional Association ( PCMCIA ) , and / or the like . cussed herein may be achieved through implementing Storage interfaces 1109 may accept , communicate , and / or FPGAs , which are a semiconductor devices containing connect to a number of storage devices such as , but not programmable logic components called “ logic blocks ” , and limited to : storage devices 1114 , removable disc devices , programmable interconnects , such as the high performance 45 and / or the like . Storage interfaces may employ connection FPGA Virtex series and / or the low cost Spartan series protocols such as , but not limited to : ( Ultra ) ( Serial ) manufactured by Xilinx . Logic blocks and interconnects can Advanced Technology Attachment ( Packet Interface ) ( ( Ul be programmed by the customer or designer , after the FPGA tra ) ( Serial ) ATA ( PI ) ) , ( Enhanced ) Integrated Drive Elec is manufactured , to implement any of the AD - SURVEY tronics ( ( E ) IDE ) , Institute of Electrical and Electronics features . A hierarchy of programmable interconnects allow 50 Engineers ( IEEE ) 1394 , fiber channel , Small Computer logic blocks to be interconnected as needed by the AD Systems Interface ( SCSI ) , Universal Serial Bus ( USB ) , SURVEY system designer / administrator , somewhat like a and / or the like . one - chip programmable breadboard . An FPGA ' s logic Network interfaces 1110 may accept , communicate , and / blocks can be programmed to perform the operation of basic or connect to a communications network 1113 . Through a logic gates such as AND , and XOR , or more complex 55 communications network 1113 , the AD - SURVEY controller combinational operators such as decoders or mathematical is accessible through remote clients 1133b ( e . g . , computers operations . In most FPGAs , the logic blocks also include with web browsers ) by users 1133a . Network interfaces may memory elements , which may be circuit flip - flops or more employ connection protocols such as , but not limited to : complete blocks of memory . In some circumstances , the direct connect , Ethernet ( thick , thin , twisted pair 10 / 100 / AD - SURVEY may be developed on regular FPGAs and then 60 1000 Base T , and / or the like ) , Token Ring , wireless connec migrated into a fixed version that more resembles ASIC tion such as IEEE 802 . 11a - x , and / or the like . Should pro implementations . Alternate or coordinating implementations cessing requirements dictate a greater amount speed and / or may migrate AD - SURVEY controller features to a final capacity , distributed network controllers ( e . g . , Distributed ASIC instead of or in addition to FPGAs . Depending on the AD - SURVEY ) , architectures may similarly be employed to implementation all of the aforementioned embedded com - 65 pool , load balance , and / or otherwise increase the commu ponents and microprocessors may be considered the “ CPU ” nicative bandwidth required by the AD - SURVEY controller . and / or “ processor ” for the AD - SURVEY . A communications network may be any one and / or the

| HAO HATA KAMA MARIA DE LA CANTATA TA TTI

Documents

Transcript of | HAO HATA KAMA MARIA DE LA CANTATA TA TTI