Continue

Probabilityandstochasticprocessesyates3rdeditionpdfdownload

T\vOortlireeliriesofl\IIATLABcodearest1fficienttosirriulateanarbitrarynumberofsequentialtrials.vVerecallfrorriSection1.7tliatrand(1,m)[56CHAPTER2SEQUENTIALEXPERIMENTSy=Columns47Columns42Columns60Columns56Columns501through1252484613through2452404025through3659494737through4854605349through60465250Figure2.4c:::==--54484748594449484748484853494561494548514853525352515847504844495151575049564456Thesin1ulationoutputof60repeatedexperin1entsof100coinflips.Example2.25Usingl\/IATLAB,perform60experiments.Ineachexperiment,flipacoin100timesandrecordthenumberofheadsinavectorYsuchthatthejthelementY:'iisthenumberofheadsinsubexperimentj.TheMATLABcodeforthistaskappearsontheleft.The100x60matrixXhas'i>jthelementX(i,j)=O(tails)orX(i,j)=1(heads)toindicatetheresultofflipiofsubexperimentj.SinceYsumsXacrossthefirstdimension,Y(j)isthenumberofheadsinthejthsubexperiment.EachY(j)isbetween0and100andgenerallyintheneighborhoodof50.TheoutputofasamplerunisshowninFigure2.4.>>X=rand(100,60)>Y=surn(X,1)Example2.26Simulatethetestingof100microprocessorsasdescribedinExample2.23.Youroutputshouldbea4x1vectorXsuchthatX,iisthenumberofgradeimicroprocessors.%chiptest.mG=ceil(4*rand(1,100));T=1:4;X=hist(G,T);ThefirstlinegeneratesarowvectorGofrandomgradesfor100microprocessors.ThepossibletestscoresareinthevectorT.Lastly,X=hist(G,T)returnsahistogramvectorXsuchthatX(j)countsthenumberofelementsG(i)thatequalT(j).Notethat"helphist"willshowthevarietyofwaysthatthehistfunctioncanbecalled.Morever,X=hist(G,T)doesmorethanjustcountthenumberofelementsofGthatequaleachelementofT.Inparticular,hist(G,T)createsbinscenteredaroundeachT(j)andcountsthenumberofelementsofGthatfallintoeachbin.Notethatin1\IIA.TLABallvariablesareassurr1edtobernatrices.InwritingMATLABcode,Xrr1a:ybeann,xrn,rnatrix>ann,x1colurr1nvector,a1x1T1,rO\iVvector>ora1x1scalar.In1\IIATLAB,wewriteX(i,j)toindexthei,jthelerr1er1t,Bycontrast,int11istext,we\rarythenotationdependir1gon\iVhetherv.rehavea[PROBLEMS57scalarX)oravectororrr1atrixX.Inaddition)v.ret1seXi,jtodenotethe'i)jthelement.T11us)XandX(inaM.A.TLABcodefragrnent)ma}'bothrefertothesarr1evariable.Quiz2.5Theflipofathickcoin}'ieldsheadswithprobability0.4,tailsvvithprobability0.5)orlandsonitsedgevvithprobability0.1.Sirnulate100thickcoinflips.Yol1routpl1tsl1ol1ldbea3x1·vectorXsuchthatX1)X2)andX3aret11enurr1berofoccurrencesofheads,tails)andedge.ProblemsDifficulty:Easy2.1.1Supposeyouflipacoint'vice.Onanyflip,thecoincomesupheadswithprobability1/4.UseHiandTitodenotetheresultofflipi.(a)Whatistheprobability,P[H1IH2],thatthefirstflipisheadsgiventhatthesecondflipisheads?(b)Whatistheprobabilit:ythatthefirstflipisheadsandthesecondflipistails?2.1.2ForExample2.2,supposeP[G1)=1/2,P[G2IG1)=3/4,andP[G2IR1]=1/4.FindP[G2),P[G2IG1),andP[G1IG2).2.1.3Attheendofregulationtime,abasketballteamistrailingbyonepointandaplayergoestothelinefort\vofreethrovvs.Iftheplayerinakesexactlyonefreethrow,thegamegoesintoovertime.Theprobabilitythatthefirstfreethrowisgoodis1/2.However,ifthefirstattemptisgood,theplayerrelaxesandthesecondattemptisgood\Vithprobability3/4.However,iftheplayermissesthefirstattempt,theaddedpressurereducesthesuccessprobabilityto1/4.Whatistheprobabilitythatthegamegoesintoovertime?2.1.4Youhavet\vobiasedcoins.CoinAcomesupheadswithprobability1/4.CoinBcomesupheads\vithprobability3/4.However,youarenotsurewhichis\Vhich,soyouchooseacoinrandomlyandyouflipit.Iftheflipisheads,youguessthattheflippedcoinisB;otherwise,youguessthattheflippedcoinis.4.\tVhatistheprobabilityP[C)thatyourguessiscorrect?ModerateDifficultExpertsOnly2.1.5Supposethatforthegeneralpopulation,1in5000peoplecarriesthehumanimmunodeficiencyvirus(HIV).AtestforthepresenceofHIVyieldseitherapositive(+)ornegative(-)response.Supposethetestgivesthecorrectans,ver993ofthetime.WhatisP[-IHJ,theconditionalprobabilitythatapersontestsnegativegiventhatthepersondoeshavetheHIVvirus?WhatisP[HI+],theconditionalprobabilitythatarandomlychosenpersonhastheHIVvirusgiventhatthepersontestspositive?2.1.6Amachineproducesphotodetectorsinpairs.Testsshowthatthefirstphotodetectorisacceptablewithprobability3/5.Whenthefirstphotodetectorisacceptable,thesecondphotodetectorisacceptablewithprobability4/5.Ifthefirstphotodetectorisdefective,thesecondphotodetectorisacceptable\vithprobability2/5.(a)Findtheprobabilitythatexactlyonephotodetectorofapairisacceptable.(b)Findtheprobabilitythatbothphotodetectorsinapairaredefective.2.1.7•Youhavetwobiasedcoins.Coin.4comesupheads\Vithprobability1/4.CoinBcomesupheadswithprobability3/4.Ho,vever,youarenotsurewhichiswhichsoyouflipeachcoinonce,choosingthefirstcoinrandomly.UseHiandTitodenotetheresultofflipi.Let.41betheeventthatcoinAwasflippedfirst.LetB1betheeventthatcoinBwasflippedfirst.\tVhatisP[H1H2)?[58CHAPTER2SEQUENTIALEXPERIMENTSAreH1andH2independent?Explainyouranswer.2.1.8Aparticularbirthdefectoftheheartisrare;ane,vborninfantwillhavethedefectD'vithprobabilityP[D)=10-4.Inthegeneralexa1nofane,vborn,aparticularheartarrhythmiaAoccurswithprobability0.99ininfants'viththedefect.However,thearrhythmiaalsoappears,;vithprobability0.1ininfantswithoutthedefect.\!\!henthearrhythmiaispresent,alabtestforthedefectisperformed.Theresultofthelabtestiseitherpositive(eventr+)ornegative(eventT-).Inanewborn'viththedefect,thelabtestispositive'vithprobabilityp=0.999independentfromtesttotest.Inane,vborn,;vithoutthedefect,thelabtestisnegative'vithprobabilityp=0.999.Ifthearrhythmiaispresentandthetestispositive,thenheartsurgery(eventH)isperformed.(a)Giventhearryth1niaAispresent,'vhatistheprobabilitytheinfanthasthedefectD?(b)Giventhataninfanthasthedefect,whatistheprobabilityP[HIDJthatheartsurgeryisperformed?(c)Giventhattheinfantdoesnothavethedefect,whatistheprobabilityq=P[HIDc)thatanunnecessaryheartsurgeryisperformed?(d)FindtheprobabilityP[H)thataninfanthasheartsurge1yperformedforthearrythmia.(e)Giventhatheartsurgeryisperformed,whatistheprobabilitythatthenewborndoesnothavethedefect?2.1.9SupposeDagwood(Blondie'shusband)wantstoeatasandwichbutneedstogoonadiet.Dagwooddecidestolettheflipofacoindetermine'vhetherheeats.·usinganunbiasedcoin,Da~voodwillpostponethediet(andgodirectlytotherefrigerator)ifeither(a)heflipsheadsonhisfirstflipor(b)heflipstailsonthefirstflipbutthenproceedstogett'voheadsoutofthenextthreeflips.Notethatthefirstflipisnotcountedintheattempttowint'voofthreeandthatDag,voodneverperformsanyunnecessaryflips.LetHibetheeventthatDag,voodflipsheadsontryi.LetTibetheeventthattailsoccursonflipi.(a)Drawthetreeforthisexperiment.Labeltheprobabilitiesofalloutcomes.(b)\i\fhatareP[H3)andP[T.1)?(c)LetDbetheeventthatDag,voodmustdiet.WhatisP[D)?\i\!hatisP[H1IDJ?(d)AreH3andH2independentevents?2.1.10ThequalityofeachpairofphotodetectorsproducedbythemachineinProblem2.1.6isindependentofthequalityofeveryotherpairofdetectors.(a)\!\!hatistheprobabilityoffindingnogooddetectorsinacollectionofnpairsproducedbythemachine?(b)Howmanypairsofdetectorsmustthemachineproducetoreachaprobabilityof0.99thatthere'villbeatleastoneacceptablephotodetector?2.1.11InStevenStrogatz'sNewYorkTimesbloghttp:IIopinionator.blogs.nytirnes.corn/2010/04/25/chances-are/?ref=opinion,thefollo,vingproblem'vasposedtohighlighttheconfusingcharacterofconditionalprobabilities.Beforegoingon1;acationfora71Jeek,youaskyo1J,rspaceyfriendto71Jateryo1J,railingplant.Without111ater,theplanthasa90percentchanceofdying.E1;en71Jithproper111atering,ithasa20percentchanceofdying.Andtheprobabilitythatyourfriend1Dillforgetto71Jateritis30percent.(a)What'sthechancethatyo1J,rplant7Dillsurvivethe111eek?{b)Ifit'sdead71Jhenyoureturn,71Jhat'sthechancethatyourfriendforgotto71Jaterit?(c)Ifyo1J,rfriendforgotto11Jaterit,71Jhat'sthechanceit'llbedead1Dhenyoureturn?Solveparts(a),(b)and(c)ofthisproblem.2.1.12•Eachtimeafishe1mancastshisline,afishiscaught,;vithprobabilityp,independentof'vhetherafishiscaughtonanyothercastoftheline.Thefishermanwillfishalldayuntilafishiscaughtand[PROBLEMSthenhe'villquitandgohome.LetCidenotetheeventthatoncastithefishermancatchesafish.DrawthetreeforthisexperimentandfindP[C1),P[C2],andP[Cn]asfunctionsofp.Oneachturnoftheknob,agumballmachineisequallylikelytodispenseared,yellow,greenorbluegumball,independentfromturntoturn.Aftereightturns,whatistheprobabilityI>[R2Y2G2B2]thatyouhavereceived2red,2yellow,2greenand2bluegumballs?2.2.1AStarburstcandypackagecontains12individualcandypieces.Eachpieceisequallylikelytobeberry,orange,lemon,orcherry,independentofallotherpieces.2.2.2(a)WhatistheprobabilitythataStarburstpackagehasonlyberryorcherrypiecesandzeroorangeorlemonpieces?(b)WhatistheprobabilitythataStarburstpackagehasnocherrypieces?(c)\i\fhatistheprobabilityP[F1]thatalltwelvepiecesofyourStarburstarethesameflavor?YourStarburstcandyhas12pieces,threepiecesofeachoffourflavors:berry,le1non,orange,andcherry,arrangedinarandomorderinthepack.Youdrawthefirstthreepiecesfromthepack.2.2.3(a)Whatistheprobabilitytheyareallthesameflavor?(b)Whatistheprobabilitytheyarealldifferentflavors?YourStarburstcandyhas12pieces,threepiecesofeachoffourflavors:berry,lemon,orange,andcherry,arrangedinarandomorderinthepack.Youdrawthefirstfourpiecesfromthepack.2.2.4(a)WhatistheprobabilityP[F1]theyareallthesameflavor?(b)WhatistheprobabilityP[F4]theyarealldifferentflavors?(c)\i\fhatistheprobabilityP[F2]thatyourStarbursthasexactlytwopiecesofeachoft'vodifferentflavors?59Inagameofrummy,youaredealtaseven-cardhand.2.2.5(a)WhatistheprobabilityP[R7]thatyourhandhasonlyredcards?(b)\i\fhatistheprobabilityP[F]thatyourhandhasonlyfacecards?(c)\tVhatistheprobabilityP[R1F]thatyourhandhasonlyredfacecards?(Thefacecardsarejack,queen,andking.)Inagameofpoker,youaredealtafive-cardhand.2.2.6(a)\t\fhatistheprobabilityI>[R5]thatyourhandhasonlyredcards?(b)\i\fhatistheprobabilityofa"fullhouse"withthree-of-a-kindandtwo-ofa-kind?Considerabinarycode'vith5bits(0or1)ineachcode'vord.Anexampleofacodewordis01010.Howmanydifferentcodewordsarethere?Ho'vmanycodewordshaveexactlythreeO's?2.2.7Consideralanguagecontainingfourletters:A,B,C,D.Ho'vmanythree-letterwordscanyouforminthislanguage?Ho'vmanyfour-letter'vordscanyouformifeachletterappearsonlyonceineachword?2.2.8OnanAmericanLeaguebaseballteam'vith15fieldplayersand10pitchers,themanagerselectsastartinglineupwith8fieldplayers,1pitcher,and1designatedhitter.Thelineupspecifiestheplayersforthesepositionsandthepositionsinabattingorderforthe8fieldplayersanddesignatedhitter.Ifthedesignatedhittermustbechosenamongallthefieldplayers,howmanypossiblestartinglineupsarethere?2.2.9SupposethatinProble1n2.2.9,thedesignatedhittercanbechosenfromamongalltheplayers.Howmanypossiblestartinglineupsarethere?2.2.10Atacasino,theonlygameisnumberlessroulette.Onaspinofthe'vheel,theballlandsinaspacewithcolorred(r),green(g),orblack(b).Thewheelhas19redspaces,19greenspacesand2blackspaces.2.2.11[60CHAPTER2SEQUENTIALEXPERIMENTS(a)In40spinsofthewheel,findtheprobabilityoftheeventoftheCelticswinningeightstraightchampionshipsbeginningin1959?Also,whatwouldbetheprobabilityoftheCelticswinA={19reds,19greens,and2blacks}.ningthetitlein10outof11years,startingin1959?Givenyouranswers,doyoutrust(b)In40spinsofthe'vheel,findtheprob-thissimpleprobabilitymodel?abilityofG19={19greens}.2.3.3Supposeeachdaythatyoudriveto(c)Theonlybetsallowedareredandworkatrafficlightthatyouencounteriseigreen.Giventhatyourandomlychoosethergreen\Vithprobability7/16,redwithtobetredorgreen,'vhatistheprobaprobability7/16,oryello\v'vithprobabilitybilitypthatyourbetisavvinner?1/8,independentofthestatusoftheliaht0onanyotherday.Ifoverthecourseoffive2.2.12Abasketballteamhasthreepuredays,G,Y,andRdenotethenumberofcenters,fourpurefor\vards,fourp1u·etimesthelightisfoundtobegreen,yello,v,guards,andoneswingmanwhocanplayorred,respectively,\vhatistheprobabilityeitherguardorforward.ApurepositionthatP[G=2,Y=1,R=2]?_Also,'vhatisplayercanplayonlythedesignatedpositheprobabilityP[G=R]?tion.Ifthecoachmuststartalineupwithonecenter,t\vofor,vards,andtwoguards,2.3.4•Inagamebetweent\voequalteams,howinanypossiblelineupscanthecoachthehometeam\Vins\Vithprobabilityp>choose?1/2.Inabestofthreeplayoffseries,a2.2.13Aninstantlotteryticketconsistsofacollectionofboxescoveredwithgray\Vax.Forasubsetoftheboxes,thegraywaxhidesaspecialmark.Ifaplayerscratchesoffthecorrectnu1nberofthemarkedboxes(andnoboxes'vithoutthemark),thenthatticketisa\Vinner.Designaninstantlotterygamein'vhichaplayerscratchesfiveboxesandtheprobabilitythataticketisa\vinnerisapproximately0.01.team'viththehomeadvantagehasagameathome,followedbyagamea\vay,followedbyahomegameifnecessary.Theseriesisoverassoonasoneteam\vinst\vogames.\tVhatisP[H],theprobabilitythattheteamwiththeho1neadvantagewinstheseries?Isthehomeadvantageincreasedbyplayingathree-gameseriesratherthanaone-gameplayoff?Thatis,isittruethatP[HJ>pforallp>1/2?2.3.12.3.5Considerabinarycode'vith5bits(0or1)ineachcode\vord.Anexampleofacodewordis01010.Ineachcodeworda'bitisazerowithprobability0.8,independentofanyotherbit.(a)Whatistheprobabilityofthecodeword00111?(b)Whatistheprobabilitythatacodewordcontainsexactlythreeones?2.3.2TheBostonCelticshavewon16NBi\.championshipsoverapproximately50years.ThusitmayseemreasonabletoassumethatinagivenyeartheCeltics\Vinthetitle\Vithprobabilityp=16/50=0.32,independentofanyotheryear.Givensuchamodel,what\VotlldbetheprobabilityAcollectionoffieldgoalkickersaredividedintogroups1and2.Groupihas3ikickers.Onanykick,akickerfro1ngroupivvillkickafieldgoalwithprobability1/(i+l),independentoftheoutcomeofanyotherkicks.(a)Akickerisselectedatrandomfromamongallthekickersandattemptsonefieldgoal.LetKbetheeventthatafieldgoaliskicked.FindP[K].(b)T'vokickersareselectedatrandom·'J{J·istheeventthatkickerjkicksafieldgoal.AreJ{iandJ{2independent?(c)_A.kickerisselectedatrandomandattempts10fieldgoals.LetMbethenumberofinisses.FindP[M=5].[PROBLEMS2.4.1Aparticularoperationhassixcomponents.Eachcomponenthasafailureprobabilityq,independentofanyothercomponent.Asuccessfuloperationrequiresbothofthefollowingconditions:•Components1,2,and3all\Vork,orcomponent4\Vorks.•Component5orcomponent6works.Dra'vablockdiagramforthisoperationsimilartothoseofFigure2.2onpage53.DeriveaformulafortheprobabilityP[W]thattheoperationissuccessful.2.4.2WewishtomodifythecellulartelephonecodingsysteminExample2.21inordertoreducethenumberoferrors.Inparticular,iftherearet\voorthreezeroesinthereceivedsequenceof5bits,\Ve\villsaythatadeletion(eventD)occurs.Other,vise,ifatleast4zeroesarereceived,thereceiverdecidesazero\Vassent,orifatleast4onesarereceived,thereceiverdecidesaonewassent.Wesaythatanerroroccursifi\Vassentandthereceiverdecidesjf=.i\Vassent.Forthismodifiedprotocol,\vhatistheprobabilityP[E]ofanerror?WhatistheprobabilityP[D]ofadeletion?2.4.3Supposea10-digitphonenumberistransmittedbyacellularphoneusingfourbinarysymbolsforeachdigit,usingthemodelofbinarysymbolerrorsanddeletionsgiveninProblem2.4.2.LetCdenotethenumberofbitssentcorrectay,Dthenumberofdeletions,andEthenumberoferrors.FindP[C=c,D=d,E=e]forallc,d,ande.2.4.4ConsiderthedeviceinProblem2.4.l.Supposewecanreplaceanyonecomponent\vithanultrareliablecomponentthathasafailureprobabilityofq/2=0.05.\i\Thichcomponentshouldwereplace?2.5.1BuildaIVIATLABsimulationof50trialsoftheexperimentofExample2.3.Youroutputshouldbeapairof50x1vectorsCandH.Fortheithtrial,Hiwill61record'vhetherit\Vasheads(Hi=1)ortails(Hi=0),andCiE{1,2}\Villrecordwhichcoin\Vaspicked.2.5.2•FollowingQuiz2.3,supposethecommunicationlinkhasdifferenterrorprobabilitiesfortrans1nitting0and1.\tVhena1issent,itisreceivedasa0withprobability0.01.\tVhena0issent,itisreceivedasa1'vithprobability0.03.Eachbitinapacketisstillequallylikelytobea0or1.Packetshavebeencodedsuchthatiffiveorfewerbitsarereceivedinerror,thenthepacketcanbedecoded.Simulatethetransmissionof100packets,eachcontaining100bits.Countthenumberofpacketsdecodedcorrectly.2.5.3Forafailureprobabilityq=0.2,simulate100trialsofthesix-componenttestofI>roblem2.4.l.Ho\vmanydeviceswerefoundtowork?Perform10repetitionsofthe100trials.Whatdoyoulearnfrom10repetitionsof100trialscomparedtoasimulatedexperimentvvith100trials?2.5.4\i\1riteaJVIATLABfunctionN=countequal(G,T)thatduplicatestheactionofhist(G,T)inExample2.26.Hint:Usendgrid.2.5.5Inthisproblem,\VeuseaMATLABsimulationto"solve"Problem2.4.4.Recallthataparticularoperationhassixcomponents.Eachcomponenthasafailureprobabilityqindependentofanyothercomponent.Theoperationissuccessfulifboth•Components1,2,and3allwork,orcomponent4\vorks.•Component5orcomponent6\Vorks.\tVithq=0.2,simulatethereplacementofacomponent\Vithanultrareliablecomponent.Foreachreplacementofaregularcomponent,perform100trials.Are100trialssufficienttodecidewhichcomponentshouldbereplaced?[DiscreteRandomVariables3.1DefinitionsArandorr1variableassignsnurr1berstooutcorr1esinthesamplespaceofanexperiment.Chapter1defir1esaprobabilityrr1odel.Itbeginswithaphysicalrr1odelofanexperirnent.Anexperimentconsistsofaprocedureandobservations.T.hesetofallpossibleobservations,S,ist11esarr1plespaceoftheexperirr1ent.Sisthebeginningofthernathernaticalproba,bilitymodel.InadditiontoS,therr1athernaticalmodelincludesarl1leforassigr1ingntlrnbersbetv.reen0arid1tosetsAinS.ThtlSforeveryAcS,therr1odelgivesusaprobabilityP(A],vvhere0<P(A)<1.Inthisc11apterandformostoftherernainderofthisbook,'"'eexarr1ineprobabilityrr1odelsthatassignr1t1mberstot11eot1tcomesir1thesarr1plespace.\Nl1er1weobserveoneofthesent1rnbers,werefertotheobservationasaran,dorn'variable.Ir1ournotation,thenameofarai,ndorr1variableisalwaysacapitalletter,forexarr1ple,X.ThesetofpossiblevaluesofXist11eran,geofX.Sincev.reoftenconsiderrnorethanonerar1domvariableatatime,wedenotetherangeofarar1domvariableb}'theletterS\A.TithaSl1bscriptthatisthenarr1eoftherandornvariable.T11usSxist11erangeofrandorr1va,riableX,Syist11erangeofrar1domvariable·y,andsoforth.\Ve11seSxtodenotetherangeofXbecausethesetofallpossiblevalt1esofXisanalogol1StoS,t11esetofallpossibleoutcornesofar1experiment.Aprobabilitymodelal\va}'Sbeginswit11anexperiment.Eachrar1dornvariableisrelateddirectlytothisexperimer1t.Therearethreetypesofrelationships.1.Therandorr1·variableist11eobservation.---Example3.1--TheexperimentistoattachaphotodetectortoanopticaIfiberandcountthenumberofphotonsarrivinginaone-microsecondtimeinterval.Eachobservation62[3.1DEFINITIONS63isarandomvariableX.TherangeofXisSx={O,1,2,...}.Inthiscase,Sx,therangeofX,andthesamplespaceSareidentical.2.Therar1dom·variableisafur1ctionoft11eobservation.---Example3.2'--Theexperimentistotestsixintegratedcircuitsandaftereachtestobservewhetherthecircuitisaccepted(a)orrejected(r).Eachobservationisasequenceofsixletterswhereeachletteriseitheraorr.Forexample,s8=aaraaa.ThesamplespaceSconsistsofthe64possiblesequences.ArandomvariablerelatedtothisexperimentisN,thenumberofacceptedcircuits.Foroutcomes8,J\T=5circuitsareaccepted.TherangeofNisSN={O,1,...,6}.3.Therandorr1·variableisaft1nctionofanotherrandomvariable.Example3.3InExample3.2,thenetrevenueRobtainedforabatchofsixintegratedcircuitsis$5foreachcircuitacceptedminus$7foreachcircuitrejected.(Thisisbecauseforeachbadcircuitthatgoesoutofthefactory,itwi11costthecompany$7todealwiththecustomer'scomplaintandsupplyagoodreplacementcircuit.)WhenNcircuitsareaccepted,6-NcircuitsarerejectedsothatthenetrevenueRisrelatedtoNbythefunctionR=g(N)=5N-7(6-J\T)=12N-42dollars.(3.1)SinceS1v={O,...,6},therangeofRisSR={-42,-30,-18,-6,6,18,30}.(3.2)Therevenueassociatedwiths8=aaraaaandallotheroutcomesforwhichN=5ISg(5)=12x5-42=18dollars(3.3)Ifv.reha·veaprobabilityrnodelfortheintegratedcircuitexperirr1entinExarnple3.2,wecanuset11atprobabilit}'rnodeltoobtainaprobabilityrr1odelfortherandornvariable.Thererr1ainderofthischapterwillde·veloprnethodstoc11aracterizeprobabilitymodelsforrandomvariables.Weobservethatintheprecedingexarr1ples,theval11eofar.a,ndorr1variablecar1alvvaysbederivedfrorr1theoutcorneoftheur1derlyingexperirn.er1t.Thisisnotacoincidence.T11eformaldefinitionofarandorr1variablereflectsthisfact.[64CHAPTER3DISCRETERANDOMVARIABLESDefinition3.1RandomVariableArandomvariableco'nsistsofan,experim,en,t'tuithaprobabilityrneasv,reP[·]defin,edon,asarnplespaceSan,daf11,rict'ior1,thatassign,sarealri11,rnbertoeachoutcornein,thesamJJlespaceoftheex;perirnen,t.Thisdefir1itionacknowledgest11atarandomvariableist11eresultofar1underlyir1gexperirnent,butitalsoperrr1itsllStoseparatetheexperiment,inparticl1lar,theobserva.tior1s,frorr1theprocessofassigningnumberstoOl1tcomes.As-vvesawir1Exarnple3.1,theassigr1rn.er1trr1aybeirr1plicitinthedefinitionoft11eexperirr1er1t,oritmayrequirefurtheranalysis.Insornedefir1itionsofexperirr1ents,theprocedurescontainvariablepararneters.Intheseexperirnents,therecanbevaluesofthepararnetersforwhichitisirr1possibletoperformtheobserva,tionsspecifiedintheexperiments.Int11esecases,theexperimentsdonotproducerandomvariables.·•-;.¥erefertoexperirr1ents-vvithpararnetersettingsthatdoriotproducerandornvariablesas'irnproperexperirnen,ts.Example3.4TheprocedureofanexperimentistofirearocketinaverticaldirectionfromEarth'ssurfacewithinitialvelocityVkm/h.TheobservationisTseconds,thetimeelapseduntiltherocketreturnstoEarth.Underwhatconditionsistheexperimentimproper?Atlowvelocities,V,therocketwi11returntoEarthatarandomtimeTsecondsthatdependsonatmosphericconditionsandsmalldetailsoftherocket'sshapeandweight.However,whenV>v*~40,000km/hr,therocketwillnotreturntoEarth.Thus,theexperimentisimproperwhenV>v*becauseitisimpossibletoperformthespecifiedobservation.Onoccasion,itisimportanttoidentifytherandorr1·variableXbythefunctionX(s)thatrr1apsthesampleoutcornestothecorrespondir1gvalueoftherar1domvariableX.Asneeded,we-vvillwrite{X=;i;}toemphasizethatthereisasetofsarr1plepointssESforw11ichX(s)=x.Thatis,"'irveha;veadoptedtheshorthandnotation{X=x}={sESIX(s)=x}.(3.4)Herearesomerr1orerar1domvariables:•A,thenumberofstl1dentsasleepinthenextprobabilitylecture;•C,t11ent1rr1beroftextsyourecei·veinthenexthour;•M,thent1rr1berofminutes:you-vvaituntilthenexttextarrives.RandornvariablesAandCarediscreterandomvariables.Thepossiblevaluesoftheserandomvariablesforrr1acot1ntableset.Theunderlyingexperimentshavesarnplespacesthatarediscrete.Therandorr1·variableMcanbear1:yrnonnegativerealr1l1rr1ber.Itisacon,tin/uo11,sran,dorn'oariable.Itsexperirr1enthasacontir1uous[3.2PROBABILITYMASSFUNCTION65sarr1plespace.Inthisc11apter,west11dythepropertiesofdiscreterar1dom·variables.Chapter4co·verscontinuousrandorr1·variables.Definition3.2DiscreteRandomVariableXisadiscreterar1,dornvariableiftherarigeofXisaco11n,tablesetThedefiningc11aracteristicofadiscreterar1domvariableisthatt11esetofpossiblevaluescan(ir1principle)belisted,e·venthoug11thelistrr1aybeir1finitelylor1g.Often,b11tnotalvva:ys,adiscreterandorr1variabletakesonir1tegervalues.Anexceptionistherandornvariablerelatedtoyourprobabilitygrade.T11eexperirr1er1tistotakethisco11rseandobserveyourgrade.AtRutgers,thesarnplespaceisS={F,D,c,c+,B,B+,A}.(3.5)vVeuseafunctionG1(-)torr1apthissarr1plespaceintoarar1dorn·variable.Forexarr1ple,G1(A)=4andG1(F)=0.Thetable011tcornesF0D1cc+22.5A33.54isacor1cisedescriptior1oftheentiremapping.G1isadiscreterandorn·variablewithrangeSa1={O,1,2,2.5,3,3.5,4}.Haveyot1tho11ghtaboutwhywetransformlettergradestonumericalvalues?Webelievetheprincipalreasonisthatitallowsustocomputeaverages.Tllisisalsoanimportantrr1otivationforcreatingrandorr1variablesbyassignir1gnurnberstotheoutcomesinasarr1plespace.Unlikeprobabilitymodelsdefinedonarbitrarysarnplespaces,randorr1variableshaveexpectedval'ues,vv11icharecloselyrelatedtoa·veragesofdatasets.Weintroduceexpectedvaluesformallyir1Section3.5.Quiz3.t=::::::..Astudenttakestvvocotu·ses.Ir1eachcourse,thestudentv.rillearneitheraBoraC.Toca.lc11lateagradepointaverage(GPA),aBisv.rort113pointsandaCisvvorth2poir1ts.Thestudent'sGPAG2isthesurnofthepoir1tsearnedforeachcoursedividedb}'2.}/lakeatableofthesarr1plespaceoftheexperirr1entaridt11ecorrespondingvaluesoft11eGPA,G2.3.2ProbabilityMassFunctionThePJ\!fFofrandornvariableXexpressestheprobabilityrnodelofar1experimentasarr1athematicalf11nctiori.T11ef11nctior1istheprobabilityP[X=3';]fore·verynumberx;.[66CHAPTER3DISCRETERANDOMVARIABLESRecallthattheprobabilitymodelofadiscreterandorrivariableassignsant1mberbetvveen0arid1toeachot1tcorrieiriasarnplespace.vVhenweha\readiscreterandorrivariableX,weexpresstheprobabilit}'rnodelasaprobabilityrriassfunction(PMF)Px(x).Tlieargl1rr1entofaP1!{Frangesoverallrealnurnbers.Definition3.3ProbabilityMassFunction(PMF)Theprobabilitymassfunction(PMF)ofthediscreteran,dornvariableXisPx(x)=P[X=x;]NotetliatX=xisarieventcorisistingofallOl1tcomessoftheunderlyingexperimentforvvhicliX(s)=x;.Ontheotherhand,Px(x)isafunctionrangingoverallrealnl1rnbersx.Forar1yvall1eofx;,thefunctioriPx(x;)istheprobabilit}'oftlieeventX=x.ObserveOllrnotationforarandorri\rariableanditsPMF.vVellSeanuppercaseletter(Xintheprecedingdefinition)fortlienarneofarandornvariable.Weust1all}'t1sethecorrespondinglowercaseletter(x)todenoteapossiblevalueoftheraridomvariable.ThenotationfortheP1!{FistheletterPv.rithast1bscriptiridicatingthenarneoftlierandornvariable.ThusPR(r)istlienotationfortheP1!{FofraridomvariableR.Intheseexamples,randxared11rnmyvariables.TliesarrierandornvariablesandP1!{FsCOl1ldbedenotedPR(v,)andPx(v,)or,indeed,PR(-)aridPx(-).vVederi\rethePMFfromthesarriplespace,theprobabilit}'rnodel,andtlierulethatmapsoutcorriestovaluesoftherandom\rariable.vVethengraphaPMFbyrriarkingonthehorizontalaxiseacli\ralue\vithnorizeroprobabilit}'anddravvingaverticalbarwithlengthproportionaltotheprobability.Example3.5WhenthebasketballplayerWiltChamberlainshottwofreethrows,eachshotwasequallylikelyeithertobegood(g)orbad(b).Eachshotthatwasgoodwasworth1point.WhatisthePMFofX,thenumberofpointsthathescored?Therearefouroutcomesofthisexperiment:gg,gb,bg,andbb.Asimpletreediagramindicatesthateachoutcomehasprobability1/4.ThesamplespaceandprobabilitiesoftheexperimentandthecorrespondingvaluesofXaregiveninthetable:OutcomesP[·]bbbggbgg1/41/41/41/40112xTherandomvariableXhasthreepossiblevaluescorrespondingtothreeevents:{X=O}={bb},{x=1}={gb,bg}'{x=2}={gg}.(3.6)Sinceeachoutcomehasprobability1/4,thesethreeeventshaveprobabilitiesP[X=O]=l/4,P[X=1]=1/2,P[X=2]=1/4.(3.7)[3.2PROBABILITYMASSFUNCTION67Wecanexpresstheprobabilitiesoftheseeventsintermsoftheprobabilitymassfunction(1/4.T,--01/2.T,--11/4;i;=2,otherwise.0)IPx(x))=ItisoftenusefuIorconvenienttodepictplotorasatable.(3.8)Px(;r;)intwootherdisplayformats:asabar0.5Px(x).T,Px(x)0-1012301/4121/21/4xEachPMFdisplayformathasitsuses.Thefunctiondefinition(3.8)isbestwhenPx(x;)isgivenintermsofalgebraicfunctionsofxforvarioussubsetsofSx.Thebarplotisbestforvisualizingtheprobabilitymasses.ThetablecanbeaconvenientcompactrepresentationwhenthePMFisalonglistofsamplevaluesandcorrespondingprobabilities.Nomatter11ovvthePx(x)isforrr1atted,thePMFofXstatest11e·valueofPx(x;)foreveryrealr1urr1berx.Thefirstthreelir1esofEquation(3.8)givethefunctionforthe·valuesofXassociatedwithnor1zeroprobabilities:x;=0,x=1andx;=2.Thefinallir1eisnecessarytospecifytheft1nctionatallothernt1rnbers.Althoughitrnaylooksill}'tosee"Px(x;)=0otherwise"includedir1rr1ostforrr1l1lasforaP1!{F,itisanessentialpartofthePMF.Itis11elpfultokeept11ispartofthedefinitioninmindvvhenvvorkingwiththeP11F.Howe·ver,inthebarplotandtablerepresentatior1softhePNIF,itisur1derstoodthatPx(x;)iszeroexceptforthoseva1t1esxexplicitlyshown.ThePNIFcontainsallofourir1formationabot1ttherandomvariableX.BecausePx(x)istheprobabilityoftheevent{X=x},Px(x;)hasant1rr1berofirr1portantproperties.T11efollowingtheorernappliesthethreeaxiomsofprobabilitytodiscreterandorr1·variables.---Theorem3.l---==::Foradiscreteran,dornvariableX1JJithPMFPx(x)aridrarigeSx:(a)Foran,yx;,Px(x){b)l::i:ESxPx(;i;)>0.=1.(c)Foran,yeven,tBCSx,theprobabil'itythatXisin,thesetBisP[B]=LPx(x).xEBProofAllthreepropertiesareconsequencesoftheaxio1nsofprobability(Section1.3).[68CHAPTER3DISCRETERANDOMVARIABLESFirst,Px(x)>0sincePx(x)=P[X=:::r].Next,v.;eobservethateveryoutcomesESisassociated'vithanumberxESx.Therefore,P[:::rESx]=L:xESxPx(x)=P[sESJ=P[S]=1.Sincetheevents{X=x}and{X=y}aremutuallyexclusivewhen:::i;-=f.y,BcanbewrittenastheunionofmutuallyexclusiveeventsB=UxEB{X=x}.ThuswecanuseAxiom3(ifBiscountablyinfinite)orTheorem1.3(ifBisfinite)towrite(3.9)xEBxEBQuiz3.2TherandornvariableJV11asP1!lFc/n,ri0=1,2,3,otherwise.(3.10)Find(a)Thevalueoftheconstantc(b)P[JV=1)(c)P[N>2)(d)P[N3.3>3)FamiliesofDiscreteRandomVariablesInapplicationsofprobability,rr1anyexperirnentshavesimilarprobabilityrnassfunctions.Ir1afamil}'ofrandornvariables,theP11IFsoftherandorr1variableshavethesarr1ematherr1aticalform,differingonlyinthevaJt1esofoneortv.ropararneters.Thusfarinourdiscussior1ofrandornvariableswehavedescribedhoweachrandorr1variableisrelatedtotheot1tcomesofanexperirr1ent.Wehavealsointroducedtheprobabilit}'rnassf\1r1ctior1,whichcontair1stheprobabilityrr1odeloftheexperirnent.Inpracticalapplications,certainfarr1iliesofrandomvariablesappearoverandoveragaininrr1anyexperirr1ents.Ineacl1farnily,theprobabilityrnassfunctionsofalltherandorr1variablesha:vethesarr1ematherr1aticalforrri.The}'differonl}'int11evalt1esofoneortwoparameters.T11iser1ablest1stostt1d}'inadvar1ceeacl1fa.mil}'ofrandorr1variablesaridlaterappl}'t11eknovvledgeviegaintospecificpracticalapplications.Inthissection,wedefinesixfamiliesofdiscreterandorn·variables.T11ereisoneform11laforthePNIFofalltherandorr1·variablesinafarr1il}'.Dependingonthefarnily,theP1!lFformulacontainsoneortvvopararr1eters.Byassigningn11rr1erica.lvaluestothepararr1eters,vveobtainaspecificrar1dornvariable.Ourr1ornenclatureforafamilyconsistsofthefarr1ilynarr1efolloV\redbyor1eortv.roparametersinparentheses.Forexarr1ple,bin,ornial(ri,p)refersingeneraltothefarr1ilyofbinorr1ialrandornvariables.[3.3FAMILIESOFDISCRETERANDOMVARIABLES69Bin,ornial(7,0.1)referstothebiriorriialrandorrivariablewit.hparametersn,=7aridp=0.1.AppendixAsummarizesirnportantpropertiesof17familiesofrandornvariables.===-Example3.6----===1Considerthefollowingexperiments:•Flipacoinandletitlandonatable.Observewhetherthesidefacingupisheadsortails.LetXbethenumberofheadsobserved.•Selectastudentatrandomandfindouthertelephonenumber.LetX=0ifthelastdigitiseven.Otherwise,letX=1.•ObserveonebittransmittedbyamodemthatisdownloadingafilefromtheInternet.LetXbethevalueofthebit(0or1).Allthreeexperimentsleadtotheprobabilitymassfunction1/2x;1/2x;Px(x)=0=0,=1,(3.11)otherwise.Becauseallthreeexperirrieritsleadtothesarneprobabilit:yrnassfunctiori,t.heycanallbearial}rzedthesarrievvay.ThePl\/IFiriExarnple3.6isamerriberofthefarriilyofBern,o'ullirandornvaria,bles.Definition3.4Bernoulli(rJ)RandomVariableXisaBernoulli(IJ)ran,dornvariableifthePMFofXhastheforrnPx(x)=p.X=l'0other'tlJ'ise,1JJherethepararneter'[Jisin,theran,ge0<'[J<1.l\/Iariypracticalapplicationsofprobabilityproduc,eseqt1entialexperirrientswithindependenttrialsiriv.rhicheachst1bexpe1irnenthastv.ropossibleoutcomes.ABernot1lliPMFrepreseritstheprobabilityrnodelforeachst1bexperirnent.\¥erefertosubexperirrientswitlitwopossibleoutcorriesasBerrio'ullitrials.Irithefollovvingexarriples,vierefertotestsofintegratedcircuitsv.rithtvvopossibleoutcorries:accept(a,)aridreject(r).Eachtestinasequenceoftestsisaniridependenttrialvvithprobabilit}'pofareject.Dependingoritheobservation,seqt1entialexperimentswithBerrioullitrialsliaveprobabilit}'rriodelsrepreseritedbyBern,o'tJ,lli,biriornial,geornetric,andPascalrandomvariables.Otlierexperirrientsproducediscreteun,iforrnrandorrivariablesandPoisson,randorn·variables.Thesesixfarriiliesofrandorrivariablesoccurofteninpracticalapplications.c:::==Example3.7TestonecircuitandobserveX,thenumberofrejects.WhatisPx(x)thePMFofrandomvariableX?[70CHAPTER3DISCRETERANDOMVARIABLESBecausethereareonlytwooutcomesinthesamplespace,andX=0withprobability1-p,Px(x)=X=1withprobabilitypo.1-px=p.x=l0otherwise.I(3.12))Therefore,thenumberofcircuitsrejectedinonetestisaBernoulli(r>)randomvariable.Example3.8Ifthereisa0.2probabilityofareject,thePMFoftheBernoulli(0.2)randomvariableIS1Px(x)0.50-1i::::::==Px(x;)=I012x0.80.2x=0,x=1,0otherwise.(3.13)Example3.9Inasequenceofindependenttestsofintegratedcircuits,eachcircuitisrejectedwithprobabilityr>.LetYequalthenumberoftestsuptoandincludingthefirsttestthatresultsinareject.WhatisthePMFofY?Theprocedureistokeeptestingcircuitsuntilarejectappears.Usingatodenoteanacceptedcircuitandrtodenoteareject,thetreeisr•Y=lr•Y=2pa......_____1-pFromthetree,weseethatP[Y=1)=r>.P[Y=2)=p(land,ingeneral,P[Y=y)=r>(l-p)Y-1.Therefore,Py(y)=a...-p),P['Y=3)=r>(l-r>)2,p(l-r>)y-1y=1)2,...0otherwise.(3.14)YisreferredtoasageometricrandomvariablebecausetheprobabilitiesinthePMFconstituteageometricseries.Ingeneral,thenumberofBerr1oullitrialsthattakeplaceuntilthefirstobservationofoneoft11etwooutcorr1esisageornetricrar1domvariable.[3.3FAMILIESOFDISCRETERANDOMVARIABLES71Geometric(p)RandomVariableXisageometric(p)ran,dornvariableifthePM.FofXhastheforrnDefinition3.5Px(;x;)=r>(1-pr1;-1x;=1,2,...0other111ise.111herethepararneterI>'isiritheran,ge0<I><1.Example3.10Ifthereisa0.2probabilityofareject,thePMFofthegeometric(0.2)randomvariableIS0.2Py(y)0.1Py(y)=(0.2)(0.8)y-ly=1,2,...0otherwise.1111••....00IO20yExample3.11--1nasequenceofn,independenttestsofintegratedcircuits,eachcircuitisrejectedwithprobabilityJ>.LetKequalthenumberofrejectsinthen,tests.FindthePMFP1o::::::==AdoptingthevocabularyofSection2.3,wecalleachdiscoveryofadefectivecircuitasuccess,andeachtestisanindependenttrialwithsuccessprobabilityJ>.TheeventK=kcorrespondstoksuccessesinritrials.WerefertoTheorem2.8todeterminethatthePMFofJ(is(3.15)Kisanexampleofabinomialrandomvariable.vVedonotstatethevaluesofkforvvhichPK(k)=0inEquation(3.15)becat1se(~)=0forkti{0,1,...,r1,}.Definition3.6Binomial(ri,,p)RandomVariableXisabinomial(ri,I>)ran,dornvariableifthePMFofXhastheform,Px(x)111here0=('"x.'i)px(l-r>)n-:i:<p<1an,dn,isan,in,tegerS'IJ,Chthatn,>1.vVhenevervvehaveaseql1enceofn,independentBernot1llitrialseachv.rithsuccessprobabilit}'p,thenurnberofSl1ccessesisabinornialrandorr1variable.Notet.hataBernoullirandomvariableisabinorr1ialrandornvariablevvith'n=1.[72CHAPTER3DISCRETERANDOMVARIABLESExample3.12Ifthereisa0.2probabilityofarejectandweperform10tests,thePMFofthebinomial(10,0.2)randomvariableis0.4PK(k)0.2I0()(3.16)I105kExample3.13Performindependenttestsofintegratedcircuitsinwhicheachcircuitisrejectedwithprobabilityp.ObserveL,thenumberoftestsperformeduntiltherearekrejects.WhatisthePMFofL?Forlargevaluesofk,itisnotpracticaltodrawthetree.Inthiscase,L=lifandonlyiftherearek-1successesinthefirstl-1trialsandthereisasuccessontriallsothatP[L=l]=Pk-1rejectsinl-1attempts,rejectonattemptl(3.17)BATheeventsAandBareindependentsincetheoutcomeofattemptlisnotaffectedbythepreviousl-1attempts.NotethatP[A]isthebinomialprobabilityofk-1successes(i.e.,rejects)inl-1trialssothatP[A]=(zk--1)Pk-1(1-r>)~-1-(k-1)l(3.18)(~~~)rl(l-r>)l-k(3.19)Finally,sinceP[B]=p,PL(l)=p[A]p[B]=LisanexampleofaPascalrandomvariable.Definition3.7Pascal(k,p)RandomVariableXisaPascal(k,p)ran,dornvariableifthePMFofXhastheforrn1JJhere0<p<1an,dkisan,in,tegersv,chthatk>1.Ingeneral,ther1l1rr1berofBe1noullitrialsthattakeplaceuntiloneofthet\vooutcornesisobservedktimesisaPascalrandorr1variable.ForaPascal(k,p)[3.373FAMILIESOFDISCRETERANDOMVARIABLESrandorr1variableX,Px(x)isnonzeroonlyforx=k,k+1,....Definition3.7doesnotstatethevalt1esofkforwhichPx(x;)=0becauseinDefir1ition3.6weha;ve(~)=0forxtf.{O,1,...,ri}.Alsonotethatt11ePascal(l,JJ)rar1dom·variableisthegeometric(p)randorr1variable.Example3.14Ifthereisa0.2probabilityofarejectandweseekfourdefectivecircuits,therandomvariableListhenumberoftestsnecessarytofindthefourcircuits.ThePMFofthePascal(4,0.2)randomvariableis0.1~-------0.050......-.....................................................................02040lExample3.15Inanexperimentwithequiprobableoutcomes,therandomvariableNhastherangeSN={k,k+1,k+2,···,l},wherekandlareintegerswithk<l.Therangecontainsl-k+1numbers,eachwithprobability1/(Z-k+1).Therefore,thePMFofJVisc::::==l/(l-k+l)0T/,=k,k+l,k+2,...,lotherwise(3.20)JVisanexampleofadiscreteuniformrandomvariable.DiscreteUniform(k,l)RandomVariableXisadiscreteuniform(k,l)ran,dornvariableifthePMFofXhastheform,Definition3.8Px(x)=l/(l-k+l)0x=k,k+l,k+2,...,lother'tlJ'ise1JJherethepararneterskan,dlarein,tegerss'uchthatk<l.Todescribethisdiscrete11r1iforrnrandorr1variable,vveusetheexpressior1"Xisuniformlydistribt1tedbetvveer1kandl."Example3.16Rollafairdie.TherandomvariableNisthenumberofspotsonthesidefacingup.Therefore,JVisadiscreteuniform(1,6)randomvariablewithPMF[74CHAPTER3DISCRETERANDOMVARIABLES0.2,...-------.....1/600.__..___..___.___.___,___..___,50n,=1,2,3,4,5,6,otherwise.(3.21)T/,Theprobabilityrr1odelofaPoissonrandornvariabledescribespl1enornenathatoccurrandornlyintime.Whilet11etimeofeachoccurrenceiscompletelyrandorn,thereisakr1ovvnaveragent1rr1berofoccurrencesperunittime.ThePoissonrnodelisl1sed\videlyir1rr1anyfields.Forexample,thearrivalofinforrnatior1requestsata\¥orld'\¥ide\¥ebser\rer,theinitiationoftelephonecalls,andtheerr1issionofparticlesfrorr1aradioactivesourceareoftenrr1odeledasPoissonrandornvariables.vVewillrett1rntoPoissonrandorr1variablesrnanytimesinthistext.Atthispoint,"''ecor1sideronl:ythebasicproperties.===-Definition3.91----==:.....iPoisson(a)RandomVariableXisaPoisson(a)raridornvariableifthePMFofXhastheform,Px(:i;)=axe-a./x!x=o,1,2,...,0other'tuise,1JJherethepararnetera'isiritherarigea>0.TodescribeaPoissonrar1dom\rariable,\Ve\villcalltheoccurrenceoft11epl1enornenonofinterestar1arrival.APoissonrr1odeloftenspecifiesanaveragerate,,.\arrivalspersecond,andatirneir1terval,Tseconds.Inthistirr1einterval,then11rnberofarri\ralsXhasaPoissonPl\/IFV1ritha=,.\T.i::::::==ExampIe3.lri___,;;:::::::::11-ThenumberofhitsatawebsiteinanytimeintervalisaPoissonrandomvariable.Aparticularsitehasonaverage,.\=2hitspersecond.Whatistheprobabilitythattherearenohitsinanintervalof0.25seconds?Whatistheprobabilitythattherearenomorethantwohitsinanintervalofonesecond?Inanintervalof0.25seconds,thenumberofhitsHisaPoissonrandomvariablewitha=,.\T=(2hits/s)x(0.25s)=0.5hits.ThePMFofn·isI--------o----·--0240.5he-0·5/h!0fi=0,1,2,...otherwise.hTheprobabilityofnohitsisP[H=O]=PH(0)=(0.5)0e-0·5/O!=0.607.(3.22)[3.3FAMILIESOFDISCRETERANDOMVARIABLES75Inanintervalof1second,a=>..T=(2hits/s)x(1s)=2hits.Letting.]denotethenumberofhitsinonesecond,thePMFof.]isP.J(j)0.20.10I024682.ie-2/j!j=0,1,2,...0otherwise.JTofindtheprobabilityofnomorethantwohits,wenotethat{J<2}={.J=O}u{J=1}u{J.=2}(3.23)istheunionofthreemutuallyexclusiveevents.Therefore,+P[.J=1]+P[.J=2]PJ(0)+PJ(1)+PJ(2)e-2+21e-2/1!+22e-2/2!=0.677.P[J.<2]=P[.J=O]==i::::::==(3.24)Example3.18Thenumberofdatabasequeriesprocessedbyacomputerinany10-secondintervalisaPoissonrandomvariable,K,withcv.=5queries.Whatistheprobabilitythattherewillbenoqueriesprocessedina10-secondinterval?Whatistheprobabilitythatatleasttwoquerieswillbeprocessedina2-secondinterval?ThePMFofJ(is0.20III055ke-5/k!k=0,1,2,...0otherwise..1015kTherefore,P[I<=O]=P1..TwithT=10seconds.Therefore,>..=0.5queriespersecond.IfNisthenumberofqueriesprocessedina2-secondinterval,a=2>..=1andNisthePoisson(1)randomvariablewithPMFe-1/ri!ri=0,1,2,...0otherwise.(3.25)Therefore,P[N>2]=1-P1v(O)-PN(l)=1-e-1-e-1=0.264.(3.26)Notethattheunitsof>..aridThavetobeconsistent.Ir1steadof>..=0.5ql1eriespersecondforT=10seconds,v.recouldt1se>..=30ql1eriesperrr1int1teforthetirne[76CHAPTER3DISCRETERANDOMVARIABLESintervalT=1/6rninl1testoobtainthesarr1eo~=5qlleries,andthereforethesarneprobabilit}'rr1odel.Inthefollovvingexarr1ples,vveseethatforafixedrate,\,theshapeofthePoissonPMFdependsonthedl1rc.itionToverw11icharrivalsarecounted.Example3.19Callsarriveatrandomtimesatatelephoneswitchingofficewithanaverageof,\=0.25calls/second.ThePMFofthenumberofcallsthatarriveinaT=2-secondintervalisthePoisson(0.5)randomvariablewithPMF1------P.J(j)(o.5).ie-0·5Ij!j0.50o----·--0=o,1,...,otherwise.42JNotethatweobtainthesamePMFifwedefinethearrivalrateas,\=60·0.25=15callsperminuteandderivethePMFofthenumberofcallsthatarrivein2/60=1/30minutes.Example3.20Callsarriveatrandomtimesatatelephoneswitchingofficewithanaverageof,\=0.25callspersecond.ThePMFofthenumberofcallsthatarriveinanyT=20-secondintervalisthePoisson(5)randomvariablewithPMF0.2P.J(j)0.10p.J(j)II.I05=5.ie-5/j!0j=0,1,...,otherwise.•1015JQuiz3.3Eachtirnearnoderntrar1srnitsonebit,thereceivir1grr1oderr1anal}rzesthesignalthatarrivesanddecides\vhet11erthetransmittedbitis0or1.Itrr1akesanerrorvvithprobabilityp,independentofwhet11eran}'ot11erbitisreceivedcorrectly.(a)Ifthetransmissioncontinuesuntilt11ereceivingrr1odemrnakesitsfirsterror,whatisthePJVIFofX,thenl1rr1berofbitstransrr1itted?(b)IfIJ=0.1,whatistheprobabilityt11atX=10?\Vhatistheprobabilitythatx>10?(c)Ifthernoderr1transmits100bits,whatisthePMFofY,thenl1mberoferrors?(d)If[J=0.01andthernoderr1trar1srnits100bits,whatistheprobabilityofY=2errorsattherecei\rer?\Vhatistheprobabilitythat·y<2?(e)Ifthetransmissioncontir1uesuntiltherecei\ringmodemrr1akesthreeerrors,'\vhatisthePJVIFofZ,t11enurnberofbitstrar1srnitted?[3.4CUMULATIVEDISTRIBUTIONFUNCTION(CDF)77(f)If'[J=0.25,vvhatist11eprobabilit}'ofZ=12bitstransmittedt1ntilthemoderr1rnakesthreeerrors?3.4CumulativeDistributionFunction(CDF)LiketheP}v1F,theCDFofrandomvariableXexpressestheprobabilityrr1odelofanexperimentasarnatherr1aticalfunction.ThefunctionistheprobabilityP[X<1';]foreverynurr1berx.ThePNIFandCDFarecloselyrelated.Eac11canbeobtainedeasil}'frorr1theot11er.===--Definition3.10CumulativeDistributionFunction(CDF)Thecumulativedistributionfunction(GDF)ofran,dornvariableXisFx(x;)=P[X<1';].Foranyrealr1urnberx,t11eCDFistheprobabilitythatt11erandorr1variableXisnolargerthanx.Allrandorr1·variableshavec11mulativedistributionfur1ctions,b11tonl}'discreterandorn·variablesha;veprobabilit}'rr1assfunctior1s.ThenotationconventionfortheCDFfollovvsthatofthePNIF,exceptthatvveusetheletterFvvithasubscriptcorrespondingtothenarr1eoftherandorr1variable.BecauseFx(x)describest11eprobabilityofanevent,theCDFhasanurnberofproperties.3.2ForariydiscreteraridornvariableXv1ithran,geBx=~---TheoremX2{x;1,x2,...}satisfyin,g1';1<<...,(a)Fx(-oo)=0an,dFx(oo)=1.{b)Forall1';'>x,Fx(x')>Fx(x;).(c)ForxiEBxaridE,ariarbitrarilysrnallpositive'nurnber,{d)Fx(x)=Fx(x;i)forallxs'uchthatx;i<x<x;,i+I·EachpropertyofTheorerr13.2hasar1equivalentstaterr1er1tir1v.rords:(a)Goingfromlefttorightonthex-axis,Fx(x)startsatzeroarider1dsator1e.(b)TheCDFneverdecreasesasitgoesfromlefttoright.(c)Foradiscreterandorr1variableX,thereisajurnp(discor1tinuity)ateachvalueofxiEBx.T11eheightofthejt1rnpatx,i:isPx(xi)·[78CHAPTER3DISCRETERANDOMVARIABLES(d)Bet\veenjumps,thegraphoftheCDFofthediscreterandorr1variableXisahorizontalline.Anotherirnportantcor1seql1er1ceofthedefinitionoft11eCDFisthatthedifferencebetvveentheCDFevah1atedattwopointsist11eprobabilitythattherandornvariabletakesonavall1ebet\veenthesetvvopoir1ts:Theorem3.3Forallb>a,7Fx(b)-Fx(a)=P[a<X<b].<X<b}asapartofaunionofinutuallyexclusiveevents.StartwiththeeventEb={X<b}.NotethatEbcanbeProofToprovethistheorem,expresstheeventEc,ib={a\VrittenastheunionEb={X<b}={X<a}U{a<X<b}=Ec,iUEab(3.27)NotealsothatEaandEabaremutuallyexclusivesothatI>[Eb]=I>[Ea]P[Eb]=Fx(b)andP[Ea]=Fx(a),\Vecan\VriteFx(b)=Fx(a)+P[a<X+P[Eab]·Since<b].Therefore,P[a<X<b]=Fx(b)-Fx(a).InvvorkingwiththeCDF,itisnecessar}'topaycarefulattentiontothenatureofineqt1alities,strict(Example3.21InExample3.5,randomvariableXhasPMF0.51/4x=O1/2x=1,1/4x=2,Px(x))Px(x;)0.....___.__.......__...._.....-11230=x0(3.28)otherwise.FindandsketchtheCDFofrandomvariableX.ReferringtothePMFPx(x;),wederivetheCDFofrandomvariableX:10IFx(x;)0.50-1Fx(x;)I0123x=P[X<x;]=x<0,0<x<1.1/43/412.1-I[3.4CUMULATIVEDISTRIBUTIONFUNCTION(CDF)79Keepinmindthatatthediscontinuitiesx;=0,x=1andx=2,thevaluesofFx(x)aretheuppervalues:Fx(O)=1/4,Fx(l)=3/4andFx(2)=l.MathtextscallthistherighthandlimitofFx(x).Consideranyfiniterar1dorr1·variableXvvithallelernentsofSxbetvveenx;rriinandx;rnax·Forthisrandomvariable,thenurr1erica.lspecificationoftheCDFbeginsvvithFx(x)=0,X<Xrnin,andendswithFx(x)=1,Liketl1estatement"Px(x;)=0otherwise,"thedescriptionoftheCDFisincorr1pletevvithoutt11esetwostatements.Thenextexampledispla}'StheCDFofaninfinitediscreterandomvariable.---Example3.22~--lnExample3.9,lettheprobabilitythatacircuitisrejectedequalp=1/4.ThePMFofY,thenumberoftestsuptoandincludingthefirstreject,isthegeometric(1/4)randomvariablewithPMFPy(y)==(1/4)(3/4)Y-ly1,2,...0otherwise.(3.29)WhatistheCDFof·y7Randomvariable·yhasnonzeroprobabilitiesforallpositiveintegers.Foranyintegern,>1,theCDFisFy(n)=~Py(j)=.~~(~);-i(3.30)Equation(3.30)isageometricseries.Familiaritywiththegeometricseriesisessentialforcalculatingprobabilitiesinvolvinggeometricrandomvariables.AppendixBsummarizesthemostimportantfacts.Inparticular,MathFactB.4implies(1x)~~·1x;.i-l=1-xn.Substitutingx;=3/4,weobtainFy(ri)=1-(~)''.(3.31)ThecompleteexpressionfortheCDFof·ymustshowFy(y)forallintegerandnonintegervaluesofy.Foraninteger-valuedrandomvariableY,wecandothisinasimplewayusingthefloorfunctionlYJ,whichisthelargestintegerlessthanorequaltoy.Inparticular,ifn,<y<n,-1forsomeintegerri,thenlYJ=n,andFy(y)=P['Y<y]=P(Y<n,]=Fy(n,)=Fy(lYJ).(3.32)[80CHAPTER3DISCRETERANDOMVARIABLESIntermsofthefloorfunction,wecanexpresstheCDFofYas1------Fy(y)0.5Fy(y)=o........._____.....051001-(3I4)LYJyy<1,>1.(3.33)yTofindtheprobabilitythatYtakesavalueintheset{4,5,6,7,8},werefertoTheorem3.3andcomputeP[3<·y<8]Fy(8)-Fy(3)=(3/4)3=-(3/4)8=0.322.(3.34)Quiz3.4UsetheCDFFy(y)tofindt:hefollowir1gprobabilities:1Fy(y)I0.80.60.40.2I003.512345(a)P[Y<1](b)P[Y<1](c)P[Y>2](d)P[Y>2](e)P[Y=1](f)P['Y=3]yAveragesandExpectedValueAnaverageisanurnberthatdescribesasetofexperirnentalobservations.Theexpectedvalueisanumberthatdescribestheprobabilitymodelofanexperiment.Theaveragevalt1eofasetofrint1rr1bersisastatisticofthethesetofr1urr1bers.Theaverageisasingler111mbert11atdescribestheentireset.Statisticiansv.rorkvvithseveralkindsofaverages.Theonest11ataret1sedt11ernostaret11ernean,,thernedian,,andthernode.T11err1ean·valueofn,nurnbersisthesurr1ofthen,r1urnbersdividedb}'ri.Anexarr1pleistherr1ean·vah1eofther1urnericalgradesoft11estudentstakingarr1idtermexarri.Therneanir1dicatestheperforrr1anceoftheentireclass.T11emedianisanotherstatisticthatdescribesasetofnurnbers.T11emedianisanurnberir1t11emiddleofadataset.Thereisar1equalnurnberofdataiternsbelovvtherr1ediar1aridabovetherr1edian.Athirdaverageistherr1odeofasetofr1urnbers.T11emodeistherr1ostcomrnonnt1rr1berintheset.Thereareasmanyorrr1orenurnbers"\viththatvaluethar1an}'otherval11e.Iftherearetwoorrnorenumbersvviththisproperty,thesetofnurr1bersiscalledrn11,ltirnodal.[3.5AVERAGESANDEXPECTEDVALUE81===-ExampIe3.23:---==::::::1Foronequiz,10studentshavethefollowinggrades(onascaleof0to10):(3.35)Findthemean,themedian,andthemode.Thesumofthetengradesis68.Themeanvalueis68/10=6.8.Themedianis7,becausetherearefourgradesbelow7andfourgradesabove7.Themodeis5,becausethreestudentshaveagradeof5,morethanthenumberofstudentswhoreceivedanyothergrade.Exarr1ple3.23aridtheprecedingcomrnentsonaveragesapplytoasetofr1urnbersobservedir1apracticalsituation.Theprobabilityrnodelsofrandorn·variablescharacterizeexperirner1ts-vvithr1urr1erica.1outcornes,andinpracticalapplicationsofprobability,weassumethattheprobabilit}'rnodelsarerelatedtot11enl1mbersobservedinpractice.Justasastatisticdescribesasetofr1urnbersobservedinpractice,apararneterdesc1ibesaprobabilit}'rnodel.Eachpararr1eterisar1urr1berthatcanbecornputedfromt11ePl\/IForCDFofarandorr1·variable.·\¥hen"'ir..reuseaprobabilit}'rnodelofarandornvariabletorepresentanapplicationt11atresl1ltsinasetofnumbers,theexpectedvalv,eoftherandornvariablecorrespondstotherr1eanvalueofthesetofnl1rnbers.Expected·valuesappearthro11ghouttheremainderofthistextbook.Tv.ronotationsfortheexpectedvalueofrar1domvariableXareE[X)and11,x.Correspor1dingtotheot11ertwoaverages,v.rehavethefollo-vvir1gdefinitions:Definition3.11ModeAmodeofraridornvariableXisarl/Urnberalli;.~DrnodsatisfyirigPx(Xrnod)>Px(i;)forDefinition3.12MedianAmedian)XrnodJofra:ndornvariableXisarl/LJ,rnberthatsatisfiesP[X<Xrned)P[X>>1/2,Xrned)>1/2.Neitherthernodenorthernedian,ofarandorr1variableXisnecessarilyunique.Therearerandom·variablesthat11a·vese·veralrnodesormedians.---Definition3.13:---ExpectedValueTheexpectedvalueofXisE[X)=µ,x=L:r;ESxxPx(i;).[82CHAPTER3DISCRETERANDOMVARIABLESEx1Jectation,isaS}rnor1}rrnforexpected·vall1e.Sometirnesthetermrnean,valv,eisalsousedasasynon}rmforexpectedvalue.\¥eprefertotlserneanvaluetorefertoastat'isticofasetofexperirnentaldata(thesurr1dividedbythenumberofdataiterns)todistinguishitfrornexpected·value,"''hichisapararneterofaprobabilit}'rnodel.Ifyourecallyourstudiesofrr1echanics,theforrnofDefinition3.13maylookfamiliar.Thinkofpointmassesor1aline"''itharr1assofPx(x)kilogramsatadistanceofxmetersfromtheorigin.Int11isrnodel,µ,xinDefinition3.13isthecenterofrnass.Thisis"''hYPx(x)iscalledprobabilityrnassfunction.---Example3.24:---=""'RandomvariableXinExample3.5hasPMF0.5x=O1/2x=1,1/4x=2,otherwise.01/4Px(x)Px(x;)=0...___.__....___.._....-10123x)(3.36)WhatisE[X]?+1·Px(1)+2·Px(2)0(1/4)+1(1/2)+2(1/4)=1.E[X]=11,x=0·Px(0)=(3.37)Tot1nderstandhovvthisdefir1itionofexpected·valuecorrespondstothenotionofaddingupasetofrneas.l1rernents,Sllpposeviehaveanexperirr1er1tthatproducesarar1dom·variableXaridweperforrr1'nindeper1denttrialsofthisexperiment.vVedenotet11evaluethatXtakesonthe'it11trialb}'x(i).vVesaythatx;(l),...,x('n)isasetofn,samplevall1esofX.·vvehave,aftern,trialsoftheexperiment,thesarnpleaverage1rn,n=-n~x;(i).ri~(3.38)i=lEachx(i)takesvaluesinthesetSx.Outoft11eritrials,ass11rr1ethateac11xESxoccursN:i:tirr1es.Thenthesurr1(3.38)becorr1es(3.39)RecallourdiscussioninSection1.3oftherelativefrequencyinterpretationofprobability.Therevvepoiintedoutthatifinn,observationsofanexperirr1er1t,theeventAoccl1rsNAtimes,\rvecaninterprettheprobabilityofAasP[A]=lirr1n-700]\TAn,.(3.40)[3.5AVERAGESANDEXPECTEDVALUE83JVA/riistherelativefrequenC}'ofA.Inthenotationofrandornvariables,v.reha;vethecorrespondingobservationt11atPx(x)=N~1;.11m-n-+oori,(3.41).FrornEquation(3.39),thisSl1ggeststhatlirr1rn,n=n-+ooLx(lirr1xESx!!2:_)n-+oo'T/,=L:i;Px(:.r;)=E[X].(3.42)xESxEquation(3.42)saysthatthedefinitior1ofE[X]correspor1dstoarr1odelofdoingthesarr1eexperirnentrepeatedly.Aftereachtrial,addupalltheobservationstodateanddivideb}'thenumberoftrials.·\'¥eproveir1Chapter10thattheresultapproachestheexpectedvalueasthenurnberoftrialsincreaseswithoutlirnit.'\¥ecanuseDefinition3.13toderivetheexpected·valueofeachfarnilyofrandornvariablesdefinedir1Section3.3.Theorem3.4.---===TheBern,oulli(rJ)ran,dornvariableXhasexpected1;alv,eProofE[X)=0·Px(O)+lPx(l)=0(1-p)+l(p)E[X]=p.=p.Theorem3.5Thegeornetric(p)ra/ndorn,1;ar~iableXhasexpectedvalueE[X]=l/'[J.ProofLetq=1-p.ThePlv1FofXbecomesPx(x)pqx-1={0x=l,2,...other\vise.(3.43)TheexpectedvalueE[X)istheinfinitesum0000(3.44)x=lx=lApplyingtheidentityofMathFactB.7,wehave00xE[x]=P'"'"'Lxqx=l00l=xPqP1qP'"'"'L:i;q=q1_q2=p-2=p.(3.45)x=lThisrestlltisintuitiveif}'Ollrecalltheir1tegratedcirctlittestingexperirnentsandconsidersornenurnericalvalues.Iftheprobabilityofrejectinganintegratedcircuitis'[J=1/5,thenor1average,}'OUhavetoperforrnE[.Y]=l/p=5teststlntil[84CHAPTER3DISCRETERANDOMVARIABLESyouobservet11efirstreject.Ifr>=1/10,theaverager1urnberoftestsllntilthefirstrejectisE[Y]=l/p=10.Theorem3.6ThePoisson,(a)ran,dom,variablein,Defin,ition,3.9hasexpectedval'LJ,eE[X]=o~.Proof(3.46)V/eobservethatx/1;!=l/(1;-1)!andalsothatthex=0terminthesumiszero.Inaddition,'vesubstituteax=ex·nx-ltofactorexfromthesumtoobtain(3.47)Next'vesubstitutel=x-1,'viththeresultl00E[X]=QLfie-a=Q.(3.48)l=O--......,,...--_./1\'\fecanconcludethatthesuminthisformulaequals1eitherbyreferringtotheidentityea=I:~0al/l!orbyapplyingTheorem3.1(b)tothefactthatthesumisthesumofthePlVIFofaPoissonrandomvariableLoverallvaluesinSLandP[SL]=1.Ir1Section3.3,-vvernodeledthenurnberofrandornarrivalsinanintervalofdl1rationTbyaPoissor1randorn·variable-vvithpararnetera=A.T.WereferredtoA.astheaveragerateofarrivals-vvithlittlejustification.T11eorern3.6providesthejustificationb:yshov.rir1gthatA.=cY./Tistheexpectedr1urr1berofarri·va1sperunittirne.Thenexttheorernprovides,-vvithoutderivatior1s,theexpected·valuesofbinomial,Pascal,ariddiscreteur1iforrr1randorr1variables.===Theorem3.7(a}Forthebin,ornial(n,,p)ran,dornvariableXofDefin,it'ion,3.6;E[X]{b}ForthePascal(k,p)ran,dornvar~iable=n,p.XofDefin/it'io'n3.1;E[X]=k/r>.(c)Forthediscrete'un,iforrn(k,l)ra/ndornE[X]1;ar~iable=(k+l)/2.XofDefin,ition,3.8,[3.5AVERAGESANDEXPECTEDVALUE85Inthefollovvir1gtheorem,weshovvthatthePoissonP1!fFisalimitingcaseofabinorr1ialPNIFvvhenthent1mberofBernoullitrials,'n,growsvvithoutlirnitbuttheexpectedn11mberofst1ccessesTl/fJremainsconstantatcv.,theexpectedvalueofthePoissonPMF.Ir1thetheorern,vveleta=>.Tanddividet11eT-secondintervalintoritirneslotseac11v.rithdurationT/ri.Ineachslot,asr1,grov.rswithoutlirr1itandtheduration,T/'n,ofeachslotgetssmallerandsrnaller\Veassurnet11atthereiseitheronearrival,wit11probabilityp=>..T/r1,=a/11,,orthereisnoarrivalinthetimeslot,wit11probability1-'[J.Theorem3.8Perforrn11,Bernovilitrials.Jn,eachtrial)lettheprobabilityofs'u,ccessbea/n,;1JJherea>0isacon,stan,tar1,dn,>cv..Lettheran,dornvariableIPKn(k)=n,)(a/n)k(1-a)n-kkn.((3.49)Fork=0,...,n,,\Vecanwritea)n-k.(1--'r/,(3.50)Noticethatinthefirstfraction,therearektermsinthenumerator.Thedenominatorisnk,alsoaproductofkterms,allequalton,.Therefore,wecanexpressthisfractionastheproductofkfractions,eachoftheform(n-j)/n,.Asn--too,eachofthesefractionsapproaches1.Hence,.n,(ri-l)···(n,-k+1)lnnk=1.n'n-+oo(3.51)Furthermore,wehave(3.52)Asn,gro'ivs'ivithoutbound,thedenominatorapproaches1and,inthenumerator,\Verecognizetheidentitylimn-+oo(l-a/n,)71=e-a.Puttingthesethreelimitstogetherleadsustotheresultthatforanyintegerk>0,pKn.(k)'-_.1llTIn-+oo'ivhichistheJ:>oissonPMF.{ake-0'./k',.k=0,1,...0otherwise,(3.53)[86CHAPTER3DISCRETERANDOMVARIABLES==--Quiz3.5,__...;;=:::iInapay-as-}rougocellpl1oneplan,thecostofsendinganSl\IIStextrr1essageis10cer1tsandthecostofreceivingatextis5cents.ForacertainSl1bscriber,theprobabilityofser1dingatextis1/3andt11eprobabilityofrecei·vir1gatextis2/3.LetCeqllalt11ecost(ir1cents)ofonetextrr1essageandfind(a)TheP11!FPc(c)(b)T11eexpected·vall1eE[C](c)Theprobabilitythatthesubscriberrecei·vesfourtextsbeforesendingatext.3.6(d)Theexpectednumberoftextsreceivedbythesubscriberbeforet11esubscriberser1dsatext.FunctionsofaRandomVariableAfunction·y=g(X)ofrandorr1variableXisar1otherrar1domvariable.ThePMFPy(y)canbederivedfromPx(x)andg(X).Ir1rnanypracticalsituations,weobservesarnple·valuesofarandorn·variableandllSet11esesarnple·valllestocornputeotherquar1tities.Oneexamplethatoccursfrequer1tlyisanexperirr1er1tinvvhichtheprocedureistornor1itort11edataactivityofacellulartelephor1esubscriberforamonthandobserve;i;thetotalr1urr1berofrnegabytessentandreceived.T11etelepl1onecornpar1yreferstothepriceplanoftheSllbscriberaridcalculatesydollars,theamol1nttobepaidbythesubscriber.Ifxisasarnplevallleof:1randorn·variableX,Defir1itior13.1impliesthatyisasarnplevall1eofarandorr1variableY.BecausevveobtainYfrornanotherrar1domvariable.werefertoYasaderivedran,dornvariable.I.Definition3.14DerivedRandomVariableEachsarnplevalv,eyofaderivedrandomvariable·yisarnathernaticalf'u'nction,g(x)ofasarnpleval'Ue;i;ofariotherran,dornvariableX.Weadoptthen,otationY=g(X)todescribetherelat'iorishipofthet'tJJoran,dornvariables.Example3.25Aparcelshippingcompanyoffersachargingplan:$1.00forthefirstpound,$0.90forthesecondpound,etc.,downto$0.60forthefifthpound,withroundingupforafractionofapound.Forallpackagesbetween6and10pounds,theshipperwillcharge$5.00perpackage.(Itwillnotacceptshipmentsover10pounds.)FindafunctionY=g(X)forthechargeincentsforsendingonepackage.WhenthepackageweightisanintegerXE{1,2,...,10}thatspecifiesthenumberofpoundswithroundingupforafractionofapound,thefunction·y=g(X)=105X-5X2500xx=1,2,3,4,5=6,7,8,9,10.(3.54)[3.6FUNCTIONSOFARANDOMVARIABLE87correspondstothechargingplan.Ir1thissectionwedeterrn1ir1etheprobabilityrnodelofaderivedrandorr1·variablefrornt11eprobabilityrr1odeloftheoriginalrandorr1variable.·\'¥estartvvithPx(x)andaft1nctionY=g(X).WellSethisinforrnatior1toobtainPy(y).Before"''epresenttheprocedureforobtair1ingPy(y))v.realertstudentstothedifferentnatureoftheft1nctionsPx(:i;)aridg(:i;).Althot1gl1the}'arebothft1nctionsV1ritht11eargurr1entx)theyareentirelydifferent.Px(x)describestheprobabilityrnodelofarandornvariable.Ithasthespecialstructt1represcribedinTheorem3.1.Ontheotherhand)g(x)car1beanyfunctionatall.Whenwecornbir1ePx(:i;)andg(x)toderivetheprobabilityrnodelforY)wearriveataPMFthatalsoconforrnstoTheorem3.1.Todescribe·yinterrr1sofourbasicrnodelofprobability,vvespecifyanexperirner1tconsistingofthefollowingprocedureandobservation:SamplevalueofY=g(X)Performanexperimentandobserveanoutcomes.Froms,findx,thecorrespondingvalueofrandomvariableX.Observeybycalculatingy=g(x).Thisproced11rernapseachexperirner1ta.1outcornetoar1t1mber)y,asample·val11eofarar1dom·variable)Y.ToderivePy(y)frornPx(:i;)andg(-),"''econsiderallofthepossibleva.lt1esofx.Foreach;i;EBx)wecomputey=g(:i;).Ifg(x)trar1sforrnsdifferentvalt1esofxintodifferent·valuesofy(g(x1)=Jg(x2)ifx1=/=;i;2)wesirr1pl}'ha:ve(3.55)Thesit11ationisalittlernorecornplicatedvvheng(:r;)transforrr1sseveral·valuesofxtothesarney.ForeachyESy)"'irveaddtheprobabilitiesofalloft11evalues;r;EBxforv.rhichg(:.c)=y.Theorern3.9appliesir1general.ItreducestoEquation(3.55)"''heng(;i;)isaone-to-onetransformation.Theorem3.9Foradiscreteran,dornvariableX)thePMFofY=g(X)'tsPy(y)=LPx(x).'.J;:g(:i;)=yIfvve·vieVI'X=xa.sthe011tcomeofanexperirr1er1t)thenTheorerr13.9sa}'St11atPy(y)ist11esurr1oftheprobabilitiesofalltheoutcornesX=;i;forV1rhichY=y.===-ExampIe3.26---==:::::aInExample3.25,supposeallpackagesweigh1,2,3,or4poundswithequalprobability.FindthePMFandexpectedvalueofY,theshippingchargeforapackage.[88CHAPTER3DISCRETERANDOMVARIABLES.15X=Y=lOOxY=l90xx-x"'Y=270Y=340Y=400-X=.10X=Y=500-X=Figure3.1ThederivedrandomvariableY=g(X)forExan1ple3.27.Fromtheproblemstatement,theweightPx(;r:)=XhasPMF1/4x=1,2,3,4,0otherwise.(3.56)Thechargeforashipment,·y,hasrangeSy={100,190,270,340}correspondingtoSx={1,...,4}.Theexperimentcanbedescribedbythefollowingtree.HereeachvalueofYderivesfromauniquevalueofX.Hence,wecanuseEquation(3.55)tofindPy(y).Py(y)=X=l•Y=lOO1/4y=100,190,270,340,0otherwise.Theexpectedshippingbillis1/4E['Y]=_!(100+190+270+340)X=4•Y=3404=225cents.Example3.27SupposetheprobabilitymodelfortheweightinpoundsXofapackageinExample3.25IS0.2~--~--~Px(x)0.1Px(x;)=00................_........._____.05100.15x=1,2,3,4,0.1x=5,6,7,8,otherwise.xForthepricingplangiveninExample3.25,whatisthePMFandexpectedvalueofY,thecostofshippingapackage?NowwehavethreevaluesofX,specifically(6,7,8),transformedbyg(-)into·y=500.[3.6FUNCTIONSOFARANDOMVARIABLE89ForthissituationweneedthemoregeneralviewofthePMFofY,givenbyTheorem3.9.Inparticular,Y6=500,andwehavetoaddtheprobabilitiesoftheoutcomesX=6,X=7,andX=8tofindPy(500).Thatis,Py(500)=Px(6)+Px(7)+Px(8)(3.57)=0.30.ThestepsintheprocedureareillustratedinthediagramofFigure3.1.ApplyingTheorem3.9,wehavePy(y)0.2Py(y)=o-----100270500y=100,190,270,340,=400,=500,0.15y0.10y0.30y0otherwise.Forthisprobabilitymodel,theexpectedcostofshippingapackageisE[Y]=0.15(100+190+270+340)+0.10(400)+0.30(500)=325cents.i::::==Example3.28TheamplitudeV(volts)ofasinusoidalsignalisarandomvariablewithPMF0.2.-----------.Pv(v)0.11/7Pv(v)=0v=-3,-2,...)3,otherwise.0....._...................................__.-505vLet·y=V2/2wattsdenotethepowerofthetransmittedsignal.FindPy(y).ThepossiblevaIuesofYareSy={0,0.5,2,4.5}.Since·y=ywhen·v=./2f;or·v=-./2f;,weseethatPy(O)=Pv(O)=1/7.Fory=0.5,2,4.5,Py(y)=Pv(./2'i;)+Pv(-./2'i;)=2/7.Therefore,Py(y)0.2Py(y)0_,______...._____.012345=y=0,1/72/7y0otherwise.y=0.5,2,4.5,(3.58)Quiz3.6l\/IonitorthreeCl1storr1erspurchasingsrnartphonesatthePhonesrnartstoreandobservewhetl1ereachbuysanApricotphor1efor$450oraBananaphonefor$300.Thera.ndornvariableNisthenurnberofcustomerspurchasinganApricotphone.AssumeNhasPMFPN(n,)==00.4T/,0.2T/,=lO'1,2,3,otherwise.(3.59)[90CHAPTER3DISCRETERANDOMVARIABLESMdollarsist11earr1ol1ntofrr1oneypaidbythreecustorr1ers.(a)ExpressMasafur1ctionofl'l.(b)Fir1dP1w(rn)andE[M].3.7ExpectedValueofaDerivedRandomVariableIfY=g(X),E[Y]canbecalculatedfrornPx(x)andg(X)v.rithoutderivingPy(y).Weencour1terrnar1:yrsituationsinv.rhichv.reneedtokr1ovvonlytheexpectedvalueofaderivedrar1domvariableratherthantheer1tireprobabilityrr1odel.Fortur1ately,toobtainthisaverage,itisnotnecessar:yrtocomputethePl\/IForCDFofthenevvrandorr1variable.Ir1stead,'ivecanusethefollo'ivingpropertyofexpected\ralues.---Theorem3.lrl===Given,aran,dornvariableX'tvithPMFPx(x)an,dthederivedran,dorn'VariableY=g(X))theexpected'oalv,eofYisE[Y]L=µ,y=g(x)Px(x;).~i;ESxProofFromthedefinitionofE[Y]andTheorem3.9,wecan'ivriteE[Y]=LyESyLyPy(y)=yESyyLPx(x)=x:g(x)=yLL(3.60)g(x)Px(x),yESyx:g(x)=y'ivherethelastdoublesummationfollowsbecauseg(x)=yforeachxintheinnersum.Sinceg(x)transformseachpossibleoutcomexESxtoavalueyESy,theprecedingdoublesummationcanbe'ivrittenasasinglesumoverallpossible

valuesxESx.ThatIS,E[Y]=L(3.61)g(x)Px(x).xESx---Example3.291~-lnExample3.26,Px(x)=1/4x=1,2,3,4,0WhatisE['Y]?otherwise,andY=g(X)=105X-5X21<x<5,5006<x<10.(3.62)[3.7EXPECTEDVALUEOFADERIVEDRANDOMVARIABLE91ApplyingTheorem3.10wehave4LPx(x)g(x)E['Y]=·,1,·-1-=(1/4)[(105)(1)-(5)(1)2]+(1/4)[(105)(2)-(5)(2)2]=+(1/4)[(105)(3)-(5)(3)2]+(1/4)[(105)(4)-(5)(4)2](1/4)[100+190+270+340]=225cents.(3.63)Thisofcourseisthesarneans-vverobtair1edir1Exarnple3.26b}'firstcalculatingPy(y)andt11er1applyingDefinition3.13.Asanexercise)youmight"''anttocornputeE[Y]inExarnple3.27directlyfromT11eorerr13.10.Frornthist11eorerr1\Vecanderivesorr1eimportantpropertiesofexpectedva.lt1es.Thefirstor1ehastodowiththedifferencebetv.reenarandorr1variableanditsexpectedva.111e.·vVher1studentslearnt11eirO'ivngradesonarr1idterrr1exarn,theyarequicktoaskaboutt11eclassaverage.Let'ssayonestudenthas73andtheclassaverageis80.Shemaybeinclir1edtothinkofhergradeas"se'irenpointsbelo-vvaverage,"ratherthar1"73."Interrnsofaprobabilityrnodel,-vve"''ouldsaythattherandornvariableXpointsor1thernidterrnhasbeer1transformedtotherandornvariableY=g(X)=X-µxpointsaboveaverage.(3.64)Theexpectedvalt1eofX·-µ,xiszero,regardlessoftheprobabilityrr1odelofX.===Theorem3.ll--Forariyran,dornvariableX)E[X-µ,x]=O.ProofDefiningg(X)E[g(X)]==XL-µxandapplyingTheorem3.10yields(x-xESxfl·X)Px(x)=L1;Px(1;)-fl·XLPx(x).(3.65)xESxThefirsttermontherightsideisµxbydefinition.Inthesecondterm)2=xESxPx(x)=1,sobothtermsontherightsideareJ.lxandthedifferenceiszero.Anotherpropertyoftheexpected\ralueofafur1ctionofarandornvariableappliestolineartransforrnations.11WecallthetransformationaX+blinearalthough,strictlyspeaking,it,shouldbecalledaffine.[92CHAPTER3DISCRETERANDOMVARIABLES3.12Forariyran,dornvariableX;~-==TheoremE[aX+b]=aE(X]+b.Thisfollov.rsdirectlyfrorr1Defir1ition3.13aridTheorem3.10.Alir1eartransformationisessentiallyascalechangeofaqt1antity,likeatransforrr1ationfrorr1inchestocentimetersorfromdegreesFahrenheittodegTeesCelsius.Ifv.reexpressthedata(randomvariableX)innevvt1r1its,thenewaverageisjusttheoldaveragetrar1sformedtothenevvunits.(Iftheprofessoraddsfivepointstoe·veryone'sgrade,theaveragegoesupbyfi·vepoints.)T11isisarareexarnpleofasitl1atior1inv.r11ichE(g(X)]=g(E(X]).Itisternpt'in,g,[yu,t11,sually'UJrorig,toapplyittoothertransform,ation,s.Forexample,if·y=X2)itist1suallyt11ecasethatE(Y]=I=(E(X])2.Expressingthisir1generalterrr1s,itist1suallyt11ecasethatE(g(X)]~g(E(X]).Example3.30RecallfromExamples3.5and3.24thatXhasPMF0.5Px(x)Px(x;)=0...___..____,.._____,-11032xx=Ox=l'1/4x=2,1/41/20)(3.66)otherwise.WhatistheexpectedvalueofV=g(X)=4X+7?FromTheorem3.12,E('V]=E[g(X)]=E(4X+7]=4E(X]+7=4(1)+7=11.(3.67)WecanverifythisresultbyapplyingTheorem3.10:E['VJ=g(O)Px(0)+g(l)Px(1)+g(2)Px(2)=7(1/4)+11(1/2)+15(1/4)=11.(3.68)Example3.31ContinuingExample3.30,letW=h,(X)=X2.WhatisE(T1V]?Theorem3.10givesE(W]=Lh(x)Px(x;)=(1/4)02+(1/2)12+(1/4)22=1.5.Notethatthisisnotthesameash,(E(W])=(1)2=1.(3.69)[3.8VARIANCEANDSTANDARDDEVIATION93..---Quiz3.7·r--:=~TheriurriberofrnernorychipsMneededinapersonalcornputerdependsonhovvrnanyapplicationprograrr1s,A,theovvnerv.rantstorunsirnultaneously.Tlienl1rriberofchipsMandtheriurriberofapplicationprogramsAaredescribedb:y(4J\lf=468chipschipschipschipsforforforfor1234progTa.rn,prograrns,prograrns,prograrns,PA(a)=0.1(5-a)a,=0otherv.rise.1)2,3,4,(a)vVhatistheexpectednurnberofprograrrisµA=E[A]?(b)ExpressJ\lf,tlienurn.berofrriernorychips,asafunctionJ\lfntlmberofapplicationprograrnsA.(c)FiridE[M]=E[g(A)].DoesE[M]=g(E[A])?3.8(3.70)g(A)oftheVarianceandStandardDeviationThevarianceVar[X]rrieasuresthedispersionofsamplevaluesofXarol1ndtlieexpectedvalueE[X].Whenwe·viev.rE[X]asanestirnateofX,Var[X]istherrieansquareerror.InSectiori3.5,v.redescribeanaverageasatypical·valueofarandomvariable.Itisonentlrnberthatsummarizesanentireprobabilityrriodel.Afterfiridingana·verage,sorneonev.rhov.raritstolookfurtherintotlieprobabilit}'modelrnightask,"Howtypicalistheavera,ge?"or"'\i\Thatarethecha.ricesofobserviriganeventfarfrorntheaverage?"Iritheexampleofthemidterrnexarri,after}'OllfiridOtltyourscoreis7pointsaboveaverage,yol1arelikelytoask,"Howgoodisthat?Isitnearthetopoftlieclassorsomewhereneartherriiddle?"Arrieastlreofdispersionisananswertothesequestionsv.rrappedupina.siriglenumber.Ifthismeasureissrriall,observatior1sarelil12]?(d)\iVhatisP[8<Y<12]?Inabagof25~1I&Ms,eachpieceisequallylikelytobered,green,orange,blue,orbro,vn,independentofthecolorofanyotherpiece.FindthethePMFofR,thenumberofredpieces.\i\fhatistheprobabilityabaghasnoredM&~lfs?3.3.2Supposewhenabaseballplayergetsahit,asingleistwiceaslikelyasadouble,'vhichistwiceaslikelyasatriple,'vhichist'viceaslikelyasahomerun.Also,theplayer'sbattingaverage,i.e.,theprobabilitytheplayergetsahit,is0.300.LetBdenotethenumberofbasestouchedsafelyduringanat-bat.Forexample,B=0vvhentheplayermakesanout,B=1onasingle,andsoon.\i\fhatisthef>lVIFofB?3.3.3\i\fhensomeonepressesSENDonacellularphone,thephoneattemptstosetupacallbytransmittingaSET.UPmessagetoanearbybasestation.Thephonewaitsforaresponse,andifnonearriveswithin0.5secondsittriesagain.Ifitdoesn'tgetaresponseaftern,=6tries,thephonestopstransmittingmessagesandgeneratesabusysignal.(b)Assumep=0.8.Whatistheminimumvalueofnthatproducesaprobabilityof0.95ofreceivingthemessageatleastonce?3.2.103.2.11\i\fhenaconventionalpagingsystemtransmitsamessage,theprobabilitythatthemessagewillbereceivedbythepageritissenttoisp.Tobeconfidentthatamessageisreceivedatleastonce,asystemtransmitsthemessagen,times.(a)_Assumingalltransmissionsareindependent,'vhatisthePMFofK,thenumberoftimesthepagerreceivesthesamemessage?Yourollapairoffairdiceuntilyouroll"doubles"(i.e.,bothdicearethesame).\iVhatistheexpectednumber,E[N],ofrolls?3.3.4[108CHAPTER3DISCRETERANDOMVARIABLES3.3.5\i\fhenyougofishing,youattach1nhookstoyourline.Eachtimeyoucastyourline,eachhookwillbesv;,rallo,vedbyafish'vithprobabilityh,independentofwhetheranyotherhookiss'vallowed.WhatisthePMFofI(a)WhatisthePMFPx(x)?(b)Ifp=0.2,whatistheprobabilitythatthechild'villthrowtheFrisbeemorethanfourtimes?3.3.7\i\fhenat'vo-,vaypagingsyste1ntransmitsamessage,theprobabilitythatthemessage'villbereceivedbythepageritissenttoisp.Whenthepagerreceivesthemessage,ittrans1nitsanacknowledgmentsignal(ACK)tothepagingsystem.IfthepagingsystemdoesnotreceivetheACK,itsendsthemessageagain.(a)WhatisthePMFofN,thenumberoftimesthesyste1nsendsthesamemessage?(b)Thepagingco1npany'vantstoli1nitthenumberoftimesithastosendthesamemessage.IthasagoalofP[N<3]>0.95.\tVhatistheminimumvalueofpnecessarytoachievethegoal?3.3.8ThenumberofbytesBinanHTMLfileisthegeometric(2.5·10-5)randomvariable.\i\fhatistheprobabilityP[B>500,000]thatafilehasover500,000bytes?K,thenumberofticketsyoubuyuptoandincludingyourfifth'vinningticket.(b)Listhenumberofflipsofafaircoinuptoandincludingthe33rdoccurrenceoftails.\t\fhatisthePMFofL?(c)Startingonday1,youbuyonelotteryticketeachday.Eachticketisawinner'vithprobability0.01.LetJV!equalthenumberofticketsyoubuyuptoandincludingyourfirstwinningticket.\i\fhatisthePMFofM?3.3.10ThenumberofbusesthatarriveatabusstopinTminutesisaPoissonrandomvariableBwithexpectedvalueT/5.(a)\t\fhatistheP~1IFofB,thenumberofbusesthatarriveinTminutes?(b)\i\fhatistheprobabilitythatinat'vominuteinterval,threebuses'villarrive?(c)Whatistheprobabilityofnobusesarrivingina10-minuteinterval?(d)Ho'vmuchtimeshouldyouallo'vsothat'vithprobability0.99atleastonebusarrives?3.3.11Inawirelessautomaticmeterreadingsystem,abasestationsendsouta'vake-upsignaltonearbyelectricmeters.Onhearingthe'vake-upsignal,ametertransmitsamessageindicatingtheelectricusage.Eachmessageisrepeatedeighttimes.(a)Ifasingletransmissionofainessageissuccessful'vithprobabilityp,'vhatisthePMFofN,thenumberofsuccessfulmessagetransmissions?(b)IisanindicatorrandomvariablesuchthatI=1ifatleastonemessageistransmittedsuccessfully;otherwiseI=0.FindtheP~1FofI.3.3.12AZipf(77,,n=1)randomvariableXhasPMFPx(x)={~(n)/xconstantc(77,)3.3.9(a)Startingonday1,youbuyonelotteryticketeachday.Eachticketisawinnerwithprobability0.1.FindthePMFofTheI:;=1Px(x)77,=1,2,...)6.1.x=1,2,...'77,'other,vise.issetsothatCalculatec(77,)for[PROBLEMS3.3.13Inabagof64"holidayseason"M&~/[s,each~1I&Misequallylikelytoberedorgreen,independentofanyotherM&Minthebag.(a)IfyourandomlygrabfourM&Ms,'vhatistheprobabilityP[E]thatyougrabanequalnumberofredandgreenM&~l[s?(b)WhatisthePMFofG,thenumberofgreen~![&Msinthebag?(c)Youbegineatingrandomlychosen~![&Msonebyone.LetRequalthenumberofredM&~/[syoueatbeforeyoueatyourfirstgreenM&M.\i\!hatisthePMFofR?3.3.14Aradiostationgivesapairofconcertticketstothesixthcallerwhokno,vsthebirthdayoftheperformer.Foreachperson'vhocalls,theprobabilityis0.75ofkno,vingtheperformer'sbirthday.Allcallsareindependent.(a)WhatisthePMFofL,thenumberofcallsnecessarytofindthe\Vinner?(b)Whatistheprobabilityoffindingthewinneronthetenthcall?(c)\i\fhatistheprobabilitythatthestationwillneednineormorecallstofindawinner?3.3.15Inapacketvoicecommunicationssystem,asourcetransmitspacketscontainingdigitizedspeechtoareceiver.Becausetransmissionerrorsoccasionallyoccur,anackno,vledgment(ACK)oranegativeackno,vledgment(NAK)istransmittedbacktothesourcetoindicatethestatusofeachreceivedpacket.\i\!henthetransmittergetsaNAK,thepacketisretransmitted.Voicepacketsaredelaysensitive,andapacketcanbetransmittedamaximumofdtimes.IfapackettransmissionisanindependentBernoullitrialwithsuccessprobabilityp,'vhatistheP~l[Fof'Ithenumberoftimesapacketistransmitted?1,3.3.16AtNewarkairport,yourjetjoinsalineasthetenthjet\vaitingfortakeoff.AtNe,vark,takeoffsandlandingsaresynchronizedtotheminute.Ineachone-minute109interval,anarrivingjetlands\Vithprobabilityp=2/3,independentofanarrivingjetinanyotherminute.Suchanarrivingjetblocksany\Vaitingjetfromtakingoffinthatone-minuteinterval.However,ifthereisnoarrival,thenthe\Vaitingjetattheheadofthelinetakesoff.Eachtake-offrequiresexactlyoneminute.(a)LetL1denotethenumberofjetsthatlandbeforethejetatthefrontofthelinetakesoff.FindtheP~IIFPL1(l).(b)LetWdenotethenumberofminutesyou\Vaituntilyourjettakesoff.FindP[vV=10].(Notethatifnojetslandfortenminutes,thenonewaitingjet\villtakeoffeachminuteandvV=10.)(c)WhatisthePMFofvV?3.3.17Supposeeachday(startingonday1)youbuyonelotteryticketvvithprobability1/2;othervvise,youbuynotickets.Aticketisa\vinnerwithprobabilitypindependentoftheoutcomeofallothertickets.LetNibetheeventthatondayiyoudonotbuyaticket.LetWibetheeventthatondayi,youbuyawinningticket.LetLibetheeventthatondayiyoubuyalosingticket.(a)\!\ThatareP[vV33],P[L81],andP[Ngg]?(b)LetJ{bethenumberofthedayon\vhichyoubuyyourfirstlotteryticket.FindtheP~l[FPK(k).(c)FindthePMFofR,thenumberoflosinglotteryticketsyouhavepurchasedinmdays.(d)LetDbethenumberofthedayon'vhichyoubuyyourjthlosingticket.\i\fhatisPD(d)?Hint:Ifyoubuyyourjthlosingticketondayd,ho\vmanylosersdidyouhaveafterd-1days?3.3.18TheSixersandtheCelticsplayabestoutoffiveplayoffseries.Theseriesendsassoonasoneoftheteamshaswonthreegames.Assumethateitherteamisequallylikelytowinanygameindependentlyofanyothergameplayed.Find(a)TheP~1IFPN(n)forthetotalnumber1'lofgamesplayedintheseries;[110CHAPTER3DISCRETERANDOMVARIABLES(b)ThePlVIFPw(?D)forthenu1nberWofCeltics'vinsintheseries;(c)TheP~1IFPL(l)forthenumberLofCelticslossesintheseries.Forabino1nialrandomvariableKrepresentingthenumberofsuccessesinn,trials,E~=oPK(k)=1.·usethisfacttoprovethebinomialtheoremforanya>0andb>0.Thatis,showthat3.3.19(a)DrawagraphoftheCDF.(b)\tVritePx(1;),thePMFofX.3.4.4FollowingExample3.22,sho'vthatageometric(p)randomvariableJ{hasCDFFK(k)={~-(1-p)lkJk<1,k>1.AttheOneTopPizzaShop,apizzasoldhasmushrooms'vithprobabilityp=2/3.Onadayin'vhich100pizzasaresold,letNequalthenumberofpizzassoldbeforethefirstpizzawithmushroomsissold.vVhatisthePMFofN?WhatistheCDFofN?3.4.5DiscreterandomvariableYhastheCDFFy(y)asshown:3.4.1IFy(y)0.750.50.25I0'----'--------()I2345yUsetheCDFtofindthefollo,vingprobabilities:<1]andP[Y<1]P[Y>2]andP[Y>2]P[Y=3]andP[Y>3](a)P[Y(b)(c)(d)Py(y)3.4.2TherandomvariableXhasCDF(0Fx(x)=0.20.71x<-1,-1<x<0,0<x<1,x>1.(a)Dra'vagraphoftheCDF.(b)WritePx(x),thePMFofX.BesuretowritethevalueofPx(x)forallxfrom-ootooo.3.4.3TherandomvariableXhasCDFFx(x)=00.40.81x<-3,-3<x<5,5<x<7,x>7.InProblem3.2.10,findandsketchtheCDFofB,thenumberofbasestouchedsafelyduringanat-bat.3.4.6IInProble1n3.2.6,findandsketchtheCDFofY,thenumberofpointsscoredina1and1forp=1/4,p=1/2,andp=3/4.3.4.7InProblem3.2.11,findandsketchtheCDFofN,thentunberofatte1nptsinadebythecellularphoneforp=1/2.3.4.83.5.1LetXhavetheuniformPx(x)=0.01{0(a)FindamodeP~!IFx=1,2,...,100,other,vise.ofX.Ifthemodeisnotunique,findthesetX1110c1ofallmodesofX.1;1nod(b)Findamedianx111ec1ofX.Ifthemedianisnotunique,findthesetX1neclofallnumbersxthataremediansofX.Itcosts20centstoreceiveaphotoand30centstosendaphotofromacellphone.Cisthecostofonephoto(eithersentorreceived).Theprobabilityofreceivingaphotois0.6.Theprobabilitysendingaphotois0.4.3.5.2(a)FindPc(c),thePMFofC.[PROBLEMS(b)WhatisE[CJ,theexpectedvalueofC?(b)\tVhatisP[X3.5.33.5.11(a)Thenumberoftrains.]thatarriveatthestationintimetminutesisaPoissonrandomvariable'~ithE[.J]=t.FindtsuchthatP[.J>O]=0.9.(b)ThenumberofbusesI<thatarriveatthestationinonehourisaPoissonrandomvariablewithE[K]=10.FindP[K=lO].(c)Ina1insinterval,thenumberofhitsLona\i\febserverisaPoissonrandomvariable'vithexpectedvalueE[L]=2hits.WhatisP[L<1]?Yousimultaneouslyflipapairoffaircoins.Yourfriendgivesyouonedollarifbothcoinscomeupheads.YourepeatthistentimesandyourfriendgivesyouXdollars.FindE[X],theexpectednumberofdollarsyoureceive.\tVhatistheprobabilitythatyoudo'\vorsethanaverage"?3.5.4i\packetreceivedbyyours1nartphoneiserror-free'vithprobability0.95,independentofanyotherpacket.3.5.5(a)Outof10packetsreceived,letXequalthenumberofpacketsreceived'vitherrors.\i\fhatisthePMFofX?(b)Inonehour,yoursmartphonereceives12,000packets.LetXequalthenumberofpacketsreceivedwitherrors.\i\fhatisE[X]?FindtheexpectedvalueoftherandomvariableYinProblem3.4.1.3.5.6FindtheexpectedvalueoftherandomvariableXinProblem3.4.2.3.5.7>E[X]]?Kisthegeometric(1/11)randomvariable.(a)WhatisP[K=E[K]]?>E[IAtacasino,peoplelineuptopay$20eachtobeacontestantinthefollowingga1ne:Thecontestantflipsafaircoinrepeatedly.Ifsheflipsheads20timesinarow,shewalksawaywithR=20milliondollars;other,viseshe'valksaway'vithR=0dollars.3.5.12(a)FindthePtv1FofR,there,vardearnedbythecontestant.(b)Thecasinocounts"losingcontestants"whofailto'vinthe20milliondollarprize.LetLequalthenumberoflosingcontestantsbeforethefirstwinningcontestant.WhatisthePMFofL?(c)Whydoesthecasinoofferthisgame?Giveexamplesofpracticalapplicationsofprobabilitytheorythatcanbeinodeledbythefollo,vingPMFs.Ineachcase,stateanexperiment,thesamplespace,therangeoftherandomvariable,thePl\1Foftherandomvariable,andtheexpectedvalue:3.5.13(a)Bernoulli(b)Binomial(c)Pascal(d)Poissonl\1akeupyotu·o'vnexamples.(Don'tcopyexamplesfromthetext.)3.5.14FindP[K<E[K]]whenFindtheexpectedvalueoftherandomvariableXinProblem3.4.3.(a)Kisgeometric(1/3).UseDefinition3.13tocalculatetheexpectedvalueofabino1nial(4,1/2)randomvariableX.(c)KisPoisson(3).3.5.83.5.9Xisthediscreteuniform(1,5)randomvariable.3.5.10(a)WhatisP[X=E[X]]?111(b)J{isbinomial(6,1/2).(d)J{isdiscreteuniform(0,6).Supposeyougotoacasinowithexactly$63.Atthiscasino,theonlygameisrouletteandtheonlybetsallo,vedareredandgreen.Thepayoffforawinningbet3.5.15[112CHAPTER3DISCRETERANDOMVARIABLESistheamountofthebet.Inaddition,the'vheelisfairsothatP[red]=P[green)=1/2.Youhavethefollo,vingstrategy:First,youbet$1.Ifyou\Vinthebet,youquitandleavethecasino'vith$64.Ifyoulose,youthenbet$2.Ifyouwin,youquitandgohome.Ifyoulose,youbet$4.Infact,'vheneveryoulose,youdoubleyourbetuntileitheryou\Vinabetoryouloseallofyourmoney.However,assoonasyouwinabet,youquitandgohome.LetYequaltheamountofmoneythatyoutakehome.FindPy(y)andE[Y).\i\10uldyouliketoplaythisgameeveryday?3.5.16InaTVgamesho,v,therearethreeidentical-lookingsuitcases.Thefirstsuitcasehas3dollars,thesecondhas30dollarsandthethirdhas300dollars.Youstartthegamebyrandomlychoosingasuitcase.Bet1Deenthet11107lnchosens1litcases,thegameshowhostopensthesuitcase\Vithmoremoney.Thehostthenasksyouifyou\VanttokeepyoursuitcaseorS\Vitchtotheotherremainingsuitcase._Afteryoumakeyourdecision,youopenyoursuitcaseandkeeptheDdollarsinside.Shouldyouswitchsuitcases?Toans,verthisquestion,solvethefollo,vingsubproblemsandusethefollo\vingnotation:•Ciistheeventthatyoufirstchoosethesuitcase\Vithidollars.•oidenotestheeventthatthehostopensasuitcase\vithidollars.Inaddition,youmaywishtogobackandreviewthel\/IontyHallprobleminExample2.4.(a)Supposeyounevers\vitch;youal,vaysstickwithyouroriginalchoice.·useatreediagramtofindtheI>MFPn(d)andexpectedvalueE[D).(b)Supposeyoualwaysswitch.·useatreediagramtofindthePlVIFPn(d)andexpectedvalueE[D).(c)Perhapsyourruleforswitchingshoulddependonho\vmanydollarsareinthesuitcasethatthehostopens?\i\ThatistheoptimalstrategytomaximizeE[D)?Hint:Considermakingarandomdecision;ifthehostopensasuitcase'vithidollars,letaidenotetheprobabilitythatyous\vitch.3.5.17'YouareacontestantonaTVgameshow;therearefouridentical-lookingsuitcasescontaining$100,$200,$400,and$800.Youstartthegamebyrando1nlychoosingasuitcase.Amongthethree1lnchosensuitcases,thegamesho\vhostopensthesuitcasethatholdsthemedianamountofmoney.(Forexample,iftheunopenedsuitcasescontain$100,$400and$800,thehostopensthe$400suitcase.)Thehostthenasksyouifyouwanttokeepyoursuitcaseorswitchoneoftheotherremainingsuitcases.Foryouranalysis,usethefollo\vingnotationforevents:•Ciistheeventthatyouchooseasuitcase\Vithidollars.•oidenotestheeventthatthehostopensasuitcasewithidollars.•Ristherewardindollarsthatyoukeep.(a)Yourefusethehost'sofferandopenthesuitcaseyoufirstchose.FindthePMFofRandtheexpectedvalueE[R].(b)YouahvaysS\vitchandrandomlychooseoneofthet\voremainingsuitcases\vithequalprobability.'YoureceivetheRdollarsinthischosensuitcase.Sketchatreediagramforthisexperiment,andfindthePMFandexpectedvalueofR.(c)Canyoudobetterthaneitheral,vaysS\vitchingoral\vaysstayingwithyouroriginalchoice?Explain.3.5.18'YouareacontestantonaTVgamesho,v;therearefouridentical-lookingsuitcasescontaining$200,$400,$800,and$1600.Youstartthegamebyrandomlychoosingasuitcase.Amongthethreeunchosens1litcases,thegamesho\vhostopensthesuitcasethatholdstheleastmoney.Thehostthenasksyouifyou\Vanttokeep[PROBLEMSyoursuitcaseorsv;ritchoneoftheotherremainingsuitcases.Forthefollo,vinganalysis)usethefollowingnotationforevents:•Ciistheeventthatyouchooseasuitcase'vithidollars.•oidenotestheeventthatthehostopensasuitcase'vithidollars.•Risthere,vardindollarsthatyoukeep.(a)Yourefusethehost'sofferandopenthesuitcaseyoufirstchose.FindthePMFofRandtheexpectedvalueE[R).(b)Youswitchandrandomlychooseoneofthet'voremainings11itcases.YoureceivetheRdollarsinthischosensuitcase.Sketchatreediagramforthisexperiment,andfindth.ePlVIFandexpectedvalueofR.3.5.19LetbinomialrandomvariableX11denotethenumberofsuccessesinn,Bernoullitrials'vithsuccessprobabilityp.ProvethatE[X11)='np.Hint:UsethefactthatI:~-~Pxn_1(1~)=1.3.5.20ProvethatifXisanonnegativeinteger-valuedrandomvariable,then113'veightliftingwork.Whatmassm,intherange1<m,<100shouldsheusetomaximizeherprobabilityof'vinningtheinoney?Forthebestchoiceofm,'vhatistheprobabilitythatshe'vinstheinoney?3.5.22Atthegym,aweightliftercanbenchpressamaximumof200kg.Foramassofmkg,(1<m,<200),themaximumnumberofrepetitionsshecancompleteisR,ageometricrandomvariablewithexpectedvalue200/1n.(a)Intermsofthemassm,,whatisthePMFofR?(b)\i\fhensheperformsonerepetition,sheliftsthem,kgmassaheighth=4/9.8metersandthusdoeswork71J=m,gh=4mJoules.ForRrepetitions,shedoesW=4m,RJoulesof'vork.\i\fhatistheexpectedworkE[W)thatshewillcomplete?(c)Afriendofferstopayher1000dollarsifshecanperform1000Joulesof'veightlifting'vork.\i\fhatmassm,intherange1<1n<200shouldsheusetomaximizeherprobabilityofwinningthemoney?00E[X)=Lp[X>k].k=O3.5.21i\.tthegym,avveightliftercanbenchpressamaximumof100kg.l(a)Intermsofthemassm,,whatisthePlVIFofR?3.6.1Giventherando1nvariableYlnProblem3.4.1,letU=g(Y)=Y2.(a)FindPu(11,).(b)FindFu(7L).(c)FindE[U).3.6.2GiventherandomvariableXlnProblem3.4.2,letV=g(X)=IXI.(a)FindPv(v).(b)FindFv(v).(b)Whensheperformsonerepetition,sheliftsthemkgmassaheighth=5/9.8metersandthusdoes'vork11;=1ngh=51nJoules.ForRrepetitions,shedoesW=5m,RJoulesof\vork.\tVhatistheexpected'vorkE[W)thatshe'villco1nplete?3.6.3GiventherandomvariableXlnProblem3.4.3,letW=g(X)=-X.(c)Afriendofferstopayher1000dollarsifshecanperform1000Joulesof(c)FindE[vV).(c)FindE[V).(a)FindPw(11;).(b)FindFw(11;).[114CHAPTER3DISCRETERANDOMVARIABLESAtadiscountbrokerage,astockpurchaseorsale\Vorthlessthan$10,000incursabrokeragefeeof1%ofthevalueofthetransaction.Atransactionworthmorethan$10,000incursafeeof$100plus0.5%oftheamountexceeding$10,000.Notethatforafractionofacent,thebrokeragealwayschargesthecustomerafullpenny.You\vishtobuy100sharesofastockwhosepriceDindollarshasPMF3.6.4Po(d)={~/3d=99.75,100,100.25,other,vise.\i\fhatisthePlVIFofC,thecostofbuyingthestock(includingthebrokeragefee)?Asourcetransmitsdatapacketstoareceiveroveraradiolink.Ther&ceiveruseserrordetectiontoidentifypacketsthathavebeencorruptedbyradionoise.\i\fhenapacketisreceivederrorfree,thereceiversendsanackno\vledg1nent(ACK)backtothesource.\i\fhenthereceivergetsapacketwitherrors,anegativeacknowledgment(NAK)messageissentbacktothesource.EachtimethesourcereceivesaNAK,thepacketisretransmitted.Voleassumethateachpackettransmissionisindependentlycorruptedbyerrors\Vithprobabilityq.3.6.5(a)FindthePlVIFofX,thenumberoftimesthatapacketistransmittedbythesource.(b)Supposeeachpackettakes1millisecondtotransmitandthatthesourcewaitsanadditionalmillisecondtoreceivetheackno\vledgmentmessage(ACKorNAK)beforeretransmitting.Let7.,equalthetimerequireduntilthepacketissuccessfullyreceived.WhatistherelationshipbetweenTandX?\i\fhatisthePMFofT?Supposethatacellularphonecosts$20permonthwith30minutesofuseincludedandthateachadditionalminuteofusecosts$0.50.IfthenumberofminutesyouuseinamonthisageometricrandomvariableM\Vithexpectedvalueof3.6.6E[M]=l/p=30minutes,\vhatisthePlVIFofC,thecostofthephoneforonemonth?Aprofessortriestocountthenumberofstudentsattendinglecture.Foreachstudentintheaudience,theprofessoreithercountsthestudentproperly(\vithprobabilityp)oroverlooks(anddoesnotcount)thestudentwithprobability1-p.Theexactnumberofattendingstudentsis70.3.6.7(a)ThenumberofstudentscountedbytheprofessorisarandomvariableN.WhatisthePMFofN?(b)LetU=70-Ndenotethenumberofuncountedstudents.\t\fhatisthePlVIFofU?(c)\t\fhatistheprobabilitythattheundercountUis2ormore?(d)For'vhatvalueofpdoesE[U]=2?Aforgetfulprofessortriestocountthel\1&Msinapackage;ho,vever,theprofessoroftenloseshisplaceanddoublecountsanl\!I&M.Foreachl\!I&Minthepackage,theprofessorcountsthel\1&Mandthen,\Vithprobabilitypcountsthel\!I&l\!Iagain.Theexactnumberofl\II&Msinthepackis20.3.6.8(a)FindthePl\l[FofR,thenumberofdouble-countedM&Ms.(b)FindthePl\IIFofN,thenumberofM&Mscountedbytheprofessor.3.7.1Startingondayn,=1,youbuyonelotteryticketeachday.Eachticketcosts1dollarandisindependentlyawinnerthatcanbecashedfor5dollars\vithprobability0.1;otherwisetheticketisworthlessLetXnequalyournetprofitaftern,days.WhatisE[Xn]?3.7.2Forrandomvariable'J'inQuiz3.6,firstfindtheexpectedvalueE[T]usingTheorem3.10.Next,findE[T]usingDefinition3.13.3.7.3Inacertainlotterygame,thechanceofgettingavvinningticketisexactlyoneinathousand.Supposeapersonbuysoneticketeachday(exceptontheleapyeardayFebruary29)overaperiodoffiftyyears.[PROBLEMS\i\fhatistheexpectednumberE[T]ofwinningticketsinfiftyyears?Ifeachwinningticketis'vorth$1000,whatistheexpectedamountE[R]collectedonthesewinningtickets?Lastly,ifeachticketcosts$2,'vhatisyourexpectednetprofitE[Q]?3.7.4SupposeanNBAbasketballplayershootinganuncontested2-pointshotwillmakethebasketwithprobability0.6.However,ifyoufoultheshooter,theshot'villbemissed,butt'vofreethro,vswillbea'varded.Eachfreethro'visanindependentBernoullitrial'vithsuccessprobabilityp.Basedontheexpectednumberofpointstheshooter'villscore,forwhatvaluesofpmayitbedesirabletofoultheshooter?3.7.5Itcantakeuptofourdaysafteryoucallforservicetogetyourcomputerrepaired.Thecomputercompanychargesforrepairsaccordingtohov;;longyouhaveto'vait.ThenumberofdaysDuntiltheservicetechnicianarrivesandtheservicechargeC,indollars,aredescribedby230.40.340.1andC=907040for1-dayservice,40for4-dayservice.for2-dayservice,for3-dayservice,(a)Whatistheexpectedwaitingtimeµ,n=E[D]?(b)WhatistheE[D-µ,n]?expecteddeviation(c)ExpressCasafunctionofD.(d)WhatistheexpectedvalueE[C]?3.7.6TrueorFalse:ForanyrandomvariableX,E[l/X]=1/E[X].3.7.7ForthecellularphoneinPro~lem3.6.6,expressthemonthlycostCasafunctionofM,thenumberofminutesused.\i\fhatistheexpectedmonthlycostE[C]?1153.7.B•Anewcellularphonebillingplancosts$15permonthplus$1foreachminuteofuse.Ifthenumberofminutesyouusethephoneinamonthisageometricrandomvariablev1ithexpectedvaluel/p,'vhatistheexpectedmonthlycostE[CJofthephone?For'vhatvaluesofpisthisbillingplanpreferabletothebillingplanofProblem3.6.6andProblem3.7.7?3.7.9Aparticularcircuitworksifall10ofitscomponentdeviceswork.Eachcircuitistestedbeforeleavingthefactory.Eachworkingcircuitcanbesoldforkdollars,buteachnonworkingcircuitisworthlessandmustbethrownaway.Eachcircuitcanbebuiltwitheitherordinarydevicesorultrareliabledevices.Anordinarydevicehasafailureprobabilityofq=0.1andcosts$1.Anultrareliabledevicehasafailureprobabilityofq/2butcosts$3.i\.ssumingdevicefailuresareindependent,shouldyoubuildyourcircuitwithordinarydevicesorultrareliabledevicesinordertomaximizeyourexpectedprofitE[R]?I-\.eepinmindthatyourans,ver'villdependonk.3.7.10IntheNewJerseystatelottery,each$1tickethassixrandomlymarkednumbersoutof1,...,46.Aticketisa'vinnerifthesixmarkednumbersmatchsixnumbersdra,vnatrandomattheendofaweek.Foreachticketsold,50centsisaddedtothepotforthewinners.Iftherearekwinningtickets,thepotisdividedequallyamongthekwinners.Supposeyouboughtawinningticketinaweekinwhich2riticketsaresoldandthepotisndollars.(a)\t\lhatistheprobabilityqthatarandomticketwillbeawinner?(b)FindtheP~l[FofKn,thenumberofother(besidesyouro'vn)winningtickets.(c)WhatistheexpectedvalueofWn,theprizeforyourwinningticket?3.7.11IfthereisnowinnerforthelotterydescribedinProblem3.7.10,thenthepotiscarriedovertothenext'veek.Supposethatinagiven'veek,anrdollarpotiscarriedoverfromtheprevious'veekand[116CHAPTER3DISCRETERANDOMVARIABLESSho\vthatthevarianceofYaX+bisVar[Y]=a2Var[X].2n,ticketssold.Ans,verthefollowingquestions.3.8.7(a)Whatistheprobabilityqthatarandomticket'villbea\Vinner?Givenarando1nvariableX'vithexpectedvalueJJ,xandvariancea~,findtheexpectedvalueandvarianceof(b)Ifyouownoneofthe2n,ticketssold,whatistheexpectedvalueofV,thevalue(i.e.,theamountyouwin)ofthatticket?IsiteverpossiblethatE[V]>1?(c)Supposethatintheinstantbeforetheticketsalesarestopped,youaregiventheopportunitytobuyoneofeachpossibleticket.Forwhatvalues(ifany)ofriandrshouldyoudoit?Inanexperimenttomonitort\vopackets,thePI\l[FofN,thenumberofvideopackets,is3.8.110.720.1FindE[N],E[N2],Var[J\T],andaN.FindthevarianceoftherandomvariableYinProblem3.4.l.3.8.2FindthevarianceoftherandomvariableXinProblem3.4.2.3.8.3FindthevarianceoftherandomvariableXinProblem3.4.3.3.8.43.8.5LetXhavethebino1nialPI\l[FPx(x)=(~)(1/2)4(a)FindthestandarddeviationofX.(b)WhatisP[µx-ax<X<JJ,x+ax],theprobabilitythatXiswithinonestandarddeviationoftheexpectedvalue?3.8.6Xisthebinomial(5,0.5)randomvariable.(a)FindthestandarddeviationofX.(b)Find1=>[1),x-ax<X<JJ,x+ax],theprobabilitythatXis\vithinonestandarddeviationoftheexpectedvalue.3.8.83.8.9•Inreal-timepacketdatatransmis-sion,thetimebetweensuccessfullyreceivedpacketsiscalledtheinterarrivaltim,e,andrandomnessinpacketinterarrivaltimesiscalled.fitter.Jitterisundesirable.Onemeasureofjitteristhestandarddeviationofthepacketinterarrivaltime.FromProblem3.6.5,calculatethejitterar.Howlargemustthesuccessfultransmissionprobabilityqbetoensurethatthejitterislessthan2milliseconds?RandomvariableKhasaPoisson(a)distribution.DerivethepropertiesE[K]=Var[K]=a.Hint:E[K2]=E[K(I<-1)]+E[IForthedelayDinProblem3.7.5,whatisthestandarddeviationanofthewaitingtime?3.8.11LetXbethebinomial(100,1/2)randomvariable.LetE2denotetheeventthatXisaperfectsquare.CalculateP[E2].3.9.13.9.2WriteaMATLABfunctionx=shipweight8(m)thatproducesmran-domsamplevaluesofthepackage\veightXwithPI\l[FgiveninExample3.27.·usetheuniquefunctionto\vritealVIATLABscriptshipcostpmf.mthatoutputsthepairofvectorssyandpyrepresentingthePMFPy(y)oftheshippingcostYinExample3.27.3.9.3Form=10,m,=100,andm=1000,useI\IIATLABtofindtheaveragecostofsendingm,packagesusingthemodelofExample3.27.Yourprograminputshouldhavethenumberoftrialsm,astheinput.TheoutputshouldbeY=-:I::n1Yi,whereYiisthecostoftheithpackage.Asmbecomeslarge,'vhatdoyouobserve?3.9.4[PROBLEMS3.9.5TheZipf(ri,n=]_)randomvariableXintroducedinProblem3.3.12isoftenusedtoinodelthe"popularity"ofacollectionofnobjects.Forexample,aWebservercandeliveroneofnWebpages.Thepagesarenumberedsuchthatthepage1isthemostrequestedpage,page2isthesecondmostrequestedpage,andsoon.Ifpagekisrequested,thenX=k.Toreduceexternalnet\vorktraffic,anISPgatewaycachescopiesofthekmostpopularpages.Calculate,asafunctionofnfor1<n,<1000,ho'vlargekmustbetoensurethatthecachecandeliverapage\Vithprobability0.75.3.9.6GeneratenindependentsamplesofthePoisson(5)randomvariableY.ForeachyESy,letn,(y)denotethenumberoftimesthatywasobserved.Thusl:yESy'n(y)=n,andthere]ativefrequencyofyisR(y)=n,(y)/n,.ComparetherelativefrequencyofyagainstPy(y)byplottingR(y)andPy(y)onthesamegraphasfunctionsofyforn=100,n,=1000andri117=10,000.Howlargeshouldn,betohavereasonableagreement?3.9.7TesttheconvergenceofTheorem3.8.l·use3.9.8theresultofProblem3.4.4andtheRandomSampleAlgorithmonPage102towriteal\IIATLABfunctionk=geometricrv(p,m)thatgeneratesm,samplesofageometric(p)randomvariable.3.9.9Findn*,thesmallestvalueofriforwhichthefunctionpoissonpmf(n,n)sho,vninExample3.37reportsanerror.Whatisthesourceoftheerror?\i\friteafunctionbigpoissonpmf(alpha,n)thatcalculatespoissonpmf(n,n)forvaluesofn,muchlargerthann,*.Hint:ForaPoisson(n)randomvariableK,PK(k)=exp(-a+kln(n)-~ln(i)).[ContinuousRandomVariables4.1ContinuousSampleSpaceArandorr1variableXisco'ntin,11,o'usiftherangeSxconsistsofoneorrnoreintervals.Foreacr1x;ESx)P[X=x;]=0.Untilnovv)viehavestudieddiscreterandorn·variables.Bydefir1ition,tr1erangeofadiscreterandomvariableisacountablesetofnurnbers.Thischapterar1al}rzesrandornvariablesthatrangeo·vercontint1oussetsofnl1mbers.Acontir1t1oussetofnurnbers,sornetirr1esreferredtoasaniriter'val)contair1salloftherealnumbersbetweentvvolirnits.Marl}'experirnentsleadtorandorn·variablesvvitharar1gethatisacontinuousinterval.Exarr1plesincluderr1easl1ringT)thearrivaltirr1eofaparticle(Sr={tlO<t<oo});rneasl1ring·v)thevoltageacrossaresistor(Sv={vi-oo<v<oo});andrr1east1ringthephaseangleAofasinusoidalradiov.rave(SA={alO[4.1CONTINUOUSSAMPLESPACE119ismicroscopicallysrnall(ontheorderof10-15),andtheprobabilityofaprecise9:00arrivaliszero.Onewa}'tot11inkaboutcontinl10t1srandornvariablesist11atthearnov,n,tofprobabihtyinanintervalgetssrr1allerandsmallerastheir1tervalshrinks.Thisisliket11emassinacontir1uousvolume.E·venthoughan}'finite·volumehassorr1ernass,thereisnornassatasinglepoint.Inphysics,v.reanal}rzethissituationbyreferringtodensitiesofmatter.Sirnilarl}',vverefertoprobabilityden,sityfun,ction,stodescribeprobabilitiesrelatedtocor1tinuousrandomvariables.T11enextsectionintroducestheseideasforrnallyb}'describinganexperirnentinvvhicht11esamplespacecor1tair1sallr1urnbersbetvveenzeroandone.Inrnanypracticalapplicatior1sofprobability,vveencour1teruniforrnrandorr1·variables.Therangeofauniforrr1randorr1·variableisaninterV'a.lv.rithfinitelirr1its.Theprobabilityrnodelofaur1iformrandorn·variablestatesthatanytv.roir1tervalsofequalsizevvithintherangehaveeql1alprobability.Tointroducerr1anyconceptsofcontinl1ousrandornvariables,v.revvillreferfrequentlytoauniforrnrar1dornvariablev.rithlimits0and1.}v'Iostcornputerlar1gl1agesincludearandornnumbergenerator.Inl\IIATLAB,t11isist11erandfunctionintroducedinCl1apter1.Theserandornnl1rnbergeneratorsproduceasequer1ceofpseudorar1domnt1mbersthatapproxirr1atethepropertiesofoutcornesofrepeatedtrialsofanexperirner1tv.rithaprobabilityrnodelthatisacontinuousuniformrar1domvariable.Inthefollov.ringexarr1ple,\Veexaminethisrandorr1·variablebydefininganexperirnentinvvl1ichtheprocedureistospinapointerinacircleofcirc11mferenceonerneter.T11ismodelisver}'sirr1ilartothemodelofthephaseangleofthesigr1althatarri·vesattheradioreceiverofacellulartelephone.Ir1steadofapointer\vitl1stoppingpointsthatcanbean}rvvherebetweer10and1rneter,thephaseanglecanha,reany\raluebetween0and27rradians.Byreferringtothespir1nir1gpointerint11eexamplesint11iscl1apter,vvearri,reatrr1atl1ernaticalexpressionsthatillustratethernainpropertiesofcontinuousrandornvariables.Theforrr1l1lasthatarisefrornanalyzingphasear1glesincornrnunicationsengineeringrnodelsha,refactorsof27rthatdonotappearir1theexarr1plesinthischapter.Exarnple4.1definesthesarnplespaceofthepointerexperimentanddemonstratesthatalloutcorr1esha\reprobabilit}'zero.---Example4.1--Supposewehaveawheelofcircumferenceonemeterandwemarkapointontheperimeteratthetopofthewheel.InthecenterofthewheelisaradiaIpointerthatwespin.Afterspinningthepointer,wemeasurethedistance,Xmeters,aroundthecircumferenceofthewheelgoingclockwisefromthemarkedpointtothepointerpositionasshowninFigure4.1.Clearly,0<X<l.Also,itisreasonabletobelievethatifthespinishardenough,thepointerisjustasIikelytoarriveatanypartofthecircleasatanyother.Foragivenx;,whatistheprobabilityP[X=x;]?Thisproblemissurprisinglydifficult.However,giventhatwehavedevelopedmethodsfordiscreterandomvariablesinChapter3,areasonableapproachistofindadiscreteapproximationtoX.AsshownontherightsideofFigure4.1,wecanmarktheperimeterwithn,equal-lengtharcsnumbered1ton,andlet·ydenotethenumber[120CHAPTER4CONTINUOUSRANDOMVARIABLESI••Figure4.1Therandompointerondiskofcircurnference1.ofthearcinwhichthepointerstops.YisadiscreterandomvariablewithrangeSy={1,2,...,ri}.Sinceallpartsofthewheelareequallylikely,allarcshavethesameprobability.ThusthePMFof·yisPy(y)l/n,y=1,2,...,n,,=0(4.1)otherwise.FromthewheelontherightsideofFigure4.1,wecandeducethatifX=x;,thenY=jn,x;l,wherethenotationIalisdefinedasthesmaIlestintegergreaterthanorequaltoa.Notethattheevent{X=x}c{Y=jn,xl},whichimpliesthatP[X=x;]<P['Y=In,xl]=l.(4.2)'nWeobservethisistruenomatterhowfinelywedivideupthewheel.TofindP[X=x),weconsiderlargerandlargervaluesofr1,.Asn,increases,thearcsonthecircledecreaseinsize,approachingasinglepoint.Theprobabilityofthepointerarrivinginanyparticulararcdecreasesuntilwehaveinthelimit,ln-+ooriP[X=x)<lirr1P[Y=Frixl)=lirr1n-+oo=0.(4.3)ThisdemonstratesthatP[X=x]<0.ThefirstaxiomofprobabilitystatesthatP[X=x;)rel="nofollow">0.Therefore,P[X=x)=0.Thisistrueregardlessoftheoutcome,x.Itfollowsthateveryoutcomehasprobabilityzero.Justasint11edisct1ssionoft11eprofessorarrivinginclass,sirnilarreasoningcanbeappliedtootherexperimentstosho-vvthatforanycontinuousrar1domvariable,theprobabilityofanyir1dividualoutcomeiszero.Thisisafundamentallydifferentsituationthantheone-vveer1counteredinOllrstl1dyofdiscreterandornvariables.Clearl}'aprobabilitymassfunctiondefinedinterrr1sofprobabilitiesofindi·vidualoutcorneshasnorr1eaninginthiscontext.Foracor1tinuousrandorr1variable,their1terestingprobabilitiesapplytointervals.[4.24.2THECUMULATIVEDISTRIBUTIONFUNCTION121TheCumulativeDistributionFunctionTheCDFFx(x)isaprobabilitymodelforanyrandorrivariable.TheCDFFx(x)isacontinuousf\1nctionifandorilyifXisacor1tinuousrandorr1variable.Exarnple4.1shov.rsthat"''henXisacontinuousrandorrivariable,P(X=x]=0forxESx.ThisirnpliesthatvvhenXiscontiriuol1s,itisirnpossibletodefineaprobabilit}'rnassfurictionPx(;i;).Oritheotherliand,vvevvillseethatthecurnulativedistributionft1nction,Fx(x)inDefinition3.10,isaveryusef\11probabilit}'rnodelforacontiriuousrandorn\rariable.·vverepeattliedefinitionhere.Definition4.1CumulativeDistributionFunction(CDF)Thecumulativedistributionfunction(GDF)ofran,dornvariableXisFx(::c)=P(X<J";].Theke}'propertiesoftheCDF,describedinTheorerri3.2andTheorem3.3,applytoallrandorri\rariables.Graphsofallcurriulati·vedistributionfunctiorisstartatzeroontlieleftaridendatorieontlieright.Allarenondecreasing,a.nd,rnostirriportaritly,theprobabilitythattherandorrivariableisinaniritervalisthedifferenceintheCDFeva1t1atedattheeridsoftheinterval..----Theorem4.Ir---Forariyran,dornvariableX;{a)Fx(-oo)=0{c)P[x1<X<x;2]={b)Fx(oo)=1Fx(x2)-Fx(x1)AlthoughtliesepropertiesapplytoanyCDF,tliereisoneirriporta.ntdifferericebetvveeritheCDFofadiscreterandorrivariablearidtheCDFofacontinuousraridorri\rariable.Recallthatforadiscreteraridorri·variableX,Fx(J";)liaszeroslopee\reryv.rhereexceptatvaluesofx;withnonzeroprobability.Atthesepoirits,thefunctionhasadiscor1tiriuityintlieforrnofajl1rripofrnagnitudePx(x).Bycontrast,thedefiningpropertyofacontiriuousrandom\rariableXisthatFx(x)isacoritinuousfunctionofX.Definition4.2ContinuousRandomVariableXisacontinuousrandomvariableiftheCDFFx(x;)isacon,t'iriv,ov,sfv,n,ction,.-=Example4.2Inthewheel-spinningexperimentofExample4.1,findtheCDFofX.[122CHAPTER4CONTINUOUSRANDOMVARIABLESWebeginbyobservingthatanyoutcomexEBx=(0,1).ThisimpliesthatFx(x)=0forx<0,andFx(:i;)=1for;i;>1.TofindtheCDFforxbetween0and1weconsidertheevent{X<;i;},withx;growingfrom0to1.EacheventcorrespondstoanarconthecircleinFigure4.1.Thearcissmallwhenx~0anditincludesnearlythewholecirclewhenx;~1.Fx(x)=P[X<x;]istheprobabilitythatthepointerstopssomewhereinthearc.Thisprobabilitygrowsfrom0to1asthearcincreasestoincludethewholecircle.Givenourassumptionthatthepointerhasnopreferredstoppingplaces,itisreasonabletoexpecttheprobabilitytogrowinproportiontothefractionofthecircleoccupiedbythearcX<x.Thisfractionissimplyx;.Tobemoreformal,wecanrefertoFigure4.1andnotethatwiththecircledividedinton,arcs,{Y<frix;l-1}C{X<;i;}C{Y<fn,xl}.(4.4)Therefore,theprobabilitiesofthethreeeventsarerelatedby(4.5)NotethatYisadiscreterandomvariablewithCDFFy(y)=ThusforxE(0,1)0y<0,k/n,(k-1)/n,1.(4.6)andforalln,,wehave1rT/,Xl---<Fx(x)<.frixl'n,(4.7)1iInProblem4.2.3,weaskthereadertoverifythatlirr1n-+CX)fn,xl/n,thatasri---+oo,bothfractionsapproachx.TheCDFofXis=x.Thisimplies1Fx(x;)0.5Fx(x;)000.51=0x<0,xO1.(4.8)xQuiz4.2Thecurnulativedistribt1tionfunctionoft11erandorr1variableYis0Fy(y)=y<0,y/4Oy>4.SketchtheCDFof·yandcalculatethefollov.rir1gprobabilities:(4.9)[4.3PROBABILITYDENSITYFUNCTION123P2Figure4.2ThegraphofanarbitraryCDFFx(x).(a)P[Y<-1](b)P[Y<1](c)P[2(d)P[Y>1.5]4.3ProbabilityDensityFunctionLiketheCDF,thePDFf'x(x)isaprobabilitymodelforacontinuo11srandornvariableX.fx(x;)isthederi·vativeoftheCDF.ItisproportionaltotheprobabilitythatXisclosetox.TheslopeoftheCDFcontainst11err1ostinterestingir1forrr1ationaboutacontir1uousrar1dorr1variable.Tl1eslopeatan:ypointxindicatestheprobabilitythatXisn,ear:i;.Tounderstandthisir1tuitively,considerthegraphofaCDFFx(:i;)giveninFigure4.2.Theorern4.l(c)statesthattheprobabilitythatXisint11eintervalofvvidt11~totherightofx1is(4.10)Noteir1Fig11re4.2thatthisislessthantheprobabilityoftheintervalofwidt11~totherightofx2,(4.11)Thecomparisonrr1akessensebecausebothintervals11avet11esarr1elength.Ifvveredt1ce~tofocusot1rattentiononoutcornesnearerandnearertox1andx2,bot11probabilitiesgetsmaller.Hovvever,theirrelativevaluesstilldependont11eaverageslopeofFx(x)atthetwopoints.Thisisapparentifv.rerewriteEq11atior1(4.10)ir1theforrn(4.12)Heret11efractionontherightsideistheaverageslope,andEquation(4.12)statesthattheprobabilitythata,randornvariableisinanintervalnearx1istheaverage[124CHAPTER4CONTINUOUSRANDOMVARIABLESf'x(x)-5Figure4.35ThePDFofthen1oden1receivervoltageX.slopeO\rertheintervaltirnesthelengthoftheir1terv·a.l.B:ydefinition,thelimitofthea\rerageslopeas.6.-+0isthederi,rativeofFx(;r;)e\raluatedat~D1.WeconcludefromthediscussionleadingtoEquatior1(4.12)thattheslopeoftheCDFinaregionnearanynurnberx;isar1indicatoroftheprobabilityofobservingtherandomvariableXnearx;.Justast11earr1ountofrnatterinasmallvolumeisthedensityofthemattertimesthesizeofvolurne,thearr1ountofprobabilit:yinasrr1allregionistheslopeoftheCDFtirnesthesizeoftheregion.Thisleadstothetermprobabilityderisity,defir1edastheslopeoftheCDF.---Definition4.3ProbabilityDensityFunction(PDF)Theprobabilitydensityfunction(PDF)ofaco'ntin,?J,O'USran,dornvariableXisj.x(x.)=dFx(x).dxThisdefinitiondisplaysthecor1\rer1tionalnotationforaPDF.Thenarr1eoft11efunctionisalowercasef'\x.rithasubscriptthatisthenarneoft11erandorr1variable.AswiththePMFandtheCDF,theargt1rr1er1tisadl1mrrl}'variable:fx(x),fx(v,),andf'x(-)areallt11esarr1ePDF.ThePDFisacompleteprobabilityrnodelofacor1tint1ousrandorr1\rariable.vV11ilet11ereareotherft1nctionsthatalsoprovidecornpleterr1odels(theCDFandthemornentgeneratingf\1nctionthatviestudyinC11apter9),thePDFisthernostt1seful.Onereasor1fort11isist11atthegraphofthePDFprovidesagoodindicationofthelikelyvaluesofobservations.-=-Example4.3Figure4.3depictsthePDFofarandomvariableXthatdescribesthevoltageatthereceiverinamodem.WhatareprobablevaluesofX?NotethattherearetwoplaceswherethePDFhashighvaluesandthatitislowelsewhere.ThePDFindicatesthattherandomvariableislikelytobenear-5V(correspondingtothesymbol0transmitted)andnear+5V(correspondingtoa1transmitted).Valuesfarfrom±5V(duetostrongdistortion)arepossiblebutmuchlesslikely.AnotherreasonthePDFisthemostusefulprobabilitymodelisthatitpla}'Sa[4.3PROBABILITYDENSITYFUNCTION125keyroleincalct1latingt11eexpectedvalueofacontinuousrandorrivariable,thesubjectofthenextsection.IrnportaritpropertiesofthePDFfollovvdirectl}'frornDefinition4.3aridthepropertiesoftheCDF.----Theorem4.2---Foracon,tin,uov,sran,dornvariableXv1ithPDFf'x(x;)){a)fx(x)>0forallx,{c)1:Jx(x)dx1'=J{b)Fx(x)=x(u)du,=1.ProofThefirststatementistruebecauseFx(:i;)isanondecreasingfunctionofxandthereforeitsderivative,fx(.r,),isnonnegative.Thesecondfactfollo,vsdirectlyfromthedefinitionoffx(x)andthefactthatFx(-oo)=0.ThethirdstatementfollowsfromthesecondoneandTheorem4.1(b).GiventhesepropertiesoftliePDF,viecanpro\rethenexttheorerri,vvhichrelatesthePDFtotheprobabilitiesofevents.===Theorem4.3P[x1<X<x2]=~"'fx(x)l:i;ldx.ProofFromTheorem4.l(c)andTheorem4.2(b),P[:i;1l.I~'--·~0.5010fx(x)=0.50O---===InExample4.4,wefoundthatthestoppingpointwasauniformrandomvariablewithPDFfx(x)=Xofthespinningwheelexperiment1Oa.Expressior1sthataresynonyrnous"''ithXisa'Un,iforrnran,dorn'/JariableareXisun,iforrnlydistrib'utedandXhasa'U'niforrndistribution,,IfXisauniforrnrar1dom·variablethereisar1equalprobabilit}'offindinganoutcomex;inanyintervaloflength~<b-awithinSx=[a,b).·\"!\fecanuseTheorerr14.2(b),Theorerr14.4,andT11eorerr14.5toderivethefollovvingpropertiesofallniforrnrandorn·variable.[4.5FAMILIESOFCONTINUOUSRANDOMVARIABLES133Theorem4.6·--.....,IfXisa?J,riiforrn(a,b)ran,dorn'/Jariable)(0Fx(x;)=•TheGDFofXis(x;-a)/(b-a)1xb.E(X]=(b+a)/2.•Theexpected'ual'/j,eofXisVar(X]=(b-a)2/12.•The'/Jarian,ceofXisExample4.11Thephaseangle,8,ofthesignalattheinputtoamodemisuniformlydistributedbetween0and27rradians.WhatarethePDF,CDF,expectedvalue,andvarianceofi::::::==8?Fromtheproblemstatement,weidentifytheparametersoftheuniform(a,b)randomvariableasa=0andb=27r.ThereforethePDFandCDFof8are1·e(e)=1/(27r)0<()<27r.0otherwise,-TheexpectedvalueisE[8](27r)2/12=7r2/3rad2.Ib/2Fe(B)=0()<0,B/(27r)10<x;<27r,x>27r.(4.37)7rradians,andthevarianceisVar(8]Therelatior1shipbetv.reent11efarr1ilyofdiscreteuniforrnrar1domvariablesandt11efarr1ilyofcontir1uousuniformrandornvariablesisfairlydirect.Thefollov.rir1gtheorerr1expressestherelatior1sl1ipforrnally.---Theorem4.7LetXbea'IJ,rl,iforrn(a,b)ran,dorn'/Jariable;vJhereaaridbarebothin,tegers.LetK=IXl.Then,Kisadiscrete?J,n,iforrn(a+1,b)ran,dorn'/Jariable.I:rl=k}=ProofRecallthatforanyx,followsthattheevent{Kisthesmallestintegergreaterthanorequaltox.It{k-1<::e<k}.Therefore,P[I<=k]=PK(k)={kPx(x)dx={l/(b-a)Jk-10k=a~l,a+2,...,b,(4.38)other,v1se.ThisexpressionforPK(k)conformstoDefinition3.8ofadiscreteuniform(a+1b)PMF.Thecontinuousrelati·vesofthefarr1ilyofgeornetricrar1dom·variables,Definition3.5,arethernembersoft11efarr1ilyofexpon,en,tialran,dorn'/Jariables.[134CHAPTER4CONTINUOUSRANDOMVARIABLES==;;...Definition4.6ExponentialRandomVariableXisa,n,exponential(,A)randomvariableifthePDFofXisfx(x)=x>O'other'uJise,01JJherethepararneterA>0.Example4.12TheprobabilitythatatelephonecalllastsnomorethantminutesisoftenmodeledasanexponentialCDF.1...-------Fr(t)0.501-e-t/3t>0.otherwise.0-Fr(t)=~-___........_-5___05ItWhatisthePDFofthedurationinminutesofatelephoneconversation?Whatistheprobabilitythataconversationwilllastbetween2and4minutes?WefindthePDFofTbytakingthederivativeoftheCDF:0.4--------.fr(t)_dFT(t)_fT(t)-0.20...____-5___..____0__,5-dt(1/3)e-t/3t>00otherwisetFromDefinition4.6,werecognizethatTisanexponential(,A=1/3)randomvariable.Theprobabilitythatacalllastsbetween2and4minutesis(4.39)Example4.13InExample4.12,whatisE[T],theexpecteddurationofatelephonecall?WhatarethevarianceandstandarddeviationofT?Whatistheprobabilitythatacalldurationiswithin±1standarddeviationoftheexpectedcallduration?UsingthePDFf'r(t)inExample4.12,wecalculatetheexpecteddurationofacall:E[T)=Jootfr(t)dt=-()()100t-1e-t/303dt.(4.40)Integrationbyparts(AppendixB,MathFactB.10)yieldsE[T]=-te-t/3000+100e-t/03dt=3minutes.(4.41)[4.5FAMILIESOFCONTINUOUSRANDOMVARIABLES135Tocalculatethevariance,webeginwiththesecondmomentofT:E[T2]=j(X)t2fr(t)rlt=-CX){CX)t21e-t/3rlt.3(4.42)loAgainintegratingbyparts,wehaveE[T2]=-t2e-t/3CX)+f(X)(2t)e-t/3rlt=oloWiththeknowledgethatE[T]=3,weobservethatE[T2]=6E[T]=18and2f(X)te-t/3rlt.(4.43)lo.f0CX)te-t/3rlt=3E[T]=9.ThusVar[T]=E[T2]-(E[T])=18-32=9minutes2.2(4.44)Thestandarddeviationisay=jVar[T]=3minutes.Theprobabilitythatthecalldurationiswithin1standarddeviationoftheexpectedvalueisP[O<T<6]=Fr(6)-Fr(O)=1-e-2=0.865(4.45)ToderivegeneralexpressionsfortheCDF,theexpectedvalue,andthevarianceofar1exponentialrar1domvariable,weapplyT11eorerr14.2(b),Theorern4.4,andTheorern4.5totheexpor1entialPDFinDefinition4.6.--·Theorem4.8:---===IfXisariex;pon,en,tial(;\)ran,dorn'variable;Fx(x)•TheGDFofXis•Theex;pectedvalv,eofXis•Thevarian,ceofXis1=e-.A~r;J;>0,other·1uise.0E[X]=l/;\.Var[X]=1/;\2.Thefollovvingtheoremshowstherelations11ipbetweenthefarr1ilyofexponentialrandorr1variablesandthefarnilyofgeometricrandornvariables.Theorem4.9IfXisan,ex;pon,en,tial(;\)ran,dornvariable,then,Kro:ndornvariablev1ith'[J=1-e->-.IXlProofAsintheTheorem4.7proof,thedefinitionofJ{impliesPK(k)isageornetric(IJ)=P[k-1<X<k].ReferringtotheCDFofXinTheorem4.8,weobservePK(k)=Fx(k)-F'E(k-1){~-.X(k-t)_e-.>..kk=1,2,...otherwise,={(e-->-)k-1(1-e->-)0k=1)2,...other\vise.(4.46)[136CHAPTER4CONTINUOUSRANDOMVARIABLESIfweletp=1-e->-,v;.rehavek=1)2,...otherwise,(4.47)\VhichconformstoDefinition3.5ofageometric(p)randomvariablewithp=1-e->-.Example4.14PhonecompanyAcharges$0.15perminutefortelephonecalls.Foranyfractionofaminuteattheendofacall,theychargeforafullminute.PhoneCompanyBalsocharges$0.15perminute.However,PhoneCompanyBcalculatesitschargebasedontheexactdurationofacaII.IfT,thedurationofaca11inminutes,isanexponentiaI(.\=1/3)randomvariable,whataretheexpectedrevenuespercallE[RA]andE[RB]forcompaniesAandB?BecauseTisanexponentialrandomvariable,wehaveinTheorem4.8(andinExample4.13)E[T)=1/.\=3minutespercall.Therefore,forphonecompanyB,whichchargesfortheexactdurationofacall,E(RB)=0.15E[T)=$0.45percall.(4.48)CompanyA,bycontrast,collects$0.15IT1foracallofdurationTminutes.Theorem4.9statesthatK=!TlisageometricrandomvariablewithparameterI>=1-e-1/3.Therefore,theexpectedrevenueforCompanyAisE[RA)=0.15E[K)=0.15/p=(0.15)(3.53)=$0.529percall.(4.49)InTheorern9.9,"''es11ovvt11atthesumofasetofir1dependentidenticallydistributedexponentialrandorr1variablesisar1Erlan,grar1domvariable.Definition4.7ErlangRandomVariableXisan,Erlang(n,,.\)ran,dornvariableifthePDFofXis(n,-1)!fx(;i;)=0x>0.-Iother'tuise,1JJherethepararneterA>0)an,dthepararnetern,>1isan,in,teger.Thepararr1eterri,isofter1calledtheorderofanErlar1grandorr1va1iable.Problern4.5.16outlinesaproceduretoverifythattheintegraloftheErlar1gPDFo·verall;i;is1.TheErlang('n,=1,.\)rar1dom·variableisidenticaltotheexponential(.\)rar1domvariable.Justastheexponer1tial(.\)randorr1variableisrelatedtot11e[4.5ProcedureFAMILIESOFCONTINUOUSRANDOMVARIABLESObservation,Parameter(s)RandomvariableJ\/IonitorXisthefirsttimecustornerarrivalsintervalinwl1ichatone-rr1inuteor1eormorecustornersarriveintervalsContint1ouslyTisthetimethatmonitorthefirstcustornercustornerarrivalsarrivesJ\/Ionitort11ecustornerarrivalsatone-rr1inuteintervalsContint1ouslyrnonitorcustornerarrivalsJ\/Ionitort11earrivalofcustornersforT=50mint1tes.Yisthefifthintervalwithor1eorrr1orearrivals·visthearrivaltimeofthefiftl1custornerNisthenumberofct1stornerswhoarriveir1T=50rnint1tes.JJ=0.095probabilit}'ofor1eorrnorearrivalsinaone-rninuteinterval1/A=10minutesistheexpectedarrivaltirneofthefirstcustornerJJ=0.095probabilit}'ofor1eorrnorearrivalsinaone-rninuteinterval1/A=10minutesistheexpectedarrivaltirneofthefirstcustornera=AT=5ist11eaveragen11rr1berofarrivalsin50rnir1utes137ProbabilityModelXrvGeorr1etric(0.095:E(X]--l/'[J=10.5one-min11teintervalsTrvExponential(0.1),E[T]=10rr1inutes.YrvPascal(5,0.095),E[.Y]=5/IJ=52.5one-minuteintervalsVrvErlang(5,0.1)E[V]=50minutes.NrvPoissor1(5),E[J\T]=5customers.Table4.1Fiveprobabilitymodelsalldescribingthesan1epatternofarrivalsatthePhonesn1artstore.'rheexpectedarrivalrateis.A=0.1customers/n1inute.Whenwen1onitorarrivalsindiscreteone-n1inuteintervals,theprobabilityvveobserveanonemptyinterval(v.rithoneorn1orearrivals)isp=1-e->.=0.095.geornetric(1-e->-)randorr1·variable,theErlang(n,,.\)continuousrandorr1·variableisrelatedtothePascal(n,,1-e->-)discreterandomvariable.===Theorem4.10IfXisan,Erla/ng(ri,A)ran,dornvariable,then,'n,ri{b)Var[X]=-:\2.(a)E[X]=.\'B}'corr1paringTheorern4.8andTheorerr14.10,vieseeforX,ar1Erlang(ri,.\)randorr1variable,aridY,a,r1exponential(A)rar1dom·variable,t11atE[X]=n,E['Y]andVar[X]=riVar[Y].Ir1thefollovvingtheorern,"''ecar1alsoconnectErlangandPoissonrandorr1variables.[138CHAPTER4CONTINUOUSRANDOMVARIABLES==;;;;Theorem4.11-iiiiiiiiiiiiiLetIFx(x)=1-FK>-x(ri-1)=10-'\:"'n-1(>.:i;)ke->-xLtk=Ok!>0,theGDFofan,>0·x;-'other111ise.Problerr14.5.18outlinesa.proofofTheorerr14.11.Theorerr14.11statesthatt11eprobabilitythattheErlang(n,,A)randornvariableis<xistheprobabilit:ythatthePoisson(AX)random·variableis>n,becausethesuminTheorerr14.11istheCDFofthePoisson(Ax)rar1dom·variablee·valuatedatn,-1.Therr1athernaticalrelationsl1ipsbetweer1t11egeornetric,Pascal,exponential,Erlang,andPoissonrandom·variablesderi·vefrorr1thevvidely-usedPoisson,processrnodelforarriv·a.lsofCl1stornerstoaservicefacility.Formaldefinitionsandtheoremsfort11ePoissonprocessappearinSectior113.4.Thearrivingcustomerscar1be,forexarnple,shoppersatthePhonesmartstore,packetsatanInterr1etrouter,orrequeststoa\i\Tebserver.Inthismodel,ther1l1rr1berofcustornersthatarriveir1aT-rninutetirneperiodisaPoisson(AT)rar1dom·variable.Undercontinuousrnonitoring,thetimetha,twewaitforonearri·valisanexponential(A)rar1domvariableandthetirne\rvewaitforn,arri\ralsisanErlang(n,,A)randorn\rariable.Ont11eotherhand,wher1vvemonitorarrivalsindiscreteone-rninuteinter\rals,thenurnberofinter\ralswe\r..raituntil"''eobserveanonemptyinterval(vvithoneorrr1orearri\rals)isageornetric(p=1-e->.)random\rariablearidther1l1rnberofinter\rals"''e\r..raitfor'nr1onerr1ptyintervalsisaPascal(n,,[J)randomvariable.Table4.1surnmarizesthesepropertiesforexperirnentst11atrr1onitorcustorr1erarrivalstothePhonesrnartstore.Quiz4.5ContinuOllSrar1domvaria,bleX11asE[X]=3andVar[X]=9.Fir1dthePDF,fx(x),if(a)Xisanexponentialrandorr1\rar(b)Xisacontinuousuniforrr1randorn\rariable.iable,(c)XisanErlangrar1dornvariable.4.6GaussianRandomVariablesThefamil}'ofGal1ssianrandomvariablesappearsinmorepracticalapplicationsthar1anyotherfarnily.ThegraphofaGaussianPDFisabell-shapedc11rve.Bell-shapedc'urvesappearinrnanyapplicationsofprobabilit}'tl1eory.Theprobabilityrnodelsintheseapplicationsarernernbersofthefarr1ilyofGav,ssian,ran,dorn[4.60.8139GAUSSIANRANDOMVARIABLES0.8I'0.60.6f'x(x;)f'x(x;)0.40.40.20.2\)0'---------____;-_____.-20426-2.x(a)µ,=2,(J=0246x1/2(b)µ=2,(J=2Figure4.5Tv.roexamplesofaGaussianrandon1variableX\:vithexpectedvaluep,andstandarddeviationa.variables.Chapter9containsarnatherr1aticalexplanationfortheprevalenceofGaussianrandorr1variablesir1modelsofpracticalphenomer1a.Becausetheyoccursofrequentlyinpractice,Gaussianrar1dom·variablesaresornetirr1esreferredtoasriorrnalrandorr1·variables.Definition4.8GaussianRandomVariableXisaGa?J,SS'ta'n(µ,,(J)ran,dornvariable'ifthePDFofX'is1JJherethepararneterµ,can,bean,yreal'nv,rnberan,dtheJJararneter(J>0.~v1ar1ystatisticstextst1set:henotationXisN[µ,,(J2]asshorthandforXisaGa'tJ,ssiari(11,,(J)ran,dornvariable.Inthisnotation,theNdenotesn,orrnal.Thegraphoff'x(x)hasabellshape,"''herethecer1terofthebellisx=µ,and(Jreflectsthewidt11ofthebell.If(Jissrr1a.ll,t11ebellisnarTOV\',vvithahigh,pointypeak.If(Jislarge,thebelliswide,"''ithalo"'''fiatpeak.(T11eheig11tofthepeakis1/(J,/2;.)Figt1re4.5containstwoexarnplesofGat1ssianPDFsv.rith/J,=2.InFigure4.5(a),(J=0.5,andinFigt1re4.5(b),(J=2.Ofcourse,theareaunderan}'Gaussiar1PDF00is.[00f'x(x;)dx=1.Furtherrnore,thepararnetersofthePDFaretheexpectedvalueofXandt11estand[-trddeviationofX.Theorem4.12IfXisaGa'tJ,SSian,(11,,(J)raridornvariable,E[X]=11,TheproofofTheorem4.12,aswellastheproofthattheareaunderaGaussianPDFis1,err1ploysintegrationbypartsandothercalc11h1stechniques.WeleavethemasanexerciseforthereaderinProblerr14.6.13.[140CHAPTER4CONTINUOUSRANDOMVARIABLESItisimpossibletoexpresstheintegralofaGat1ssiar1PDFbetweennonir1fir1itelirnitsasaf\1nctionthata.ppearsonrnostscientificcalculators.Instead,v..ret1suall}'findintegralsoft11eGaussianPDFbyreferringtotables,suchasTable4.2(p.143),thathavebeenobtainedbyr1urr1ericalintegration.Tolearnhov..rtollSethistable,v..reintroducethefollovvingimportantpropert}'ofGaussianrandorr1·variables.Theorem4.13IfXisGa/ussian,(µ,,a),·y=aX+bisGaussiari(aµ+b,aa).Thetheorernstatesthata.n}'lineartrar1sforrnatior1ofaGaussianrandorr1variableproducesanotherGaussianrandomvariable.T11ist11eorerr1allov.rsustorelatethepropertiesofanarbitraryGaussianrar1domvariabletothepropertiesofaspecificrandorr1·variable.---Definition4.9---StandardNormalRandomVariableThestandardnormalrandomvariableZistheGa/tJ,ssian,(0,1)raridornvar1iable.Theorern4.12indicatest11atE[Z]=0andVa.r[Z]=1.ThetablesthatwellSetofindir1tegralsofGat1ssianPDFscontain·valuesofFz(z),theCDFofZ.\Veintroducet11especialnota.tion(z)forthisfunction.Definition4.10StandardNormalCDFTheGDFofthestan,dardn,orrnalran,dornvariableZis(z)=1y2;jz-2e-'U,/2du.CX)Givenatableofval11esof(z),vveusethefollov..ringtheorerr1tofir1dprobabilitiesofaGaussianrandorr1variablevvithpararnetersµ,anda.Theorem4.14IfXisaGa'ussian,(µ,,a)raridornvariable,theGDFofXisxFx(:i;)=(aµ,).TheprobabilitythatXisin,thein,terval(a,b]isInllSingthistheorem,vietransforrr1valuesofaGaussianrandornvariable,X,toequi·valentval11esofthestandardnorrnalrar1dornvariable,Z.Forasarnple·valuexoftherandomvariableX,thecorrespondingsarnpleva.lt1eofZisZ=x-µ,(4.50)[4.60.50.50.40.40.3.--...~~.--...~~........,0.20.10-4141GAUSSIANRANDOMVARIABLES0.30.21-0.1()0-2z24-4-2-zx(a)Figure4.60xz24(b)Syn1metrypropertiesoftheGaussian(0,1)PDF.Notet11atzisdirnensionless.Itrepresentsxasanumberofstandardde·viationsrelativetot11eexpected,ralueofX.Table4.2presents===Example4.15;==::::aSupposeyourscoreonatestisx=46,asamplevalueoftheGaussian(61,10)randomvariable.Expressyourtestscoreasasamplevalueofthestandardnormalrandomvariable,Z.Equation(4.50)indicatesthatz=(46-61)/10=-1.5.Thereforeyourscoreis1.5standarddeviationslessthantheexpectedvalue.TofindprobabilitiesofGa.t1ssianrar1domvariables,-vvet1sethevaluesof(z)presentedir1Table4.2.Notethatthistablecor1tainsentriesonl}'forz>0.Fornegativevalt1esofz,\Veapplythefollo-vvingpropertyof===Theorem4.15=1-Fig11re4.6displa}'Sthesymrnetrypropertiesof(z).Bothgrap11scontainthestandardnorrr1alPDF.InFigure4.6(a),theshadedareaunderthePDFis(z).SincetheareaunderthePDFequals1,the11nshadedareat1ndert11ePDFis1-(z).InFigt1re4.6(b),theshadedareaor1therightis1-(-z).Thisgra.p11dernor1stratesthat(-z)=1-Example4.16IfXistheGaussian(61,10)randomvariable,whatisP[X<46)?[142CHAPTER4CONTINUOUSRANDOMVARIABLESApplyingTheorem4.14,Theorem4.15,andtheresultofExample4.15,wehaveP[X<46]=Fx(46)=(4.51)Thissuggeststhatifyourtestscoreis1.5standarddeviationsbelowtheexpectedvalue,youareinthelowest6.7%ofthepopulationoftesttakers.Example4.17'IfXisaGaussian(11,=61,0"=10)randomvariable,whatisP[514.75.Therefore,if~>4.75,orr>45,theprobabilityoferrorislessthan10-6.Although10C-6)seemsaverysmallnumber,mostpracticalopticalfibertransmissionsystemshaveconsiderablylowerbinaryerrorrates.KeepinmindthatQ(z)istheprobabilitythataGaussianrandomvariableexceedsitsexpected·vall1eb:ymorethanzstandardde·viations.WecanobservefrornTable4.3,Q(3)=0.0013.ThismearisthattheprobabilitytliataGaussianrandorrivariableisrriorethanthreestandardde·viationsabo·veitsexpectedvalueisapproxirnatel:yorieinatliousand.Incoriversationwerefertotheevent{X-11,x>3ax}asathree-sigrnae'ven,t.Itisunlikelytooccur.Table4.3indicatestliattlieprobabilityofa5aeventisontheorderof10-7.Quiz4.6XistheGaussian(0,1)randorn·variableand·yistheGaussian(0,2)raridornvariable.SketchthePDFsfx(x)aridj'.y(y)onthesameaxesandfind:(a)P[-1<X<1],(b)P[-1<·y<1],(c)P[X>3.5],4.7(d)P[Y>3.5].DeltaFunctions,MixedRandomVariablesXisarnixedrandornvariableifSxhasatleastonesarnplevaluevvithnorizeroprobabilit}'(likeadiscreterandomvariable)andalsohassamplevaluesthatcoveraninterval(likeacontinuousrandornvariable).TliePDFofarriixedrandomvariablecontainsfinitenorizerovaluesariddeltah1rictiorismultipliedb}'probabilities.Thusfar,ouranalysisofcontinuousraridornvariablesparallelsot1raritilysisofdiscreterandornvariablesinCliapter2.Becal1SeofthedifferentnatureofdiscreteandcontinuOllSrandornvariables,\VerepresenttheprobabilityrnodelofadiscreterandomvariableasaPNIFandwerepresenttheprobabilitymodelofacontin11ousrandornvariableasaPDF.Theseft1nctionsareimportaritbecausetlieyenablet1stocalct1lateprobabilitiesofe\rentsandpararrietersofprobabilityrnodels(st1chastheexpectedvalt1eandthe\rariarice).CalculationscontairiingaP1!{Firivol\resurns.ThecorrespondingcalculationsforaPDFcontairiintegrals.Inthissection,weintroducetheuriitirnpt1lsef\1nction6(1';)asarriathematicaltoolthaturiitestheanalysesofdiscreteandcontinuousrandomvariables.Thet1nitirnpulse,oftencalledthedeltafun,ction,,allowsustousethesarneformulastodescribecalculatiorisvvithbothtypesofrandorrivariables.Itdoesnotalterthecalculations,itjl1stprovidesanevvnotationfordescribingthern.Tliisisespeciallyconvenientvvhenvverefertoarn'ixedra/ndornvariable,vvhichliaspropertiesofbothcontinl1ousanddiscretera.ndorrivariables.[146CHAPTER4CONTINUOUSRANDOMVARIABLES6e(x)'•••E---161---E181E=--•••••4E--lI')'I-1/2E=1-1/2-xFigure4.7AsE---+0,df.(x)approachesthedeltafunction6(::r).Foreachthecurveofd€(x)equals1.E,theareaunderThedeltafurictionisriotcompletelyrespectablerriathematicall}'becal1seitiszeroe·verywhereexceptatoriepoint,andthereitisinfinite.Thusatitsmostinterestingpointithasnonurriericalvalueatall.Wliile6(x)issomewhatdisreputable,itisextremel}'useful.TlierearevariOllSdefinitionsofthedeltafunction.AllofthemsharethekeypropertypresentedinTheorem4.16.Hereistliedefiriitionadoptedinthisbook.Definition4.12UnitImpulse(Delta)FunctionLet1/E-E/2<X<E/2,0other111ise.TheunitimpulsefunctionisTherriatherriaticalproblerriwitliDefiriitiori4.12isthatde(x)hasnolimitat;x:=0.AsindicatedinFigure4.7,de(O)justgetsbiggerandbiggerasE---+0.AlthoughthisrriakesDefinition4.12sorriev.rliatunsatisfactor}',tlieusefltlpropertiesofthedeltafunctionarereadil}'derrionstratedvvhen6(x)isapproxirnatedbyde(x;)forverysrriallE.Wenov.rpresentsomepropertiesofthedeltafunction.'\'Vestatethesepropertiesastlieorerriseventhoughtheyareriottheorerrisinthetlsualsenseofthistextbecausewecannotprovetherri.Insteadoftlieorerriproofs,werefertode(x)forsmallvaluesofEtoindicatevvhyt:hepropertieshold.AlthoughdE(O)blovvsupasE---+0,theareaunderdE(x)istlieintegralJOOJE/21dE(x)dx=-dx=1.(4.54)-E/2E-ooThatis,theareauriderde.(x)isalways1,riorriatterliovvsrriallthevalueofconcludethattheareaunder6(x;)isalso1:Joo-oo6(x;)dx=1.E.'\'Ve(4.55)[4.7DELTAFUNCTIONS,MIXEDRANDOMVARIABLES147Thisresultisaspecialcaseofthefollo\vir1gpropertyoft11edeltafunction.--·Theorem4.16---==::::::iForan,ycon,tinv,o?J,sj?J,riction,g(x);Joog(;i;)=g(x;o)·-()()Theorem4.16isoftencalledthesiftin,gpropertyoft11edeltafur1ction.\Vecar1seethatEquation(4.55)isaspecialcaseofthesiftingpropertyforg(x)=1aridxo=0.Tounderstar1dTheorern4.16>cor1sidertheintegral(4.56)Ont11erightside,wehavethea\reragevalueofg(x)overtheir1terval[x;0-E/2,x0+E/2].AsE---+0,thisa\reragevalueml1stcon\rergetog(x0).Thedeltafunctior1hasacloseconnectiontotheunitstepfunction.UnitStepFunctionDefinition4.13Theunitstepfunctionisv,(x;)=x<0)1x>o.0===Theorem4.17i.........:::==j'=O(v)dv='u(x).To11nderstandTheorem4.17,v.reobservethatforanyx;>0,\VecanchooseE<2xsothatj:r:de(v)dv=l.(4.57)-()()Th11sforan}'x':I0,inthelirr1itasE---+O>J:r:oodc(v)rlv=v,(x).Notethatweha\renotyetconsideredx=0.Infact>itisnotcornpletel}'clearvvhatthe\ralue0ofJ001/2>or1.vVehaveadoptedtheconver1tionthat.f000[148CHAPTER4CONTINUOUSRANDOMVARIABLESTheorem4.17allowsustowriteo(x)d'n(x).dx=(4.58)Eqt1ation(4.58)ernbodiesacertainkindofconsistencyinitsinconsistency.Thatis,o(x)doesnotreallyexistatx;=0.Sirnilarly,thederi·vativeof'u(x;)doesriotreall}'existat;i;=0.However,Equation(4.58)allovvsustouse6(x)todefineageneralizedPDFthatappliestodiscreterandorr1variablesasv.rellastocontir1uo11srandorr1variables.Cor1sidertheCDFofa.discreterandomvariable,X.Recallthatitisconstar1teverywhereexceptatpoir1tsXiEBx,v.r11ereit11asjt1rr1psofheightPx(x;i)·Usir1gthedefinitionoftheunitstepft1nction,wecanwritet11eCDFofXasFx(x)L=Px(;i;i)v,(x;-x,i)·(4.59);i:iESxFrornDefir1itior14.3,wetaket11ederivativeofFx(x)tofindt11ePDFfx(x).ReferringtoEquation(4.58),tl1ePDFofthediscreterandomvariableXisf'x(x;)L=Px(xi)o(;i;-x;i)·(4.60)xiESxWhenthePDFir1cludesdeltafur1ctionsoftheforrno(x;-x;i),v.resaythereisanirr1pulseatxi.When\VegraphaPDFfx(x;)t11atcontainsar1irr1pulseatxi,-vvedrawaverticalarro-vvlabeledbytheconstantthatmultipliestheirr1pulse.Wedra\veacharro\vrepresentinga.nirr1pulseatthesarr1eheightbecausethePDFisal-vvaysinfiniteat

eachsuchpoint.Forexarnple,t11egraphoff'x(x)frorr1Equation(4.60)lSfx(x)'.I•••x,x,Usingdeltafunctior1sint11ePDF,\Vecanappl}'theforrr1t1lasinthischaptertoallrandornvariables.Inthecaseofdiscreterandomvariables,theseformulasareequi\ralenttotheonespresentedinC11apter3.Forexample,ifXisadiscreterandorr1variable,Defir1itior14.4becorr1esE[X]=1=Lx-CX)Px(x;)O(x-Xi)dx.(4.61)~r:iESxB}'-vvritingtheintegralofthesurnasasumofir1tegralsandusingthesiftingpropert}'oft:hedeltafur1ction,E[X]=LxiESx1=xPx(x;)O(x-x,i)dx;-CX)=LxiESxxiPx(x;i),(4.62)[4.7DELTAFUNCTIONS,MIXEDRANDOMVARIABLES149ThePl\!lF,CDF,andPDFofthediscreterandomvariableY.Figure4.8vvhichisDefir1itior13.13.---Example4.191--SupposeYtakesonthevalues1,2,3withequalprobability.ThePMFandthecorrespondingCDFof·yarey<1,1/31<y<2,2/32<y<3,y>3.10Py(y)1/3y=1,2,3,=Fy(y)otherwise,0=(4.63)Usingtheunitstepfunctionv,(y),wecanwriteFy(y)morecompactlyas111Fy(y)=-v,(y-1)+-v,(y-2)+-'u,(y-3).3.33(4.64)ThePDFofYisJy(y)=dFy(y)dy111=-o(y-1)+-o(y-2)+-o(y-3).333(4.65)WeseethatthediscreterandomvariableYcanberepresentedgraphicallyeitherbyaPMFPy(y)withbarsaty=1,2,3,byaCDFwithjumpsaty=1,2,3,orbyaPDFfy(y)withimpulsesaty=1,2,3.ThesethreerepresentationsareshowninFigure4.8.TheexpectedvalueofYcanbecalculatedeitherbysummingoverthePMFPy(y)orintegratingoverthePDFfy(y).UsingthePDF,wehaveE[Y]==J:Jyjy(y)dyoo;_Yo(y-1)dy+3=1/3+2/3+1=2.-ooJoo;_Yo(y-oo32)dy+Joo-Yo(y-3)dy-oo3(4.66)[150CHAPTER4CONTINUOUSRANDOMVARIABLESWhenFx(x)hasadiscontir1uityatx,-vvellSeFx(x+)andFx(:i;-)todenotethellpperandlov.rerlimitsat;i;.Thatis,Fx(:i;-)=limFx(:i;-h),(4.67)h-+O+Usingthisnotation,-vveca,nsaythatiftheCDFFx(x)11asajumpat:i;0,then1·x(x)hasanirr1pulseat;i;0v.reigl1tedb}'theheightofthediscontir1uityFx(xci)-Fx(x0).~=Example4.20FortherandomvariableYofExample4.19,(4.68)---Theorem4.18Foraran,dorn'/JariableX,1J.Jeha'/Jethefollo'uJin,geq11,i'IJalen,tstaternen,ts:(a)P[X=:i;o]=q{b)Px(xo)=q(c)Fx(xci)-Fx(:i;0)=q{d)1·x(xo)=qb(O)InExarr1ple4.19,v.resa,v.rthatj'.y(y)consistsofaseriesofimpulses.T11evalueoffy(y)iseither0oroo.Bycontrast,thePDFofacontinuousrandom·variablehasnonzero,finitevaluesoverintervalsof;i;.Ir1thenextexarnple,-vveencour1terarandorr1variablethathascontinuOllSpartsandirnpulses.---Definition4.14----MixedRandomVariableXisamixedran,dornvariable'ifan,don,lyiffx(;i;)co'ntain,sbothirnp11,lsesan,drion,zero7fin,ite'/Jalv,es.Example4.21--0bservesomeonedialingatelephoneandrecordthedurationofthecall.Inasimplemodeloftheexperiment,1/3ofthecallsneverbegineitherbecausenooneanswersorthelineisbusy.Thedurationofthesecallsis0minutes.Otherwise,withprobability2/3,acalldurationisuniformlydistributedbetween0and3minutes.LetYdenotethecallduration.FindtheCDFFy(y),thePDFfy(y),andtheexpectedvalueE(Y].LetAdenotetheeventthatthephonewasanswered.P(A]=2/3andP(Ac]=1/3.Since·y>0,weknowthatfory<0,Fy(y)=0.Similarly,weknowthatfory>3,Fy(y)=l.For0<y<3,weapplythelawoftotalprobabilitytowrite(4.69)[4.7DELTAFUNCTIONS,MIXEDRANDOMVARIABLESWhenAcoccurs,Y=0,sothatfor0<y<3,P[Y<ylAc]=l.occurs,thecalldurationisuniformlydistributedover[O>3],sothatfor0P[Y<ylA]=y/3.So,for0<y<3,Py(y)=(1/3)(1)+(2/3)(y/3)=151WhenA<y<3,1/3+2y/9.(4.70)ThecompleteCDFof·yis1y<0,1/3+2y/90<y<3,1y>3.0Py(y)Fy(y)=1/300213yConsequently,thecorrespondingPDFfy(y)is1/3'•fy(y)2/9j'y(y)00I23=b(y)/3+2/90<y<3,otherwise.0yForthemixedrandomvariable·y,itiseasiesttocalculateE[Y]usingthePDF:JCX)E[Y]=-CX)1y-b(y)dy313+2-ydy09=02y23+--:-92=1minute.(4.71)0InExarr1ple4.21,weseethatwit.hprobability1/3,Yresernblesadiscreterandorr1variable;otherv.rise,Ybe11aveslikeacontinuOllSrandomvariable.Thisbeha·viorisreflectedint11eirnpl1lseinthePDFof·y.Inmanypracticalapplicationsofprobabilit}'>rr1ixedrar1dornvariablesariseasfunctionsofcontinl1ousrandorr1·variables.Electroniccircuitsperforrr1rnanyofthesefl1nctions.Exarnple6.8inSection6.3givesoneexarnple.Beforegoir1ganyfurther,-vvere·view-vvhatwehavelearnedabo11trar1dornvariables.Foranyrandorr1v.a.riableX,•Xal-vva}'ShasaCDFFx(x)=P[X<:c].•IfFx(x;)ispiecewisefiat-vvithdiscontinuousjurnps,thenXisdiscrete.•IfFx(x)isacontinuousfunctior1>ther1Xiscontir1uous.•IfFx(x;)isapiece\visecontinuousfunctionv.rithdiscontinuities,ther1Xisrr1ixed.•WhenXisdiscreteorrr1ixed,thePDFfunctions.fx(x)containsor1eorrnoredelta[152CHAPTER4CONTINUOUSRANDOMVARIABLESQuiz4.7Thecurnulativedistribt1tionfunctionofrar1domvariableXisFx(x)=<0;i;(x+1)/4-1<;i;<1,1;i;-1,>1.Sketcht11eCDFandfindthefollovving:(a)P[X<1](b)P[X<1](c)P[X=1](d)t11ePDFfx(x)4.8(4.72)l\ilATLABBuilt-inJ\IIATLABfunctior1s,eitheraloneorv.rithadditionalcode,canbet1sedtocalculatePDFsandCDFsofseveralrandornvariablefarr1ilies.Therandandrandnfunctionssimulateexperirnentsthatgeneratesarnplevaluesofcontinuot1st1niform(0,1)rar1domvariablesandGaussiar1(0,1)randorr1·variables,respectively.ProbabilityFunctionsTable4.4describesJ\!I.A.TLABft1nctionsrelatedtofourfamiliesofcontinuot1srandornvariablesintroducedinthischapter:t1niform,exponential,Erlang,andGaussian.Thefunctior1scalculatedirectlyt11eCDFsaridPDFsoft1niformandexponentialrandorr1·variables.ForErlangandGaussiar1rar1dornvariables,thePDFscar1becalculateddirectlybuttheCDFsrequirenumericalintegration.ForErlangrandornvariables,erlangcdfusesT11eorern4.11.FortheGaussianCDF,vveuset11ebuilt-ir1MATLABerrorfunctionfunctionF=erlangcdf(n,lambda,x)F=1.0-poissoncdf(lambda*x,n-1);erf(x)2r::.='.J;12e-v,du.(4.73)+-1erf(-x·)22J2)(4.74)y7r0ItisrelatedtotheGaussiar1CDFbyvvhichishovvweirr1plerner1tt11eJ\11.A.TLABfunctionphi(x).IneachfunctionXrn]'.ThepdffunctiondescriptioninTable4.4,xdenotesavectorx=[x;1outputisavectorysuc11thatYi=f'x(x;i).Thecdffunctionoutputisa·vectory[4.8MATLABRandomVariableMATLABFunctionFunctionOutputXUniforrn(a,b)Yi=f'x(x;i)Yi=Fx(xi)X=[X1Yi=f'x(x;i)Yi=Fx(x,i)X=[X1Yi=f'x(x;i)Yi=Fx(xi)X=[X1Yi=f'x(x;i)Yi=Fx(xi)X=[X1XExponential(.A)XErlang(ri,A)XGal1ssiar1(11,,0"2)Table4.4y=uniformpdf(a,b,x)y=uniformcdf(a,b,x)x=uniformrv(a,b,m)y=exponentialpdf(lambda,x)y=exponentialcdf(lambda,x)x=exponentialrv(lambda,m)y=erlangpdf(n,lambda,x)y=erlangcdf(n,lambda,x)x=erlangrv(n,lambda,m)y=gausspdf(mu,sigma,x)y=gausscdf(mu,sigma,x)x=gaussrv(mu,sigma,m)l\/IATLAB1531Xrn]1X1n]1Xrn]1Xrn]functionsforcontinuousrandon1variables.suchthaty,i=Fx(xi)·T11ervf\1nctionoutputisavectorX=[X1Xrn]'suchthateachX,iisasan1ple·valueoftherandom·variableX.Ifm,=1,thentheoutputisasinglesarnpleva1t1eofrandorn·variableX.RandomSamplesNov.rt11atwehaveintroducedcontinuot1srandorr1·variables,v.;recansayt11a.tthebt1ilt-inf\1nctiony=rand(m,n)isM.A.TLAB'sapproxirr1ationtoauniforrn(0,1)ra.r1dorr1variable.Itisar1approxirr1ationfortworeasons.First,randproducespset1dora.ndornr1urr1bers;thent1rr1bersseernrandorr1bt1ta.reactuallytheot1tpt1tofadeterrr1inistica.lgorithrri.Second,randprodl1cesadot1bleprecisionfioatir1gpoir1tnt1rr1ber,representedinthecomputerby64bits.ThusJ\IIATL.A.Bdistir1gt1ishesr10rr1orethan264uniquedot1bleprecisionfioa.tir1gpointnt1rr1bers.B}'corr1parision,thereareuncour1tablyinfir1iterealnumbersin(0,1).E·vent11oughrandisriotrandorr1anddoesnothaveacontinuOllSrange,wecar1forallpracticalpl1rposesuseitasasourceofindependentsample·valuesoftheuniforrr1(0,1)randorr1variable.Weha;vealread}'ernployedt11erandfur1ctiontogeneraterandorr1sarr1plesoft1niform(0,1)randorr1variables.Corrver1iently,MATL.A.Balsoir1cludesthebt1ilt-ir1functionrandntoger1eraterar1domsarr1plesofstandardr1ormalrandom·variables.T11usgaussrvgeneratesGaussian(µ,,O")randornvariablesb}'stretcl1ingandsl1iftir1gstandardr1ormalrandornvariables.Forothercontinl1ousrar1dornvariables,we11sea.techr1iquedescribedinTheorern6.5thattra.nsforrr1sat1niform(0,1)rar1dom·variableUintoothertypesofra.ndorr1·variables.Thisisexplainedir1theMATL.A.BsectionofChapter6.functionx=gaussrv(mu,sigma,m)x=mu+(sigma*randn(m,1));[154CHAPTER4CONTINUOUSRANDOMVARIABLES==--Quiz4.8__,,;;=::::::iWriteaMATLABf\1nctior1t=t2rv(m)thatger1eratesrnsarr1plesofarar1dornvariablewitr1thePDFf'rir>2(t)asgi·veninExarnple7.10.Difficulty:Easy4.2.1ThecumulativedistributionfunctionofrandomvariableXisFx(x)x<-1,-l1.0(x+1)/2=1Moderate4.2.4+DifficultTheCDFofrandomvariableWis(o<-5,-5<71)<-3'-3<71J<3,71)w±."i8Fw('111)=l414+3(w-3)81(a)WhatisP[X35.>1/2]?(b)WhatisP[-1/2<X<3/4]?(d)\i\fhatisthevalueofP[Xa,suchthat4.2.2TheCDFofthecontinuousrandomvariableVis0=c(v+5)21(a)\!\ThatisP[W<4]?(b)\i\fhatisP[-2<W<2]?(c)vVhatisIrel="nofollow">[IXI<1/2]?Fv(v)ExpertsOnlyv<-5,-5<v<7,v>7.(c)\tVhatisP[W>OJ?(d)WhatisthevalueofasuchthatI>[vV<a]=1/2?4.3.1TherandomvariableXhasprobabilitydensityfunctionexfx(x)={00<x<2,otherwise.·usethePDFtofind(a)Whatisc?(b)WhatisP[V(a)theconstantc,>4]?(b)P[O<X<1],(c)\i\1hatisP[-3a]=2/3?(c)P[-1/2<a,suchthat4.2.3Inthisproblem,weverifythatlimn-tooIn,xl/n,=x.(a)Verifythatn;i;<fnxl<n,x+1.(b)Usepart(a)tosho'vlimn-toorn,xl/n,=x<1;2J,(d)theCDFF:x:(x).4.3.2ThecumulativedistributionfunctionofrandomvariableXis(0Fx(x)=~(1;+l)/2l1x<-1,-l1.x.(c)Useasimilarargumenttoshowthatlimn-tooln,xJ/ri=x.FindthePDFfx(x)ofX.4.3.3FindthePDFfu(7J,)oftherandomvariableUinProblem4.2.4.[PROBLEMS4.3.4Foraconstantparametera>0,aRayleighrandomvariableXhasPDF()-_j·.xx22axe{2-ax/24.4.3x>0,otherwise.0155RandomvariableXhasCDFFx(x)=0x<0,x/20<x<2,1>2.x\i\fhatistheCDFofX?4.3.5RandomvariableXhasaPDFoftheformfx(x)=~f1(::r)+~f2(x),'vherej·.1(x)-_{.()_j2x-0<X<2,othervvise,C10{c2e-x04.3.6Forconstantsaandb,randomvariableXhasPDFa:i;2+b:i;TheprobabilitydensityfunctionofrandomvariableYis4.4.4fy(y)0o<y~2,={yI20other,v1se.WhatareE[Y]andVar[Y]?ThectunulativedistributionfunctionoftherandomvariableYis4.4.50<1;<1,other,vise.\i\fhatconditionsonaandbarenecessaryandsufficienttoguaranteethatfx(x)isavalidPDF?4.4.1(b)\iVhatisVar[X]?x>0,otherwise.\i\fhatconditionsmustc1andc2satisfysothatfx(x)isavalidPDF?.()_{jxx-(a)\!\ThatisE[X]?y(y+1)/2-l1yfX(,,,'")-1<x<3,other,vise.-1,>1.\iVhatareE[Y]and·var[Y]?ThecumulativedistributionfunctionofrandomvariableVis4.4.6RandomvariableXhasPDF1/4-{0<0Fv(v)=0(v+5)2I1441v<-5,-5<v<7)v>7.DefinetherandomvariableYbyYh(X)=X2.(a)\!\ThatareE[V]and·var['!]?(a)FindE[X]andVar[X].(b)\iVhatisE[V3]?(b)Findh(E[X])andE[h(X)].(c)FindE[Y]andVar[Y].ThecumulativedistributionfunctionofrandomvariableUis4.4.74.4.2LetXbeacontinuousrandomvariable'vithPDF<-5,-5<'IJ,<-3,-371,·()--{1/8l0issomeconstant.Theaircraftiscorrectlyidentifiedbytheradarifthereflectedpo,veroftheaircraftislargerthanitsaveragevalue.\tVhatistheprobabilityP[C)thatanaircraftiscorrectlyidentified?Yisanexponentialrandomvariable'vithvarianceVar[Y)=25.4.5.4(a)WhatisthePDFofY?Uisazeromeancontinuousuniformrandomvariable.\tVhatisP[U2<\far[U))?4.5.8UisacontinuousuniformrandomvariablesuchthatE[U]=10andP[U>12)=1/4.WhatisP[U<9)?4.5.94.5.10Xisacontinuousuniform(-5,5)randomvariable.(a)\t\fhatisthePDFfx(1;)?(b)\tVhatistheCDFFx(x)?(c)\t\fhatisE[X)?(d)\tVhatisE[X5)?(e)WhatisE[ex)?4.5.11•Xisacontinuousunifor1n(-a,a)randomvariable.FindP[IXI4.5.12•<Var[X]).XisauniformrandomvariablewithexpectedvalueJJ,x=7andvarianceVar[X)=3.WhatisthePDFofX?TheprobabilitydensityfunctionofrandomvariableXis4.5.13.fx(x)=2{(1/2)e-x/20x>0,other,vise.(b)WhatisE[Y)?(c)\i\fhatisP[Y>5)?ThetimedelayY(inmilliseconds)thatyourcomputerneedstoconnecttoanaccesspointisanexponentialrandomvariable.4.5.5>E[Y]).(b)FindP[Y>2E[Y]).(a)FindP[Y4.5.6XisanErlang(ri,>.)randomvar-iable'vithparameter,\=1/3andexpectedvalueE[X)=15.(a)Whatisthevalueoftheparametern,?(b)WhatisthePDFofX?(c)\i\fhatisVar[X)?(a)\t\fhatisP[l<X<2)?(b)\tVhatisFx(x),thecumulativedistributionfunctionofX?(c)WhatisE[X],theexpectedvalueofX?(d)\t\fhatisVar[X],thevarianceofX?Verifyparts(b)and(c)ofTheorem4.6bydirectlycalculatingtheexpectedvalueandvarianceofauniformrandomvariablewithparametersa<b.4.5.14Long-distancecallingplanAoffersflat-rateserviceat10centsperminute.CallingplanBcharges99centsforeverycallunder20minutes;forcallsover20minutes,thechargeis99centsforthefirst20minutesplus10centsforeveryadditional4.5.15[PROBLEMSminute.(Notethattheseplansmeasureyourcalldurationexactly,\vithoutroundingtothenextminuteorevensecond.)Ifyourlong-distancecallshaveexponentialvalue1/Ahasn,thmomentdistributionwithexpectedvalueTminutes,Hint:\Vhichplanoffersalo,verexpectedcostpercall?pendixB,MathFactB.10).4.5.16InthisproblemweverifythatanErlang(ri,.A)PDFintegratesto1.Lettheintegralofthen,thorderErlangPDFbedenotedby_1=In-0.An:::en-1e->-x(n,_1)I.dx.First,showdirectlythattheErlangl=>DF\Vithn,=1integratesto1byverifyingthatIl=1.Second,useintegrationbyparts(AppendixB,lV.IathFactB.10)tosho'vthatIn=In-1·4.5.17CalculatethekthmomentE[Xk]ofanErlang(ri,A)randomvariableX.UseyourresulttoverifyTheorem4.10.Hint:RememberthattheErlang(ri+k,.A)PDFintegratesto1.(b)Applyintegrationbyparts(seeAppendixB,MathFactB.10)tothisintegraltosho'vthatforx>0,(.\;:i;)n-le-.AxFxn(x)=Fxn-i(x)-(n_l)!(c)UsethefactthatFx1(x)=1-e-.Axforx>0toverifytheclaimofTheorem4.11.4.5.19ProvebyinductionthatanexponentialrandomvariableX\vithexpectedUseintegrationbyparts(Ap-4.5.20ThisproblemoutlinesthestepsneededtoshowthatanonnegativecontinuousrandomvariableXhasexpectedvalueE[X]=fo00[1-Fx(x)]dx.(a)l:0,showthat00rP[X>r]<[xfx(x)dx.(b)Usepart(a)toarguethatifE[X]<oo,thenlimrP[Xr-+oo>r]=0.(c)Nowuseintegrationbyparts(AppendixB,lVIathFactB.10)toevaluate1=[1-Fx(x)]dx.4.5.18Inthisproblem,weoutlinetheproofofTheorem4.11.(a)LetXndenoteanErlang(ri,A)randomvariable.UsethedefinitionoftheErlangPDFtoshowthatforanyx>0,15704.6.1Thepeaktemperature7.,,asmeasuredindegreesFahrenheit,onaJulydayinNewJerseyistheGaussian(85,10)randomvariable.WhatisP[T>100],P[T<60],andP[70<T<100]?4.6.2\i\!hatisthePDFofZ,thestandardnormalrandomvariable?4.6.3Findeachprobability.(a)VisaGaussian(JL=0,a=2)randomvariable.FindI>[V>4].(b)vVisaGaussian(µ=2,a=5)randomvariable.WhatisP[W<2]?(c)l:65].[158CHAPTER4CONTINUOUSRANDOMVARIABLES4.6.4Ineachofthefollowingcases,YisaGaussianrandomvariable.Findtheexpectedvalueµ=E[Y].(a)Yhasstandarddeviationa=10andP[Y<10]=0.933.(b)Yhasstandarddeviationa=10andP[Y<OJ=0.067.(c)YhasstandarddeviationaandP[Y<10]=0.977.(FindJLasafunctionofa.)(d)P[Y>5]=1/2.4.6.5YourinternalbodytemperatureTindegreesFahrenheitisaGaussian(JL=98.6,a=0.4)randomvariable.Intermsofthe100].Doesthismodelseemreasonable?4.6.6ThetemperatureTinthisthermostaticallycontrolledlecturehallisaGaussianrandomvariablewithexpectedvalueJL=68degreesFahrenheit.Inaddition,P[T<66]=0.1587.\i\fhatisthevarianceofT?4.6.7XisaGaussianrandomvariablewithE[X]=0andP[IXI<10]=0.1.\i\fhatisthestandarddeviationax?4.6.8_A..functioncommonlyusedincomm1u1icationstextbooksforthetailprobabilitiesofGaussianrandomvariablesisthecomplementaryerrorfunction,definedaserfc(z)=J;.f,00e-2xdx.Sho'vthatQ(z)=~erfc(72).ofn,yearsfilledwithblackboarderrors,thetotalamountindollarspaidcanbeapproximatedbyaGaussianrandomvariabley,;iwithexpectedvalue40nandvariancelOOn,.\tVhatistheprobabilitythatY20exceeds1000?HowmanyyearsnmusttheprofessorteachinorderthatP[Y'.;i>1000]>0.99?4.6.11Supposethatoutof100millionmenintheUnitedStates,23,000areatleast7feettall.Supposethattheheightsof·u.S.menareindependentGaussianrandomvariableswithaexpectedvalueof5'10".Let1'lequalthenumberofmen'vhoareatleast71611tall.(a)Calculateax,thestandarddeviationoftheheightof·u.s.men.(b)IntermsoftheInthisproblem,weverifythatforx>0,(x)+21erfJ2..()(a)LetYhaveaGaussian(0,1/J2)distributionandshowthatFy(y)=y/_-oo1fy(u)(b)ObservethatZ=(0,1)andsho'vthatd?L=J2yif?(z)=Fz(z)=Fv4.6.9ThepeaktemperatureT,indegreesFahrenheit,onaJulydayinAntarcticaisaGaussianrandomvariable'vithavarianceof225.Withprobability1/2,thetemperature'J'exceeds-75degrees.WhatisP[T>OJ?\i\fhatisP[T<-100]?4.6.10Aprofessorpays25centsforeachblackboarderrormadeinlecturetothestudentwhopointsouttheerror.Inacareer2+erf(y).isGaussian(72).4.6.13Thisproblemoutlinesthestepsneededtosho'vthattheGaussianPDFintegratestounity.ForaGaussian(µ,,a)randomvariablevV,we,villsho'vthatI=1_:f1tv('UJ)d'UJ=1.[PROBLEMS=(a)Usethesubstitutionxshowthat1=100r.;-/_v211('ID-µ)/atoedx.Pe=E[Pe(Y)]=-oo(b)Showthat001_:Q(v12if)fy(y)dy.FindasimpleformulafortheBERPeasafunctionoftheaverageSNR'Y.001/_-oo/_-ooe12=211Theaverageprobabilityofbiterror,alsokno\vnasthebiterrorrateorBER,is.,-x-/2159(x2+y2)/2d:i~dy.(c)Changetopolarcoordinatestoshowthat12=1.4.6.14Attimet=0,thepriceofastockisaconstantkdollars.Attimet>0thepriceofastockisaGaussianrandomvariableX'vithE[X]=kand·var[X]=t.Attimet,aCallOptionatStrikekhasvalueV=(X-k)+,'vheretheoperator(-)+isdefinedas(z)+=max(z,0).(a)FindtheexpectedvalueE[V].(b)Supposeyoucanbuythecalloptionforddollarsattimet=0.Attimet,youcansellthecallforVdollarsandearnaprofit(orlossperhaps)ofR=V-ddollars.LetdodenotethevalueofdsuchthatP[R>O]=1/2.Yourstrategyisthatyoubuytheoptionifd<dosothatyourprobabilityofaprofitisP[R>OJ>1/2.Finddo.(c)Letd1denotethevalueofdsuchthatE[R]=0.01xd.Nowyourstrategyistobuytheoptionifd<d1sothatyourexpectedreturnisatleastonepercentoftheoptioncost.Finddi.(d)Arethestrategies"B11ytheoptionifd0.\iVou]dyoubeinterestedinsellingtheshortstraddle?Areyougettingsomething,namelyE[R']dollars,fornothing?(a)WhatisF-vv('UJ)?4.7.1CDF4.7.7For80%oflectures,ProfessorXarrivesontimeandstartslecturing\vithdelayT=0.\tVhenProfessorXislate,thestartingtimedelay'J'isuniformlydistributedbet,veen0and300seconds.FindtheCDFandPDFofT.LetXbearandomvariable\Vith(0Fx(x)=x/3+1/3x/3+2/31x<-1,-11]andP[X>l].4.7.3Forrando1nvariableXofProblem4.7.2,findfx(1;),E[X],andVar[X].4.7.4XisBernoullirandomvariable\vithexpectedvaluep.\iVhatisthePDFfx(1;)?4.7.5Xisageometricrandomvariable\Vithexpectedvaluel/p.WhatisthePDFfx(x)?4.7.6\tVhenyoumakeaphonecall,thelineisbusy\Vithprobability0.2andnooneanswers\vithprobability0.3.TherandomvariableXdescribestheconversationtime(inminutes)ofaphonecallthatisanswered.XisanexponentialrandomvariablewithE[X]=3minutes.LettherandomvariableWdenotetheconversationtime(inseconds)ofallcalls(W=0whenthelineisbusyorthereisn.oans\ver.)(b)\tVhatisfw('llJ)?(c)\tVhatareE[vV]andVar[vV]?4.7.8\tVithprobability0.7,thetossofanOlympicshot-puttertravelsD=60+Xfeet,'vhereXisanexponentialrandomvariable\vithexpectedvalueµ,=10.Otherwise,\Vithprobability0.3,afouliscommittedbysteppingoutsideoftheshot-putcircleand\VesayD=0.\tVhataretheCDFandPDFofrandomvariableD?4.7.9For70%oflectures,ProfessorYarrivesontime.WhenProfessorYislate,thearrivaltimedelayisacontinuousrandomvariableuniformlydistributedfrom0to10minutes.Yet,assoonasf>rofessorYis5minuteslate,allthestudentsgetupandleave.(ItisunknownifProfessorYstillconductsthelecture.)Ifalecturestarts\vhenI>rofessorYarrivesandal\vaysends80ininutesafterthescheduledstartingtime,'vhatisthePDFofT,thelengthoftimethatthestudentsobservealecture.4.8.1Writeafunctiony=quiz31rv(m)thatproducesm,samplesofrandomvariableYdefinedinQuiz4.2.4.8.2FortheGaussian(0,1)complementaryCDFQ(z),ausefulnumericalapproximationforz>0isAQ(z)=(~ant5n)-z2/2e,wheret=---1---=a4=a21+0.231641888z-0.142248368-0.7265760135a1=0.127414796=0.7107068705a5=0.5307027145.a3[PROBLEMSTocomparethisapproximationtoQ(z)1usel\IIATLABtograph_Q(z)-Q(z))e(zQ(z).161Ifwegeneratealargenumbern,ofsamplesofrandomvariableX,letnidenotethenumberofoccurrencesoftheevent{ill<x<(i+1)fl}.vVewouldexpectthat4.8.3·useexponentialrv.rnandTheorem4.9andto\Vriteal\IIATLABfunctionk=georv(p,rn)thatgeneratesm,samplesofageometric(p)randomvariableJ{.ComparetheresultingalgorithmtothetechniqueemployedinProblem3.9.8forgeornetricrv(p,rn).limni=n--+oorifx(ill)fl,orequivalently,limn:=n--+ooriufx(ill).·usel\IIATLABtoconfirmthis\Vithflfor=0.014.8.4ApplyingEquation(4.14)\vithxreplacedbyillandd1~replacedbyll1weobtain(a)anexponential(.A=1)randomvariableXandfori=0,...,500,P[ill<X(b)aGaussian(3,1)randomvariableXandfori=0,...,600.[MultipleRandomVariablesChapter3andChapter4analyzeexperirr1entsinvvhichanoutcomeisonenl1rr1ber.Begirrr1ingvviththisc11apter,weanalyzeexperirnentsinwhichanoutcorr1eisacollectionofnurnbers.Eachr1urr1berisasarnplevalueofarandorr1variable.T11eprobabilityrr1odelforsucr1anexperirnentcontainst11epropertiesoftheir1dividualrandorr1variablesariditalsocor1tainstherelationshipsamongtherandorn·variables.Chapter3considersonlydiscreterandorr1\rariablesandChapter4considersor1lycontinuousrandornvariables.Thepreser1tchapterconsidersallrandornvariablesbecausea11igl1proportionofthedefinitionsandt11eorerr1sappl}'tobothdiscreteandcontinl1ousrandom\rariables.Hov.rever,justasvvithir1dividualrandomvariables,thedetailsofr1umericalcalculatior1sdependor1v.rhetherrandomvariablesarediscreteorcontinuOllS.Conseql1ently,vvefindthatrnanyforrr1ltlascorneinpairs.Oneforrnula,fordiscreterandorn\rariables,containssums,andtheotherformula,forcontir1uousrandornvariables,contair1sir1tegrals.Ir1thisc11apter,\Vecor1siderexperimentsthatproduceacollectionofrandornvariables,X1,X2,...,Xn,v.rheren,canbean}'ir1teger.Formostofthischapter,v.restlld}''n=2randorr1va,riables:Xarid·y.Apairofrandorr1variablesisenoughtoshov.rt11eimportantcor1ceptsandusefulproblern-sol\ringtechr1iques.Moreo\rer,thedefinitionsaridtheorernsweintroduceforXandYgeneralizeton,randornvariables.Thesegeneralizeddefinitionsappea,rneartheendofthisc11apterinSection5.10.vVealsonotet11atapa,irofrar1domvariablesXandYisthesameasthet\vodirr1ensior1alvector[XYJ'.Sirr1ilarly,therandomvariablesX1,...,Xncar1be1v.rrittenast11en,dirr1er1sior1alvectorX=[X1Xn]•Sincethecorr1ponentsofXarerar1dornvariables,Xiscalledaran,dorn'Uector.ThllSt11ischapterbeginsourstudyofrandorn\rectors.Thissubjectiscontir1uedinChapter8,whichusestechniquesoflinearalgebratodevelopfurtherthepropertiesofrandomvectors.Webeginhere\vitht11edefinitionofFx,y(;i;,y),thejoin,tc'u,rn:tJ,lati'vedistrib11,tion,f11,n,ction,oftvvorandornvariables,ageneralizationoft11eCDFintroducedin162[5.1JOINTCUMULATIVEDISTRIBUTIONFUNCTION163Section3.4aridagairiiriSection4.2.ThejointCDFisacorripleteprobabilit:yrriodelforanyexperimentthatprod11cestv.rorandorrivariables.Ho,vever,itnotveryt1sefulforarialyzingpracticalexperiments.MorellSefulrriodelsarePx,y(x;,y),thejo'iritprobabilityrnassf11,n,ction,fort'\vodiscreterandomvariables,presentedinSections5.2and5.3,andfx,y(x,y),thejoiritprobab,ilityden,sityf'un,ctiorioft'\vocontin11ot1sraridomvariables,presentediriSections5.4and5.5.Sectiori5.7considersft1nctionsoftworar1dornvariablesandexpectations,respectivel:yr.Weextendthedefinitionofindependenteventstodefineindependentraridom·variables.ThesubjectofSection5.9istliespecialcasein'\vhichXand·yareGaussian.Pairsofraridomvariablesappearinawide·varietyofpracticalsituations.AnexarripleisthestrengthofthesignalatacellulartelephonebasestationreceiverCY)andthedistance(X)ofthetelephoriefrorntliebasestation.Anotlierexampleoftvvoraridom·variablesthat\Veencounterallthetirneinourresearcliisthesigrial(X),emittedbyaradiotransrriitter,andtliecorrespondingsignal(Y)tliateventuallyarri·vesatareceiver.Inpractice\Veobserve·y,butvvereall:yrv.ranttoknowX.NoiseanddistortionpreveritusfromobservingXdirectl:yr,andv.reuseaprobabilit:yrrriodeltoestirriateX.Example5.1Wewouldliketomeasurerandomvariabley=X,butweinsteadobservex+z.(5.1)ThenoiseZpreventsusfromperfectlyobservingX.Insomesettings,Zisaninterferingsignal.Inthesimplestsetting,ZisjustnoiseinsidethecircuitryofyourmeasurementdevicethatisunrelatedtoX.Inthiscase,itisappropriatetoassumethatthesignalandnoiseareindependent;thatis,theeventsX=;r;andZ=zareindependent.Thissimplemodelproducesthreerandomvariables,X,YandZ,butanypaircompletelyspecifiestheremainingrandomvariable.Thuswewillseethataprobabilitymodelforthepair(X,Z)orforthepair(X,Y)willbesufficienttoanalyzeexperimentsrelatedtothissystem.5.1JointCumulativeDistributionFunctionThejointCDFFx,Y(x;,y)=P[X<x,Y<y]isacompleteprobabilityrriodelforanypairofraridomvariablesXand·y.Inanexperimenttliatprodt1cesoneraridomvariable,eventsarepointsoriritervalsonaline.Inariexperimentthatleadstotworandorri·variablesXandY,eaclioutcorne(x,y)isapointinaplaneandeventsarepoiritsorareasintheplane.J11stastlieCDFofonerandornvariable,Fx(;r:),istheprobabilityoftheintervaltotheleftofx,tliejointCDFFx,Y(x,y)oft'\vorandorrivariablesistheprobabilit:yroftheareabelowandtotheleftof(x,y).Thisistheinfiniteregionthatinch1destheshadedareainFig11re5.1ande\rerythingbelowaridtotheleftofit.[164CHAPTER5MULTIPLERANDOMVARIABLESy{X7.0Fy(y)=(5.4)ReferringtoTheorem4.6,weseefromEquation(5.4)thatXisacontinuousuniform(5,6)randomvariableand·yisacontinuousuniform(6>7)randomvariable.Inthisexarr1ple,\Veneedtorefertosixdifferentregionsint11ex,yplaneandthreedifferentforrnulastoexpressaprobabilityrr1odelasajointCDF.Section5.4introducesthejointprobabilitydensit}'functionasar1otherrepresentationoftheprobabilitymodelofapairofrandorn·variablesfx,y(x;,y).Forchildrens'agesXandYinExarr1ple5.2>"''e'\villsl10V1rinExample5.6thattheCDFFx,y(x,y)irr1pliesthatt11ejoir1tPDFisthesimpleexpression1·x,Y(x;,y)=15<;r:<6,60otl1erV1rise.(5.5)Togetar1otherideaofthecomplexityofllSingt11ejointCDF,tryprovingt11efollowingt11eorern,whichexpressestheprobabilitythatanoutcorneisir1arectar1gleint11eX,Yplaneintermsoft11ejointCDF.---Theorem5.2'---P[x;1<X<x2,Y1<Y<Y2]=Fx,Y(:r:2,Y2)-Fx,Y(x2,Y1)-Fx,Y(x1,Y2)+Fx,Y(x1,Y1).[166CHAPTER5MULTIPLERANDOMVARIABLESThestepsr1eededtoprovethetheoremareoutlinedinProblern5.1.5.Thetheorerr1sa}'Sthattofindt11eprobabilitythatar1outcomeisinarectangle,itisnecessarytoevaluatethejoir1tCDFatallfourcorners.vVher1t11eprobabilityofinterestcorrespondstoanonrecta,r1gl1lararea,usingthejointCDFisevenmorecomplex.Quiz5.1--=Expresst11efollovvingextrernevaluesofthejoir1tCDFFx,Y(x;,y)asnl1mbersorintermsoftheCDFsFx(x;)andFy(y).(a)Fx,y(-oo,2)(b)Fx,Y(oo,oo)(c)Fx,y(oo,y)(d)Fx,y(oo,-oo)5.2JointProbabilityMassFunctionFordiscreterandomvariablesXandY,thejointPl\IIFPx,y(x,y)istheprobabilitythatX=xarid·y=y.Itisacorr1pleteprobabilit}rmodelforXandY.CorrespondingtothePJ\l[Fofasinglediscreterar1dom·variable,v.reha;veaprobabilityrnassfunctionoftvvo·variables.Definition5.2JointProbabilityMassFunction(PMF)Thejointprobabilitymassfunctionofdiscreteran,dorn'variablesXaridYisPx,Y(x,y)=P[X=x;,Y=y).Forapairofdiscretera,r1domvariables,thejointPMFPx,Y(x,y)isacompleteprobabilityrnodel.Fora,nypairofrealntrrnbers,thePMFistheprobabilityofobservir1gt11eser1urnbers.Ther1otationisconsistentv.riththatofthejointCDF.TheuppercaseSl1bscriptsofP,separatedbyacornma,arethenamesofthetv.rorandorr1variables.Weusually'ivritetheargurnentsofthef\1nctior1ast11elo'ivercaselettersassociatedwitl1therandorn\rariablenames.CorrespondingtoSx,t11erangeofasir1glediscreterandorr1variable,weuset11enotationSx,Ytodenotethesetofpossiblevaluesoft11epair(X,Y).Thatis,Sx,Y={(x;,y)IPx,y(x,y)>O}.(5.6)Keepinrnindt11at{X=x;,·y=y}isaneventinanexperimer1t.Thatis,forthisexperirnent,thereisasetofobservationsthatleadstobothX=x;and·y=y.Foranyxandy,wefindPx,Y(x;,y)byStlrr1rningtheprobabilitiesofalloutcornesoft11eexperirr1entforwhicl1X=xandY=y.[5.2JOINTPROBABILITYMASSFUNCTION167Therearevariousvvaystorepreser1tajointP1!{F.Weuset11reeoft11err1inthefollovvir1gexarr1ple:agraph,alist,andatable.---Example5.3·--Testtwointegratedcircuitsoneaftertheother.Oneachtest,thepossibleoutcomesarea,(accept)andr(reject).Assumethatallcircuitsareacceptablewithprobability0.9andthattheoutcomesofsuccessivetestsareindependent.CountthenumberofacceptablecircuitsXandcountthenumberofsuccessfultestsYbeforeyouobservethefirstreject.(Ifbothtestsaresuccessful,let·y=2.)DrawatreediagramfortheexperimentandfindthejointPMFPx,Y(x,y).0.9a•aaX=2,Y=2•arX=l,Y=la,ra•ra•rrS={aa,,a,r,ra,,rr}.X=l,Y=OrrTheexperimenthasthetreediagramshowntotheleft.ThesamplespaceoftheexperimentisX=O,Y=O(5.7)Observingthetreediagram,wecomputeP[aa]=0.81,P[ar]=0.09,(5.8)P[ra]=0.09,P[rr]=0.01.(5.9)EachoutcomespecifiesapairofvaluesXand·y.Letg(s)bethefunctionthattransformseachoutcomesinthesamplespaceSintothepairofrandomvariables(X,'Y).Theng(aa)=(2,2),g(a,r)=(1,1),g(ra)=(1,0),g(rr)=(0,0).(5.10)Foreachpairofvaluesx,y,Px,Y(x,y)isthesumoftheprobabilitiesoftheoutcomesforwhichX=x;andY=y.Forexample,Px,y(l,1)=P[ar].Pxy(x,y)y=Oy=lx;=0x;=1x;=20.010.09000.090y=2000.81ThejointPMFcanberepresentedbythetableonleft,or,asshownbelow,asasetoflabeledpointsinthex;,yplanewhereeachpointisapossiblevalue(probability>0)ofthepair(x,y),orasasimplelist:y•2•1010.81x=2,y=2,0.09x=1,y=1,.09..._0_1~--·_09_0.81Px,Y(x,y)=_,,_2x0.09x=1,y=0,0.01x=0,y=o.0otherwiseNotethatalloftheprobabilitiesaddt1pto1.Thisreflectsthesecondaxiorr1[168CHAPTER5MULTIPLERANDOMVARIABLESofprobability(Sectior11.3)thatstatesP[S]=l.Usingt11enotationofrandornvariables.wewritethisasILLPx,y(x,y)=l.(5.11):i:ESxyESyAsdefinedinChapter3,t11erangeSxisthesetofallvaluesofXv.rithnonzeroprobabilityandsimilarlyforSy.Itiseasytoseetheroleofthefirstaxiornofprobabilityint11ePMF:Px,Y(x,y)>0forallpairs;r;,y.T11ethirdaxiorr1,whichhastodowiththe11r1ior1ofrn11tt1a.1lyexcl11siveevents,takesustoar1otherirnportantpropert}'ofthejointPMF.Werepresentar1ever1tBasaregionintheX,Yplar1e.Figure5.2s11ovvst\vOexarr1plesofever1ts.\¥evvot1ldliketofindtheprobabilit}'thatthepairofrandorr1variables(X,Y)isir1t11esetB.vVhen(X,Y)EB,wesa}'thee·ventBoccurs.]\/Ioreover,vve\vriteP[B]asashorthandforP[(X,·y)EB].Ther1exttheorerr1saysthatwecanfindP[B]bya.ddingtheprobabilitiesofallpoints(x;,y)thatareinB.Theorem5.3Fordiscreteran,dornvariablesXan,dYan,dan,ysetBin,theX,Yplan,e,theprobabilityofthee'verit{(X,Y)EB}isP[B]=LPx,Y(x,y).('.1;,y)EBThefollowir1gexamplel1sesTheorern5.3.Example5.4ContinuingExample5.3,findtheprobabilityoftheeventBthatX,thenumberofacceptablecircuits,equalsY,thenumberoftestsbeforeobservingthefirstfailure.Mathematically,Bistheevent{Xare=Y}.TheelementsofBwithnonzeroprobabilityBnSx,Y={(0,0),(1,1),(2,2)}.(5.12)Therefore,P[B]=Px,Y(0,0)+Px,Y(1,1)+Px,Y(2,2)=0.01+0.09+0.81=0.91.(5.13)Ifvvevievvx;,ya.stheoutcomeofar1experiment,thenTheorern5.3sirnpl}'saysthattofir1dtheprobabilityofane\rent,vvesumO\reralltheoutcorr1esir1thatevent.Inesser1ce,Theorern5.3isarestaternentofTheorerr11.5intermsofrandornvariablesXandYandjointPMFPx,Y(x,y).[5.3y•••••••••••••••••••169yB={X-•+y-•<9}B={X-t}'<3}••MARGINALPMFx•••••••••••••••••••xFigure5.2SubsetsBofthe(X,Y)plane.Points(X,Y)ESx,Yaren1arkedbybullets.Quiz5.2ThejointPMFPQ,G(q,g)forrar1domvariablesQandGisgi·ver1inthefollovvingtable:P.c(q,g)q=0q=1g=Og=1g=20.060.180.240.040.120.16g=30.120.08Calculatet11efollov.ringprobabilities:(a)P[Q=0)(b)P[Q=G](c)P[G>1)(d)P[G>Q)5.3MarginalPMFFordiscreterandomvariables,thernarginalPMFsPx(x;)andPy(y)areprobabilitymodelsfort11eindividualrandorr1variablesXa.ndYbuttheydonotprovideacornpleteprobabilitymodelforthepairX,Y.InanexperirnentthatproducestworandornvariablesXandY,itisalwa}'Spossibletoconsideroneoftherar1domvariables,Y,aridignoretheotherone,X.Int11iscase,wecar1t1sethemethodsofChapter3toanal}rzetheexperirr1entandderivePy(y),vvhichcontainstheprobabilityrr1odelfortherandorr1variableofinterest.Ont11eotl1er11ar1d,ifvvehavealreadyanalyzedtheexperirnenttoderi·vethejointPJVIFPx,Y(x;,y),itv.ro11ldbecorrvenienttoderivePy(y)fromPx,Y(x,y)vvithoutreexarr1inir1gthedetailsoftheexperirr1er1t.Todoso,'""'evie\vx;,yasthe011tcorneofar1experimentandobservethatPx,y(x,y)istheprobabilit}'ofanoutcorr1e.Moreover,{Y=y}isar1event,sothatPy(y)=P[Y=y)istheprobabilityofar1event.Theorern5.3relatesthe[170CHAPTER5MULTIPLERANDOMVARIABLESprobabilit}'ofaneventtothejointP1!fF.ItimpliesthatwecanfindPy(y)bysurnmingPx,y(x;,y)overallpointsinSx,Ywitl1t11epropertyY=y.Inthesurn,yisaconstant,andeachterrr1correspondstoavalueofxEBx.Sirnilarl}',wecanfindPx(x)b}'surr1mingPx,Y(x,y)o·verallpointsX,Ys11chthatX=x;."\"/Vestatethisrr1atl1err1aticallyinthenexttheorerri.---Theorem5.4Fordiscretera/ndornvariablesXan,dY'UJithjo'iritPM.FPx,Y(x;,y),Px(x;)=LPx,Y(x,y),Py(y)L=yESyPx,Y(x;,y).:r;ESxT11eorem5.4showst1showtoobtaintheprobabilityrnodel(PNIF)ofX,andtheprobabilitymodelof·ygivenaprobabilityrr1odel(jointPMF)ofXandY.vVhenarar1dom·variableXispartofa.r1experirr1entthatproducest-vvora.r1domvariables,wesorr1etirr1esrefertoitsPMFasarnargin,alprobabilityrnassf11,rict'ion,.Thisterminologycomesfrorr1t11ematrixrepresentatior1ofthejoir1tP1!fF.B}'addingrowsandcolurnnsandv.rritingtheresultsintherr1argir1s,-vveobtainthernarginalP1!fFsofXandY.vVeillustratethisbyreferer1cetot11eexperirnentinExample5.3.i::::::==Example5.5y=Oy=1y=2Px.Yx,yx;=0x;=1x;=20.010.090Wenotethatboth00.090000.81InExample5.3,wefoundXandYhavethejointPMFshowninthistable.FindthemarginalPMFsfortherandomvariablesXandY.Xand·yhaverange{O,1,2}.Theorem5.4gives2Px(0)=2LPx,Y(0,y)=0.01Px(l)y=O=LPx,Y(l,y)=0.18(5.14)y=O2Px(2)=LPx,Y(2,y)=0.81Px(x;)=0x;-=J0,1,2(5.15)y=OReferringtothetablerepresentationofPx,Y(x,y),weobservethateachvalueofPx(x)istheresultofaddingalltheentriesinonerowofthetable.Similarly,theformulaforthePMFofYinTheorem5.4,Py(y)=2=xESxPx,Y(x;,y),isthesumofalltheentriesinonecolumnofthetable.WedisplayPx(x)andPy(y)byrewritingthetableandplacingtherowsumsandcolumnsumsinthemargins.Px.y(;r;,y)x;=0.T,--1.T,--2Py(y)y=Oy=l0.010.0900.1000.0900.09y=2Px(x)000.810.810.010.180.81[5.4JOINTPROBABILITYDENSITYFUNCTION171ThusthecolumnintherightmarginshowsPx(x;)andtherowinthebottommarginshowsPy(y).Notethatthesumofalltheentriesinthebottommarginis1andsoisthesumofa11theentriesintherightmargin.ThisissimplyaverificationofTheorem3.l(b),whichstatesthatthePMFofanyrandomvariablemustsumto1.===Quiz5.3~==:::::::1TheprobabilitymassfurictioriPH,B(h)b)forthetv.rorandornvariablesHaridBisgivenintliefollov.ringtable.FindthernargirialPl\/IFsPH(h,)andPB(b).PHB(h,)b)h,=-1h,=0h,=15.4b=000.10.1b=20.400.1b=40.20.10(5.16)JointProbabilityDensityFunctionTherriostl1sef\1lprobabilityrnodelofcontinuousrandomvariablesXarid·yisthejointPDF1·x,y(x)y).ItisageneralizationofthePDFofasiriglerandorrivariable.Definition5.3JointProbabilityDensityFunction(PDF)Thejoin,tPDFofthecor1,tir1/UO'USraridorn'VariablesXan,dYisafv,n,ction,fx,y(x)y)111iththepropertyFx,Y(x;,y)=J'.r;JYfx,Y(v,)'v)dvdv,.-()()-()()GiveriFx,Y(x,y),Definition5.3irripliesthat1·x,Y(x;,y)isaderivativeoftheCDF.===Theorem5.5;-:==2,)_8Fx,Y(x)y).fXY(X,y:.:i:.:i'uxuyForasiriglerandorrivariableX,thePDF1·x(x)isameasureofprobabilityperunitlength.FortvvorandomvariablesXaridY)thejointPDFfx,y(x,y)rneasuresprobabilityperunitarea.Inparticular)frornthedefinitionoftliePDF,P[x<X<x+dx,y<Y<y+dy)=fx,Y(;r;)y)dxdy.(5.17)Definition5.3andTheorem5.5derrioristratetliattliejointCDFFx,Y(x,y)andthejointPDFfx,Y(x;,y)representtliesameprobabilityrriodelforraridom·variablesX[172CHAPTER5MULTIPLERANDOMVARIABLESandY.Intliecaseofonerandomvariable>vvefot1ndinChapter4thatthePDFistypicall}'rrioreusefulforproblerrisol-virig.Thead·vantageisevenstrongerforapairofrandomvariables.c:::==Example5.6UsethejointCDFforchildrens'agesXand·ygiveninExample5.2toderivethejointPDFpresentedinEquation(5.5).ReferringtoEquation(5.3)forthejointCDFFx,y(:i;,y),wemustevaluatethepartialderivative82Fx,y(;r;,y)/f};,r;f}yforeachofthesixregionsspecifiedinEquation(5.3).However,82Fx,y(;r;,y)/8x8yisnonzeroonlyifFx.Y(:i;.y)isafunctionofbothxandy.Inthisexample,onlytheregion{582fxy(:i;,y)=,.[(:i;-5)(y-6))='8x8y88-[:i;-5)-[y-6)=1.ox&y(5.18)Overallotherregions,thejointPDFfx,Y(x,y)iszero.Ofcourse,rioteveryfunctiorif"x,y(;r;>y)isajointPDF.Properties(e)arid(f)ofTheorern5.1fortheCDFFx,Y(x,y)irriplycorrespondingpropertiesforthePDF.~--Theorem5.6AjoiritPDFfx,Y(;i;,y)hasthefollo'uJin,gpropertiescorrespon,d'in,gtofirstan,dsecon,dax;iornsofprobab'ility(seeSection,1.3):(a)fx,y(:i;,y)>0forall(:i;,y);(b)1:1:Jx,Y(x,y)dxdy=1.Givenanexperirrienttl1atproducesapairofcoritin11ousra.ndornvariablesXandY,ane\rentAcorresporidstoaregionoftheX,Yplane.TheprobabilityofAisthedoubleintegraloffx,·y(:i;,y)overtheregionAoftheX>Yplane.===Theorem5.7--Theprobabilitythatthecoritin,uo'usran,dorn'variables(X,Y)arein,Aisp[A]=JJfX,Y(x,y)dxdy.AExample5.7RandomvariablesXandYhavejointPDFc00<x<5,0<y<3,otherwise.FindtheconstantcandP[A)=P[2<X<3,1<Y<3).(5.19)[5.4JOINTPROBABILITYDENSITYFUNCTION173Thelargerectangleinthediagramistheareaofnonzeroprobability.Theorem5.6statesthattheintegralofthejointPDFoverthisrectangleis1:1151=y0A3cdydx=15c.(5.20)0Therefore,c=1/15.ThesmalldarkrectangleinthediagramistheeventA={2<X<3>1<Y<3}.P[A]istheintegralofthePDFoverthisrectangle,whichisrrJ_1211153p[Al=3dvdv,=2I15.(5.21)ThisprobabilitymodelisanexampleofapairofrandomvariablesuniformlydistributedoverarectangleintheX,Yplane.Thefollo-vvingexarr1plederi·vest11eCDFofapairofrandorr1variablesthathasajointPDFthatiseasyto'ivriterr1atherr1atically.Thepurposeoftheextur1pleistointroducetechniquesforanalyzir1garnorecorr1plexprobabilit}'rnodelthantheoneinExample5.7.Typically,'iveextractir1terestingir1forrr1ationfromarnodelb}'integratingt11ePDForafur1ctionofthePDFO\rersorneregionintheX,Yplane.Ir1perforrningthisintegration,t11err1ostdifficulttaskistoider1tifyt11elimits.ThePDFintheexarr1pleisverysimple,justaconstantO\reratriangleint11eX,Yplar1e.However,toevalui1teitsir1tegralovertheregioninFigure5.1-vveneedtoconsiderfivedifferentsituationsdependir1gonthe\raluesof(x;,y).Thesoh1tionoftheexarnpledernonstratesthepointthatt11ePDFisusuallyamoreconciseprobabilityrr1odelthatoffersrnoreinsightsintothenatureofar1experimentthantheCDF.Example5.8FindthejointCDFFx,Y(x;,y)whenXandYhavejointPDFy-1=fx)'(x,y)=2\f'x,Y(x;,y)=20Ootherwise.(5.22)1WecanderivethejointCDFusingDefinition5.3inwhichweintegratethejointPDFfx,Y(x>y)overtheareashowninFigure5.1.ToperformtheintegrationitisextremelyusefultodrawadiagramthatclearlyshowstheareawithnonzeroprobabilityandthentousethediagramtoderivethelimitsoftheintegralinDefinition5.3.Thedifficultywiththisintegralisthatthenatureoftheregionofintegrationdependscriticallyonxandy.Inthisapparentlysimpleexample,therearefivecasestoconsider!ThefivecasesareshowninFigure5.3.First,wenotethatwithx<0ory<0,thetriangleiscompletelyoutsidetheregionofintegration,asshowninFigure5.3a.Thus[174CHAPTER5MULTIPLERANDOMVARIABLESyyx.'(.r;<0or!JO<y(a)yyI:-:}.':-:...xxI00<.r<y<.f<lO<y<l>l(d).l'(c)}'xx>1andy>l(e)Figure5.3FivecasesfortheCDFFx,Y(x,y)ofExample5.8.[5.4JOINTPROBABILITYDENSITYFUNCTION175wehaveFx,Y(x,y)=0ifeitherx<0ory<0.Anothersimplecaseariseswhenx>1andy>l.Inthiscase,weseeinFigure5.3ethatthetriangleiscompletelyinsidetheregionofintegration,andweinferfromTheorem5.6thatFx,y(x,y)=1.Theothercaseswemustconsideraremorecomplicated.Ineachcase,sincef'x,y(x,y)=2overthetriangularregion,thevalueoftheintegralistwotimestheindicatedarea.When(::c,y)isinsidetheareaofnonzeroprobability(Figure5.3b),theintegralisFx,y(x;,y)={Y{x2dv,dv=2xy-y2(Figure5.3b).JoJ.u(5.23)InFigure5.3c,(x;,y)isabovethetriangle,andtheintegralisFx,Y(x,y)=rxJ.rx2dv,dv=u;i;2(Figure5.3c).(5.24)}0TheremainingsituationtoconsiderisshowninFigure5.3d,when(x,y)istotherightofthetriangleofnonzeroprobability,inwhichcasetheintegralis1"11Fx,Y(x,y)=2dudv=2y-y2(Figure5.3d)(5.25)TheresultingCDF,correspondingtothefivecasesofFigure5.3,is<0ory<00;r;2x;y-y2Ol;i;>l,y>lx22y-y21(a),(b),(c)'(d),(e).(5.26)InFigure5.4,thesurfaceplotofFx,y(x;,y)showsthatcases(a)through(e)correspondtocontoursonthe"hill"thatisFx,Y(x;,y).Intermsofvisualizingtherandomvariables,thesurfaceplotofFx,Y(x;,y)islessinstructivethanthesimpletrianglecharacterizingthePDFf'x,Y(x,y).BecausethePDFinthisexampleisf'x,Y(x,y)=2over(x;,y)ESx,Y,eachprobabilityisjusttwotimestheareaoftheregionshowninoneofthediagrams(eitheratriangleoratrapezoid).Youmaywanttoapplysomehighschoolgeometrytoverifythattheresultsobtainedfromtheintegralsareindeedtwicetheareasoftheregionsindicated.Theapproachtakeninoursolution,integratingoverSx,YtoobtaintheCDF,worksforanyPDF.InExarnple5.8,ittakescarefulstudytoverifythatFx,y(;r:,y)isavalidCDFthatsatisfiesthepropertiesofTheorern5.1,ore\renthatitisdefinedforall\raluesxandy.Comparingthejoir1tPDFvvit11thejointCDF,weseethatthePDFindicatesclearl}'t11a.tX,Yoccurswithequalprobabilityinallareasoft11esarnesizeinthetriangl1larregion0<y<x<1.T11ejointCDFcorr1pletel}'hidesthissirnple,importantpropertyoftheprobabilitymodel.Inthepreviousexample,t11etriar1gl1la.rs11apeoft11eareaofnonzeroprobabilitydernandedourcarefulattention.Inthenextexarr1ple,theareaofnonzeroprobabilityisarectangle.Hovvever,t11eareacorrespondingtothee\rer1tofinterestisrnorecorr1plicated.[176CHAPTER5MULTIPLERANDOMVARIABLES..•I•I1..••I.........•......•o•I•.0.5200Figure5.40.5121.5xAgraphofthejointCDFFx,y(:i~,0yy)ofExan1ple5.8.Example5.9AsinExample5.7,randomvariablesXandYhavejointPDFOX]?ApplyingTheorem5.7,weintegratefx,Y(x;,y)overthepartoftheX,·yplanesatisfyingY>X.Inthiscase,yY>X~)JoJxlo3P[A]=((31-03((5.28)dydx3-x(3-::r;)2dT-15,.,30303-10(5.29)Inthisexarr1ple,itmakeslittledifferencewl1ether-vveir1tegratefirstoveryandthenoverxortheotl1er'\Va.Jraround.Ingeneral,110'\vever,aninitialefforttodecidethesimplest-vva:ytointegrateoveraregioncanavoidalotofcomplicatedrnathematicalmaneu'ireringinperforrningtheintegration.Quiz5.4Thejointprobabilitydensityf\1nctionofrandorr1variablesXand·yiscxy0<0otherwise.;i;<1,0<y<2,Findt11econstantc.Whatistheprobabilityoft11ee\rentA=X2(5.30)+Y2<1?[5.55.5MARGINALPDF177MarginalPOFForcontint1ousrandorr1variables,t.hemarginalPDFsf'x(x)andjy(y)areprobabilit}'rnodelsfortheindividualrandomvariablesXand·y,buttheydonotprovideacompleteprobabilitymodelforthepairX,Y.Suppose-vveperforrnanexperirnentthatproducesapairofrandorn·variablesXandYwithjointPDFfx,Y(x,y).Forcertainpurposesv.rernaybeinterestedonl}'intherandorr1variableX.\l\fecanirnaginethatweignoreYandobserveor1lyX.SinceXisarandornvariable,ithasaPDFfx(x).Itshouldbeapparer1tthatthereisarelationshipbet-vveenfx(x)andfx,y(::c,y).Inparticular,iff'x,y(x,y)corr1pletelysurnmarizesourknowledgeofjointe\rentsoftheformX=x,·y=y,thenwes11ouldbeabletoderivet11ePDFsofXandYfrorr1f'x,y(x;,y).Thesituatior1parallels(-vvithintegralsreplacingsurr1s)therelationshipinTheorern5.4betv.reenthejoir1tP11.IFPx,y(x,y),andtherr1arginalPMFsPx(x)aridPy(y).Therefore,vierefertofx(x)andjy(y)asthernargin,alprobabilityden,sityfuriction,soff'x,Y(x;,y).--=--Theorem5.8:--IfXa/ndYareraridornvariablesVJithjoin,tPDFfx,Y(x;,y),fx(x)=1:fx,Y(x,y)dy,fy(y)=1:fx,Y(x,y_)_d_,x_.---ProofFromthedefinitionofthejointPDF,wecan'vrite(5.31)Takingthederivativeofbothsideswithrespecttox('vhichinvolvesdifferentiatingan00integralwithvariablelimits),'veobtainfx(x)=J00fx,Y(x,y)dy._Asimilarargumentholdsforfy(y).c::::==Example5.10ThejointPDFofXand·yisfx,y(x,y)=5yI40-1<x<1,x;2<y<1,otherwise.(5.32)FindthemarginalPDFsfx(x)andj'.y(y).WeuseTheorem5.8tofindthemarginalPDFfx(x;).InthefigurethataccompaniesEquation(5.33)below,thegraybowl-shapedregiondepictsthosevaluesofXandYforwhichf'x,Y(x,y)>0.Whenx<-1orwhenx>1,fx,Y(;r;,y)=0,andthereforefx(x)=0.For-11,j'y(y)=0.For0<y<1,weintegrateoverthehorizontalbarmarked·y=y.Theboundariesofthebararex;=-Jf;andx=Therefore,for0<y<1,yJY.JY5y5yJ;=./Yfy(y)=-dx=-x=5y312/2.-JY44J;=-fi;J(5.35)-1-y112y1121ThecompletemarginalPDFofYis3-----(5/2)y3120-100<y<1,otherwise.(5.36)1yQuiz5.5Thejointprobabilitydensityfunctionofrandorr1variablesXand·yis1·x,Y(x,y)=6(x0+y2)/5o<x;<1,o<y<1,otherwise.(5.37)Findf·x(x)andj'y(y),themarginalPDFsofXandY.5.6IndependentRandomVariablesRandornvariablesXand·yareindependentifandonlyiftheevents{X=x;}a.nd{'Y=y}areindependentforallx,yinSx,Y.Discreterandorr1variablesXand·yareindeper1dentifandor1lyifPx,Y(x;,y)=Px(:I;)Py(y).ContinuousrandornvariablesXandYareindependentifandonlyiffx,Y(x;,y)=1·x(x;)j.y(y).Chapter1presentsthecor1ceptofindeper1dentevents.Definition1.6statest11ateventsAandBareir1dependentifandonl}'iftheprobabilityoftheintersectionistheprodl1ctoft11eindividl1alprobabilities,P[AB]=P[A]P[B].[5.6INDEPENDENTRANDOMVARIABLES179Applyir1gtheideaofindependencetorar1dornvariables,v.resaythatXandYareindependentrandorr1variablesifandor1lyiftheevents{X=x}and{'Y=y}areindependentforall;i;EBxandallyESy.Ir1termsofprobabilitymassfur1ctionsandprobabilitydensityfunctions,""'ehavethefollovvingdefinition.===Definition5.4IndependentRandomVariablesRan,dorn'/JariablesXan,dYareindependentifarido'nlyifPx,y(x,y)Discrete:=Px(:i;)Py(y);Con,tin/u,011,s:f"x,Y(x,y)=fx(x)fy(y).Example5.11Arethechildrens'agesXand·yinExample5.2independent?InExample5.2,wederivedtheCDFsFx(x)andFy(y),whichshowedthatXisuniform(5,6)andYisuniform(6,7).ThusXandYhavemarginalPDFs15<-0;i;<6.160,v>0,'U,+v<1,otherwise.(5.40)[180CHAPTER5MULTIPLERANDOMVARIABLESAreUandVindependent?Sincefu,v(v,>v)lookssimilarinformtof'x,Y(x,y)inthepreviousexample,wemightsupposethatUandVcanalsobefactoredintomarginalPDFsf'u(v,)andf'v(v).However,thisisnotthecase.Owingtothetriangularshapeoftheregionofnonzeroprobability,themarginalPDFsaref'u(v,)=12v,(1-v,)2O0otherwise,--f'v(v)I=12v(l-v)2O0otherwise.--IClearly,Uand·varenotindependent.LearningUchangesourknowledgeofV.Forexample,learningU=1/2informsusthatP[V<1/2]=1.Inthesetwoexarriples,-vveseethattheregioriofnonzeroprobabilityplaysacrucialroleindeterrniningwhetherrandorn·variablesareindependent.Onceagain>-vveempliasizethattoinferthatXandYareindependerit,itisnecessarytoverifythefunctionalequalitiesinDefiriition5.4forall;_r;EBxandyESy.Therearernanycasesinv.rhiclisorneeventsoftheforrri{X=x}and{'Y=y}areiridependentandothersareriotindependent.Ifthisisthecase>therandorn·variablesXandYarenotindependent.InExarnples5.12and5.13,\VearegiveriajointPDFaridaskedtodetermine-vvhethertherandorrivariablesareindependent.Bycontrast,irirnan}'applicationsofprobability>thenatureofanexperimentleadstoarnodelinwhicliXand·yareindependent.Intheseap1Jlicationsweexarnineanexperirrieritanddeterminetliatitisappropriatetorriodelapairofrandorn\rariablesXandYasindependent.Toanalyzetlieexperirnent,-vvestart\A.TiththePDFsfx(i;)andj'y(y),andthenconstructtliejointPDFf'x,Y(x,y)=fx(x)fy(y).Example5.14ConsideragainthenoisyobservationmodelofExample5.1.SupposeXisaGaussian(0,ax)informationsignalsentbyaradiotransmitterandY=X+Zistheoutputofalow-noiseamplifierattachedtotheantennaofaradioreceiver.ThenoiseZisaGaussian(0,az)randomvariablethatisgeneratedwithinthereceiver.WhatisthejointPDFf'x,z(i;,z)?Fromtheinformationgiven,weknowthatf'x(x)=1~e-x2XandZhavePDFs/20"~'(5.41)ThesignalXdependsontheinformationbeingtransmittedbythesenderandthenoiseZdependsonelectronsbouncingaroundinthereceivercircuitry.Asthereisnoreasonforthesetoberelated,wemodelXandZasindependent.Thus,thejointPDFis1-1(x.;+r~)!727ieaxC7z.fx,z(i;,z)=f'x(i;)f'z(z)=27ryaxaz(5.42)c::=====[5.7EXPECTEDVALUEOFAFUNCTIONOF1WORANDOMVARIABLES181==--Quiz5.6---==:::2(A)RandornvariablesXand·yinExarnple5.3andrandorn·variablesQaridGinQuiz5.2havejointPMFs:Px,Y(x,y)x=Ox=lx=2y=Oy=l0.010.09000.090PQ,c(q,g)q=0q=1y=2000.81g=Og=l0.060.040.180.12g=20.240.16g=30.120.08(b)AreQaridGindeperident?(a)AreXandYir1dependerit?(B)RandorrivariablesX1andX2areindependentandideriticallydistributedvvithprobabilityder1sityf\1rictionfx(;r:)=5.7x;/200-<x-<2otlier\vise.)(5.43)ExpectedValueofaFunctionofTwoRandomVariablesg(X,Y),afunctionoftworandomvariables,isalsoarandomvariable.Asvvithonerandomvariable,itisconveriienttocalculatetheexpectedvalue,E[g(X,Y)],\vithoutderivingaprobabilityrnodelofg(X,Y).Thereareman:yrsitl1ationsiriwhichvieobservetworandornvariablesandusetheirvaluestocomptlte[182CHAPTER5MULTIPLERANDOMVARIABLES===Theorem5.91____.;;==Forro/ndornvariablesXan,dY,theeJ;pectedval?J,eofWDiscrete:E[l!Tl)=LL~r;ESxContinuous:E[W]==g(X,Y)isg(x;,y)Px,Y(x,y);yESyl:l:g(x,y)fx,Y(x,y)dxdy.Theorern5.9issurprisinglypowerful.Forexarr1ple,itletsuscalculateeasilytheexpected·valueofalinearcorr1binationofseveralfunctior1s.===Theorem5.10E[a1g1(X,Y)ProofLetg(X,Y)+···+a,,gn(X,Y))=a1E[g1(X,Y))+···+anE[gn(X,'Y)).11=a1g1(X,Y)+···+°'n9n(X,Y).FordiscreterandomvariablesX,Y,Theorem5.9statesE[g(X,Y)]=LL(a1g1(x,y)+···+angn(x,y))Px,y(x,y).(5.44)xESxyESy\'Alecanbreakthedoublesummationinton,'veighteddoublesummations:LLE[g(X,Y)]=a1g1(x,y)Px,Y(x,y)+···+anxESxyESyLL9n(x,y)Px,y(x,y).xESxyESyByTheorem5.9,theithdoublesum1nationontherightsideisE[gi(X,Y)];thus,E[g(X,Y)]=a1E[g1(X,Y)]+...+°'nE[gn(X,Y)].(5.45)Forcontinuousrandomvariables,Theorem5.9saysE[g(X,Y)]=1:1:(a1g1(x,y)+···+angn(x,y))fx,Y(x,y)d1;dy.(5.46)Tocompletetheproof,'veexpressthisintegralasthesumofnintegralsandrecognizethateachofthenewintegralsisaweightedexpectedvalue,a,iE[gi(X,Y)].Inv.rords,Theorerr15.10sayst:hattheexpectedva.lt1eofalinearcorr1binationequalsthelinearcombinatior1oftr1eexpected·values.Wewillhaverr1anyoccasionstoapplythistheorem.Thefollovvir1gtheorerndescribestheexpectedsurnoft\vorandomvariables,aspecialcaseofTr1eorerr15.10.===Theorem5.11--Forariyt'tlJOran,dornvariablesXan,dY,E[X+·y)=E[X)+E['Y).[5.7EXPECTEDVALUEOFAFUNCTIONOF1WORANDOMVARIABLES183Thistheoremirnpliesthat"if..Tecanfindtheexpectedsurr1oftv.rorandornvariablesfrorntheseparateprobabilityrr1odels:Px(x)andPy(y)orf'x(x)andfy(y).'\''!\!edonotneedacornpleteproba,bilit:yrnodelerr1bodiedinPx,y(x,y)orfx,y(x,y).Bycontrast,thevariar1ceofX+Ydependsont11eentirejointPJVIForjointCDF:---==Theorem5.12The'oarian,ceofthes'urnoft1110ran,dornvariablesisVar[X+Y]ProofSinceE[X+Y]·var[X+Y]=Var[X]+Var[Y]+2E[(X-µ,x)CY-µ,y)].=JLx+µ,y,=E[(X+Y-(µx+µy))2]=E[((X-µx)+(Y-µ,y))2]=E[(X-µ,x)2+2(X-JLx)(Y-µ,y)+(Y-µy)2].(5.47)\\!eobservethateachofthethreetermsintheprecedingexpectedvaluesisafunctionofXandY.Therefore,Theorem5.10impliesVar[X+Y]=E[(X-µx)2]+2E[(X-µ,x)(Y-JLY)]+E[(Y-JLY)2].(5.48)Thefirstandlasttermsare,respectively,\far[X]and\far[Y].TheexpressionE[(X-µ,x)(Y-µ,y)]inthefinaltermofTheorerr15.12isa.pararneteroftheprobabilit:ymodelofXarid·y.Itrevealsimportar1tpropertiesoftherelations11ipofXandY.Thisquar1tityappeExample5.15Acompanywebsitehasthreepages.Theyrequire750kilobytes,1500kilobytes,and2500kilobytesfortransmission.Thetransmissionspeedcanbe5Mb/sforexternaIrequestsor10Mb/sforinternalrequests.Requestsarriverandomlyfrominsideandoutsidethecompanyindependentlyofpagelength,whichisalsorandom.Theprobabilitymodelsfortransmisionspeed,R,andpagelength,L,are:PR(r)=0.4r=50.6TlO(0.3)=10)otherwise,PL(l)=~0.5l~2l=750,l=1500,l=2500(5.49))otherwise.Writeanexpressionforthetransmissiontimeg(R,L)seconds.DerivetheexpectedtransmissiontimeE[g(R,L)].DoesE[(g(R,L)]=g(E[R],E[L])?ThetransmissiontimeTsecondsisthethepagelength(inkb)dividedbythetrans-[184CHAPTER5MULTIPLERANDOMVARIABLES=8L/1000R.missionspeed(inkb/s),orTPR,L(r,l)=PR(r)PL(l)andE[g(R,L)]=~~P1z(r)PL(l)~~lr81000r·8(0.40.6)5+10=1000=1.652s.Bycomparison,E[R]ThisimpliesBecauseRandLareindependent,SllOOOr~PL(l)l~l(0.3(750)+(0.51500)+0.2(2500))=l:rrPR(r)=8Mb/sandE[L]=l:tlPL(l)=1475kilobytes._8E[L]_r.::g(E[R],E[L])-lOOOE[R]-1.470s#E[g(R,L)].5.8(5.50)(5.51)Covariance,CorrelationandIndependenceThecovariariceCov[X,Y],thecorrelationcoefficientPx,Y,andthecorrelatiorirx,YareparametersoftheprobabilitymodelofXand·y.ForindependentrandorrivariablesXand·y,Cov[X,Y]=PX,Y=o.Definition5.5CovarianceThecovarianceoft1110raridorn'variablesXarid·yisCov[X,Y]=E[(X-µ,x)(Y-µ,y)].Sometirnes,thenotationCJxyisusedtoderiotetheco·varianceofXand·y.'\'!Vehavealread:ylearriedtliattheexpectedvaluepararneter,E[X],isatypicalvalueofXaridthattlievarianceparameter,Var[X],isasingleriurriberthatdescribeshovvsarnplesofXteridtobespreadaroundtlieexpectedvalueE[X].Inananalogous"''ay,thecovariariceparameterCo·v[X,·y]isasiriglenumberthatdescribeshov.rthepairofrandomvariablesXaridYvarytogether.ThekeytounderstandingCO\rariariceistherandomvariableW=(X-µ,x)CY-µ,y).(5.52)SinceCo,r[X,Y]=E[vV],vveobservethatCov[X,Y]>0tellsusthatthetypicalvaluesof(X-1;,x)(Y-µ,y)arepositive.Hovvever,thisisequivalenttosayingtliatX-11,xandY-µytypicallyha\rethesarnesigri.Thatis,ifX>1;,xthen"''ewould[5.8COVARIANCE,CORRELATIONANDINDEPENDENCE185typicall}'expectY>µ,y;:1ndifX<11,xtlien"''evvouldexpecttoobserveY<µy.Irishort,ifCov[X,Y]>0,v.rev.rol1ldexpectXarid·ytogoupordov.rritogether.Ontlieotherhand,ifCov[X,·y]<0,"''ewouldexpectX-µ,xandY-µytotypicallyha\reoppositesigns.Inthiscase,whenXgoesup,Yt}rpicallygoesdovvri.Finally,ifCov[X,Y]~0,vvemiglitexpectthattliesignofX-1),xdoesn'tprovidernucliofaclueaboutthesignofY-µ,y.Whilethiscasualarg1lllleritrnaybereasonablyclear,itrriayalsobesome"''hatunsatisfacto1y.Forexample,v.rol1ldCov[X,·y]=0.1befairlydescribedasCov[X,Y]~O?TheansvvertothisquestioridependsontherrieasurementunitsofXaridY.Example5.16SupposeweperformanexperimentinwhichwemeasureXandYincentimeters(forexampletheheightoftvvosisters).However,ifwechangeunitsandmeasureheightinmeters,wewillperformthesameexperimentexceptweobserveX=X/100andY=Y/100.Inthiscase,andYhaveexpectedvaluesµx_=µ,x/100m,µ,y=µ,y/100mandXCO\T[x,f]=E[(x-µ,x_)(Y-µ,y)]-E[(X-µ,x)('Y-µ,y)]-Co''[X,Y]m2-10,000-10,000.(5.53)Changingtheunitofmeasurementfromcm2tom2reducesthecovariancebyafactorof10,000.However,thetendencyofX-µ,xandY-µytohavethesamesignisthesameasthetendencyof/J,xandY-11,ytohavethesamesign.(Bothareanindicationofhowlikelyitisthatagirlistallerthanaverageifhersisteristallerthanaverage).X-Apararrieterthatindicatestherelationshipoftworandorri\rariablesregardlessofmeasurerrienturiitsisa.norrrializedversionofCov[X,Y],calledtliecorrelationcoefficient.Definition5.6CorrelationCoefficientThecorrelationcoefficientoft1110ro/ndorn'/JariablesXa'nd·y'isPX,Y=Cov[X,Y]JVar[X]·var['Y]Cov[X,Y]NotetliatthecovariancehasunitsequaltotlieproductoftheunitsofXaridY.Thus,ifXliasunitsofkilogramsarid·yhasunitsofseconds,theriCov[X,Y]hastlnitsofkilograrri-seconds.Bycontrast,Px,Yisadirrierisionlessql1antitytliatisnotaffectedbyscalechanges.Theorem5.13IfX=aX+ba'ndY=cY{a)pxy=px,y,AA)+d,then,{b)Co\r[X,Y]=acCo\r[X,Y].[186CHAPTER52MULTIPLERANDOMVARIABLES...•••.....•••••••••....-...'t~r.:,~2••.:t.........#.•..,,~·.'0·•,.......1.........·.::.·_....~'"...0~~••,,,..0(a)PX,Y•••-2•-2~-22x=-0.9'.tld•.••.,,..:.,1J41:••;.••••fr.1•.:.•9'C-220.r!..!W~.~~·•••f.-22x(b)PX,Y=0:.#"'·:t~::0-2.•••0x2(c)PX,Y=0.9Figure5.5Eachgraphhas200samples,eachn1arkedbyadot,oftherandon1variablepair(X,Y)suchthatE[X]=E[Y]=0,Var[X]=Var[Y]=1.Theproofstepsareoutlir1edinProblem5.8.9.RelatedtothisinsensitivityofPX,Ytoscalechariges,anirriportantpropert:yofthecorrelationcoefficientisthatitisboundedby-1and1:===Theorem5.14-1<PX,Y<1.ProofLeta~anda~denotethevariancesofXandY,andforaconstanta,letW=X-aY.Then,Var[W]=E[(X-aY)2]-(E[X-aY])2.(5.54)SinceE[X-aY]=p,x-aJJ,y,expandingthesquaresyieldsVar[W]=E[X2-2aXY+a2Y2]-(p,x2-2ap,xp,y+a2µy2)=Var[X]-2aCov[X,Y]+aVar[Y].2(5.55)SinceVar[W]>0foranya,wehave2aCov[X,Y]<·var[X]+a2Var[Y].Choosinga=ax/ayyieldsCov[X,Y]<ayax,'vhichimpliespx,·y<1.Choosinga=-ax/ayyieldsCov[X,Y]>-ayax1whichimpliespx,Y>-1.vVhenPX,Y>0,wesaytriatXand·yarepositivelycorrelated,aridvvhenPx,Y<0vveSa}'XaridYare'negativelycor;·elated.IfIPx,YIiscloseto1,sayIPx,YI>0.9,thenXandYarehighlycorrelated.Notethat.highcorrelationcanbepositiveornegative.Figl1re5.5shO\¥SoutcornesofindeperidenttrialsofanexperirnentthatproducesrandorrivariablesXand·yforrandornvariablepairsvvith(a)riegati·vecorrelation,(b)zerocorrelation,and(c)positivecorrelatiori.TliefollowingtheorernderrioristratesthatIPx,YI=1whenthereisaliriearrelationsliipbetweenXand·y.[5.8COVARIANCE,CORRELATIONANDINDEPENDENCETheorem5.15IfXa'ndYareran,dom,variablessuchthatY-1PX,Y==aX187+b;(],<0,0(],=1(],>0,o.Theproofisleftasanexerciseforthereader(Problern5.5.7).Sorneexarriplesofpositive,negative,aridzerocorrelationcoefficientsiriclude:•Xisastuderit'sheight.·yisthesarnestudent'sweiglit.0<Px,Y<1.•Xisthedistanceofacellularphonefrorritheriearestbasestatiori.·yisthepov.rerofthereceivedsignalatthecellularphone.-1<Px,Y<0.•Xistheternperatureofaresistorrneasl1redindegreesCelsius.YisthetemperatureofthesarneresistormeasuredinKelvins.PX,Y=1.•Xisthegainofanelectricalcircuitmeasuredindecibels.Yistheattenuation,rneasuredindecibels,ofthesarnecircuit.PX,Y=-1.•Xisthetelephonenumberofacellularphorie.·yistheSocialSecurit}'nl1rnberoftliephone'sowner.Px,Y=0.Thecorrelation,oftworandom·variables,deriotedrx,Y,isanotherpararneteroftheprobabilityrnodelofXarid·y.rx,Yisacloserelati·veofthecovariarice.Definition5.7CorrelationThecorrelationofXan,d·yisrx,Y=E[XY]Thefollovvingtlieorerricoritairisusefulrelatiorishipsarriongthreeexpected·values:thecovarianceofXandY,tliecorrelationofXaridY,andthe·varianceofX+·y.===Theorem5.16=rx,Y-µ,xµ,y.{b)Var[X+Y]=Var[X]+Var[Y]+2Cov[X,Y].(c)IfX=·y)Cov[X,Y]=·var[X]=Var[Y]an,drx,Y=E[X2]=E[Y2].(a)Co·v[X,Y]ProofCross-multiplyinginsidetheexpectedvalueofDefinition5.5yieldsCov[X,Y)=E[XY-p,xY-/LYX+p,xp,y).(5.56)Sincetheexpectedvalueofthesumequalsthesumoftheexpectedvalues,Cov[X,Y)=E[XY)-E[fLXY)-E[µyX)+E[!LY/Lx).(5.57)[188CHAPTER5MULTIPLERANDOMVARIABLESNotethatintheexpressionE[µ,yX],JLYisaconstant.ReferringtoTheorem3.12,weseta=JL·yandb=0toobtainE[µyX]=JLyE[X]=µ,yµ,x.ThesamereasoningdemonstratesthatE[µ,xY]=JLXE[Y]=µxµ,y.Therefore,Cov[X,Y]=E[XY]-µ,x/LY-/LY/Lx+/LYµx=rx,Y-µ,x/LY.(5.58)Theotherrelationshipsfollo'vdirectlyfromthedefinitionsandTheorem5.12.-=Example5.11FortheintegratedcircuitstestsinExample5.3,wefoundinExample5.5thattheprobabilitymodelforXandYisgivenbythefollowingmatrix.Px,Y(x,y)0=1=2:r;=:r;:r;Py(y)y=Oy=l0.010.0900.1000.0900.09Px(x)y=2000.810.810.010.180.81Findrx,YandCov[X,Y].ByDefinition5.7,22rx,Y=E[X.Y]=LLxyPx,Y(x;,y)(5.59)x=Oy=O=(1)(1)0.09+(2)(2)0.81=3.33.(5.60)TouseTheorem5.16(a)tofindthecovariance,wefind+(2)(0.81)=(1)(0.09)+(2)(0.81)=E[X]=(1)(0.18)1.80,E[Y]=1.71.(5.61)Therefore,byTheorem5.16(a),Cov[X,·y]=3.33-(1.80)(1.71)=0.252.T11ete1TI1sorthogorialandurico'rrelateddescriberar1dorr1·variablesforvv11icl1rx,Y=0andrar1domvariablesforwhichCo·v[X,·y]=0respectivel}'·OrthogonalRandomVariablesDefinition5.8Ran,dornvariablesXan,dYareorthogonalifrx,Y=0.---Definition5.9---UncorrelatedRandomVariablesRan,dornvariablesXan,dYareuncorrelated'ifCov[X,Y]=0.1Thisterrr1inolog}',v.rhilewidelyused,issomewhatcor1fusing,sinceorthogon,alrr1ear1szerocorrelationandv,n,correlatedrr1eanszerocovariance.[5.8COVARIANCE,CORRELATIONANDINDEPENDENCE189Weha·vealreadyriotedthatifXand·yarehighlycorrelated,thenobservirigXtellsusalotabot1ttheaccorripanyingobservatioriY.Graphically,thisisvisibleinFigt1re5.5v.rlienwecorriparethecorrelatedcases(a)and(c)totlieuncorrelatedcase(b).Ontheotherhand,ifCo·v[X,Y]=0,itisoftenthecasethatlearningXtellsuslittleaboutY.\A/ehaveusedriearlythesarriev.rordstodescribei'Tl,depe'Tl,deritrandorri·variablesXaridY.Thefollovvingtheorerncontainsseveralirriportantpropertiesofexpected·valuesofindependeritraridom\r[triables.Itstatesthatindependeritraridomvariablesareuncorrelatedbutnotnecessaril}'orthogonal.---Theorem5.11--Foriridepe'Tl,de'Tl,tra'Tl,dorn'variablesXaridY;(a)E[g(X)h,(Y)]=E[g(X)]E[h(Y)],{b)rx,Y=E[X.Y]=E[X]E[Y];(c)co,r[X,Y]=pX,Y=O;{d)Var[X+·y]=Var[X]+Var'["Y],ProofV/epresenttheprooffordiscreterandomvariables.ByreplacingP~l[Fsandsumsarriveatessentiallythesameproofforcontinuousrandom'vithPDFsandintegrals\Vevariables.SincePx,y(1;,y)=Px(1;)Py(y),E[g(X)h(Y)]=LLg(x)h(y)Px(x)Py(y)xESxyESyLg(x)Px(1;)xESxLh(y)Py(y)=E[g(X)]E[h(Y)].(5.62)yESyIfg(X)=X,andh(Y)=Y,thisequationimpliesrx,Y=E[XY]=E[X]E[Y].ThisequationandTheorem5.16(a)implyCov[X,Y]=0._Asaresult,Theorem5.16(b)impliesVar[X+Y]=Var[X]+Var[Y].Furthermore,px,Y=Cov[X,Y]/(crxcry)=0.Tlieseresultsallfollovvdirectl}'fromthejointP11:Fforindependentrandorrivariables.WeobservethatTlieorerri5.17(c)statesthati'Tl,depe'Tl,de'Tl,traridorn'variablesare'LJ.'Tl,correlated.SN"evvillha,rerrianyoccasionstorefertotliispropert}'·Itisimportanttoknovvthat\¥hileco,r[x,Y]=0isanecessarypropert}Tforindeperidence,itisriotsufficient.Therearerrianypairsofuncorrelatedrandorn\rariablesthatare'Tl,otindependent.===Exampie5.1a:==::::::aForthenoisyobservationY=X+ZofExample5.1,findthecovariancesCo,r[X,Z]andCov[X,·y]andthecorrelationcoefficientsPx,zandPx,Y.WerecallfromExample5.1thatthesignalXisGaussian(O,o-x),thatthenoiseZisGaussian(O,o-z),andthatXandZareindependent.WeknowfromTheorem5.17(c)[190CHAPTER5MULTIPLERANDOMVARIABLESthatindependenceofXandZimpliesCov[X,Z]=Px,z=0.(5.63)Inaddition,byTheorem5.17(d),Var[Y]=Var[X]+Var[Z]=o-1+o-~.(5.64)SinceE[X]=E[Z]=0,Theorem5.11tellsusthatE['Y]=E[X]+E[Z]=0andTheorem5.17)(b)saysthatE[XZ]=E[X]E[Z]=0.ThispermitsustowriteCov[X,Y]=E[XY]=E[X(X+Z)]2=E[X+XZ]=E[X2J+E[XZ]=E[X2J=o-1.ThisimpliespX,y=o-1Io-~Cov[X,Y](5.65)l+o-xIO"z2·--:=~=~=2JVar[X]Var[Y]WeseeinExample5.18thattheco·variancebetV\reentlietransrriittedsignalXandtherecei·vedsigrialYdependsontheratioo-~.Thisratio,referredtoastliesign,al-to-rioiserat'io,liasastrongeffectoricomrriunicationquality.If/o-~07JJ1{b}FX1,...,Xn(;i;1'···'Xn)JJ00{c):i;-oo•··:r;n-oof'x1,...,Xn('lJ,1'···''U,n)r17J,1···r17J,n'00.••-oo=f'x1,...,xn(x;1,...,Xn)rlx;1···rlx;n=1.-ooOftenvveconsiderane·ver1tAdescribedintermsofapropertyofX1,...,Xn,suc11asIX1+X2+···+Xnl<1,orrr1ax,i:X,i:<100.TofindtheprobabilityoftheeventA,wesumthejoir1tP11.IForintegratethejointPDFoverall;r;1,...,;r;nthatbelongtoA.Theorem5.24Theprobabilityofarieven,tAex;rJressed'iriterrnsoftheran,dorn'uariablesX1,...,XnisDiscrete:p[A]=(x1,...,xn)EAContinuous:P[A]=j···jfx,,...,x.(xi,...,Xn)dx1dx2...dxn.AAlthol1ghv.re11avewrittenthediscreteversionofTheorem5.24vvithasinglesurnmation,v.rernustrerr1emberthatinfactitisaml1ltiples11movertheri·variablesX1,...,x;n·[5.10xyz(1Page)(2Pages)(3Pages)000001111222334012340123012010432103210210100Table5.1197MULTIVARIATEPROBABILITYMODELSPx,Y,z(x,y,z)TotalPagesEvents1/12961/1081/241/121/161/1621/181/61/61/541/91/62/812/271/811211109810987876654BBBBABBBABBABBBThePMFPx,Y,z(x,y,z)andtheeventsAandBforExample5.22.Example5.21Considerasetofn,independenttrialsinwhichtherearerpossibleoutcomess1,...,Srforeachtrial.Ineachtrial,P[s,i]='[J,i.LetN,iequalthenumberoftimesthatoutcomeSioccursovern,trials.WhatisthejointPMFofN1,...,Nr?ThesolutiontothisproblemappearsinTheorem2.9andisrepeatedhere:(5.73)Example5.22Foreachproductthatacompanysells,acompanywebsitel123hasatechsupportdocumentavailablefordownload.The1/31/21/6PMFofL,thenumberofpagesinonedocument,isshowninthetableontheright.Forasetoffourindependentinformationrequests,find:(a)thejointPMFoftherandomvariables,X,Y,andZ,thenumberof1-page,2-page,and3-pagedownloads,respectively,(b)P[A]=P[totallengthoffourdownloadsis8pages],(c)P[B]=

P[atleasthalfofthefourdownloadshavemorethan1page].Thedownloadsareindependenttrials,eachwiththreepossibleoutcomes:L=1,L=2,andL=3.Hence,theprobabilitymodelofthenumberofdownloadsofeach[198CHAPTER5MULTIPLERANDOMVARIABLESlengthinthesetoffourdownloadsisthemultinomialPMFofExample5.21:(l):i;(1)(l)z4)(71Pxy''z(x,y,z)=.-x,y,z-32-:6(5.74)ThePMFisdisplayednumericallyinTable5.1.ThefinalcolumnofthetableindicatesthattherearethreeoutcomesineventAand12outcomesineventB.Addingtheprobabilitiesinthetwoevents,wehaveP[A]=107/432andP[B]=8/9.Inanal:yzir1ganexperiment,-vvernightv.rishtostudysorneoft11erandorr1variablesandignoreotherones.Toaccornplishthis,\Vecar1derivernarginalP1![Fsorrr1argir1alPDFsthatareprobabilit}'modelsforafractionoftherandomvariablesinthecompleteexperirner1t.Considerar1experirner1t-vvit11fol1rrandorr1variablesl;Tl,X,Y,Z.Theprobabilitymodelfortheexperirnentisthejoir1tP1![F,Pw,x,Y,z('w,x,y,z)orthejoir1tPDF,f'w,x,Y,z(w,x,y,z).Thefollowir1gtheorernsgiveexamplesofrr1argir1alP1![FsaridPDFs.==;;;;:Theorem5.25__....;;;=;;;;iForajoiritPMFP11v,x,Y,z('w,x,y,z)ofd'iscretera'ndornvariablesW,X,Y,Z,sornernarg'in,alPMFsarePx,Y,z(x,Y,z)=LPv11,x,Y,z(x,y,z),LLP'v\1,x,Y,z('w,x,y,z),'llJ,'wESwP11v,z('ID,z)=:i:ESxyESy---Theorem5.26Forajoin,tPDFfw,x,Y,z('w,x,y,z)ofcon,tin,uousran,dornvariablesW,X,Y,Z,sornernarg'irialPDFsarefw,x,Y(w,x,y)=fx(x)=1:1:1:1:fw,x,Y,z(w,x,y,z)dz,fw,x,Y,z(w,x,y,z)dwdydz.Theorerns5.25and5.26canbegeneralizedinastraig11tforwardwaytoanyrr1arginalP1![Forrr1argir1alPDFofa.narbitrarynurnberofrandornvariables.Foraprobabilitymodeldescribedbythesetofrar1dornvariables{X1,...,Xn},eachnonemptystrictsubsetofthoserandornvariableshasarnarginalprobabilityrnodel.Thereare2nSl1bsetsof{X1,...,Xn}·Afterexcludingtheentiresetandt11er1ullset0,-vvefir1dthatthereare211'-2marginalprobabilitymodels.[5.10MULTIVARIATEPROBABILITYMODELS199Example5.23AsinQuiz5.10,therandomvariablesY1,...,Y4havethejointPDF40<Y1<Y2<1,0<Y3<Y4<1,0otherwise.fy,,Y.(yi,'Y4)=1:1:fy.,,Y4(Yi,...,Y4)dyzdy3.(5.75)(5.76)Intheforegoingintegral,thehardpartisidentifyingthecorrectlimits.TheselimitswilldependonYIandy4.For0(5.77)ThecompleteexpressionforJy,Y(Y1,y4)14is4(1-y1)Y400otherwise.(5.78)Similarly,for0<y2<1and0<y3<1,(5.79)Thecompleteexpressionforfy2,y3(Y2,y3)isLastly,for0f'.Y.3(y3)<y3<4y2(l-y3)O0otherwise.(5.80)1,=JoofY2,Y.3(y2,y3)dy2-oo=fl4y2(llo-y3)dy2=2(1-y3).(5.81)Thecompleteexpressionis2(1-y3)0<Y3<1,0otherwise.(5.82)Exarnple5.22dernonstratesthatafairlysirnpleexperirnentcar1generateajointPMFthat,intableforrn,isperhapsst1rprisingl}'long.Ir1fact,apracticalexperirner1toftengeneratesajoir1tPMForPDFthatisforbiddinglycornplex.Theirnportar1texceptionisanexperimer1tthatprod11cesn,independentrandorr1variables.Thefollov.ringdefir1itionexter1dsthedefir1itionofir1dependenceoftworandornvariables.ItstatesthatX1,...,Xnareindeper1dentw11enthejointPJ\l.IForPDFcanbefactoredintoaprodt1ctofn,rr1argir1alPJ\l.IFsorPDFs.[200CHAPTER5MULTIPLERANDOMVARIABLESDefinition5.14NIndependentRandomVariablesRan,dornvariablesX1,...,Xnareindependentifforallx1,...,x;n,Discrete:Independer1ceofn,rar1domvariablesistypicallyapropertyofanexperirr1entconsistingofriindeper1dentst1bexperirnents,in"''hichsubexperirnentiproducestherar1domvariableX,;,.Ifallsubexperimentsfollov.rthesarneprocedt1reandhavethesarr1eobservation,alloftheXihavethesarnePMForPDF.Inthiscase,-vvesaytherar1dornvariablesX,;,areiden,ticallydistribv,ted.---Definition5.1IndependentandIdenticallyDistributed(iid)X1,...,Xnareindependentandidenticallydistributed(iid)ifDiscrete:Example5.24TherandomvariablesXLet1,...,XnhavethejointPDF10<x;,;,<1,i=1,...,n,,0otherwise.(5.83)AdenotetheeventthatrnaxiX,;,<1/2.FindP[A).WecansolvethisproblembyapplyingTheorem5.24:(5.84)grows,theprobabilitythatthemaximumislessthan1/2rapidlygoesto0.WenotethatinspectionofthejointPDFrevealsthatX1,...,X4areiidcontinuousuniform(0,1)randomvariables.TheintegrationinEquation(5.84)iseasybecauseindependenceimpliesAsriP[A)=P[X1<1/2,...,Xn<1/2)=P[X1<1/2)x···xP[Xn<1/2)=(1/2)11-.(5.85)[s.11==--Quiz5.10--==:::iTherandornvariablesY1,...,MATLAB201Y4havethejoir1tPDF40<Y1<Y2<1,0<Y3<Y4<1,0othervvise.(5.86)LetCdenotetheeventtr1atrnaxiYi<1/2.Fir1dP[C).5.11MA1.,LABItisconvenienttouseJ\IIATLABtogeneratepairsofdiscreterandornvariablesXand·ywitr1anarbitraryjointPJ\IIF.Therearenogenerallyapplicabletecr1r1iquesforgenera.tingsa.rnplepairsofacontinuousrandomvariable.TherearetechniquestailoredtospecificjointPDFs,forexarr1ple,bivariateGaussian.MATLABisausefultoolforstud}ringexperirr1entsthatproduceapairofrandomvariablesX,Y.Sirr1ulationexperimentsoftendependor1thegenerationofsarnplepairsofrandom·variableswitr1specificprobabilit}'models.Thatis,givenajointPMFPx,y(:;c,y)orPDFf'x,y(x,y),vveneedtoproduceacollectionofpairs{(x1,Y1),(:c2,Y2),...,(xrn,Yrn)}.Forfinitediscreterandornvariables,vveareabletodevelopsornegeneraltechniques.Forcontinuot1srar1domvariables,wegivesornespecificexamples.DiscreteRandomVariablesvVestartv.riththecasewhenXandYarefiniterar1domvariablesvvithrangesSy={Y1)...)Yrn}.(5.87)Inthiscase,vvecantakeadvantageofJ\IIATLABtechr1iquesforsurfaceplotsofg(x,y)overthex,yplane.Ir1J\IIATLA.B,vierepresentSxandSybythen,elernent·vectorsxandrn,elernentvectorsy.Thefunction[SX,SY]=ndgrid(sx,sy)producesthepairofn,xrn,rr1atrices,IX!SX=lx.n~11;i;.nSY=JIYIY;nll;lY~nj(5.88)WerefertornatricesSXaridSYasasarnplespacegridbecausethe}'areagridrepresentationofthejointsamplespaceSx,Y={(x,y)lxESx,YESy}.(5.89)[202CHAPTER5MULTIPLERANDOMVARIABLESThatis,[SX(i,j)SY(i,,j)]ist11epair(xi,Y.i)·Tocorr1pletetheprobabilitymodel,forXaridY,inJ\l.IATLAB,v.reerr1ployt11en,xmrnatrixPXYsucl1thatPXY(i,j)=Px,Y(x,i,Y1)·Tornakest1rethatprobabilitieshavebeengeneratedproperly,wenotethat[SX(:)SY(:)PXY(:)]isarnatrixvvhoserovvslistallpossiblepairsXi,Y.7andcorrespondingprobabilitiesPx,Y(x,i,Y.7).Gi·venafunctior1g(x,y)t11atoperatesontheelernentsofvectorsxandy,theadvantageofthisgridapproachisthattheMATLABfur1ctior1g(SX,SY)v.rillcalculateg(x,y)foreachxESxandyESy.Inparticular,g(SX,SY)producesann,xrn,matrix'\vitl1i,jthelerr1entg(x,i,Y.i)·Example5.25AnInternetphotodeveloperwebsiteprintscompressedphotoimages.Eachimagefilecontainsavariable-sizedimageofXx·ypixelsdescribedbythejointPMFPxy(x,y)x=800x=1200x=1600y=400y=800y=12000.20.0500.050.20.10.10.10.2.(5.90)ForrandomvariablesX,·y,writeascriptimagepmf.mthatdefinesthesamplespacegridmatricesSX,SY,andPXY.Inthescriptimagepmf.m,thematrixSXhas[800SYhas[400800thevariables:12001600]'foreachcolumnand1200]foreachrow.Afterrunningimagepmf.m,wecaninspect%irnageprnf.rnPXY=[0.20.050.1;0.050.20.1;00.10.2];[SX,SY]=ndgrid([80012001600],...[4008001200]);>>irnageprnf;SXsx=80012001600>>SYSY=4004004008001200160080012001600800800800120012001200Example5.26At24bits(3bytes)perpixel,a10:1imagecompressionfactoryieldsimagefileswithB=0.3x·ybytes.FindtheexpectedvalueE(B]andthePMFPB(b).%irnagesize.rnirnageprnf;SB=O.3*(SX.*SY);eb=surn(surn(SB.*PXY))sb=unique(SB)'pb=finiteprnf(SB,PXY,sb)'Thescriptimagesize.mproducestheexpectedvalueaseb,andproducesthePMF,whichisrepresentedbythevectorssbandpb.The3x3matrixSBhas.J"theIementg(Xi,Y.i)=0.3;i;iY.i.ThecaIcuIat.ioni,ofebissimplyaMATLABimplementationofTheorem5.9.SincesomeelementsofSBareidentical,sb=unique(SB)extractstheuniqueelements.AlthoughSBandPXYareboth3x3[s.11MATLAB203>>imagesizeeb319200sb96000144000192000288000384000432000576000pb0.20000.05000.05000.30000.10000.10000.2000Figure5.7Outputresultingfron1imagesize.minExample5.26.matrices,eachisstoredinternallybyl\IIATLABasa9-elementvector.Hence,wecanpassSBandPXYtothefinitepmf()function,whichwasdesignedtohandleafiniterandomvariabledescribedbyapairofcolumnvectors.Figure5.7showsoneresuItofrunningtheprogramimagesize.ThevectorssbandpbcomprisePB(b).Forexample,PB(288000)=0.3.RandomSamplePairsForfiniterandorr1variablesX,·ydescribedbySx,SyandjointPl\!IFPx,y(x;,y),orequi·valentl}'SX,SY,andPXYinl\IIATLAB,wecar1generaterandornsarr1plepairst1singthefunctionfiniterv(s,p,m)definedinChapter3.Recallt11atx=finiterv(s,p,m)returnedmsarnples(arrangedasacolumnvectorx)ofarandomvariableXsuchthatasarr1pleval11eiss(i)v.rithprobabilityp(i).Infact,tos11pportrandorr1variablepairsX,Y,thefunctionw=finiterv(s,p,m)perrr1itsstobeakx2rr1atrix-vvheret11erO'\VSofsen11merateallpairs(x,y)wit11nonzeroprobabilit}'·Giver1thegridrepresentatior1SX,SY,andPXY,wegeneratern,sarr1ple..pairsviaxy=finiterv([SX(:)SY(:)],PXY(:),m)Inparticular,the'ithpair,SX(i),SY(i),'\villoccurwit11probabilityPXY(i).Theot1tputxy-vvillbeanrn,x2matrixsuchthateachro-vvrepreser1tsasamplepairx,y.===-ExampIe5.27i"---==::::1Writeafunctionxy=imagerv(m)thatgeneratesrnsamplepairsoftheimagesizerandomvariablesX,YofExample5.26.Thefunctionimagervusestheimagesize.mscripttodefinethematricesSX,SY,andPXY.Itthencallsthefiniterv.mfunction.Hereisthecodeimagerv.mandasamplerun:functionxy=imagerv(m);imagepmf;S=[SX(:)SY(:)];xy=finiterv(S,PXY(:),m);>>xy=imagerv(3)xy80012001600400800800[204CHAPTER5MULTIPLERANDOMVARIABLESExample5.27car1begeneralizedtoproducesarnplepairsforan:ydiscreterandomvariablepairX,·y.Hovvever,giver1acollectionof,forexarnple,rn,=10,000sarr1plesofX,·y,itisdesirabletobeabletocheckv.r11etherthecodegeneratesthesarnplepairsproperly.Inparticula.r'V\Tevvishtocheckforeacr1;_r;ExandyEyvvhethertherelativefrequencyofx;,yinrn,samplesisclosetoPx,Y(x;,y).Ir1thefollo\vingexarr1ple,viedevelopaprogramtocalculatearr1atrixofrelativefrequenciesthatcorrespondstotherr1atrixPXY.ssExample5.28GivenalistxyofsamplepairsofrandomvariablesX,YwithJVI.A.TLABrangegridsSXandSY,writeal\IIATLABfunctionfxy=freqxy(xy,SX,SY)thatcalculatestherelativefrequencyofeverypairx,y.Theoutputfxyshouldcorrespondtothematrix[SX(:)SY(:)PXY(:)].[SX(:)SY(:)]infreqxyhasrowsthatlistallpossiblepairsx,y.Weappendthismatrixtoxytoensurethatthenewxyhaseverypossiblepair;r:,y.Next,theuniquefunctioncopiesalluniquerowsofxytothematrixUandalsoprovidesthevectorJthatindexestherowsofxyinU;thatis,xy=U(J).Inaddition,thenumberofoccurrencesofjinJindicatesthenumberofoccurrencesinxyofrowjinU.ThusweusethehistfunctiononJtocalculatetherelativefrequencies.Weincludethecorrectionfactor-1becausewehadappended[SX(:)SY(:)]toxyatthestart.Lastly,wereordertherowsoffxybecausetheoutputofuniqueproducestherowsofUinadifferentorderfrom[SX(:)SY(:)PXY(:)].functionfxy=freqxy(xy,SX,SY)xy=[xy;SX(:)SY(:)];[U,I,J]=unique(xy,'rows');N=hist(J,1:max(J))-1;N=N/sum(N);fxy=[UN(:)];fxy=sortrows(fxy,[213]);ThematrixMATLABprovidesthefur1ctior1stem3(x,y,z),"''herex,y,andzarelengthn,vectors,for·visualizingabi·variatePl\![FPx,Y(x;,y)orforvisualizir1grelativefrequenciesofsamplevaluesofapairofrandomvariables.Ateac11positionx(i),y(i)onthexyplane,thefunctiondrawsastemofheig11tz(i).Example5.29Generatern,=10,000samplesofrandomvariablesX,·yofExample5.26.Calculatetherelativefrequenciesandusestem3tographthem.Thescriptimagestem.mgeneratesthefollowingrelativefrequencystemplot.'/.imagestem.mimagepmf;xy=imagerv(10000);fxy=freqxy(xy,SX,SY);stem3(fxy(:,1),...fxy(:,2),fxy(:,3));xlabel('\itx');ylabel('\ity');.'.........tottIfo•I..'...0.20.1•ttIt"'t•0t..oot•t•o•IotI•16000f'ftI..fIIoIo0-~::::--_:__-__:..:.~___..1200800400y00x[s.11MATLAB205ContinuousRandomVariablesForcontinuot1srandorr1\rariables,MATLABcanbe11sefulir1avariet:yofways.Sorr1eoftheseareobvious.Forexarnple,ajointPDFfx,Y(x,y)orCDFFx,Y(x,y)canbe\rieV\redusingthefunctionplot3.Figure5.4wasgeneratedthisway.Howe\rer,forger1eratir1gsarr1plepairsofcontinuot1srandornvariables,therearenogeneraltechniquessuchasthesarnplespacegridsweerr1ployedvvithdiscreterar1dornvariables.When\rveintrod11cedcontinuo11srandorn\ra.riablesinC11apter4,wealsointroducedfarriiliesofvvidel}'usedrandorr1\rariables.InSection4.8,weprovidedacollectionofMATLABft1nctionss11chasx=erlangrv(n,lambda,m)togeneratemsarnplesfrorr1thecorrespondingPDF.HoV\rever,forpairsofcontinuousrandomvariables,"''eintrodt1cedonlyor1efamilyofprobabilityrr1odels,r1amel}'thebi\rariateGaussianrandorr1variablesXand·y.ForthebivariateGaussia.r1model,wecanuseTheorem5.21andtherandnfunctiontogeneratesamplevalues.T11ecornrnandZ=randn(2,1)ret11rnsthevectorZ=[Z1Z2\r.rhereZ1andZ2areiidGaussian(0,1)randorn\rariables.Next"''eforrr1thelir1earcorr1binationsJ'(5.91a)W1=0"1Z1W2=p0"2Z1+V(1-p2)0"~Z2(5.91b)FrornTheorerr15.21weknoV\rthatW1andl;T/2areabivariateGaussianpair.Inaddition,fromt11eforrr1ulasgiveninTheorerr15.21,\rvecansho\r.rthatE[lV1]E[W2]=0,Var[W1]=O"r,Var['VT12]=O"~andp11v1,w2=p.Thisimpliesthat(5.92)isapairofbivariateGaussianrandorr1\rariables"''ithE[X,i]=/J,i,·var[X,i]=O"f,andpx1,x2=p.Weirr1plementthisalgorithmthattransforrnst11eiidpairZ1,Z2ir1tot11ebivariateGaussiar1pairX1,X2intheMATLA.Bfunctior1xy=gauss2var(mx,my,sdx,sdy,r,m)Theoutputxyisa2xrn,rnatrixin\r.rhicheach2-elementcol11mr1isasarnpleofabi\rariateGaussianpairX,Ywithpararr1etersµ,x=mx,µ,y=my,O"x=sdx,O"y=sdyandcovariancePx,Y=r.Inthiscode,muisa2xrn,matrixinwhicheac11colurr1r1holdst11epairmx,my.Eachcolt1rr1nofrandn(2,m)isapairZ1,Z2ofindepender1tGaussian(0,1)randornvariables.T11ecalculationA*randn(2,m)irr1plementsEquation(5.91)form,difl'erer1tpairsZ1,Z2.functionxy=gauss2rv(mx,sdx,my,sdy,r,m)mu=[mxmyJ';cxy=r*sdx*sdy;C=[sdx~2cxy;cxysdy~2];xy=gaussvector(mu,C,m);[206CHAPTER5MULTIPLERANDOMVARIABLESg.--~~~....-~~~--~~~-Thesarnpleoutp11tofgauss2varshovvnhereisprod11cedv.riththecornrr1ands64>>xy=gauss2rv(3,3,5,1,0.5,500);>>plot(xy(1,:),xy(2,:),'.');vVeobservet11a.tthecenteroft11ecloudis(JJ,x,/J,y)=(3,5).Inadditior1,'""'er1otethattheXandYaxesarescaleddifferentl}'beca11se•••2,__~~......._..._~~~......_~~~~-10010CJx=3andCJy=l.20Weobservethatthisexarr1plevvithPx,Y=0.5shov.rsrar1dornvariablesthatarelesscorrelatedthantheexarnplesinFigure5.5vvithIPI=0.9.Wenotet11atbivariateGaussianrandorr1variablesareaspecialcaseofn,dimensionalGaussianrar1dornvectors,v.r11ichareintroducedinChapter8.Basedonlinearalgebratechniques,C11apter8introducesthegaussvectorfunctiontogeneratesarnplesofGaussianrandornvectorsthatgeneralizesgauss2rvtoridirnens1or1s.BeyondbivariateGaussianpairs,thereexistavarietyoftec11niquesforgeneratingsarnplevah1esofpairsofcontir1uousrar1domvariablesofspecifict}rpes.AbasicapproachistogenerateXbasedonthemarginalPDF1·x(x)andthengenerateYllSingaconditionalprobabilityrnodelthatdependsor1theval11eofX.ConditionalprobabilityrnodelsandMATLABtechniquest11atemplO}'theserr1odelsarethesubjectofC11apter7.ProblemsDifficulty:EasyModerate5.1.1RandomvariablesXandYhavethejointCDF(1-e-x)(l-e-Y)Fx,Y(x,y)=1;y>O;>0,o,v.(a)WhatisP[X<2,Y<3]?(b)WhatisthemarginalCDF,Fx(x)?(c)\i\fhatisthemarginalCDF,Fy(y)?5.1.2Expressthefollowin.gextremevaluesofFx,Y(x,y)intermsofthemarginalcumulativedistributionfunctionsFx(1;)andFy(y).(a)Fx,y(x,-oo)(b)Fx,y(1;,oo)(c)Fx,y(-00,00)(d)Fx,y(-oo,y)Difficult+ExpertsOnly(e)Fx,y(oo,y)ForcontinuousrandomvariablesX,Y'vithjointCDFFx,Y(x,y)andinarginalCDFsFx(x)andFy(y),findP[x1<X<1;2Uy1<Y<y2].Thisistheprobabilityoftheshaded"cross"regioninthefollowingdiagram.5.1.3y...................................................................................................................................~~~y;·.....~::~·:.·~:.·~:.·~:.·~:.·~:.·~:~j~j~j~j~j~j~j~.....:~·.:~·.:~·.:~·.:~·.:~·.:~·.:~.·~:.............~:~:~:~:~:~:.~.~~~~~~~.......................................................................................................................x[PROBLEMS5.1.4RandomvariablesXandYhaveCDFFx(x)andFy(y).IsF(x,y)=Fx(x)Fy(y)avalidCDF?Explainyourans,ver.5.1.5Inthisproblem,rem5.2.\VeA={X<X1'Y1<y<y2}'B={::r1<X<x2,Y<y1},={x1<X<x2,Y1<Y<y2}.(b)ExpresstheprobabilityoftheeventsA,B,andAUBUCintermsofthejointCDFFx,Y(x,y).(c)UsetheobservationthateventsA,B,andCaremutuallyexclusivetoproveTheorem5.2.5.1.6CanthefollowingfunctionbethejointCDFofrandomvariablesXandY?1-F(1;,y)=e-(x+y){O<X]?(c)\t\lhatisP[Y>X]?(b)\tVhatisP[Y(d)\tVhatisP[Y=X]?(e)WhatisP[X<1]?proveTheo-(a)SketchthefollowingeventsontheX,Yplane:C207x>O,y>O,otherwise.5.2.1RandomvariablesXandYhavethejointPl\!IFcxyx=1,2,4;P.x,·y(x,y)={other\vise.0y=1,3,5.2.3Testt\vointegratedcircuits.Ineachtest,theprobabilityofrejectingthecircuitisp,independentoftheothertest.LetXbethenumberofrejects(either0or1)inthefirsttestandletYbethenumberofrejectsinthesecondtest.FindthejointPl\!IFPx,y(1;,y).5.2.4Fortwoindependentflipsofafaircoin,letXequalthetotalnumberoftailsandletYequalthenumberofheadsonthelastflip.FindthejointPl\!IFPx,Y(x,y).5.2.5•InFigure5.2,theaxesofthefiguresarelabeledXandYbecausethefigtu·esdepictpossiblevaluesoftherandomvariablesXandY.Ho,vever,thefigureattheendofExample5.3depictsPx,y(x,y)onaxeslabeled'vithlovvercasexandy.ShouldthoseaxesbelabeledwiththeuppercaseXandY?Hint:Reasonableargumentscanbemadeforbothviews.5.2.6AsageneralizationofExample5.3,consideratestofncircuitssuchthateachcircuitisacceptablewithprobabilityp,independentoftheoutcomeofanyothertest.ShowthatthejointPl\!IFofX,thenumberofacceptablecircuits,andY,thenumberofacceptablecircuitsfoundbeforeobservingthefirstreject,is(a)Whatisthevalueoftheconstantc?<X]?(c)\i\fhatisP[Y>X]?Px,Y(;i;,y)(d)WhatisP[Y=X]?Px0other,vise.(b)WhatisP[Y(e)\i\1hatisP[Y=3]?5.2.2RandomvariablesXandYhavethejointPl\!IFPx,Y(x,y)=cl1;+YIx=-2,0,2;y=-1,0,1,0otherwise.(a)Whatisthevalueoftheconstantc?(nxyyI)Px(l-p)n-x0<y<;i;<nnHint:For0{x,=y=r1,,<y<x<n,showthat=x,Y=y}=_4nBnc,whereA:Thefirstytestsareacceptable.B:Testy+1isarejection.C:Theremainingn,-y-1testsyieldx-yacceptablecircuits[208CHAPTER5MULTIPLERANDOMVARIABLES5.2.7\i\fitht'vominutesleftinafiveminuteovertime,thescore.is0-0inaRutgerssoccerinatchversusVillanova.(NotethattheovertimeisNOTs11,dden-death)Inthenext-to-lastminuteofthegame,either(1)Rutgersscoresagoalwithprobabilityp=0.2,(2)'lillanovascoreswithprobabilityp=0.2,or(3)neitherteamscoreswithprobability1-2p=0.6.Ifneitherteamscoresinthenext-to-lastminute,theninthefinalminute,either(1)Rutgersscoresagoalwithprobabilityq=0.3,(2)Villanovascoreswithprobabilityq=0.3,or(3)neitherteamscoreswithprobability1-2q=0.4.However,ifateamscoresinthenext-to-lastminute,thetrailingteamgoesforbrokesothatinthelastminute,either(1)theleadingteamscoreswithprobability0.5,or(2)thetrailingteamscores'vithprobability0.5.Forthefinaltwominutesofovertime:(a)SketchaprobabilitytreeandconstructatableforPR,v(r,v),thejointPMFofR,thenumberofRutgersgoalsscored,andV,thenumberofVillanovagoalsscored.(b)Whatistheprobabilit)rP[T)thattheovertimeendsinatie?(c)\i\fhatistheIY~l[FofR,thenumberofgoalsscoredbyRutgers?(d)WhatisthePMFofG,thetotalnumberofgoalsscored?5.2.8Eachtestofanintegratedcircuitproducesanacceptablecircuit'vithprobabilityp,independentoftheoutcomeofthetestofanyothe.rcircuit.Intestingn,circuits,letJ{denotethenumberofcircuitsrejectedandletXdenotethenumberofacceptablecircuits(either0or1)inthelasttest.FindthejointPMFPK,x(k,1;).5.2.9Eachtestofanintegratedcircuitproducesanacceptablecircuit'vithprobabilityp,independentoftheoutcomeofthetestofanyothercircuit.Intestingn,circuits,letJ{denotethenumberofcircuitsrejectedandletXdenotethenumberofacceptablecircuitsthatappearbeforethefirstrejectisfound.FindthejointPMFPK,x(k,x).5.3.1GiventherandomvariablesXandYinProblem5.2.1,find(a)ThemarginalPMFsPx(x)andPy(y),(b)TheexpectedvaluesE[X)andE[Y],(c)ThestandarddeviationsO'xandO'y.5.3.2GiventherandomvariablesXandYinProblem5.2.2,find(a)ThemarginalPMFsPx(x)andPy(y),(b)TheexpectedvaluesE[X)andE[Y],(c)ThestandarddeviationsO'xandO'y.5.3.3Forn,=0,1,...and0<k<100,thejointPtv1FofrandomvariablesNandJ{isPN,K(n,,k)=lOOne-lOOIri.(100)i.K:pk(l_p)100-k.Other,vise,PN,K(n,k)=0.FindthemarginalPlVIFsPN(r1,)andP11/21Px,Y(x,y)=X=0,1,2,3,4,5;y=0,1,...'x'0otherwise.FindthemarginalPMFsPx(:i;)andPy(y)andtheexpectedvaluesE[X)andE[Y).5.3.5RandomvariablesNandJ{havethejointP~IIFk=l,...,n;n=l,2,...other,vise.FindthemarginalPMFsPN(n,)andP1n=0,1,...other,vise.[PROBLEMS209FindthemarginalPMFPN(n).Sho'vthatthemarginalPMFPK(k)satisfiesPK(k)=P[N>k]/100.Sketchtheregionofnonzeroprobabilityandansvverthefollowingquestions.5.4.1RandomvariablesXandYhavethejointPDF(b)'\i\fhatis.fx(x)?.fx,Y(x,y)={c0(c)WhatisE[X]?x>0,y>0,xotherwise.+y<1,(a)Whatisthevalueoftheconstantc?(b)WhatisP[X<Y]?(c)WhatisP[X+Y<1/2]?5.4.2RandomvariablesXandYhavejointPDFfx,Y(x,y)={cxy2OY]andP[Y<X2].(c)FindP[min(X,Y)<1/2].(d)FindP[max(X,Y)exJustify5.5.3XandYarerandomvariables'viththejointPDF(a)Findtheconstantc.(b)FindP[X>OJ?(a)\tVhatisP[Xfx,Y(x,y)={<3/4].5.4.3RandomvariablesXandYhavejointPDF~x+y<1)1;>0,y>0,otherwise.(a)\tVhatisthemarginalPDFfx(x)?(b)\tVhatisthemarginalPDFfy(y)?.{6efx,·y(x,y)=0(2x+3y)x>O,y>O,other,vise.>Y]andP[X+Y<l].(b)FindP[1nin(X,Y)>l].5.5.4OverthecircleX2+Y2<r2,randomvariablesXandYhavetheuniformPDF(a)FindP[X(c)FindP[max(X,Y)<l].5.4.4RandomvariablesXandYhavejointPDF.y)-_(X.YJ.,,.t·{2+y2.<r2,1/(7rr2)x0otherwise.(a)WhatisthemarginalPDFfx(x)?(b)\tVhatisthemarginalPDFfy(y)?f.x,Y(x,y)={8xy0O<yFollo,vingthemethodofExample5.8,findthejointCDFFx,y(:i;,y).5.5.1RandomvariablesXandYhavethejointPDF1/2fx,y(x,y)={0-l5.5.5XandYarerandomvariableswiththejointPDF.fX,Y(:i;,y)={5x/22Q-l0,other,vise.\i\fhatisP[X25.7.3ArandomECEsophomorehasheightX(roundedtothenearestfoot)andGPAY(roundedtothenearestinteger).TheserandomvariableshavejointPMFPx,y(1;,y)l11=ly=2y=3y=4<X1]?5.6.7Intermsofapositiveconstantk,randomvariablesXandYhavejointPDF.,y)={k+3xfx,Y(x20-l/2Var[Y1).5.7.11XandYarerandomvariables'vithE[X)=E[Y)=0suchthatXhasstandarddeviationax=2\vhileYhasstandarddeviationay=4.(a)ForV=X-Y,whatarethesmallestandlargestpossiblevaluesofVar[V)?(b)ForvV=X-2Y,\\rhatarethesmallestandlargestpossiblevaluesofVar[vV)?5.7.12RandomvariablesXandYhavejointPDF.fx,Y(X,y)={4:Q:eyO38,thentheperson'sWBCcountismeasured.IfvV>10,thepersonisdeclaredill(eventI).(a)SupposeWandTareuncorrelated.WhatisP[I)?Hint:Drawatreediagramfortheexperiment.(b)No\vsupposeWandThavecorrelationcoefficientpw,'r=1/J2.FindtheconditionalprobabilityP[IIT=t]thatapersonisdeclaredillgiventhattheperson'stemperatureisT=t.5.9.8SupposeyourgradeinaprobabilitycoursedependsonyourexamscoresX1andX2.Theprofessor,afanofprobability,releasesexamscoresinanormalizedfashionsuchthatX1andX2areiidGaussian(Jl,=0,a=J2)randomvariables.YoursemesteraverageisX=0.5(X1+X2).(a)YouearnanAgradeifXisP[A)?>1.\i\fhat(b)ToimprovehisSIRS(StudentInstructionalRatingService)score,theprofessordecidesheshouldawardmoreA's.NowyougetanAifmax(X1,X2)>1.\i\fhatisP[-4)no,v?(c)Theprofessorfoundoutheisunpopularatratemyprofessor.comanddecidestoa\vardanAifeitherX>1ormax(X1,X2)>1.No\vwhatisP[A)?(d)·undercriticismofgradeinflationfromthedepart1nentchair,theprofessoradoptsanewpolicy.AnAisa\vardedifmax(X1,X2)>1andmin(X1,X2)>0.NO\V'vhatisP[A)?[PROBLEMS2155.9.9Yourcoursegradedependsont'votestscores:X1andX2.'YourscoreXiontestiisGaussian(µ=74,a=16)randomvariable,independentofanyothertestscore.where(a)\i\lithequalweighting,gradesaredeterminedbyY=X1/2+X2/2.YouearnanAifY>90.WhatisP[-4]=P[Y>90]?(a)FindtheCDFFy1(y1)intermsofthe(b)Astudentaskstheprofessortochoosea'veightfactor'llJ,0<'l.JJ<1,suchthatFindP[-4]asafunctionoftheweight'l.1J.WhatvalueorvaluesofVJmaximizeP[A]=P[Y>90]?(c)AdifferentstudentproposesthatthebetterexamistheonethatshouldcountandthatgradesshouldbebasedonM=max(X1,X2).Inafitofgenerosity,theprofessoragrees!No'v'vhatisP[A]=P[M>90]?(d)Howgenerouswastheprofessor?Inaclassof100students,'vhatistheexpectedincreaseinthenumberofA'sa'varded?5.9.10·underwhatconditionsontheconstantsa,b,c,anddisajointGaussianPDF?5.9.11ShowthatthejointGaussianPDFfx,·y(x,y)givenbyDefinition5.10satisfies1_:1_:fx,Y(x,y)d:i;dy=1.sgn(x)={l-1xx>0,<o.(b)Sho'vthatY1andY2arebothGaussianrandomvariables.(c)i\reY1andY2bivariateGaussianrandomvariables?5.10.1Everylaptopreturnedtoarepaircenterisclassifiedaccordingitsneededrepairs:(1)LCDscreen,(2)motherboard,(3)keyboard,or(4)other.ArandombrokenlaptopneedsatypeirepairwithprobabilityPi=24-i/15.Let1',Tiequalthenumberoftypeibrokenlaptopsreturnedonadayinwhichfourlaptopsarereturned.(a)FindthejointPMFofNi,N2,N3,N4.(b)\t\fhatistheprobabilitythattwolaptopsrequireLCDrepairs?(c)\t\fhatistheprobabilitythatmorelaptopsrequiremotherboardrepairsthankeyboardrepairs?5.10.2Whenorderingapersonalcomputer,acustomercanaddthefollo,vingfeaturestothebasicconfiguration:(1)additionalmemory,(2)flatpaneldisplay,(3)professionalsoftware,and(4)wirelessmodem.ArandomcomputerorderhasfeatureiwithprobabilityPi=2-iindependentofotherfeatures.Inanhourin'vhichthreecomputersareordered,letNiequalthenumberofcomputerswithfeaturei.(a)FindthejointPMFHint:·useEquation(5.68)andtheresultofProblem4.6.13.5.9.12RandomvariablesX1andX2areindependentidenticalGaussian(0,1)randomvariables.Let(b)\tVhatistheprobabilityofsellingaco1nputer'vithnoadditionalfeatures?(c)Whatistheprobabilityofsellingacomputer,\rithatleastthreeadditionalfeatures?[216CHAPTER5MULTIPLERANDOMVARIABLES5.10.3TherandomvariablesX1,...,X11havethejointPDF1fx1,...,Xn(1;1,...,Xn)=0Ti.•Otherwise,yourejectsuitcaseiandopensuitcasei-1.•Ifyouhaverejectedsuitcasesn,downthrough2,thenyoumustaccepttheamountX1insuitcase1.ThusthethresholdTi=0sinceyouneverrejecttheamountinthelastsuitcase.(a)Supposeyourejectsuitcasesn,throughi+1,butthenyouacceptsuitcasei.FindE[XilXi>Ti].(b)LetWkdenoteyourre\vardgiventhattherearekunopeneds11itcasesremaining.\i\fhatisE[vV1)?(c)AsafunctionofTk,findarecursiverelationshipforE[vVk)intermsofTkandE[Wk-1).(d)Forn,=4suitcases,findthepolicy(T{,...,T;),thatmaximizesE[vV4).Giventheset{U1,...,Un}ofiiduniform(0,T)randomvariables,wedefine5.10.112175.11.1ForrandomvariablesXandYinExample5.26,usel\IIATLABtogeneratealistoftheformX1X2Y1Y2Px,Y(x1,Y1)Px,Y(x2,Y2)thatincludesallpossiblepairs(x,y).5.11.2ForrandomvariablesXandYinExample5.26,uselVIATLABtocalculateE[X),E[Y),thecorrelationE[XY),andthecovarianceCov[X,Y).5.11.3YougeneraterandomvariablevV=WbytypingW=sum(4*randn(1,2))ina1VIATLABCo1nmand\vindo\v.\i\fhatisVar[vV)?5.11.4\i\fritetrianglecdfplot.m,ascriptthatgraphsFx,Y(x,y)ofFigure5.4.5.11.5Problem5.2.6extendedExample5.3toatestofricircuitsandidentiasthekth"smallest"elementoftheset.Thatis,X1istheminimumele1nent,X2isthesecondsmallest,andsoon,uptoXn,whichisthemaximumelementof{U1,...,Un}.NotethatX1,...,Xnarekno\vnastheorderstatisticsofU1,...,Un.Provethat={~!/T"0<::r1<···<otherwise.Xn<T,fiedthejointPDFofX,thenumberofacceptablecircuits,andY,thenumberofsuccessfultestsbeforethefirstreject.\i\fritea1VIATLABfunction[SX,SY,PXY]=circuits(n,p)thatgeneratesthesamplespacegridforthen,circuittest.Checkyourans,veragainstEquation(5.11)forthep=0.9andn,=2case.Forp=0.9andn,=50,calculatethecorrelationcoefficientpx,Y.[ProbabilityModelsofDerivedRandomVariablesTherearerr1anysituationsinvvhic11vveobserveoneormorerandomvariablesandusetheirvaluestocorr1puteanevvrandorr1·variable.Forexarnple,vvhen·voltageacrossanroohrnresistorisarar1domvariableX,thepovverdissipatedint11atresistorisY=X2/r0.Circuitdesigr1ersneedaprobabilitymodelforYtoe·valuatethepowerconsl1rr1ptior1ofthecircuit.Similarly,ift11earr1plitude(currentorvoltage)ofaradiosignalisX,thereceivedsignalpovverisproportionalto·y=X2.AprobabilityrnodelforYisessentialinevaluatir1gtheperforrnar1ceofaradioreceiver.T11eot1tputofalirr1iterorrectifierisanot11errar1domvariablet11atacircuitdesignerrr1ayneedtoanalyze.Radiosystemsalsopro·videpracticalexarnplesofft1nctionsoftworandorr1variables.Forexarr1ple,wecandescribet11earr1plitudeofthesigT1altransrnittedbyaradiostationasarandorr1variable,X.Wecandescribetheattenuationofthesigr1alasittra·velstotheanter1naofarnovingcarasanotherrandom·variable,Y.Ir1thiscasetheamplitudeofthesignalatt11eradioreceiverinthecaristherandorr1variablevV=X/Y.Ot11erpracticalexarr1plesappearincellulartelephonebasestationsv.rithtv.roantennas.T11earr1plitudesoft11esigr1alsarrivingatthetv.roantennasarernodeledasrandornvariablesXarid·y.Theradioreceiverconnectedtot11etvvoar1tennascanusethereceivedsigr1alsinavarietyofways.•Itcanchoosethesigr1alwitht11elargerarnplitudeandignoret11eotherone.Ir1t11iscase,thereceiverproducest11erandornvariableW=XifIXI>IYIandvV=·y,ot11erwise.Thisisanexarnpleofselect'ion,di'versitycornb'iri'irig.•Thereceivercanaddt11etwosignalsanduseW=X+Y.Thisprocessisreferredtoasequalga'iricornbin,in,gbecat1seittreatsbothsignalseqt1a1ly.•Athirdalternativeistocorr1binet11etv.rosignalsunequallyinordertogivelessv.reighttothesignalconsideredtobemoredistorted.Ir1thiscaseW=aX+b"Y.Ifaandbareoptirnized,thereceiverperforrnsrnax;'irnalratiocornb'in,in,g.218[6.1PMFOFAFUNCTIONOFTWODISCRETERANDOMVARIABLES219Allthreecorribiningprocessesappearinpracticalradiorecei·vers.Forrnally,-vvehavetliefollo-vvingsituations.•·vveperforrrianexperimentandobserveasarriplevalueofrandorrivariableX.Basedonourkr1ovvledgeoftlieexperiment,v.rehaveaprobabilitymodelforXerribodiedinthePNIFPx(:i;)orPDFfx(x).Afterperforrningtlieexperirrient,viecalculateasarriplevalueoftherandomvariableW=g(X).•vVeperforrriariexperirrientandobserveasarriple·valueoft-vvoraridomvariablesXaridY.Basedonot1rkriowledgeoftlieexperirnent,viehaveaprobabilityrnodelforXaridYernbodiedinajointPMFPx,Y(x;,y)orajoiritPDFfx,Y(x,y).Afterperforrningtheexperirrient,wecalculateasarriplevalueoftherandorrivariableW=g(X,·y).Inbothcases,therriatherriaticalproblernistodeterrrlinethepropertiesof'{;Tl.Previo11schaptersaddressaspectsofthisproblerri.Theorern3.9pro·videsaforrr111laforPv.1('w),tliePMFofVf!=g(X)andTheorerri3.10providesaforrriulaforE[vV)givenPx(x)andg(X).Chapter4,oncontint1ousrandom·variables,provides,iriTheorerri4.4,aforrriulaforE[W)givenfx(x;)aridg(X)butdeferstothischapterexarriiningtheprobabilitymodelofW.Sirnilarly,Chapter5examinesE[g(X,Y))bt1tdoesnotexplainhowtofindthePNIForPDFoflV=g(X,'Y).Inthischapter,-vvedeveloprnethodstoderivetliedistributiori(PMF,CDForPDF)ofafunctionoforieortworandorrivariables.Priorchaptersha:vealotofnewideasandconcepts,eachillt1stratedbyarelativelysrriallnurriberofexarnples.Iricontrast,tliischapterhasrelativelyfev.rriewconceptsbutrnanyexarriplestoillustratethetechniq11es.Inparticular,Sections6.2and6.3advocateasingleapproach:firidtlieCDFF1,v('w)=P[W<w)byfindingthosevaluesofXsuclithatW=g(X)<w.Sirriilarly,Section6.4usesthesamebasicidea:FindthosevaluesofX,·ysuclitliatW=g(X,·y)<w.Wliiletliisideaissimple,thederivatioriscanbecornplicated.6.1PMFofaFunctionofTwoDiscreteRandomVariablesPw('tJJ),thePMFofafunctionofdiscreterandornvariablesXandYisthesurrioftheprobabilitiesofallsarnplevalues(;i;,y)for-vvhichg(;i;,y)='ID.vVhenXaridYarediscreterandorri·variables,51,v,therangeofW,isacountablesetcorrespondingtoallpossiblevaluesofg(X,·y).Therefore,WisadiscreterandornvariableandliasaP1!{FP1t\1('ID).'\N"ecanapplyTlieorerri5.3tofindPw('w)=P[W='W).Since{W='W}isariotherriamefortlieevent{g(X,·y)=tu},-vveobtairiP1¥('w)b}'addingtlievaluesofPx,Y(x;,y)corresporidingtothex,ypairsfor-vvliichg(x,y)='W.===Theorem6.l-Fordiscreteran,dornvariablesXaridY;thederivedraridornvariableW=g(X,·y)[220CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLEShasPMFPx,y(:i;,y).Pw('uJ)=(:r;,y):g(:r;,y)=·wi::::::::==Example6.1=40=60Afirmsendsouttwokindsofnewsletters.Onekindcontainsonlytextandgrayscaleimagesandrequires0.10.15l=140centstoprinteachpage.Theotherkindcontains0.20.3l=2colorpicturesthatcost60centsperpage.Newslet0.10.15l=3terscanbe1,2,or3pageslong.LettherandomvariableLrepresentthelengthofanewsletterinpages.SL={1,2,3}.LettherandomvariableXrepresentthecostincentstoprinteachpage.Bx={40,60}.Afterobservingmanynewsletters,thefirmhasderivedtheprobabilitymodelshownabove.LetW=g(L,X)=LXbethetotalcostincentsofanewsletter.FindtherangeS1andthePMFP1"1('u1).;i;;i;"1PLx(l,x;)l=1l=2l=3;i;=40x=600.150.1(\!\1=40)(\!\1=60)0.30.2(\!\!=80)0.15(l!\1=120)(l!\1=120)0.1(l!\1=180)406080120180PM1('UJ)0.150.10.30.350.1ForeachofthesixpossiblecombinationsofLandX,werecordT.V=LXunderthecorrespondingentryinthePMFtableontheleft.TherangeofWisSw={40,60,80,120,180}.WiththeexceptionofW=120,thereisauniquepairL,XsuchthatT.V=LX.ForW=120,Pw(l20)=PL,x(3,40)+PL,x(2,60).Thecorrespondingprobabilitiesarerecordedinthesecondtableontheleft.'UJ6.2FunctionsYieldingContinuousRandomVariablesToobtainthePDFofT.V=g(X),acontinuousfunctionofacoritinuousrandorrivariable,derivetrieCDFofVT!andthendifferentiate.Theprocedureisstraightforvvardv.rheng(y;,y)isalinearf\1nction.Itisrnorecomplexforotherfunctions.vVhenXaridW=g(X)arecontinuot1srandorn·variables,wedevelopatwo-stepprocedt1retoderivethePDFf'w('w):1.FiridtheCDFFw('w)=P[W<'llJ].2.ThePDFisthederivativefw(w)=dFH1('U;)/dw.Thisprocedureahnaysvvorksaridisea.S}'torernerriber.Wrieng(X)isaliriearfurictionofX,thernethodisstraightforvvard.Othenvise,asv.reshallseeiriexa.rnples,findingFw(tv)canbetricky.[6.2FUNCTIONSYIELDINGCONTINUOUSRANDOMVARIABLES221Beforeproceedir1gtotheexarr1plesandtheorerns,weaddonererninder.Itiseasiertocalct1lateE[g(X)]directlyfrornthePDF1·x(x)t1singTheorem4.4thanitistoderivet11ePDFofY=g(X)andthenusethedefir1itionofexpected·value,Definition4.4.Thissectionappliestosituationsinwhichitisnecessar}'tofindacornpleteprobabilityrr1odelofW=g(X).Example6.2InExample4.2,lVcentimetersisthelocationofthepointeronthe1-metercircumferenceofthecircle.UsethesolutionofExample4.2toderivefv,1('1D).ThefunctionW=lOOX,whereXinExample4.2isthelocationofthepointermeasuredinmeters.TofindtheCDFW=lOOX100Fw('UJ)=P[W<w].thefirststepistotranslatetheevent'ID(-3J.L'W){W<'W}intoaneventdescribedbyX.Eachoutcomeof100'theexperimentismappedtoan(X,VT!)paironthelineW=lOOX.Thustheevent{WFx(x)X.Forthisprob-0x;<0,xO1.=(6.1)(6.2)Fromthisresult,wecanusealgebratofind0Fw('w)=Fx('ID)=100'ID1001w100<O,'U)0<<1,=-100'W>1100-)w<0,0'W10010<'W<100.-I(6.3)w>100.WetakethederivativeoftheCDFofVT!overeachoftheintervalstofindthePDF:fIt\!('ID)dF1,v('UJ)=d'w<100,1/10000otherwise.(6.4)[222CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLESWeseethatT1Vistheuniform(0,100)randomvariable.WellSethistvvo-stepprocedureinthefollowingtheoremtogeneralizeExarr1ple6.2byderivir1gtheCDFandPDFforan}'scalechangeandarl}'cor1tinuousrandorr1variable.Theorem6.2IfVV=aX;1JJherea,>0,then,MlhasGDFa'ndPDFf"w(vJ)=la,f"x('w/a).<'liJ]=P[X<'11J/a]=Fx('1D/a).Fw('w)=Fx('llJ/a),ProofFirst,\VefindtheCDFofW,F\!v(vJ)=P[aX(6.5)\'VetakethederivativeofFy(y)tofindthePDF:=dF;('1D)=fw('1D)1fx(vJ/a).(6.6)a'UJTheorern6.2statesthatrr1ltltiplyingarandorr1variableb}'apositi·veconstantstretches(a>1)ors11rinlf"x(x;)=2x;0<x<1.0otherwise.--(6.7)IFindthePDFofW=aX.SketchthePDFofWfora,=1/2,1,2..................................................................................4..--~~~~~~~~-----~a=l/22'WIa,21~a=2O"'===:.L---l~~--1.~~.....J0Foranya,>0,weuseTheorem6.2tofindthePDF:21w00<'W-I(6.8)3Asaincreases,thePDFstretcheshorizontally.Forthefarr1iliesofcontinl1ousrandorr1·variablesinSectior1s4.5and4.6,wecant1seT11eorern6.2toshov.rt11atrr1ultipl}ringarandorr1variablebyaconstar1tproducesanev.rfarr1ilyrr1err1berwitl1transforrr1edpararr1eters.[6.2FUNCTIONSYIELDINGCONTINUOUSRANDOMVARIABLES223Theorem6.3W=aX,1JJherearel="nofollow">0.(a)IfXis'Un,iforrn(b,c),then,Wis'U'niforrn(ab,ac).{b)IfXisexprJ'nen,tial(A),then,Mlisex;pon,en,tial(.\/a).(c)IfXisErlarig(n,,.\),then,WisErlan,g(n,,.\/a).{d)IfXisGaussiari(µ,,F(x')impliesthatxvariableX=p-I(U),vvecan'vriteFx(x)=P[F-1(U)<x]>x'.Hence,fortherandom=P[U<F(x)]=F(x).(6.16)vVeobservethatthereq1.iirerr1entthatFx('u)haveaninversefor0<u<1lirnitstheapplicabilityofT11eorern6.5.Forexarnple,thisreqt1irementisnotrnetbythernixedrandomvariablesofSection4.7.Ager1eralizatonofthetheorerr1thatdoesholdforrnixedrandorr1variablesisgi·veninProblern6.3.13.Thefollov.ringexamplesdemonstratetheutilityofT11eorem6.5.Example6.5Uistheuniform(0,1)randomvariableandX=g(U).Deriveg(U)suchthatXistheexponential(1)randomvariable.TheCDFofXisFx(:i;)=01-e-xx<0,x>o.(6.17)Notethatifv,=Fx(x;)=1-e-~r;.thenx=-lr1(l-u).Thatis,Fx1(v,)=-ln(l-'u)for0<v,<1.Thus,byTheorem6.5,X=g(U)=-ln(1-U)(6.18)istheexponentialrandomvariablewithparameter>.=1.Problem6.2.7asksthereadertoderivethePDFofX=-ln(l-U)directlyfromfirstprinciples.---Example6.6,___Forauniform(0,1)randomvariableU,findafunctiong(-)suchthatX=g(U)hasauniform(a,b)distribution.TheCDFofXisb.ll0Fx(x)=;i;(6.19)[226CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLESForanyusatisfying0<v,<1,v,=Fx(x)=(1';-a)/(b-a)ifandonlyifx=Fx1(v,)=a+(b-a)v,.(6.20)ThusbyTheorem6.5,X=a+(b-a)Uisauniform(a,b)randomvariable.NotethatwecouldhavereachedthesameconclusionbyobservingthatTheorem6.3implies(b-a)Uhasauniform(O,b-a)distributionandthatTheorem6.4impliesa+(b-a)Uhasauniform(a,(b-a)+o,)distribution.Anotherapproach,takeninProblem6.2.11,istoderivetheCDFandPDFofa+(b-a)U.Thetechniql1eofTheorem6.5isparticl1larlyusefulvvhent11eCDFisaneasil}'invertiblefunction.Unfortunatel}',t11erearemanyrandorr1·variables,includingGaussianandErlang,invvhichtheCDFanditsirrversearedifficulttocorr1pute.Inthesecases,\rveneedtodevelopothermethodsfortransforrr1ir1gsarnplevaluesofauniformrandorr1vaiabletosample\ralliesofarandomvariableofir1terest.Quiz6.2Xisanexponer1tial(;\)PDF.Sl1owthat·y=isaRa}rleighrandorr1variable(seeAppendixA.2).Expresst11eRayleighpararneterainterrr1softheexponer1tialpararneter;\.v'X.6.3FunctionsYieldingDiscreteorMixedRandomVariablesAhardlirniterelectroniccircuithastwopossibleoutput\roltages.Iftheinputvoltageisasamplevalueofacontinuousrandorn\rariable,theoutputvoltageisasamplevalueofadiscreterar1domvariable.Theoutpl1tofasoftlimitercircuitisasarr1plevall1eofamixedrandomvariable.Theprobabilityrr1odelsofthelirnitersdependontheproba,bilityrnodeloftheinputandonthet\rvOlimitingvoltages.InSectior16.2,ourexarr1plesandtheorernsrelatetoacontinl1ousrar1domvariablederi\redfromtwocontinl1ousrandornvariables.Bycontrast,inthefollo,r..ringexarnple,thefunctiong(X)transforrr1sacor1tint1ousrandorr1variabletoadiscreterandorr1\rariable.Example6.7----===LetXbearandomvariablewithCDFFx(x).Let·ybetheoutputofaclippingcircuit,alsoreferredtoasahardlimiter,withthecharacteristic·y=g(X)where4g(x)2g(x)=()-505.T,x<0,3x>0.1(6.21)[6.3FUNCTIONSYIELDINGDISCRETEORMIXEDRANDOMVARIABLES227ExpressFy(y)andj'y(y)intermsofFx(x;)andfx(x;).Beforegoingdeeplyintothemath,itishelpfultothinkaboutthenatureofthederivedrandomvariable·y.Thedefinitionofg(x;)tellsusthatYhasonlytwopossiblevalues,Y=1and·y=3.Thus·yisadiscreterandomvariable.Furthermore,theCDF,Fy(y),hasjumpsaty=1andy=3;itiszerofory<1anditisonefory>3.Ourjobistofindtheheightsofthejumpsaty=1andy=3.Inparticular,Fy(1)=P[Y<1)=P[X<0)=Fx(0).Fy(y)0Fy(y)Fx(O)00I234y=Fx(O)1y<1,1<y<3,y>3.(6.22)(6.23)ThePDFconsistsofimpulsesaty=1andy=3.TheweightsoftheimpulsesarethesizesofthetwojumpsintheCDF:Fx(O)and1-Fx(O),respectively.fy(y)'...j'y(y)=Fx(0)6(y-1)+[1-Fx(0))6(y-3).0I234yThefollov.ringexamplecontainsafunctionthattransforrnscor1tinuousrar1dornvariablestoarr1ixedrar1dornvariable.c:::::=Example6.8TheoutputvoltageofamicrophoneisaGaussianrandomvariableVwithexpectedvalue11,v=0andstandarddeviationo-v=5V.Themicrophonesignalistheinputtoasoftlimitercircuitwithcutoffvalue±10V.TherandomvariablevT!istheoutputofthelimiter:vT!(-10W=g('V)={Vl10v<-10,-10<v<10,v>10.(6.24)WhataretheCDFandPDFofvV?TofindtheCDF,weneedtofindF11v(w)=P[W<'UJ]forallvaluesof'UJ.Thekeyisthatallpossiblepairs('V,W)satisfylV=g('V).Thisimplieseach'Wbelongstooneofthreecases:[228CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLESw(a)(a)'W'Ww<-10w(b)-10<'W<10(c)'w>lO<-10:FromthefunctionW=g(V)weseethatnopossiblepairsCV,l!\l)satisfyvl!W.(b)-10<'ID<10:Inthiscaseweseethattheevent{vV<'W},markedingrayontheverticalaxis,correspondstotheevent{V<'W},markedingrayonthehorizontalaxis.ThecorrespondingCV,W)pairsareshowninthehighlightedsegmentofthefunctionW=g(V).Inthiscase,Fw(vJ)=P[vl!<'W]=P('V<'w]=Fv('w).(c)'W>10:Hereweseethattheevent{W<'W}correspondstoallvaluesof·vandP(W<'UJ]=P[V<oo]=1.ThisisanotherwayofsayingW=10isthemaximumvll.WecombinetheseseparatecasesintheCDFFH1('1D)=P[vV<w]=0'UJ<-10,Fv('w)-10<vJ<10,'U)>10.1(6.25)Theseconclusionsarebasedsolelyonthestructureofthelimiterfunctiong(V)withoutregardfortheprobabilitymodelofV.NowweobservethatbecauseVisGaussian(0,5),Theorem4.14statesthatFv(v)=Flt\1(VJ)=<-10,0'UJiJ?('ID/5)-10<'UJ<10,1'UJ(6.26)>10.NotethattheCDFjumpsfrom0to0,weintegratethejointPDFf'x,Y(x,y)overtheregionoftheX,YplaneforwhichY<'WX,X>0,andY>0asshown:yP[Y<wX]YwX=lo=(lo""fxy(x,y)dy)dxlo=.\e-(lo"'"dy)dx=lo=.\e-(1dx1""=1w-1'Y1""=1-e-µ:wx)>..>..+µ,'llJ(6.36)Therefore,0F\;\1('IJJ)=1-'ID>..A+µ,VJw<0,>o.(6.37)[232CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLESDifferentiatingwithrespecttoVJ,weobtain'ID>0,f'w('ID)=(6.38)otherwise.Quiz6.4(A)Asmartphonerurisa,nev.rsapplicationthatdovvnloadsInternetnewsevery15mirit1tes.Atthestartofadownload,theradiornodemsnegotiateacorinectiorispeedthatdependsoritheradiochannelq11alit:y.·vVheritheriegotiatedspeedislov.r,thesmartphoneredt1cesthearnountofnevvsthatittransferstoavoidvvastingitsbattery.Tlient1mberofkilobytestransmitted,L,andthespeedBinkb/s,ha·vethejointPMFPL,B(l,b)l=256l=768l=1536b=5120.20.050b=1,0240.10.10.1b=2,0480.050.20.2LetTdenotetlienurnberofsecondsrieededforthetransfer.ExpressTasaft1nctionofLandB.WliatistheP1!{FofT?(B)FindtheCDFaridthePDFof'VT!=XYvvhenrandornvariablesXaridYhavejointPDF6.51O0othervvise.(6.39)PDFoftheSumofTwoRandomVariablesThePDFofthes11moftv.roindeperidentcontiriuousrandorrivariablesXandYistheconvolt1tionofthePDFofXandtliePDFofY.TheP1!{Fofthes11rrioftv.roindependentinteger-valuedrandornvariablesistliediscretecorrvolutionofthetwoPMFs.ywX+Y<vvX_..____•wWenovvexarriinethest1rriW=X+·yoftvvocontin11ousrandomvariables.AsweseeinTheorem6.6,thePDFofWdependsonthejointPDFfx,y(::r;,y).Inparticular,intheproofoftlienexttheorerri,v.refiridthePDFofWusingthetvvo-stepprocedureinv.rliichvvefirstfiridtheCDFFw(VJ)byintegratingthejointPDFfx,Y(x,y)o·vertlieregioriX+Y<'W,assho,;vri.[6.5----Theorem6.8ThePDFofW=Xfw(w)=PDFOFTHESUMOFTWORANDOMVARIABLES+Yis1:fxy(x,w-x)dx=J:(J':'ProofFw(w)=P[X+Y1:<w]=233fx,y(w-y,y)dy.Jx,Y(x,y)dy)dx.(6.40)TakingthederivativeoftheCDFtofindthePDF,vvehave.(11J)fvv=dFvvi(11;)G,71J=Joo(-dd(Jw-xfx,·y(x,y)dy))dx_71J00J_0000=fx,y(1;,11;-x)d1;.(6.41)-ooBymakingthesubstitutiony=11;-x,weobtainfvv(11J)=-=Example6.11FindthePDFofW=X+Yfx,Y(:i;,y)=)'I)'-\\.'-.\'\I'J:(6.42)whenXandYhavethejointPDF2O0otherwise.(6.43)ThePDFofW=X+YcanbefoundusingTheorem6.8.ThepossiblevaluesofX,Yareintheshadedtriangularregionwhere0<X+Y=W<1.Thusfv.;(w)=0forvJ<0or'ID>1.For0<'W<1,applyingTheorem6.8yieldsfw(VJ))I'fx,y(11;-y,y)dy.{'W=J2dx=2vJ,(6.44)0IThecompleteexpressionforthePDFofl;\/isfw('w)=2'WQ<'UJ0<1,otherwise.(6.45)WhenXaridYareindepender1t,thejoir1tPDFofXand·yistheproductoftherr1arginalPDFs1·x,y(x,y)=fx(x)j'y(y).ApplyingT11eorerr16.8tothisspecialcase,-vveobtainthefollowir1gtheorern.[234CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLES==;;;;Theorem6.91........;==When,XaridYarein,depen,den,tran,dornvariables7thePDFofWfw(w)=1:fx(w-y)fy(y)dy=1:fx(x)fy(w-=X+Yisx)dx.InTheorem6.9,'""'ecorr1bir1etv.rour1i·variatefunctions,f·x(-)andj'y(·),ir1ordertoproduceathirdft1nction,f'w(·).Thecombinatior1inT11eorerr16.9,referredtoasacon,volution,,arisesinmar1:yrbrar1chesofappliedrnat11ernatics.WhenXarid·yareindeper1dentir1teger-valueddiscreterandornvariables,thePMFofW=X+Yisacorrvolution(seeProblerr16.5.1).00Pw('w)=~Px(k)Py('llJ-(6.46)k).k=-ooYoumayhaveencounteredconvoll1tionsalread:yrir1stt1dyinglir1earsysterns.Sornetimes,wet1sethenotationfw(w)=fx(x)*fy(y)todenotecorrvolution.Quiz6.5LetXandYbeindeper1dentexponentialrar1domvariableswit11expected·valuesE[X)=1/3andE[Y)=1/2.FindthePDFofW=X+Y.6.61\1.IATLABTheorem6.5andtherandfunctioncanbeernployedtogeneratesamplevaluesofcontinuousrandorr1variables.Example6.12UseExample6.5towriteaJ\IIATLABprogramthatgeneratesm,samplesofanexponential(.\)randomvariable.functionx=exponentialrv(larnbda,rn)x=-(1/larnbda)*log(1-rand(rn,1));variable.ByTheorem6.3(b),X=·y;,AInExample6.5,wefoundthatifUisauniform(0,1)randomvariable,thenY=-lr1(l-U)istheexponentiaI(1)randomisanexponential(.A)randomvariable.--Example6.13UseExample6.6towriteaJ\IIATLABfunctionthatgenerates(a,b)randomvariable.functionx=uniforrnrv(a,b,rn)x=a+(b-a)*rand(rn,1);'IT/,samplesofauniformExample6.6saysthatY=a,+(b-a)Uisauniform(a,b)randomvariable.Weusethisinuniformrv.[6.6MATLAB235Theorerr19.9willdernonstratethatthesumofn,independentexponential(;\)randornvariablesisanErlar1grandornvariable.Thefunctior1erlangrvgeneratesrn,sarnplevaluesoftheErlang(n,,;\)randornvariable.Notethat-vvefirstgeneraterirn,exponentialrandorr1variables.Thereshapefunctionarrangesthesesarr1plesinanrn,xn,array.St1rr1rningacrosstherO\AJS}'ieldsmErlangsarnples.functionx=erlangrv(n,lambda,m)y=exponentialrv(lambda,rn*n);x=sum(reshape(y,m,n),2);Fina.11}',forarandornvariableXvvithar1arbitrar}'CDFFx(x;),vveimplernentthefunctionicdfrv.m,vvhichusesTheorem6.5forgeneratir1grandornsarr1ples.Theke}'istodefineal\IIA.TLABfur1ctior1x=icdfx(u)t11atcalculatesx=Fx1(v,).Thefunctionicdfx(u)isthenpassedasanargumenttoicdfrv.m\vhichgeneratessamplesofX.NotethatMATLABpassesafunctionasanargurr1enttoanotherfur1ctionusingafunctionharidle,vvhichisakir1dofpointer.Thefollowingexarr1ples11ovvsho\vtousefunctionx=icdfrv(icdfhandle,m)%Usage:x=icdfrv(©icdf,m)%returnsmsamplesofrvX%withinverseCDFicdf.mu=rand(m,1);x=feval(icdfhandle,u);icdfrv.m.Example6.14Writeal\IIATLABfunctionthatusesicdfrv.mtogeneratesamplesofY,themaximumofthreepointerspins,inExample4.5.FromEquation(4.18),weseethatfor0<y<1,Fy(y)=y3.Ifu=Fy(y)=y3,theny=Fy1(v,)='u,113.Sowedefine(andsavetodisk)icdf3spin.m.Now,thefunctioncally=icdfrv(©icdf3spin,1000)generatesavectorholding1000samplesofrandomvariableY.Thenotation©icdf3spinisthefunctionhandleforthefunctionicdf3spin.m.functiony=icdf3spin(u);y=u.-(1/3);Keepinmindthatforthel\IIATLABcodetorur1quickly,itisbestfortheinverseCDFfunction(icdf3spin.mir1thecaseofthelastexample)toprocessthevectoruvvithot1tt1singaforlooptofindtheir1verseCDFforeachelernentu(i).\'Vealsor1otethatt11issarnetechniquecar1beextendedtocases\vheretheinverseCDFFx1(v,)doesnotexistforall0<'n<1.Forexarnple,thein\rerseCDFdoesnotexistifXisarnixedrandorn\rariableoriff'x(x)isconstantoveraninterval(a,b).Ho\vtouseicdfrv.minthesecasesisaddressedinProblerr1s6.3.13and6.6.4.Quiz6.6Writeal\IIATLABft1nctionV=Vsample(m)thatret11rnsmsarr1pleofrar1domvariable·v\vithPDFf'v(v)=(v+5)/72-5<-v<-720other\vise.)(6.47)[236CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLESProblemsDifficulty:Easy6.1.1RandomvariablesXandYhavejointPMFIx+Yl/14Px,Y(x,y)=x=-2,0,2;y=-1,0,1,0FindthePlVIFofW=X-Y6.1.2ForrandomvariablesXandYinProblem6.1.1,findtheI>MFofvV=X+2Y.6.1.3Nisabinomial(n,=100,p=0.4)randomvariable.Misabinomial(n=50,p=0.4)randomvariable.GiventhatJ\lfandNareindependent,whatisthePl\/IFofL=JV!+J\T?6.1.4LetXandYbediscreterandomvariableswithjointPl\/IFPx,y(x,y)thatiszeroexcept'vhenxandyareintegers.LetW=X+Yandsho'vthatthePlVIFofvVsatisfiesModerateDifficulttExpertsOnly6.2.3Ina50kmTourdeFl.'ancetimetrial,arider'stimeT,measuredinminutes,isthecontinuousuniform(60,75)randomvariable.LetV=3000/Tdenotetherider'sspeedoverthecourseinkm/hr.FindthePDFofV.6.2.4Inthepresenceofahead,vindofnormalizedintensityW,yourspeedonyourbikeisV=g(W)=20-lOvV113mi/hr.The'vindintensityvVisthecontinuousuniform(-1,1)randomvariable.(Note:IfWisnegative,thenthehead,vindisactuallyatail,vind.)FindthePDFfv(v).6.2.5IfXhasanexponential(.A)PDF,whatisthePDFofW=X2?6.2.6LetXdenotethepositionofthepointerafteraspinonawheelofcircumference1.Forthatsa1nespin,letYdenotetheareawithinthearcdefinedbythestoppingpositionofthepointer:00Pw(1D)L=Px,Y(x,'UJ-x).x=-oo6.1.5LetXandYbediscreterandomvariableswithjointP~l[F0.01Px,Y(x,y)=0x=l,2...,10,y=l,2...,10,oth.er,vise.\i\fhatisthel")MFofW=min(X,Y)?6.1.6ForrandomvariablesXandYinProblem6.1.5,'vhatisthePMFofV=max(X,Y)?6.2.1ThevoltageXacrossa1nresistorisauniformrandomvariablewithparameters0and1.Theinstantaneouspo,verisY=X2.FindtheCDFFy(y)andthePDFfy(y)ofY.6.2.2XistheGaussian(0,1)rando1nvariable.FindtheCDFofY=IXIanditsexpectedvalueE[Y).(a)WhatistherelationshipbetweenXandY?(b)\t\fhatisFy(y)?(c)\t\fhatisfy(y)?(d)\t\fhatisE[Y)?6.2.7Uistheunifor1n(0,1)randomvariableandX=-ln(l-U).(a)\t\fhatisFx(x)?(b)·\t\fhatisfx(x)?(c)\t\fhatisE[X)?[PROBLEMS6.2.8Xistheuniform(0,1)randomvariable.Findafunctiong(::i;)suchthatthePDFofY=g(X)is6.3.1•XhasCDFFx(x)=0<y<1,237<0x::i;/3+1/3<x<0,0<x<1,1<x.::i;/3+2/3otherwise.1-1,-1Y=g(X)where6.2.9AnamplifiercircuithaspowerconsumptionYthatgrowsnonlinearly\viththeinputsignalvoltageX.\i\fhentheinputsignalisXvolts,theinstantaneouspowerconsumedbytheamplifierisY=20+15X2\i\fatts.TheinputsignalXisthecontinuousuniform(-1,1)randomvariable.FindthePDFfy(y).6.2.10UseTheorem6.2toproveTheorem6.3.6.2.11Fortheuniform(0,1)randomvariableU,findtheCDFandPDFofY=a+(b-a)U\Vitha<b.ShowthatYistheuniform(a,b)randomvariable.6.2.12Theorem6.5requiredtheinverseCDFp-1(11,)toexistfor0<11,<1.\i\fhy\Vasitnotnecessarythatp-l(11,)existateither11,=0or11,=1?6.2.13Xisacontinuousrandomvariable.Y=aX+b,wherea,bf:.0.Provethat.()_fx((y-b)/a)jYylal.Hint:Considerthecasesaseparately.<0and>06.2.14LetcontinuousrandomvariableXhaveaCDFF(x)suchthatp-1(11,)existsforall7Lin[O,l].Sho\vthatU=F(X)istheuniform(0,1)randomvariable.Hint:Uisarandomvariablesuchthat\VhenX=x',U=F(::i;').Thatis,weevaluatetheCDFofXattheobservedvalueofx.<0,>o.(a)\t\fhatisFy(y)?(b)\iVhatisfy(y)?(c)WhatisE[Y]?6.3.2Ina50kmcyclingtimetrial,arider'sexacttime'Jmeasuredinminutes,isthecontinuousuniform(50,60)randomvariable.Hovvever,arider'srecordedtimeRinsecondsisobtainedbyroundingupTtonext\vholesecond.Thatis,ifTis50minutes,27.001seconds,thenR=3028seconds.Ontheotherhand,ifTisexactly50minutes27seconds,thenR=3027.\tVhatisthePMFofR?1,6.3.3ThevoltageVattheoutputofamicrophoneisthecontinuousuniform(-1,1)randomvariable.ThemicrophonevoltageisprocessedbyaclippingrectifierwithoutputL={IVIIVI<0.5,0.5(a)\t\fhatisP[La~00g(X)={xxotherwise.=0.5]?(b)\i\fhatisFL(l)?(c)\t\fhatisE[L]?6.3.4Uistheuniformrandomvariablewithparameters0and2.TherandomvariableWistheoutputoftheclipper:vV=(U)={Ug1<1'u>1.UFindtheCDFFw(1D),thel")DFfw(1D),andtheexpectedvalueE[W].[238CHAPTER6PROBABILITYMODELSOFDERIVEDRANDOMVARIABLES6.3.5Xisarandomvariable\VithCDFFx(x).LetY=g(X)wherebyv<0,w=v0<v<10,0g(::i,')--xx{10=100'>o.ExpressFy(y)intermsofFx(x).6.3.6Supposethatacellularphonecosts$30permonthwith300min.utesofuseincludedandthateachadditionalminuteofusecosts$0.50.ThenumberofminutesyouusethephoneinamonthisanexponentialrandomvariableTwithwithexpectedvalueE[T]=200minutes.Thetelephonecompanychargesyouforexactlyho\vmanyminutesyouusewithouta.nyroundingoffractionalminutes.LetCdenotethecostindollarsofonemonthofservice.SupposetheinputVisthecontinuousuniform(-15,15)randomvariable.FindthePDFofW.6.3.10ThecurrentXacrossaresistoristhecontinuousuniform(-2,2)rando1nvariable.ThepowerdissipatedintheresistorisY=9X2\iVatts.(a)FindtheCDFandPDFofY.(b)Apowermeasurementcircuitisrangelimitedsothatitsoutputisw=(a)WhatisP[C=30]?(b)WhatisthePDFofC?(c)\i\fhatisE[C]?{yy<16,16othervvise.FindthePDFofW.6.3.7Theinputvoltagetoarectifieristhecontinuousuniform(0,1)randomvariableU.TherectifieroutputisarandomvariableWdefinedbyW=g(U)={~6.3.11Adefectivevoltmetermeasuressmallvoltagesaszero.Inparticular,whentheinputvoltageisV,themeasuredvoltageisu<0,u>o.FindtheCDFFvv(v;)andtheexpectedvalueE[W].6.3.8v>10.10IVI<0.6,otherwise.IfVisthecontinuousuniform(-5,5)randomvariable,'vhatisthePDFofW?RandomvariableXhasPDF.()-_xxf{x/20<x<2,0other\vise.Xisprocessedbyaclippingcircuitwithoutput6.3.12Xisthecontinuousuniform(-3,3)randomvariable.\i\fhenXispassedthroughalimiter,theoutputisthediscreterandomvariableX=(X)=gxx<1,>1.(a)WhatisP[Y=0.5]?(b)FindtheCDFFy(y).6.3.9GivenaninputvoltageV,theoutputvoltageofahalf-waverectifierisgiven{-ccXOwherecisanunspecifiedpositiveconstant.(a)\!\ThatistheP~1IFP.x(x)ofX?(b)\tVhe.nthelimiterinputisX,thedistortionDbet,ve~ntheinputXandthelimiteroutputXisD=d(X)=(X-g(X))2.[PROBLEMSIntermsofc,findtheexpecteddistortionE[D]=E[d(X)].\i\fhatvalueofcminimizesE[D]?(c)YisaGaussianrandomvariablev;,riththesameexpectedvalueandvarianceasX.\i\fhatisthePDFofY?(d)SupposeYispassedtl1roughthelimiteryieldingtheoutputY=g(Y).ThedistortionDbetv;,reentheinputYandthelimiteroutputYisAD=d(Y)=(Y-g(Y))2.Intermsofc,findtheexpecteddistortionE[D]=E[d(Y)].\i\fhatvalueofcminimizesE[D]?6.3.13Inthisproblem\VeproveageneralizationofTheorem6.5.GivenarandomvariableXwithCDFFx(x),defineF(11)=min{xlFx(x)>11,}.Thisproblemprovesthatforacontinuo_usuniform(0,1)randomvariableU,X=F(U)hasCDFFx(x)=Fx(x).(a)Sho\vthatwhenFx(x)isacontinuous,strictlyincreasingfunction(i.e.,Xisnotmixed,Fx(x)hasnojumpdiscontinuities,andFx(x)hasno"flat"intervals(a,b)'vhereFx(x)=cfor-1a,<x<b),thenF('IL)=Fx('IJ,)for0<'IJ,<1.(b)Showtha!ifFx(:i;)hasajumpat1;=xo,thenF('IJ,)=:i;oforall11,intheinterval(c)ProvethatX=6.4.3•XandYhavejointPDFfX.Y(X,'I/)=''{00otherwise.(a)AreXandYindependent?(b)LetU=min(X,Y).FindtheCDFandPDFofU.(c)LetV=max(X,Y).FindtheCDFandPDFofV.6.4.4RandomvariablesXandYhavejointPDF.(X,Y)fx,Y={Qx+yOLetvV=max(X,Y).(a)\t\fhatisSvv,therangeofvV?(b)FindFw(11J)andfw(11J).6.4.5RandomvariablesXandYhavejointPDF6yfx,Y(x,y)={0OLetvV=Y-X.(a)\t\fhatisSvv,therangeofvV?(b)FindFw(11J)andfw(11J).6.4.6RandomvariablesXandYhavejointPDF={~otherwise.LetvV=Y/X.6.4.1RandomvariablesXandYhavejointPDF{6x>O,y>O,:i;+y<l,2F(U)hasCDFFx(x)=Fx(:i;)..fx,y(:i;,y)=6.4.2ForrandomvariablesXandYinProblem6.4.1,findtheCDFandPDFofW=min(X,Y).fx,Y(x,JJ)A239X'lj2'OMFofvV=X+Ysatisfies00Pw('ID)=Lk=-ooPx(k)Py('UJ-k).[PROBLEMSXandYhavejointPDF6.5.2.(fx'yx,y)={20::e>O,y>O,x+y<l,otherwise.FindthePDFofvV=X+Y.FindthePDFofvV=XXandYhavethejointPDF{2f·0expectedvaluesaand(3,respectively,andsho'vthatN=J+J{isaPoissonrandomvariable'vithexpectedvaluea+(3.Hint:Showthatn6.5.3.X.Y(x,y)=241PN(n,)=+YPK(m)PJ(n,-m),ni=OvvhenOLandthensimplifythesummationbyextractingthesumofabinomialPMFoverallpossiblevalues.Useicdfrv.mtowriteafunctionw=wrv1(m)thatgeneratesm,sample.sofrandomvariableWfromProblem4.2.4.NotethatFw1('11,)doesnotexistfor7L=1/4;however,youmustdefineafunctionicdfw(u)thatret1u·nsavalueforicdfw(0.25).Doesitmatterwhatvalueyoureturnforu=O.25?6.6.1FindthePDFofvV=XXandYhavethejointPDF6.5.4.()jxyx,y'={10+YwhenO0)thescheduledlecturetime.Whenyouobservethattheprofessorisalreadytwominuteslatebuthasnotyetarrived,youhavelearnedthatX>2butyouhavenotlearnedtheprecisevalueofX.Knov.rledgeofthecor1ditioningeventBchangestheprobabilit:yofthee·ventA.Giventhisinforrr1ationandaprobabilitymodel,v.recar1llSeDefir1ition1.5tofir1dtheconditior1alprobabilityP[AIB].Astartingpointist11eever1tA={X<x};-vvev.ro11ldfindP[AIB]=P[X0,theconditionalcumulativedistributionfunctionofXisFx1s(x)=P[X<xlB].Thedefinitior1ofthecor1ditior1alCDFappliestodiscrete,contir1uous,andrnixedrandorr1variables.Hov.re·ver,justas-vve11avefoundinpriorchapters,theconditionalCDFisnottherr1ostconvenientprobabilit.Yrr1odelforman:yrcalculations.Insteadv.rehavedefinitionsforthespecialcasesofdiscreteXandcontinuousXthatarerr1oreuseful....---Definition7.2ConditionalPMFGivenanEventGiven,theeven,tB'tuithP[B]>0,theconditionalprobabilitymassfunctionofXis[244CHAPTER7CONDITIONALPROBABILITYMODELSInChapter4wedefiriedthePDFofacontinuousraridomvariableasthederivativeoftheCDF.Sirriilarly,v.riththeknovvledgetliatxEB,wedefinetheconditionalPDFasthederivativeoftheconditionalCDF.Definition7.3ConditionalPDFGivenanEventForararidornvariableXaridan,even,tB'tuithP[B]>0)theconditionalPDFofXgivenBis.)_dFx1B(x)fx1B(xdx.Thef\1nctionsPx1B(:x;)aridf"x1B(x;)areprobabilityrnodelsforanev.rraridomvariablerelatedtoX.Hereweha;veextendedourriotationcorrventioriforprobabilit:yfurictioris.'\'!Vecontinuetheoldcor1ventiontliataCDFisdenotedb:ytheletterF,aPMFbyP,andaPDFbyf,vviththesubscriptcontairiingthenameoftherandorrivariable.Hov.re·ver,vvithaconditioninge·verit,thesubscriptcoritainsthenarneoftheraridom·variablefollov.redbya·verticalbarfollov.redbyastaternentoftheconditioningevent.Theargurnentoftliefunctioriisusuallythelowercaselettercorresporidingtotlie·variablenarne.Tliearg11rrientisadurrirny·variable.Itcot1ldbeanyletter,sotha.tPx1B(:x;)arid1·YIB(Y)arethesamefunctionsasPx1B(v,)andfYIB(v).Sornetirries-vvev.rritethefunctionwitlinospecifiedargurnentatall:Px1B(-).WhenaconditionirigeveritBcBx,bothP(B]andP[AB]inEquation(7.1)arepropertiesoftliePMFPx(x;)orPDFf"x(x;).Nov.reitherthee·ventA={X=x;}iscontainedintheeventBoritisnot.IfXisdiscretearidxEB,then{AB}={X=x}nB={X=x}aridP(X=x,B]=Px(x;).Otherv;.rise,if;x;tj_B,tlien{X=x}nB=0andP(X=x;,B]=0.Sirnilarobserva.tionsapply\vhenXiscontinuous.Thenexttlieorerriusestheseobservationstocalculatetheconditionalprobabilit}'rriodels.---==Theorem7.l=--Foraran,dornvariableXan,dan,even,tBCSx'tuithP[B]>0)thecon,dition,alPDFofXg'iven,BisDiscrete:Px1B(x;)=Con,tin/uov,s:fXIB(x;)=Px(x)P[B]0otherv.Jisef'x(x)P[B]:x;EB,0other'tJJise.Thetheorerristatesthat-vvhenvvelearnthatanoutcorriexEB,theprobabilitiesofallxtj_Barezeroinourcoriditionalrriodel,andtheprobabilitiesofallxEBareproportionall}'higlierthantlieyvverebefore\Velearried;x;EB.[7.1CONDITION/NGARANDOMVARIABLEBYANEVENT245Example7,],----===Awebsitedistributesinstructionalvideosonbicyclerepair.ThelengthofavideoinminutesXhasPMFO.lx=1,2,3,4,x=5,6,7,8,0otherwise.0.15Px(x)=(7.3)Supposethewebsitehastwoservers,oneforvideosshorterthanfiveminutesandtheotherforvideosoffiveormoreminutes.WhatisthePMFofvideolengthinthesecondserver?WeseekaconditionalPMFfortheconditionxFromthedefinitionofL,EL={5,6,7,8}.FromTheorem7.1,Px(:i;)p[L]x=5,6,7,8,0otherwise.(7.4)wehave8P[L]=LPx(x;)=0.4.(7.5):1;=5WithPx(x)=0.1fora-;EL,0.1/0.4=0.25x=5,6,7,8,0otherwise.(7.6)Thusthelengthsoflongvideosareequallylikely.Amongthelongvideos,eachlengthhasprobability0.25.Sorr1etirr1esinsteadofa.lettersuchasBorLthatder1otesthesubsetofSxthatforrr1sthecor1ditior1,-vve'\VritetheconditionitselfintheP1!{F.Intheprecedingexa.rr1plewecouldt1sethenotationPx1x>5(x)fortheconditionalPNIF.Example7.4Forthepointer-spinningexperimentofExample4.1,findtheconditionalPDFofthepointerpositionforspinsinwhichthepointerstopsontheleftsideofthecircle.LetLdenotetheleftsideofthecircle.Intermsofthestoppingposition,L=[1/2,1).RecallingfromExample4.4thatthepointerpositionXhasauniformPDFover[O,1),P[L]=f1f"x(;r;)11;2dx;=f111;2dx=1/2.(7.7)Therefore,21/28.0bservingthatP[A)conditionalPMFofXas1-121/2012/200=12/20,wecanwritethex=9,10,...,20,(7.10)otherwise.--Example7.6Y32.-..-----........---........--.-.......---.___,..-.xD.2D.JD.r/2Thecontinuousuniform(-r/2,r/2)randomvariableXisprocessedbyab-bituniformquantizertoproducethequantizedoutputY.RandomvariableXisroundedtothenearestquantizerlevel.Withab-bitquantizer,therearer1,=26quantizationlevels.Thequantizationstepsizeis~=r/ri,and·ytakesonvaIuesinthesetQy={Y-n/2,Y-n/2+1,···,Yn/2-1}(7.11)whereYi=~/2+i~.Thisrelationshipisshownforb=3inthefigureontheleft.GiventheeventBithatY=Yi·findtheconditionalPDFofXgivenBi.IntermsofX,weobservethatBiP[Bi]=1={'i~<X<('i+l)~}.Thus,('i+l)~~11·x(x)d:i;=-=-.ri~ri(7.12)ByDefinition7.3,fx1Bi(x;)=1·x(x)P[Bi)0otherwise,<x<('i+l)~,1/~'i~0otherwise.(7.13)GivenB,i:,theconditionalPDFofXisuniformovertheithquantizationinterval.[7.1CONDITIONINGARANDOMVARIABLEBYANEVENT247Insorr1eapplications,'"'ebegin\vit11asetofconditior1alprobabilitymodelssuchastheP1!{FsPx1Bi(x),'i=l,2,...,rn,,v.rhereB1,B2,...,Brnisapartition.Wethent1sethelavvoftotalprobabilit}'tofindthePMFPx(x;).===Theorem7.2'===Forran,dornvariableXres11Jt'irigfrornan,experirnen,t'tlJithpart'ition,B1,...,Brn,rnDiscrete:Px(x)=2::Px1Bi(x;)P(Bi);'l=l'rn,Con,tin/u,011,s:1·x(x)=2::f·xIBi(x;)P[B,i:]i=lProofThetheoremfollo,vsdirectlyfromTheorem1.10withA={X=x}fordiscreteXor_4={x<X<::i;+dx}whenXiscontinuous.Example7.7LetXdenotethenumberofadditionalyearsthatarandomlychosen70-year-oldpersonwilllive.Ifthepersonhashighbloodpressure,denotedaseventH,thenXisageometric(IJ=0.1)randomvariable.Otherwise,iftheperson'sbloodpressureisnormal,eventN,Xhasageometric(IJ=0.05)PMF.FindtheconditionalPMFsPx1H(:J;)andPx1N(x;).If40percentofall70-year-oldshavehighbloodpressure,whatisthePMFofX?TheproblemstatementspecifiestheconditionalPMFsinwords.Mathematically,thetwoconditionalPMFsarePx1H(x)=o.1(0.gr1;-la-;=1,2,...,otherwise,0()Px1Nx=o.os(o.9sri;-la-;=1,2,...,0otherwise.SinceH,Nisapartition,wecanuseTheorem7.2towrite+pxIN(a-;)p[N)(0.4)(0.1)(0.9)x-i+(0.6)(0.05)(0.95r;-lpx(a-;)=pxIH(x)p[H.)10x=1,2,...,otherwise.(7.14)Example7.8RandomvariableXisavoltageatthereceiverofamodem.Whensymbol"O"istransmitted(eventB0),XistheGaussian(-5,2)randomvariable.Whensymbol"l"istransmitted(eventB1),XistheGaussian(5,2)randomvariable.Giventhatsymbols"O"and"l"areequallylikelytobesent,whatisthePDFofX?TheproblemstatementimpliesthatP(B0)=P(B1)=1/2and[248CHAPTER7CONDITIONALPROBABILITYMODELSfx1so(x)=2~e-fx1s,(x)=2~e-(x-5)'/8(7.15)ByTheorem7.2,f"x(x;)=f"xlBo(x)P[Bo]+fXIB1(x;)P(B1]=4~(e-(x+5)2/8+e-(x-5)2/8).(7.16)Problem7.7.1asksthereadertographfx(x)toshowitssimilaritytoFigure4.3.Quiz7.1(A)OntheInternet,dataistransmittedinpackets.Inasirnplemodelfor·vvorldvVideWebtraffic,thent1rnberofpacketsNneededtotrar1srnita'\i\Tebpagedependsonvvhetherthepagehasgraphicirnages.Ifthepage11asirnages(e\rentI),t11enJ\Tisuniforrnl}'distribt1tedbetvveen1and50packets.Ifthepageisjusttext(e\rentT),thenNisuniforrr1between1and5packets.Assurr1ingapageha,sirnagesvvithprobability1/4,findthe(a)conditior1alPl\/IFPN11(n,)(b)conditionalPMFPN1r(ri)(d)conditionalPMFPNIN<1o(n,)(B)Yisacontinuousl111iforrn(0,10)randorr1variable.Findthefollovving:7.2(a)P(Y<6](b)theconditionalPDFfYIY(c)P(Y>8](d)theconditionalPDFfYIY>s(Y)ConditionalExpectedValueGivenanEventThedefinitionsofconditionalexpectedva.lt1eE(XIE]aridconditiona1varianceE(XIB]correspondtothedefinitionsofE(X]andVar(X]v.rithPx1B(x;)replacingPx(x)orf'x1B(x;)replacingfx(x).Becausetheconditionir1ge\rentBtellsusthattheoutcomeofar1experimentisan

elerr1entofB,Px1B(x)orfx1B(x;)canbeviewedasaP1!lForPDFinar1experirr1er1tvvithsarr1plespaceB.Thisisconfirmedb}'thefollovvingtheorerr1,whichreplacessarr1plespaceSvvithBir1Theorem3.1.[7.2CONDITIONALEXPECTEDVALUEGIVENANEVENT249===Theorem7.3:----===DiscreteX:ContinuousX:(a)ForariyxEB;Px1B(x)>0.(a)Foran,yxEB1f"x1B(x;)>0.(b)2=:1;EBpXIB(;:r;)(b)=l.(c)Thecon,dition,alprobabilitythatXisin,thesetCisP[GIB]=L.rBfxIB(x)rlx=1.(c)Thecon,dition,alprobabilitythatXisin,thesetCisPx1B(x).P[CIBJ=:r;ECfcfx1a(x)dx.Conditior1alprobabilityrr1odels.haveparameterscorrespondingtot11epararr1etersof11nconditionalprobabilit}'rnodels.T11erefore,viecancorr1p11teexpectedvalt1esoftheconditionalrandom·variableXIEandexpectedvaluesoffunctionsofXIEint11esarnewayt11at"''ecompt1teexpectedva.lt1esofX.Theonlydifferenceisthatweuset11ecor1ditionalP1!{FPx1B(x;)orPDFfx1B(x)inplaceofPx(x)orfx(x).Definition7.4ConditionalExpectedValueTheconditionalexpectedvalueofraridornvariableXgiven,con,d'ition,BisDiscrete:E[XIE]=L;r;Px1B(;r;);:i;EBCon,tin/tJ,O?J,s:E[XIB]=j00xf'x1B(x;)rlx.-()()Analternati·venotationforE[XIB]isµXIB·When""'earegiventheconditionalprobabilitymodelsPx1Bi(x;)forapartitionB1,...,Brn,vvecancomputetheexpected·valueE[X]ir1terrr1softheconditionalexpectedvaluesE[XIBi]·.-----==Theorem7.4Foraran,dornvariableXresultin,gfrornan,experirneritVJithpartit'ioriB1,...,B,,n,rnE[X]=LE[XIB,i]p[B,i].'i=l[250CHAPTER7CONDITIONALPROBABILITYMODELSProof\i\fhenXisdiscrete,E[X]=L:xxPx(x),andvvecanuseTheorem7.2to'vrite1n:ri=l(7.17)i=lxi=l\i\fhenXiscontinuous,theproofusesthecontinuousversionofTheorem7.2andfollowsthesamelogic,withthesummationoverxreplacedbyintegration.Foraderivedrandom·variableYrem3.10.g(X),-vveha·vet11eequivalentofTheo----Theorem7.5TheccYndition,aleJ;pectedval11,eofYDiscrete:=g(X)given,coriditioriBisE[YIB]=E(g(X)IB]=Lg(1:)Px1B(x);xEBContinuous:E[YIB]=E(g(X)IB]=J:g(x)fx1s(x)dx.Itfollov.rsthattheconditionalvarianceandcor1ditionalstar1darddeviationconforrntoDefir1itions3.15and3.16,-vvithXIEreplacingX.==;....Definition7.5----'ConditionalVarianceandStandardDeviation1Thecon,ditiori,alvarian,ceofXgiven,e'veritBisThecon,dition,alstaridardde'viation,isO"XIB=/Var[XIB].Theconditionalvariar1ceandconditiontilstar1darddeviationarellSefulbecausetheymeastlrethespreadoftherandornvariableafterwelearnt11econditioningir1forrnationB.Iftheconditionalstandarddeviatior1O"XIBisrnuchsrr1allerthar1O"x,thenwecansa:yt11a.tlearnir1gtheoccurrenceofBreducesourur1certaintyabol1tXbecauseitshrir1l0.0otherwise.-I(7.22)tForcallsthatlastatleast2minutes,whatistheconditionalPDFofthecallduration?Inthiscase,theconditioningeventisT>2.TheprobabilityoftheeventisP[T>2]=1=fr(t)dt=e-2/3.(7.23)TheconditionalPDFofTgivenT>2is0.4------fTIT>2(t)fr(t)0.200510tP(T>2]t>2,0otherwise,~e-(t-2)/3t>2,0otherwise.Notethatfr1r>2(t)=fr(t-2),atime-shiftedversionoffr(t).AninterpretationofthisresuItisthatiftheca11isinprogressafter2minutes,thedurationoftheca11is2minutesplusanexponentialtimeequaltothedurationofanewcall.TheconditionalexpectedvalueisE[TIT>2]=00121t-e3(t-2)/3dt.(7.24)Integrationbyparts(AppendixB,MathFactB.10)yieldsE[TIT>2]=-te-(t-2)/3002+100e-(t-2)/3dt=2+3=52minutes.[252CHAPTER7CONDITIONALPROBABILITYMODELSRecallinExample4.13thattheexpecteddurationofthecallisE[T)=3minutes.WeinterpretE[TIT>2)bysayingthatifthecallisstillinprogressafter2minutes,theadditionaldurationis3minutes(thesameastheexpectedtimeofanewcall),andtheexpectedtotaltimeis5minutes.===Quiz7.2'==::::a(A)ContinuingQuiz7.1(A),find(a)E[Nl.l\1<lOL(b)Var[NIN<10).(B)Contir1uingQt1iz7.1(B),find(a)E[YIY<7.36L(b)Var['YIY<6).ConditioningTwoRandomVariablesbyanEventTheprobabilityrr1odelforrandomvariablesXand·ygiveneventBisrelatedtothellnconditiona.lprobabilit}'rr1odelforXarid·yinthesarr1e-vvaythatt.heprobabilitymodelforXgivenBisrelatedtot11eprobabilitymodelforX.Theconditionalprobabilityrr1odelcanbeusedtofindthecor1ditionalexpectedvalueandconditionalvarianceofVV=g(X,Y)inthesame-vvaythatt11eunconditionalprobabilitymodelforXand·yisusedtocomputeE[g(X,Y)andVar[g(X,'Y)).Anexperirr1entproducest-vvorandorr1·variables,XandY.\Velearnthatt11eoutcorne(:i;,y)isanelerner1tofane\rent,B.\Vet1setheir1forrr1atior1(x,y)EBtoconstrl1ctane\vproba.bilityrnodel.IfXandYarediscrete,thene\vrnodelisaconditionaljoir1tPMF,theratioofthejointPMFtoP[BJ.IfXandYarecontint1ous,thenewrr1odelisaconditionaljointPDF,definedast11eratioofthejointPDFtoP[B).Thedefir1itionsoft11esefur1ctior1sfollo\vfromt11esameintl1itionasDefinition1.5fortheconditionalprobabilityofar1e\rer1t.Definition7.6ConditionalJointPMFFordiscreteran,dornvariablesXarid·yan,dan,e'veritB111ithP[BJ>0,theconditionaljointPMFofXa/ndYgiven,BisPx,YIB(x,y)=P[X=;i;,Y=ylBJ.Thefollowir1gtheorernisa.r1irr1rnediatecor1sequenceofthedefir1ition.[7.3CONDITION/NG1WORANDOMVARIABLESBYANEVENT253r----==Theorem7.6---===1Forariyeven,tB,aregion,oftheX,Yplan,e1JJithP[B]>0,Px,YIB(x;,y)=Px,Y(x,y)p[B](x,y)EB,0other·1nise.Example7.11yPx,Y(x,y)4..l...l...l...l.RandomvariablesXandYhavethejointPMFPx,Y(x,y)asshown.LetB={X+·y<4}andfindtheconditionalPMF....l.......l...Px,YIB(x,y).e12e103.l2•8.l10....l...e1G.4.l•8e12e10e12e10T'0123I4EventB={(1,1),(2,1),(2,2),(3,1)}consistsofallpoints(x;,y)suchthatx+y<4.ByaddinguptheprobabilitiesofalloutcomesinB,wefindyPx,YIB(x,y)43..:l.2•7..:l10P[B]=Px,y(l,1)+Px,y(2,1)e14..:l.e14.l7e1+Px,Y(2,2)+Px,Y(3,1)=12·T'0123I4TheconditionalPMFPx,YIB(x,y)isshownontheleft.Inthecaseoftvvocontinuot1srandomvariables,wehavethefollowingdefinitionofthecor1ditionalprobabilitymodel.Definition7.7ConditionalJointPDFGiven,arieven,tB111ithP[B]>0,thecon,dition,aljoiritprobabilityden,sityfv,riction,ofXandYis1·x,Y(x;,Y)fx,YIB(x;,y)p[BJ=0(x,y)EB,other111ise.[CHAPTER7254CONDITIONALPROBABILITYMODELS==~ExampIe7.12t-....;=::::::aXand·yarerandomvariableswithjointPDF1/15OO;theconditionalPMFofXg1,ve'TI,Y=y'lSPx1y(xly)=P[X=xlY=y].Thefollov.ringtheorerncontainstlierelationsliipbetvveeritliejointP1!{FofXandYandthetv.roconditionalPl\/IFs,Px1y(xly)andPyix(ylx).Theorem7.8Fordiscretera'Tl,dornvariablesXa'Tl,d·y'Ul'ithjo'i'Tl,tPMFPx,y(a:;,y),a'Tl,dxaridy[CHAPTER7258CONDITIONALPROBABILITYMODELSGivenX=~r;.theconditionalPMFofYisthediscreteuniform(1,1';)randomvariable.B:ycontrastwitl1XandYdiscrete,whenXandYarecontinuous,\VecannotapplySectior17.3directl:ybecauseP[B]=P[Y=y]=0asdiscussedinChapter4.Instead,\vedefineaconditionalPDF,der1otedasf'x1y(1';ly),astheratioofthejointPDFtothernarginalPDF.---Definition7.9ConditionalPDFForys'uchthatjy(y)>0)thecon,dition,alPDFofXgiven,{'Y=y}is,y)fX.IY(X·IY)-_f'xf,Y(x()yyProblem7.4.12asksyol1toverifythatf'x1Y(1';1y)isaconditionaldensit}'fl1nctionforXgiver1theconditior1inge\renty<·y<y+~inthelirnitas~approacheszero.Defir1ition7.9implies.(I)_1·x,Y(;r;,y)jYIXyx1·x(x;).(7.38)Foreachyvvithfy(y)>0,thecor1ditionalPDFfx1y(xly)gi,resusanewprobabilityrnodelofX.'\Vecanusethisrnodelinar1y\Va}'thatvveusefx(1';),t11emodel"''ehaveir1theabsenceofkno\vledgeofY.Example7.16ReturningtoExample5.8,randomvariablesXandYhavejointPDFy1fx)'(x,y)=2\f'x,Y(x;,y)=2O0otherwise.(7.39)'-----r+X1For0<x<1,findtheconditionalPDFj'y1x(ylx;).For0PDFfx1y(xly).For0<y<1,findtheconditional<x;<1,Theorem5.8impliesf'x(x)=j00fx,y(x;,y)rly=-oor;r;2rly=2x.Jo(7.40)TheconditionalPDFofYgivenXisy)_fYIX(YIX·)-_fxf,Y(1';,x(x;)-l/xO0otherwise.(7.41)[CHAPTER7258CONDITIONALPROBABILITYMODELSGivenX=~r;.theconditionalPMFofYisthediscreteuniform(1,1';)randomvariable.B:ycontrastwitl1XandYdiscrete,whenXandYarecontinuous,\VecannotapplySectior17.3directl:ybecauseP[B]=P[Y=y]=0asdiscussedinChapter4.Instead,\vedefineaconditionalPDF,der1otedasf'x1y(1';ly),astheratioofthejointPDFtothernarginalPDF.---Definition7.9ConditionalPDFForys'uchthatjy(y)>0)thecon,dition,alPDFofXgiven,{'Y=y}is,y)fX.IY(X·IY)-_f'xf,Y(x()yyProblem7.4.12asksyol1toverifythatf'x1Y(1';1y)isaconditionaldensit}'fl1nctionforXgiver1theconditior1inge\renty<·y<y+~inthelirnitas~approacheszero.Defir1ition7.9implies.(I)_1·x,Y(;r;,y)jYIXyx1·x(x;).(7.38)Foreachyvvithfy(y)>0,thecor1ditionalPDFfx1y(xly)gi,resusanewprobabilityrnodelofX.'\Vecanusethisrnodelinar1y\Va}'thatvveusefx(1';),t11emodel"''ehaveir1theabsenceofkno\vledgeofY.Example7.16ReturningtoExample5.8,randomvariablesXandYhavejointPDFy1fx)'(x,y)=2\f'x,Y(x;,y)=2O0otherwise.(7.39)'-----r+X1For0<x<1,findtheconditionalPDFj'y1x(ylx;).For0PDFfx1y(xly).For0<y<1,findtheconditional<x;<1,Theorem5.8impliesf'x(x)=j00fx,y(x;,y)rly=-oor;r;2rly=2x.Jo(7.40)TheconditionalPDFofYgivenXisy)_fYIX(YIX·)-_fxf,Y(1';,x(x;)-l/xO0otherwise.(7.41)[7.4CONDITION/NGBYARANDOMVARIABLEGivenX=x,weseethatYistheuniform(0,x)randomvariable.For0Theorem5.8impliesj-()()f'x)y(<y<1,100j'y(y)=259;i;'y)rlx=r2rlx}y=2(1-y).(7.42)Furthermore,Equation(7.38)implies(7.43)ConditionedonY=y,weseethatXistheuniform(y,1)randomvariable.===Example7.17'==:::::1Atnoononaweekday,webeginrecordingnewcallattemptsatatelephoneswitch.LetXdenotethearrivaltimeofthefirstcall,asmeasuredbythenumberofsecondsafternoon.LetYdenotethearrivaltimeofthesecondcall.Inthemostcommonmodelusedinthetelephoneindustry,Xand·yarecontinuousrandomvariableswithjointPDF0<x<y,otherwise,0(7.44)whereA>0calls/secondistheexpectedarrivalrateofcalls.FindthemarginalPDFsfx(x)andfy(y)andtheconditionalPDFsf'x1y(xly)andfy1x(Ylx).Forx<0,f'x(x)=0.Forx>0,Theorem5.8givesfx(x):fx(x)=1=A2e-Aydy=Ae-'"'(7.45)ReferringtoAppendixA.2,weseethatXisanexponentialrandomvariablewithexpectedvalue1/.A.GivenX=x,theconditionalPDFof·yis.Ae-)...(y-:r;)fx,Y(x,y)_·(I'·)_.jYIXyxf'x(x··)-0y>x,otherwise.(7.46)NowwecanfindthemarginalPDFof·y.Fory<0,fy(y)=0.Theorem5.8impliesfo'l/.A2e-AYrlx=.A2ye-AYY>0o,otherwise.(7.47)YistheErlang(2,.A)randomvariable(AppendixA.2).GivenY=y,theconditionalPDFofXis(,,y)f·I')X,YfXIY(.xy;i_:fy(y)--l/y00<x<'Y.,otherwise.(7.48)[260CHAPTER7CONDITIONALPROBABILITYMODELSUndertheconditionthatthesecondcallarrivesattimey,thetimeofarrivalofthefirstcallistheuniform(0,y)randomvariable.InExarr1ple7.17,-vvebegin-vvitha.jointPDFandcomputetv.roconditionalPDFs.Ofteninpracticalsitl1atior1s,webeginvvit11aconditior1alPDFaridamarginalPDF.Thenweusethisir1forrr1ationtocorr1putethejointPDFandtheothercor1ditior1alPDF.Thissarneapproachalso-vvorksfordiscreterandorr1variablesusingP1!fFs.Thenecessaryforrr1ulasareinthefollov.ringtheorerr1s.---Theorem7.91--Fordiscreteran,dornvariablesXarid·yvrithjoin,tPMFPx,Y(x;,y),arids11,chthatPx(x)>0a/ndPy(y)>0,;r:aridy--·Theorem7.10---==::::::iForcoritin,11,o'usraridornvariablesXan,dY'tuithjoin,tPDFfx,Y(x,y),an,da:;aridys11,chthatf'x(;r:)>0an,dfy(y)>0,Example7.18LetRbetheuniform(0,1)randomvariable.GivenR=r,Xistheuniform(0,r)randomvariable.FindtheconditionalPDFofRgivenX.Theproblemdefinitionstatesthat10Ol/r00<x<r'otherwise.(7.49)ItfollowsfromTheorem7.10thatthejointPDFofRandXisf'R,x(r,x;)=fx1R(x;lr)fn(r)=l/r00<X<r<1,otherwise.NowwecanfindthemarginalPDFofXfromTheorem5.8.For0fx(x;)=Joo_00f'R,x(r,x)dr=11:i:dr-;:-=-lnx.(7.50)<x<1,(7.51)[2627.5CHAPTER7CONDITIONALPROBABILITYMODELSConditionalExpectedValueGivenaRandomVariableRandornvariablesXandY:havecor1ditionalprobabilitymodelsPx1y(1';ly)orfx1y(xly)that:haveconditionalexpectedvalueandvariancepararr1etersE[XIY=y)andVar[XIY=y).WhenweconsiderE[Xl'Y=y]asafunctionoftherandomobservation·y=y,-vveobtaintherandornvariableE(XIY].Theexpectedva.lt1eofE(Xl.Y]isE(X].ForeachyESy,thecor1ditionalPMFPx1y(xly)orconditionalPDFfx1y(xly)isamodifiedprobabilityrr1odelofX.Wecanusethisrr1odelinanywaythatv.reuset11eorigir1alPx(1';)orf'x(x;),therr1odel"''e11aveint11eabser1ceofkno-vvledgeofY.Mostirnportar1t,v.recanfindexpected·values-vvithrespecttoPx1y(xly)orfx1y(x;ly).Definition7.10ConditionalExpectedValueofaFunctionForan,yyESy)thecon,dition,alexpected'valv,eofg(X,Y)g'iven,·y=yisDiscrete:E[g(X,Y)IY=y]=Lg(x,y)Px1Y(xly);'.r;ESxContinu011.s:E[g(X,Y)IY=y]=1:g(x,y)JXIY(xly)dx.AspecialcaseofDefinition7.10v.rithg(x;,y)=xist11econditionalexpectedvalueE(XIY=y]=Discrete:L3';Px1Y(1';1y);xESxContinuous:E[XIY=y]===~Example1:xfXIY(xly)dx.7.llJ--==InExample7.15,wederivedconditionalPMFsPy1x(Yll),Py1x(yl2),Py1x(Yl4).FindE('YIX=3';)for3';=1,2,3,4.Py1x(yl3),andInExample7.15wefoundthatgivenX=x,Ywasadiscreteuniform(1,x)randomvariable.Sinceadiscreteuniform(1,1';)randomvariablehasexpectedvalue(1+x)/2,E('YIX=l]=E['YIX=3]=11;;13=1,=2,12E(YIX=2]=-+_=1.5,2E(YIX=4]=1+24=2.5.(7.55)(7.56)Notethatingeneral,theconditior1alexpectedvalt1eE(XIY=y]isaft1r1ctionofy[2627.5CHAPTER7CONDITIONALPROBABILITYMODELSConditionalExpectedValueGivenaRandomVariableRandornvariablesXandY:havecor1ditionalprobabilitymodelsPx1y(1';ly)orfx1y(xly)that:haveconditionalexpectedvalueandvariancepararr1etersE[XIY=y)andVar[XIY=y).WhenweconsiderE[Xl'Y=y]asafunctionoftherandomobservation·y=y,-vveobtaintherandornvariableE(XIY].Theexpectedva.lt1eofE(Xl.Y]isE(X].ForeachyESy,thecor1ditionalPMFPx1y(xly)orconditionalPDFfx1y(xly)isamodifiedprobabilityrr1odelofX.Wecanusethisrr1odelinanywaythatv.reuset11eorigir1alPx(1';)orf'x(x;),therr1odel"''e11aveint11eabser1ceofkno-vvledgeofY.Mostirnportar1t,v.recanfindexpected·values-vvithrespecttoPx1y(xly)orfx1y(x;ly).Definition7.10ConditionalExpectedValueofaFunctionForan,yyESy)thecon,dition,alexpected'valv,eofg(X,Y)g'iven,·y=yisDiscrete:E[g(X,Y)IY=y]=Lg(x,y)Px1Y(xly);'.r;ESxContinu011.s:E[g(X,Y)IY=y]=1:g(x,y)JXIY(xly)dx.AspecialcaseofDefinition7.10v.rithg(x;,y)=xist11econditionalexpectedvalueE(XIY=y]=Discrete:L3';Px1Y(1';1y);xESxContinuous:E[XIY=y]===~Example1:xfXIY(xly)dx.7.llJ--==InExample7.15,wederivedconditionalPMFsPy1x(Yll),Py1x(yl2),Py1x(Yl4).FindE('YIX=3';)for3';=1,2,3,4.Py1x(yl3),andInExample7.15wefoundthatgivenX=x,Ywasadiscreteuniform(1,x)randomvariable.Sinceadiscreteuniform(1,1';)randomvariablehasexpectedvalue(1+x)/2,E('YIX=l]=E['YIX=3]=11;;13=1,=2,12E(YIX=2]=-+_=1.5,2E(YIX=4]=1+24=2.5.(7.55)(7.56)Notethatingeneral,theconditior1alexpectedvalt1eE(XIY=y]isaft1r1ctionofy[7.5CONDITIONALEXPECTEDVALUEGIVENARANDOMVARIABLE263andtliatE["YIX=x]isafunctionofx.Hovvever,vvhenXandYareiridependent,theobservationY=yprovidesrioiriforrnationaboutX;nordoeslearriingX=:i;informt1sabo11tY.Acor1sequenceisthattheconditionalexpectedva1t1esarethesarneastheunconditionalexpectedval11eswhenXaridYareindependent.===Theorem7.12(a)E[XIY=y]=E[X]forallyESy){b)E[YIX=x]=E[Y]forallxESx.Proof\'!\lepresenttheprooffordiscreterandomvariables.ByreplacingPl'v1Fsandsums\VithPDFsandintegrals,wearriveatessentiallythesameproofforcontinuousrandomvariables.SincePx1y(xly)=Px(x),E[XIY=y)=LxPxry(xly)=LxPx(1;)=E[X).(7.57)xESx(7.58)yESyyESyWhenvieintroducedthecoriceptofexpectedval11einChapters3arid4,v.reobservedtliatE[X]isapropertyoftlieprobabilityrnodelofX.ThisisalsotrueforE[XIB]\vlienP[B]>0.Thesituationisrnorecornplex\vhenweconsiderE[XIY=y],theconditior1alexpectedvalt1egivenararidorrivariable.Inthiscase,theconditionalexpected·valueisadifferentnt1rnberforeachpossibleobservatioriyESy.TliisimpliestliatE[XIY=y]isaf11nctionoftherandomvariableY.\Vet1sethenotationE[XIY]todenotethisfunctiorioftlierandomvariableY.Sinceafunctionofarandorrivaria.bleisanotherraridomvariable,\veconcludetliatE[XIY]isaran,dornvariable!Thefollowingdefiriitionrriayhelptoclarifythispoint.Definition7.11ConditionalExpectedValueFunctionThecon,dition,ale:i;pectedvalv,eE[XI"Y]isafv,n,ctioriofran,dornvariable·yS'IJ.chthatifY=y)theriE[XIY]=E[XIY=y].c:::==Example7.20ForrandomvariablesXandYinExample5.8,wefoundinExample7.16thattheconditionalPDFofXgiven·yis.fx1y(:i;ly)=fxy(x,y)j·y(y)1/(1-y)O<y0otherwise.(7.59)[264CHAPTER7CONDITIONALPROBABILITYMODELSFindtheconditionalexpectedvaluesE[XIY=y)andE[XIY].GiventheconditionalPDFfx1y(1';ly),weperformtheintegrationJ:E[XIY=y]=f,=xfx1y(xly)dx11x2l+y21-yx;rlx=2(1-y)Y(7.60)SinceE[XIY=y)=(1+y)/2,E[Xl'Y]=(1+Y)/2.AninterestingpropertyoftherandomvariableE[XIY]isitsexpectedvall1eE[E[XIY)].vVefir1dE[E[XIY]]intvvosteps:Firstv.recalculateg(y)=E[Xl'Y=y],andthenvieapplyTheorerr14.4toevaluateE[g(Y)].T11istwo-stepprocessisknovvnasiteratedexpectation,.====Theorem7.13IteratedExpectationE[E[XIY]]=E[X).Proof\?VeconsidercontinuousrandomvariablesXandYandapplyTheorem4.4:E[E[XIY]]=1_:E[XIY=y]fy(y)dy.(7.61)ToobtainthisformulafromTheorem4.4,vvehaveusedE[XIY=y]inplaceofg(x)andfy(y)inplaceoffx(x).Next,\VesubstitutetherightsideofEquation(7.55)forE[XIY=y]:E[E[XIY]]=1_:(/_:xfx1Y(xly)dx)fy(y)dy.(7.62)Rearrangingtermsinthedoubleintegralandreversingtheorderofintegration,\VeobtainE[E[XIY]]=1_::i~1_:fx1y(xly)fy(y)dydx.(7.63)Next,\VeapplyTheorem7.10andTheorem5.8toinferthattheinnerintegralisfx(x).Therefore,E[E[XIY]]=1_::i~fx(x)dx.(7.64)TheproofiscompletebecausetherightsideofthisformulaisthedefinitionofE[X].Asimilarderivation(usingsumsinsteadofintegrals)provesthetheoremfordiscreterandomvariables.Thesarnederivationcanbeger1eralizedtoan}'functior1g(X)ofoneofthet\vo[7.6BIVARIATEGAUSSIANRANDOMVARIABLES:CONDITIONALPDFS265randorr1·variables:---Theorem7.14E[E[g(X)I'Y]]=E[g(X)].Thefollov.ringforrr1l1lasapplyT11eorern7.14todiscreteandcontinuOllSrandornvariables.Discrete:E[g(X)l=E[E[g(x)I'Y]]=LE[g(X)Iy=y]Py(y);yESyContinuous:E(g(X)]=E[E[g(X)IY]]=1:E[g(X)IY=y]fy(y)dy.Theorern7.14decornposesthecalculationofE[g(X)]ir1totvvosteps:calculatingE[g(X)IY=y)asafunctionof·yandthencalculatingtheexpectedvalueofthefunctionusingtheprobabilityrnodelofY.Quiz7.5(A)Forrandorr1·variablesAandBinQl1iz7.4(A)find:(a)E[YIX=2],(b)Var[XIY=O].(B)ForrandornvariablesXandYinQuiz7.4(B))fir1d:(a)E[YIX=1/2],7.6(b)Var[XIY=1/2).BivariateGaussianRandomVariables:ConditionalPDFsForbivariateGa11ssiar1randornvariablesXand·y,theconditionalPDFsf"x1y(xly)a.ndj'.y1x(ylx;)areGaussian.Var[XIY]<Var[X]andVar[YIX]<Var[Y).Herewereturntot11ebivariateGaussiar1randorr1variablesXandYintroducedinSection5.9.Ourstartingpoir1tisthefactorizedexpressionforthejointPDFfx,y(x,y)giveninEquation(5.68)aridrepeatedhere:1fx,Y(x;,y)=(Jxy'2;22e-(:i:-µx)/25minutes.sx=(1:8)';px=[0.15*ones(4,1);...0.1*ones(4,1)];sxL=unique(find(sx>=5));pL=sum(finitepmf(sx,px,sxL));pxL=finitepmf(sx,px,sxL)/pL;XofavideogivenWithrandomvariableXdefinedbysxandpxasinExample3.43,thiscodesolvesthisproblem.ThevectorsxLidentifiestheeventL,pListheprobabilityP[L],andpxListhevectorofprobabilitiesPx1L(:I;i)foreach;J;iEL.TlieconditionalPNIFandPDFcanalsobeusedinMATLABtosirriplifytliegenerationofsarnplepairs(X)·y).Forexample)-vvhenXaridYha;vethejoiritPDFfx,y(:I;)y))abasica.pproachistogeneratesamplevalues;,r;1)...)X1nforXusingthernargina.1PDFfx(:I;).Thenforeachsarnple;J;i)-vvegenerateYiusingtlieconditiorialPDFj"y1x(Yl:J;i)·l\IIATLABcandothisefficientl}'providedthesarnplesy1)...)Yrncanbegerieratedfrornx1)...)Xrnusing·vector-processingtechniq11es)asirithefollo-vvingexarnple.[PROBLEMS269ExampIe7.22'----==::::::1Writeafunctionxy=xytrianglerv(m)thatgeneratesm,samplepairs(X,Y)inExample7.16.InExample7.16,wefoundthatfx(:i;)=2;i;O-<;r:<l'-l/x00otherwise,0otherwise.;i;,(7.73)ForO<;i;<1,wehavethatFx(x)=x2.UsingTheorem6.5togeneratesamplevaluesofX,wedefinev,=Fx(:i;)=x2.Then,for0<v,<1,x=.JU,.ByTheorem6.5,ifUisuniform(0,1),thenVuhasPDFfx(:i;).Next,weobservethatgivenX=;r:i,Yistheuniform(0,x,i)randomvariable.Givenanotheruniform(0,1)randomvariableUi,Theorem6.3(a)statesthat~=;i;iUiistheuniform(0,x,i)randomvariable.Weimplementtheseideasinthefunctionxytrianglerv.m.functionxy=xytrianglerv(rn);x=sqrt(rand(rn,1));y=x.*rand(rn,1);xy=[xy];Quiz7.7Forrandom·variablesXa.ndYvvithjointP11:FPx,Y(:i;,y)giveninExarr1ple7.11,vvriteaMATLABfunctionxy=dtrianglerv(m)thatgeneratesrn,samplepairs.ProblemsDifficulty:7.1.1EasyRandomvariableXhasCDFx<-3,0.4-3<x<5,0.85<x<7,1x>7.0Fx(x)=FindtheconditionalCDFFx1x>o(x)andPMFPx1x>o(x).Xisthediscreteuniform(0,5)randomvariable.\i\fhatisE[XIX>E[X]]?7.1.27.1.3Difficult+ExpertsOnlyInayouthbasketballleague,aplayerisfouledintheactofshootingalayup.Thereisaprobabilityq=0.2thatthelayupisgood,scoring2points.Ifthelayupisgood,theplayerisalsoa\varded1freethrow,givingtheplayerachanceatathree-pointplay.Ifthelayupismissed,then(becauseofthefoul)theplayerisstillawardedonepointautomaticallyandisalsoawardedonefreethro\v,enablingachancetoscoretwopointsintotal.Theplayermakesafreethro'vwithprobabilityp=1/2.7.1.4(a)\t\lhatisthePJVIFofX,thenumberofpointsscoredbytheplayer?XhasPlVIFPx(x)=Moderate(~)(1/2)4FindP..x-1B(x)'vhereB={X=f.0}.(b)FindtheconditionalP~1IFPxir(x)ofXgiveneventTthatthefreethrowisgood.[CHAPTER7270CONDITIONALPROBABILITYMODELS7.1.5Everydayyouconsidergoingjogging.Beforeeachmile,includingthefirst,youwillquitwithprobabilityq,independentofthenumberofmilesyouhavealreadyrun.Ho,vever,youaresufficientlydecisivethatyouneverrunafractionofamile.Also,wesayyouhaverunamarathon\Vheneveryourunatleast26miles.Giventhatapersonishealthy(eventH),abloodsugarmeasurementXistheGaussian(90,20)randomvariable.Giventhatapersonhasdiabetes,(eventD),XistheGaussian(60,40)randomvariable.Arandomlychosenpersonishealthy\vithprobabilityP[H]=0.9orhasdiabeteswithprobabilityP[D]=0.1.(a)LetJV!equalthenumberofmilesthatyourunonanarbitraryday.FindthePMFP1VJ(m,).(a)\tVhatistheconditionalPDFfx1H(x)?(b)Letrbetheprobabilitythatyourunamarathononanarbitraryday.Findr.(c)FindP[HIT-],theconditionalprobabilitythatapersonishealthygiventheeventofanegativetest.(c)LetJbethenumberofdaysinoneyear(notaleapyear)inwh.ichyourunamarathon.FindthePIVIFPJ(j).Thisanswermaybeexpressedintermsofrfoundinpart(b).(d)DefineJ{=M-26.LetAbetheeventthatyouhaverunamarathon.FindPKIA(k).7.1.6ArandomECEstudenthasheightXininchesgivenbythePDF.f~'24e-(x-70)/8+e-(x-65)2/8~5v87T(a)Sketchfx(x)overtheinterval60<x<75.(l140;thetestisnegative(eventT-)ifX<110,andthetestisambiguous(eventT0)if110<X<140.x(b)CalculatetheconditionalprobabilitiesI>[T+IH],andP[T-IH].(d)\i\fhenapersonhasanambiguoustestresult(T0),thetestisrepeated,possiblymanytimes,untileitherapositiveT+ornegativeT-resultisobtained.LetJ\Tdenotethenumberoftimesthetestisgiven.Assumingthatforagivenpersontheresultofeachtestisindependentoftheresultofallothertests,findtheconditionalPlVIFofNgiveneventHthatapersonishealthy.NotethatJ\T=1ifthepersonhasapositiveT+ornegativeT-resultonthefirsttest.7.1.8ForthequantizerofExample7.6,thedifferenceZ=X-Yisthequantizationerrororquantization"noise."_A..sinExample7.6,assumethatXhasauniform(-r/2,r/2)PDF.(a)GiveneventBithatY=Yi=6./2+it:::.andXisintheithquantizationinterval,findtheconditionalPDFofZ.(b)Sho\vthatZisauniformrandomvariable.FindthePDF,theexpectedvalue,andthevarianceofZ.7.1.9ForthequantizerofExample7.6,weshowedinProblem7.1.8thatthequantizationnoiseZisauniformrandomvariable.IfXisnotuniform,showthatZisnonuniformbycalculatingthePDFofZforasimpleexample.7.2.1Xisthebinomial(5,1/2)randomvariable.FindPx1B(x),wherethecondi-[PROBLEMStionB={X>Var[XIBJ?7.2.2J.lX}.WhatareE[XIBJandRandomvariableXhasCDF-1(0.()Fxxx<-1,0.2-1<x<0'0.70<::e<1,l1x>1.GivenB={IXI>O},findPxis(x).\1VhatareE[XIBJandVar[XIBJ?7.2.3Xisthecontinuousuniform(-5,5)randomvariable.GiventheeventB={IXI<3},findthe(a)conditionalPDF,fx1B(x),(b)conditionalexpectedvalue,E[XIB],(c)conditionalvariance,Var[XIBJ.7.2.4Yistheexponential(0.2)randomvariable.Given_4={Y<2},find:(a)fYIA(Y),(b)E[YIAJ.7.2.5Fortheexperimentofspinningthepointerthreetimesandobservingthemaximumpointerposition,Example4.5,findtheconditionalPDFgiventheeventRthatthemaximumpositionisontherightsideofthecircle.Whataretheconditionalexpectedvalueandtheconditionalvariance?7.2.6ThenumberofpagesXinadocumenthasI>JVIF0.15=1,2,3,4,::i;=5,6,7,8,::i;otherwise.AfirmsendsalldocumentswithanevenntunberofpagestoprinterAandalldocuments'vithanoddnumberofpagestoprinterB.(a)FindtheconditionalPl\.1FofthelengthXofadocument,giventhedocumentwassenttoA.Whataretheconditionalexpectedlengthandstandard•?dev1.at,ion.271(b)FindtheconditionalPlVIFofthelengthXofadocu1nent,giventhedocument'vassenttoBandhadnomorethansixpages.\tVhataretheconditionalexpectedlengthandstandarddeviation?7.2.7Selectintegratedcircuits,testtheminsequenceuntilyoufindthefirstfailure,andthenstop.LetNbethenumberoftests.Alltestsareindependent,'vithprobabilityoffailurep=0.1.ConsidertheconditionB={N>20}.(a)FindthePMFP1v(n,).(b)FindPN18(ri),theconditionalPMFof1'lgiventhattherehavebeen20consecutivetests'vithoutafailure.(c)\t\lhatisE[NIB],theexpectednumberoftestsgiventhattherehavebeen20consecutivetests'vithoutafailure?7.2.8WistheGaussian(0,4)randomvariable.GiventheeventC={vV>O},findtheconditionalPDF,fw1c(1D),theconditionalexpectedvalue,E[vVICJ,andtheconditionalvariance,Var[WICJ.7.2.9Thetimebet,veentelephonecallsatatelephones'vitchistheexponentialrandomvariableTwithexpectedvalue0.01.(a)\t\lhatisE[I'IT>0.02),theconditionalexpectedvalueofT?(b)'\i\fhatis·var[TIT>0.02),theconditionalvarianceofT?7.2.10Asthefinalriderinthefinal60kmti1netrialoftheTourdeFrance,RoymustfinishintimeT<1hourto'vintheTour.Hehasthechoiceofbikemadeof(1)carbonfiberor(2)titanium.Onthecarbonfiberbike,hisspeedVoverthecourseisthecontinuousuniformrandomvariablewithE[V)=58km/hrand\far[V)=12.Onthetitaniumbike,Vistheexponentialrandomvariable'vithE[V)=60km/hr.(a)RoychooseshisbiketomaximizeI>[vV],theprobabilityhe'vinstheTour.\i\fhichbikedoesRoychooseandwhatisP[W)?(b)SupposeinsteadthatRoyflipsafaircointochoosehisbike.WhatisP[W)?[CHAPTER7272CONDITIONALPROBABILITYMODELS7.2.11ForthedistanceDofashot-puttossinProblem4.7.8,findtheconditionalPDFsfDID>o(d)andfDID5.FindtheconditionalPMFPx,YIA(x,y).7.3.2ContinuingProblem7.3.1,letBdenotetheeventthatmax(X,Y)<5.FindtheconditionalPMFPx,·YIB(::r,y).7.3.3RandomvariablesXandYhavejointPDFfx,y(:i;,y)=6e-(2x+3y){0x>O,y>O,otherwise.LetAbetheeventthatX+Y<1.FindtheconditionalPDFfx,YIA(x,y).7.3.4NandJ{havejointPMFPN,K(n,k)={~'-rr-·En=l,2,...k=l,...,n,otherwise.LetBdenotetheeventthatN>10.(a)FindtheconditionalPMFsPNIB(n)andPN,KIB(n,,k).\1Vhichshouldyoufindfirst?(b)FindtheconditionalexpectedvaluesE[NIB],E[KIB],E[N+IVar[NIB],Var[KIB],E[NIXandYhavejointPDF(x+y)/3Ofx,y(:i;,y)=otherwise.0LetA={Y0<1}.(b)Findfx,YIA(:i;,y).(c)Findfx1A(x)andfYIA(y).7.3.6RandomvariablesXandYhavejointPDFfx,Y(x,y)=l0+2y)/3(a)WhatisP[-4]?(b)Findfx,YIA(x,y).(c)Findfx1A(x),andfYIA(y).7.3.7•Astudyexamined'vhethertherewascorrelationbet,veenhowmuchfootballaperson\vatchedandhowbaldthepersonwas.ThetimeT\Vatchingfootballvvasmeasuredona0,1,2scalesuchthatT=0ifapersonneverwatchedfootball,T=1ifapersonwatchedfootballoccasionally,andT=2ifapersonwatchedalotoffootball.Si1nilarly,baldnessBwasineasuredonthesamescale:B=0forapersonwithafullheadofhair,B=1foraperson'viththinninghair,andB=2forapersonwhohasnotmuchhairatall.TheexperimentwastolearnBandTforarandomlychosenperson,equallylikelytobeaman(eventM)ora\voman(eventW).Thestudyfoundthatgivenaperson\Vasaman(eventM),rando1nvariablesBandTwereconditionallyindependent.Similarly,giventhatapersonwasa\Voman(eventvV),BandTwereconditionallyindependent.l\/Ioreover,BandThadconditionaljointPMFsbIo120.30.5120.60.210.320.1120.20.2(a)FindtheconditionalI=>MFPB,'r1w(b,t)ofBandTgiventhatapersonisa\voman.(a)WhatisP[A]?((4xLetA={Y<1/2}.<x<l;0<y<1,0other,vise.(b)FindtheconditionalPMFPB,'TIM(b,t)ofBandTgiventhatapersonisaman.(c)FindthejointPlVIFPB,'r(b,t).(d)FindthecovarianceofBandT.AreBandTindependent?[PROBLEMS7.3.8RandomvariablesXandYhavejointPDF273Y=X+ZisanoisyobservationofX.vVhatistheconditionalPDFfy1x(ylx)?(b)FindfyIA(Y)andE[YIA].7.4.5Abusinesstripisequallylikelytotake2,3,or4days.Afterad-daytrip,thechangeinthetraveler's\Veight,measuredasanintegernumberofpounds,isauniform(-d,d)randomvariable.Foronesuchtrip,denotethenumberofdaysbyDandthechangein\VeightbyW.FindthejointPlVIFPD,w(d,71J).(c)Findfx1A(x)andE[Xl-4].7.4.65::r2/2<x<1;Ofx,Y(x,y)=0LetA={Y-1otherwise.<1/4}.(a)FindtheconditionalPDFfx,YIA(x,y).7.3.9XandYareindependentrandomvariableswithPDFsfx(x)=2x{0fv(Y)={LetA={X~y'<x<1;(4x+2y)/300O<y<l,otherwise.fx,Y(x,y)=0<x<1,other\vise,(a)}"'orwhichvaluesofyisfx1y(xly)defined?V\fhatisfx1y(xly)?other,vise.>Y}.(a)WhatareE[X]andE[Y]?(b)WhatareE[XIA]andE[YIA]?7.4.1XandYhavejointPDFGivenX=::r,•Y1isGaussian'vithconditionalexpectedvalue::randconditionalvariance1.•Y2isGaussian'vithconditionalexpectedvalue::randconditionalvariancex2.FindtheconditionalPDFsfy11x(Y1l:::r)andfY21x(Y2Ix)·7.4.2Xisthecontinuousuniform(0,1)randomvariable.GivenX=x,Yhasacontinuousuniform(0,x)PDF.\tVhatisthejointPDFfx,Y(x,y)?SketchtheregionoftheX,Yplaneforwhichfx,Y(x,y)>0.7.4.3Xisthecontinuousuniform(0,1)randomvariable.GivenX=:::r,Yisconditionallyacontinuousuniform(0,1+x)randomvariable.\i\!hatisthejointPDFfx,·y(x,y)ofXandY?7.4.4ZisaGaussian(0,1)noiserandomvariablethatisindependentofX,and(b)For'vhichvaluesof:::risfy1x(yl:::r)defined?V\fhatisfy1x(ylx)?7.4.7Astudent'sfinalexamgradedependsonhowclosethestudentsitstothecenteroftheclassroomduringlectures.Ifastudentsitsrfeetfromthecenteroftheroom,thegradeisaGaussianrandomvariable\Vithexpectedvalue80-randstandarddeviationr.IfrisasamplevalueofrandomvariableR,andXistheexamgrade,\vhatisfx1R(xlr)?7.4.8Y=ZX'vhereXistheGaussian(0,1)randomvariableandZ,independentofX,hasPMFl-pPz(z)={Pzz=-1,=1.TrueorFalse:(a)YandZareindependent.(b)YandXareindependent.7.4.9AttheOneTopPizzaShop,mushroomsaretheonlytopping.Curiously,apizzasoldbeforenoonhasmushrooms\Vithprobabilityp=1/3\vhileapizzasoldafternoonneverhasmushrooms.Also,apizzaisequallylikelytobesoldbeforenoonas[CHAPTER7274CONDITIONALPROBABILITYMODELSafternoon.Onadayinwhich100pizzasaresold,letNequalthenumberofpizzassoldbeforenoonandletMequalthenumberofmushroompizzassoldduringtheday.\~!hatisthejointPIVIFPM,1...1(1n,n,)?AreMandNindependent?Hint:FindtheconditionalPMFofMgivenN.7.4.10RandomvariablesXandYhavethejointPMFinthefollowingtable.Px,Yx,yX=-1x=Ox=ly=-13/161/6y=O1/161/6y=l01/81/801/6(a)AreXandYindependent?(b)Theexperimentfromv.;hichXandYarederivedisperformedsequentially.First,Xisfound,thenYisfound.Inthiscontext,labeltheconditionalbranchprobabilitiesofthefollo,vingtree:?X--l~~~~-~y?Y-0?Y=-1X=O~?._-1Y=OY=l?X=l~Y=O1---Y=l7.4.11Flipacointwice.Oneachflip,theprobabilityofheadsequalsp.LetXiequalthenumberofheads(either0or1)onflipi.LetvV=X1-X2andY=X1+X2.FindPw,Y(?D,y),Pw1y('U;ly),andPY1vv(yl1D).7.4.12Sho'vthatwithprobabilityp,independentofanyotherpacket.ObservepacketsattheInternetrouteruntilyouseetwovoicepackets.LetMequalthenumberofpacketsuptoandincludingthefirstvoicepacket.LetNequalthenumberofpacketsobserveduptoandincludingthesecondvoicepacket.FindtheconditionalPlVIFsPMIN(m,ln)andPNIM(n,lm,).Interpretyourresults.7.4.14Supposeyouarriveatabusstopattime0,andattheendofeachminute,withprobabilityp,abusarrives,or\Vithprobability1-p,nobusarrives.Wheneverabusarrives,youboardthatbuswithprobabilityqanddepart.Let'I"equalthenumberofminutesyoustandatabusstop.LetNbethenumberofbusesthatarrivewhileyou\vaitatthebusstop.(a)Identifythesetofpoints(n,t)forwhichP[J\T=n,,T=t]>0.(b)FindPN,'r(n,,t).(c)FindthemarginalPMFsPN(n,)andPr(t).(d)FindtheconditionalPMFsPN1r(n,lt)andP'TIN(tln,).7.4.15EachmillisecondatanInternetrouter,apacketindependentlyarriveswithprobabilityp.Eachpacketiseitheradatapacket(d)withprobabilityqoravideopacket(v).Eachdatapacketbelongstoanemail\Vithprobabilityr.LetNequalthenumberofmillisecondsrequiredtoobservethefirst100emailpackets.LetTequalthenwnberofmillisecondsyouobservetherouter\Vaitingforthefirstemailpacket.FindtheinarginalP~!.IFPr(t)andtheconditionalI>MFPNl'r(n,lt).Lastly,findtheconditionalP~!.IFp,r11v(tlri).XandYhavejointPDF7.5.1Hint:P[::r1<X<x2,y<Y<y+~]canbe\vrittenasanintegraloffx,Y(x,y).7.4.13PacketsarrivingatanInternetrouterareeithervoicepackets(v)ordatapackets(d).Eachpacketisavoicepacket.xf'y(x,y)={2Ootherwise.FindthePDFfy(y),theconditionalPDFfx1y(xly),andtheconditionalexpectedvalueE[XIY=y].[PROBLEMS7.5.2LetrandomvariablesXandYhavejointPDFfx,Y(x,y)giveninProblem7.5.1.FindthePDFfx(x),theconditionalPDFfy1x(11lx),andtheconditionalexpectedvalueE[YIX=x].7.5.3Theprobabilitymodelforrandomvariable_4is1/3PA(a)=2/30TheconditionalprobabilitymodelforrandomvariableBgiven_4is:1/3PBIA(bl-1)=PBIA(bll)=2/301/21/2lOb=0,b=1,b=0,b=1,otherwise.(b)IfA=1,'vhatistheconditionalexpectedvalueE[BIA=l]?=1,whatistheconditionalPlVIFPAIB(all)?(d)IfB=1,whatistheconditionalvarianceVar[AIB=1]ofA?(e)\i\fhatisthecovarianceCov[A,B]?7.5.4ForrandomvariablesAandBgiveninProblem7.5.3,letU=E[BIA].FindthePMFPu(u).WhatisE[U]=E[E[BIA]]?Randomvariables1\TandJ{havethejointPMF7.5.5PN'K(,,,,r,,7~)-_'Yl{RandomvariablesXandYhavejointPDF7.5.6fX,Y(x,y)={(a)WhatislOOne-100(n+l)!n=O,l,...;k=O,l,...,n,0otherwise.(a)FindthemarginalPMFP_N(n,)andtheconditionalPMFPK1N(klr1,).(b)FindtheconditionalexpectedvalueE[KIN=n,].~/2-1fy(y)?(b)\tVhatisfx1y(xly)?(c)\t\fhatisE[XIY=y]?OverthecircleX2+Y2<r2,randomvariablesXandYhavetheuniformPDF7.5.7otherwise,(a)\i\fhatistheprobabilitymodelforrandomvariables_4andB?WritethejointI>lVIFPA,B(a,b)asatable.(c)IfB(c)ExpresstherandomvariableE[Ia=-1,a=1,other,vise.275fX,Y(:i;,y)=1/(7rr2){Ox2+112<r2,otherwise.(a)\t\fhatisfy1x(11lx)?(b)\i\!hatisE[YIX=x]?(ContinuationofProblem4.6.14)Attimet=0,thepriceofastockisaconstantkdollars.Attimet>0thepriceofastockisaGaussianrandomvariableX'vithE[X]=kandVar[X]=t.Attimet,aCallOptionatStrikekhasvalue7.5.8wheretheoperator(·)+isdefinedas(z)+=max(z,0).Supposethatatthestartofeacht=30daymonth,youcanbuythecalloptionatstrikekatapriceDthatisarandomvariablethatfluctuateseverymonth.YoudecidetobuythecallonlyifthepriceDisnomorethanathresholdd*.\i\fhatvalueofthethresholdd*maximizestheexpectedreturnE[R]?Inaweeklylottery,each$1ticketsoldadds50centstothejackpotthatstartsat$1millionbeforeanyticketsaresold.Thejackpotisannouncedeachmorningtoencouragepeopletoplay.Onthemorningoftheithdaybeforethedrawing,thecurrentvalueofthejackpotJiisannounced.7.5.9[CHAPTER7276CONDITIONALPROBABILITYMODELSOnthatday,thenumberofticketssold,1',Ti,isaPoissonrandomvariablewithexpectedvalue.Ji.Thus,sixdaysbeforethedrav;,ring,themorningjackpotstartsat$1mil-Yoususpectit'sOKtoignorethedifferencesbetweenmenandwomensinceforbothgroups,weightandspeedarenegativelycorrelated,'vithp=-0.6.Tocon-lionandN6ticketsaresoldthatday.OnvinceyourselfthisisOK,youdecidetothedayofthedrawing,theannouncedjackpotisJodollarsandNoticketsaresoldbeforetheeveningdra,ving.Whataretheexpectedvalueandvarianceof.],thevalueofthejackpottheinstantbeforethedra,ving?Hint:·useconditionalexpectations.7.6.1You'vishtomeasurerandomvariableX'vithexpectedvalueE[X)=1andvariance\far[X)=1,butyourmeasurementprocedureyieldsthenoisyobservationY=X+Z,'vhereZistheGaussian(0,2)noisethatisindependentofX.(a)FindtheconditionalPDFZgivenX=x.fzix(zlx)=X7.6.4LetX1andX2haveabivariateGaussianPDF'vithcorrelationcoefficientp12suchthateachXiisaGaussian(Jl,i,ai)randomvariable.ShowthatY=XiX2hasvarianceof(b)FindtheconditionalPDFfy1x(yl2)ofYgivenX=2.Hint:GivenX=::e,YstudythespeedXand'veightYofacyclistrando1nlychosenfromalargemixedg;·oupo~maleandfemalecyclists.HowareXandYcorrelated?Explainyouranswer.Hints:LookaheadtoProblem9.2.4andalsousetheiteratedexpectationtofind+z.7.6.2XandYarejointlyGaussianrandomvariableswithE[X)=E[Y)=0andVar[X)=·var[Y)=1.Furthermore,E[YIXJ=X/2.Findfx,y(2;,y).7.6.3Astudyofbicycleridersfoundthatamalecyclist'sspeedX(inmilesperhourovera100-mile"century"ride)and'veightY(kg)couldbemodeledbyabivariateGaussianPDFfx,Y(x,y)'vithparametersJLx=20,ax=2,JLY=75,ay=5andpx,Y=-0.6.Inaddition,afemalecyclist'sspeedX'andweightY'couldbemodeledbyabivariateGaussianPDFfX',Y'(::e',y')'vithparametersJLx'=15,aX'=2,p,y1=50,ay1=5andPx',Y'=-0.6.Formenand'vomen,thenegativecorrelationofspeedandweightreflectsthecommon'visdomthatfastcyclistsaretlhin.Asithappens,cyclingismuchmorepopularamongmenthanwomen;inamixedgroupofcyclists,acyclistisamalewithprobabilityp=0.80.7.6.5·usetheiteratedexpectationforaproofofTheorem5.19vvithoutintegrals.7.7.1ForthemodemreceivervoltageXwithPDFgiveninExample7.8,use1v1ATLABtoplotthePDFandCDFofrandomvariableX.\tVriteal\IIATLABfunctionx=modernrv(rn)thatproducesmsamplesofthemodemvoltageX.7.7.2ForthequantizerofExample7.6,weshowedinProblem7.1.9thatthequantizationnoiseZisnonuniformifXisnonuniform.Inthisproblem,weexaminewhetheritisareasonableapproximationtomodelthequantizationnoiseasuniform.ConsiderthespecialcaseofaGaussian(0,1)randomvariableXpassedthroughauniformb-bitquantizerovertheinterval(-r/2,r/2)withr=6.Doesauniformapproximationgetbetterorworseasbincreases?Writeal\IIATLABprogramtogeneratehistogramsforZtoans,verthisquestion.[RandomVectorsInt11ischapter,"''eexpandontheconceptspresentedinCl1apter5.WhileChapter5introducedtheCDFandPDFofrirandorr1variablesX1,...,Xn,thischapter1focl1sesontherandorr1·vectorX=[X1Xn]•Arandornvectortreatsacollectionofn,randorn·variablesasasingleentit}'·Thus,·vectornotationprovidesaconciserepreser1tationofrelationshipsthat"''ouldother-vvisebeextrernelydifficulttorepresent.T11efirstsectionofthiscl1apterpresentsvectornotationforasetofrandornvariablesandtheassociatedprobabilityfunctior1s.Thest1bseqt1entsectionsdefir1ernargina.1probabilit}'f\1nctionsofsubsetsofrirandorr1variables,n,indeper1dentrandom·variables,independentrar1domvectors,andexpectedvaluesoffunctionsof'nrandom·variables.Wethenintroducethecovariar1cernatrixandcorrelationrnatrix,tv.rocollectionsofexpected·valuesthatpla}'anirr1portantroleinstochasticprocessesandinestirnationofrar1dornvariables.ThefinaltV\rosectionscoverGaussiar1rar1domvectors.andtheapplicationofl\IIATLA.B,whichisespeciallyusef\11ir1workingwithrr1ultiplerar1dornvariables.8.1VectorNotationArar1domvectorwithn,dirr1ensionsisaconciserepresentatior1ofasetofn,randornvariables.Thereisacorrespondir1gnotationfortheprobabilityrr1odel(CDF,PMF,orPDF)ofarandomvector.vVhenanexperirr1er1tprodt1cest-vvoorrnorerar1dom·variables,·vectorandrr1atrixnotationprovideacor1ciserepreser1tationofprobabilityrnodelsandtheirproperties.Thissectionpresentsasetofdefir1itionsthatestablishtherr1athematicalnotationofrandomvectors.'\''f\feuseboldfacenotationxforacoll1mnvector.Ro"''·vectorsaretrar1sposedcolumnvectors;x'isaro-vv·vector.Thecorr1ponentsofacolurnnvectorare,bydefinition,\:vritteninacolurr1n.Ho-vvever,tosavespace,-vve"''illoften277[278CHAPTER8RANDOMVECTORSt1set11etransposeofaro\:vvectortodisplayacol11mnvector:yisacol11mr1vector.Definition8.1RandomVectorArandomvectorisacolurnn,vectorX=[X1Xn]I.=[y1YnJ/Eachx.iisaran,dornvariable.Arandorr1variableisarandornvectorwithn,=1.Thesarnple·valuesofthecomponentsofarandorr1vectorconstituteacolurr1n·vector.Definition8.2--VectorSampleValue1Asamplevalueofarandomvectorisacolv,rnr1,vectorx=[1';1x;n]•Theithcornpor1,er1,t)Xi;ofthevectorxisasarnple'val'ueofaran,dornvariable,Xi.Follo-vvingot1rconventionforrar1dornvariables,theuppercaseXistherar1domvectorandthelowercasexisasamplevalueofX.Ho-vvever,wealsot1seboldfacecapitalss11chasAandBtodenotematriceswitl1componentsthatarenotrar1domvariables.Itv.rillbeclearfrornthecontextwl1etherAisarnatrixofnt1rr1bers,arnatrixofrar1dom·variables,orarandornvector.TheCDF,PNIF,orPDFofarar1dom·vectoristhejointCDF,joir1tPNIF,orjointPDFofthecorr1ponents.Definition8.3RandomVectorProbabilityFunctions(a)TheGDFofarandomvectorXisFx(x)=Fx1,...,Xn(x1,···,1';n)·{b)ThePMFofadiscreterandomvectorXisPx(x)=Px1,...,Xn(x1,···,1';n)·{c)ThePDFofacontinuousrandomvectorXisvVet1sesimilarnotationforafur1ctior1g(X)=g(X1,...,Xn)ofrirandorr1variablesandaft1nctiong(x)=g(x1,...,xn)ofn,nurnbers.Justas-vvedescribedtherelationshipoftworandornvariablesinCl1apter5,wecanexploreapairofrandornvectorsbydefiningajoir1tprobabilitymodelforvectorsasajoir1tCDF,ajointPMF,orajointPDF.Definition8.4ProbabilityFunctionsofaPairofRandomVectorsForran,dornvectorsXv.1ithricornpor1,er1,tsar1,dY'luithm,cornpor1,er1,ts:[8.1VECTORNOTATION279(a)ThejointGDFofXandYis{b)ThejointPMFofdiscreteran,dornvectorsXan,dYis(c)ThejointPDFofcoritirl'IJ,OUSran,dornvectorsXaridY'isThelogicofDefinition8.4isthatthepairofrar1domvectorsXandYisthesarr1easW=[X'Y']'=[X1XnY1Y;n]',aconcatenationofXandY.Thusaprobabilit:yfunctionofthepairXandYcorrespondstothesameprobabilit}'functionofW;forexarr1ple,Fx,v(x,y)isthesarneCDFasFw(w).Ifv.reareir1terestedonl}'inX=X1,...,Xn,\Vecar1usetherr1etl1odsintroducedinSection5.10toderi\rearnarginalprobabilityrr1odelofX1,...,Xnfrorr1thecompleteprobabilit}'rr1oclelforXi,...,Xn,Y1,...,·Yrn·Thatis,ifanexperirnentproducescontir1l1ousrandornvectorsXandY,thenthejoir1t\rectorPDF1·x,v(x,y)isacorr1pleteprobabilityrnodel,whilef'x(x)andfy(y)aremarginalprobabilit}'rnodelsforXandY.Example8.1RandomvectorXhasPDFfx(x)=wherea=[126eI-ax0x>0,(8.1)otherwise,3]'.WhatistheCDFofX?Becauseahasthreecomponents,weinferthatXisathree-dimensionalrandomvector.Expandinga'x,wewritethePDFasafunctionofthevectorcomponents,fx{x)=x;i0>0,(8.2)otherwise.ApplyingDefinition8.4,weintegratethePDFwithrespecttothethreevariablestoobtainFx(x)=>0,(1-e-x1)(1-e-2:1:2)(1-e-3x3)x,;,0otherwise.(8.3)[280CHAPTER8QuizRANDOMVECTORS8.1==~Discreterandorr1vectorsX=[xix;2x3J'andY=[Y1Y=AX.FindthejointP1!{FPy(y)ifXhasjointPl\IIF(1-P)r/1:3Px(x)=8.2y3]'arerelatedb:y<X2<;r:3;X1{1,2,...},X1,X2,X3Eothervvise0Y2andA='1-1001-1001IndependentRandomVariablesandRandomVectorsTheprobabilityrr1odelofthepairofindependentrandornvectorsXandYist.heproductoftheprobabilit}'rnodelofXandtheprobabilityrr1odelofY.Inconsideringtherelationsr1ipofapairofrandorr1vectors,v.rehavethefollovvingdefinitionofindependence:Definition8.5IndependentRandomVectorsD'iscrete:Px,v(x,y)=Px(x)Pv(y);Con,t'iriv,ous:fx,Y(x,y)=1·x(x)f.y(y).Example8.2AsinExample5.23,randomvariablesY1,...,·y4havethejointPDF40LetV=['Y1Y4J'andW=[Y2·y3]0<Y1<Y2<1,0<Y3<Y4<1,otherwise.1.(8.4)AreVandWindependentrandomvectors?WefirstnotethatthecomponentsofVareV1=Y1,andV2=Y4.Also,W1=Y2,andT¥2=~·Therefore,40Q<V1<'UJ1<1;Q<W2<V2<1,otherwise.(8.5)[8.3SinceV=[Y1Y4]1andW=['Y2FUNCTIONSOFRANDOMVECTORS281Y3]',(8.6)InExample5.23.wefoundj·y11y4(Y1,y4)andfy2,y1(y2,y3)inEquations(5.78)and(5.80).FromthesemarginalPDFs,wehave1·v(v)=1·w(w)=4(1-'V1)V20<V1,V2<1,0otherwise,4vJ1(1-w2)00<'IJJ1,'W2<1,otherwise.(8.7)(8.8)Therefore,fv(v)1·w(w)=whichisnotequalto16(1-V1)V2W1(1-'UJ2)0<V1,V2,W1,'ID2<1,0otherwise,fv,w(v,w).(8.9)ThereforeVandWarenotindependent.Quiz8.2Uset11ecorr1ponentsofY=[Y1,...,Y4]'inExarr1ple8.2toconstructtvvoindependentrandorn·vectorsVandW.Pro·vethatVandWareindeper1dent.8.3FunctionsofRandomVectorsPw('tJJ),thePMFoflV=g(X),afur1ctionofdiscreterandornvectorX,istheSl1rr1oft11eprobabilitiesofallsamplevectorsxforwhichg(x)='IJJ.ToobtainthePDFofVT!,aft1nctionofacontinuot1srandorr1vector,wederivetheCDFofVT!aridthendifferentiate.Theexpectedvalueofaf\1nctionofadiscreterandornvectoristhest1movertherangeoft11erandomvectoroftheproductofthef\1nctio11andthePMF.Theexpectedvalueofafunctionofacontinuousra,r1domvectoristheintegralovertherangeoftherar1domvectoroftheprodl1ctoftheft1r1ctionandthePDF.Jl1staswedidforonerandornvariablearidtvvorandorn·variables,vvecanderivearandorr1variableW=g(X)thatisafunctionofar1arbitrar}'numberofrandornvariables.IfWisdiscrete,theprobabilitymodelcanbecalculatedasPw('ID),theprobabilityoftheeventA={W='llJ}ir1Theorerr15.24.IfWiscontinuous,theprobabilitymodelcar1beexpressedasF11v(v;)=P[W<w].[282CHAPTER8RANDOMVECTORS==;;;;TheoremB.l--Forran,dornvariablevT!=g(X),Discrete:P11v(w)=P(W=tv]=LPx(x);x:g(x)='wContinuous:Fw(w)=P[W<w]=J···Jfx(x)dx1···dx""g(x)Example8.3Consideranexperimentthatconsistsofspinningthepointeronthewheelofcircumference1meterinExample4.1n,timesandobservingYnmeters,themaximumpositionofthepointerinthe'nspins.FindtheCDFandPDFof·yn·IfXiisthepositionofthepointeronspin'i,thenYn=rr1ax{X1,X2,...,Xn}.Asaresult,Yri<yifandonlyifeachXi<y.Thisimplies(8.10)Ifweassumethespinstobeindependent,theevents{X1{Xn<y}areindependentevents.Thus<y},{X2<y},...,(8.11)Example4.2derivesEquation(4.8):Fx(x)=0x;1x;<0,01,n,yn-I00otherwise.(8.13)ThefollowingtheoremisageneralizationofExarr1ple8.3.ItexpressesthePDFoft11emaxirr1t1mandrr1inirnurr1va.lt1esofasequenceofir1dependentandidentica.11}'distributed(iid)contir1uol1Srandornvariablesinterrr1softheCDFandPDFoftheir1dividt1alrandorr1variab1es.[8.3FUNCTIONSOFRANDOMVECTORS283Theorem8.2LetXbeavectorofriiidcon,tin/tJ,O'IJ,8raridorn'variable8,eachv1ithGDFFx(x)aridPDFf'x(x).(a)TheGDFaridthePDFofY=rr1ax{X1,...,Xn}arefy(y)='n(Fx(Y))n-Ifx(y).(b)TheGDFan,dthePDFofW=rnin{X1,...,Xn}areFM!('W)=1-(1-FX('lV))n,f'w('w)=ri(l-Fx(1D))n-If'x(w).ProofBydefinition,Fy(y)=I=>[Y<y].BecauseYisthemaximumvalueof{X1,...,Xn},theevent{Y<y}={X1<y,X2<y,...,Xn<y}.BecausealltherandomvariablesXiareiid,{Y<y}istheintersectionofri,independentevents.Eachoftheevents{Xi<y}hasprobabilityFx(y).Theprobabilityoftheintersectionistheproductoftheindividualprobabilities,whichimpliesthefirstpartofthetheorem:Fy(y)=(Fx(y))n.ThesecondpartistheresultofdifferentiatingFy(y)withrespecttoy.ThederivationsofFw(1D)andfvv(v1)aresimilar.Theybegin'viththeobservationsthatF1tv(1D)=1-P[W>7D]andthattheevent{W>7D}={X1>7D,X2>71J,...X11>v1},'vhichistheintersectionofriindependentevents,eachwithprobability1-Fx(v1).Insorneapplicationsofprobabilitytheory,weareinterestedor1lyintheexpectedvalueofafur1ction,notthecorr1pleteprobabilityrr1odel.Alt11oughv.recanalvvaysfindE(vT!)byfirstderivingPw(w)orf'v,1(1D),itiseasiertofindE(W)byappl}rir1gthefollowingtheorem.--==Theorem8.3Foraran,dorn'VectorX,theran,dorn1;ariableg(X)hasexpectedval'/J,eDiscrete:E(g(X))=L···Lg(x)Px(x);XnESxnContinuous:E[g(X)]=1:···1:g(x)fx(x)dx1···dxn.IfW=g(X)istheproductofn,ur1i·variatefunctionsandthecorr1ponentsofXarerr1t1tuallyindepender1t,E(TV)isaproductofn,expectedvalues.--=Theorem8.4When,thecornpon,en,tsofXareirideper1,der1,tran,dornvariables,E(g1(x1)92(x2)...9n(Xn)]=E(g1(x1)]E(g2(x2)]...E[gn(Xn)].[284CHAPTER8RANDOMVECTORSProofWhenXisdiscrete,independenceimpliesPx(x)impliesL···LE[g1(X1)···9n(Xn))=I:(8.14)xnESxnx1ESx1=(g1(x1)···9n(xn)Px(x)I:g1(x1)Px,(x1))···(g,,(x,,)Px,,(x,,))(8.15)xnESxnx1ESx1=E[g1(X1))E[g2(X2)]..·E[gn(Xn)].(8.16)Thederivationissimilarforindependentcontinuousrandomvariables.Wehaveconsideredthecaseofasinglerandorr1variablelV=g(X)derivedfrornarandomvectorX.Sorneexperirr1entsrr1ayyieldanewrandorn·vectorY-vvit11cornponentsY1,...,·Y;it11atarefl1nctionsoft11ecorr1ponentsofX:·yk=9k(X).WecanderivethePDFofYb}'firstfindingtheCDFFy(y)aridthenapplyingDefinition5.11.Thefollo-vvingtheoremdemonstratesthistechnique.Theorem8.5Given,theco'ntin,uousran,dornvectorX,defin,ethederivedraridornvectorYsuchthatYk=aXk+bforcon,stan,tsa>0an,db.TheGDFan,dPDFofYareFy(y)=Fx(YI-b,...,Ynaab))=yy()fProofvVeobserveYhasCDFFy(y)=P[aX1Fy(y)=P[X1<Y1aJ_.x(Y1an1-ab,...,yn-ab).+b<y1,...,aXn+b<Yn)·Sincea>0,b,...,Xn<Yn-bl=Fx(Y1-b,...,Yn-b).aa(8.17)aDefinition5.13definesthejointPDFofY,71...,yn)j.Y(Y)=f}FY1,n...,Yn(Y1,nuy1···uyn=..l.1·Xan(Yl-ba,...,Yn-b)a(8.18).Theorerr18.5isaspecialcaseofatransforrnatior1oftheforrr1Y=AX+b.T11efollowingtheorerr1isaconseql1enceofthechange-of-variabletheorern(Apper1dixB,]\/Iat11FactB.13)inmultivariablecalculus.==;;;:Theorem8.6___,;;==IfXisacon,tin,urxusran,dornvectoraridA'isan,in,vertiblernatrix;,theriYhasPDF1=AX+b[8.4ProofLetB={YIYEXPECTEDVALUEVECTORANDCORRELATIONMATRIX285<y}sothatFy(y)=JBfy(y)dy.Definethevectortransformationx=7,(y)=A-I(y-b).Itfollo,vsthatYEBifandonlyifXET(B),whereT(B)={xiAx+b<y}istheimageofBundertransformationT.ThisimpliesFv(y)=P[XET(B)]=(J,r(B)fx(x)dx(8.19)Bythechange-of-variabletheorem(MathFactB.13),(8.20)'vhereldet(A-l)IistheabsolutevalueofthedeterminantofA-1.Definition8.3fortheCDFandPDFofarandomvectorcombined\vithTheorem5.23(b)implythatfy(y)=fx(A-1(y-b))ldet(A-1)1.Thetheoremfollows,sinceldet(A-1)1=1/ldet(A)I.==--Quiz8.3---===i(A)Atestofligr1tbulbsproducedbyarnachinehasthreepossibleoutcorr1es:L,longlife;A,averagelife;andR,reject.Theresultsofdifferer1ttestsareindepender1t.Alltestshavetr1efollowingprobabilitymodel:P[L]=0.3,P[A]=0.6,andP[R]=0.1.LetX1,X2,andX3betr1enumberoflightbl1lbsthatareL,A,andRrespectivelyinfi·vetests.Findtr1eP1!{FPx(x);therr1arginalP1!{FsPx1(x1),Px2(x2),andPx3(x3);andthePJ\IIFofW=rnax(X1,X2,X3).(B)Tr1erandornvectorXhasPDFfx(x)=e-:r;30<X1<0otherv.rise.X2<::r;3,FindthePDFofY=AX+b.v.rhereA=diag[2,2,2]andb=[48.4(8.21)44]'.ExpectedValueVectorandCorrelationMatrixTheexpectedvalueofarandomvectorisavectorcontair1ingtheexpectedvaluesofthecornponentsofthevector.Thecovarianceofarandornvectorisasymmetricrr1atrixcontair1ingthevariancesofthecomponentsoftherandorr1vectorandthecovariancesofallpairsofrandorr1variablesintherandornvector.Correspor1dingtotr1eexpectedvalueofasinglerandornvariable,theexpectedvall1eofarandornvectorisacoh1rnn·vectorinv.rr1ichthecornponentsaretr1eexpectedvaluesofthecomponentsoftherandorr1vector.Thereisacorrespondir1gdefir1itior1oftr1e·variar1ceandstanda.rdde·viationofarandornvector.[286CHAPTER8RANDOMVECTORS==;....Definition8.6----==;;;.iExpectedValueVectorTheexpectedvalueofarandomvectorXisacolv,rnn,'vectorThecorrelationandcovariance(Definition5.7aridDefinition5.5)arenurnberst11atcontainimportantir1forrr1ationaboutapairofrandorn·variables.Correspondir1ginformationaboutrandornvectorsisreflectedinthesetofcorrelationsandthesetofcovariar1cesofallpairsofcornpor1ents.Thesesetsarereferredtoassecon,d-orderstatistics.Theyhaveaconcisematrixnotation.Toestablishther1otation,wefirstobservethatforrandornvectorsXv.rithn,cornpor1er1tsaridYwithrn,corr1ponents,thesetofallprodt1cts,X,i~i,iscontair1edinthen,x1T1,ran,dornrnatrixXY'.IfY=X,t11erandornmatrixXX'contair1sallproducts,X,iXj,ofcomponer1tsofX.===Example8.4:===1IfX=[X1X2X3],whatarethecomponentsofXX'?XX'=X1X2[X1X3XfX3J=X2X1X3X1(8.22)InDefinition8.6,vvedefin.edt11eexpectedvalueofarandorn·vectorasthe·vectorofexpected·values.Thisdefinitior1car1beextendedtorar1dommatrices.ExpectedValueofaRandomMatrixDefinition8.7Foraran,dornm,atrixA11Jiththeran,dornvariableA,ijasitsi,jthelerneri,t)E[A)'tsarnatrixv.Jithi,jthelernen,tE[Aij].ApplyingthisdefinitiontotherandommatrixXX',v.rehaveaconcisev.raytodefinethecorrelationrnatrixofrar1dom·vectorX.==;....Definition8.8----==:...JVectorCorrelationThecorrelationofarandomvectorXisaririxrirnatrixRxv1ithi,jthelerneri,tRx('i,j)=E[XiXj].Jn,'Vectorri,otatiori)Rx=E[XX').[8.4EXPECTEDVALUEVECTORANDCORRELATIONMATRIX287Example8.5IfX=[X1X2X3J',thecorrelationmatrixofXisRx=E(X1X2]E(X1X3)E[Xr]E(X2X3)E(X2X1)E[X~JE(X3X1)E(X3X2]E[X§JE[Xr]rx2,X1rx3,X1rx1,X2E[X~JrX3,X2rx1,X3rx2,X3E[X§JThei,jthelerr1entofthecorrelationmatrixistheexpected·vall1eoftherar1domvariableXiX.i.Thecovarian,cernatrixofXisasimilarger1eralizationoft11eco·variar1ceoftvvorandomvariables.Definition8.9--VectorCovarianeeThecovarianceofarandomvectorXisan,n,xn,rnatrix;Cx·1uithcornpor1,er1,tsCx(i,j)=Cov(Xi,X.7].Jn,vectorr1,otatior1,,Cx=E((X-µx)(X-µx)']Example8.6IfX=[X1X2X3J',thecovariancematrixofXisVar(X1]Cov[X1,X2]·var[X2]Cx=Cov[X2,X1]Cov[X3,X1]Cov[X3,X2]Cov[X1,X3]Cov[X2,X3]Var[X3](8.23)T11eorerr15.16(a),w11icl1connectst11ecorrelationandcovarianceofapairofrandomvariables,canbeextendedtorandorr1vectors.--==Theorem8.7Foraran,dornvectorXVJithcorrelation,rnatrixRx,co·uariaricernatrixCx,an,dvectorexpected·ualv,eµx,ProofTheproofisessentiallythesameastheproofofTheorem5.16(a),'vithvectorsreplacingscalars.CrossmultiplyinginsidetheexpectationofDefinition8.9yieldsCx=E[XX'-Xµ'x_-µxX'+µxµ'x]=E[XX']-E[Xµ'x_J-E[µxX'J+E[µxµ'x_].(8.24)[288CHAPTER8RANDOMVECTORSSinceE[X)=µxisaconstantvector1Example8.7FindtheexpectedvalueE[X],thecorrelationmatrixRx,andthecovariancematrixCxofthetwo-dimensionaIrandomvectorXwithPDFfx(x)=2Q<0otherwise.~r;l<X2<1,(8.26)Theelementsoftheexpectedvaluevectorare(8.27)TheintegralsareE[X1]=1/3andE[X2]=2/3,sothatµx=E[X]=[1/3Theelementsofthecorrelationmatrixare2/3]'.(8.28)(8.29)(8.30)TheseintegralsareE[X12]=1/6,E[X22]=1/2,andE[X1X2]=1/4.Therefore,1/61/4]Rx=[1/41/2.(8.31)WeuseTheorem8.7tofindtheelementsofthecovariancematrix.[1/61/4][1/92/9][1/181/36]Cx=Rx-µxµx=1/41/2-2/94/9=1/361/18.I(8.32)Inadditiontothecorrelationsandco·variancesoftheelementsofor1erandornvector,itisusefultorefertothecorrelatior1saridcovariancesofelerr1entsoftworandorr1vectors.[8.4EXPECTEDVALUEVECTORANDCORRELATIONMATRIX289,.....---Definition8.10VectorCross-CorrelationThecross-correlationofrandomvectors,Xv1ithn,cornpon,en,tsan,dY111ithm,cornpon,en,ts,isaririxrn,rnatrix;Rxyv1ithi,jthelernen,tRxy(i,j)=E[X.i.YiLor,irivector'notation,,Rxy=E[XY'].---Definition8.11---VectorCross-CovarianceThecross-covarianceofapairofrandomvectorsX'UJithn,cornpon,en,tsaridYv1ithrn,cornpon,en,tsisan,n,x'IT/,rnatrix;Cxy'tuithi,jthelernen,tCxy(i,j)Cov[Xi,·y7],or,in,vectorn,otation,,Cxy=E[(X-µx)(Y-µy)'].Todistinguishthecorrelationorcovarianceofarandornvectorfromthecorrelationorco·variar1ceofapairofrandorr1vectors,wesorr1etirnesllSet11eterrr1ir1ologyautocorrelation,anda11,toco'vo,r~ian,cevv11ent11ereisor1erandorr1vectorandcrosscorrelation,andcross-covarian,cevvhent11ereisapairofrandorr1vectors.NotethatvvhenX=Yt11eautocorrelationandcross-correlationareider1tical(asarethecovariances).Recognizingthisider1tity,sornetextsusethenotationRxxaridCxxforthecorrelationandcovariar1ceofarandorr1vector.WhenYisalineartransforrr1ationofX,t11efollo\vingtheorernstatestherelationshipofthesecond-orderstatisticsofYtothecorrespondingstatisticsofX.===Theorem8.8:===Xisa'nn,-d'irnerisirJ'nalran,dornvector'tuithexpectedvalv,eµx,correlatioriRx,an,dco'var~ian,ceCx.Thern,-dirnen,sion,alran,dornvectorY=AX+b,111hereAisan,mxrirnatrix;an,dbisan,1T1,-dirnen,sion,alvector,hasex;pectedval11,eµy,correlation,rnatr~ix;Ry,an,dcovarian,cernatrixCygiveribyµy=Aµx+b,Ry=ARxA'+(Aµx)b'+b(Aµx)'+bb',Cy=ACxA'.ProofVVederivetheformulasfortheexpectedvalueandcovarianceofY.Thederivationforthecorrelationissimilar.First,theexpectedvalueofYisµy=E[AX+b]=AE[X]+E[b]=Aµx+b.ItfollowsthatY-µy=A(X-(8.33)µx)·ThisimpliesCy=E[(A(X-µx))(A(X-µx))']=E(A(X-µx))(X-µx)'A'J=AE[(X-µx)(X-µx)']A'=ACxA'.(8.34)[290CHAPTER8RANDOMVECTORSExample8.8Giventheexpectedvalueµx,thecorrelationRx,andthecovarianceCxofrandomvectorXinExample8.7,andY=AX+b,whereA=10630and-2-2b=36(8.35))findtheexpectedvalueµy,thecorrelationRy,andthecovarianceCy.FromthematrixoperationsofTheorem8.8,weobtainµy1/6Ry=13/124/313/127.59.254/3Cy=9.2512.51/185/121/3=[1/35/123.53.2523]'and1/33.253.5(8.36)Thecross-correlationandcross-covariar1ceoftvvorandomvectorscanbederivedusingalgebrasirr1ilartotr1eproofofTheorem8.8.-----Theorem8.9ThevectorsXan,dY=AX+bha'vecross-correlat'ion,Rxyaridcross-covariariceCxygiven,byRxy=RxA'+µxb',Cxy=CxA'.Inthenextexa.rnple,vveseethatco-varianceandcross-co·varia.ncerr1a.tricesallovvustoquickl:yrcalculatet11ecorrelationcoefficientbetweenan:yrpairofcorr1ponentrandorr1·variables.==;;;..ExampIe8.91__.;;=;;;;:ContinuingExample8.8forrandomvectorsXandY=AX+b,calculate(a)Thecross-correlationmatrixRxyandthecross-covariancematrixCxy.(b)Thecorrelationcoefficientspy1,y3andpx2,Y1.(a)DirectmatrixcalculationusingTheorem8.9yields1/6Rxy=[1/413/125/34/3l29/12)1/18Cxy=[1/365/121/3l1/35/12.(8.37)[8.5GAUSSIANRANDOMVECTORS291(b)ReferringtoDefinition5.6andrecognizingthatVar[Yi,]=Cv(i,'i),wehaveCy(l,3)=0.756jCy(l,l)Cy(3,3)(8.38)Cxv(2,1)=l/2.)Cx(2,2)Cv(l,1)(8.39)Similarly,Quiz8.4Thetliree-dirnensionalraridomvectorX=[X1fx(x)=6X21X3]hasPDF0(8.40)0otlierwise.FindE(X]andthecorrela.tionaridcovariancernatricesRxaridCx.8.5GaussianRandomVectorsThemultivariateGaussianPDFisaprobabilit}'rriodelforavectorinvvhicliallthecorriporientsareGaussianraridornvariables.Theparametersoftherriodelaretheexpectedvaluevectorandtliecovariancerria.trixofthecornponents.AlinearfunctionofaGaussiarirandorr1vectorisalsoaGaussianrandomvector.Thecomponentsofthestandardnormalrandorrivectoraremutuallyindependentstandardriormalrandomvariables.l\/IultipleGaussianraridom·variablesappearinrnanypracticalapplicationsofprobabilit}'theory.ThernvJtivariateGa1Lss'ian,distribv.tion,isaprobabilitymodelforn,randorrivariables'ivitlithepropert}'thattherriarginalPDFsareallGaussian.Asetofrandorrivariablesdescribedbytherril1lti'irariateGaussianPDFissaidtobejoiritlyGaussian,.AvectorwhosecorriponentsarejointlyGaussianrandorrivariablesissaidtobeaGa'ussian,ran,dornvector.TliePDFofaGaussianrandorrivectorhasapartictllarlyconcisenotation.Definition8.12GaussianRandomVectorXistheGa'ussian,(µx,Cx)ran,dornvector'tnithex;1Jectedvalv.eµxan,dcovariariceCxifan,don,lyif.1(1()'-1fx(x)=(27r)n/2(det(Cx)]l/2exp-2x-µxCx(x-µx)1JJheredet(Cx),thedeterrnin,an,tofCx,sat'isfiesdet(Cx)rel="nofollow">0.)[292CHAPTER8RANDOMVECTORSDefinition8.12isageneralizationofDefir1ition4.8andDefinition5.10.vVhenn,=1,Cxandx-µxareO'~aridx-µ,x,andthePDFir1Definitior18.12redt1cestot11eordir1aryGa11ssiar1PDFofDefinition4.8.Thatis,al-dirr1ensionalGaussian(µ,,0'2)randorr1vectorisaGaussian(µ,O')randorr1variable.1InProblem8.5.8,weask}'Olltoshowthatforn,=2,Definition8.12reducestothebi·variateGaussianPDFinDefinition5.10.Theconditiont11atdet(Cx)>0isager1eralizationoft11ereqt1irementforthebivariateGaussiar1PDFthatIPx,YI<1.Basically,det(Cx)>0reflectst11erequirernentthatnorandom·variableX,iisalinearcornbinationoftheot11errandornvariablesinX.ForaGaussianrar1dorn·vectorX,anirnportantspecialcaseisCo·v(Xi,X.i]=0forall'i-=Jj.Int11ecovariancernatrixCx,t11eoff-diagonalelerner1tsareallzeroandtheit11diagonalelerr1entissirnplyVar(X,i]=O',z.Inthiscase,'""'ewriteCx=diag(O'f,O'§,...,O'~,]..\i\Ther1thecovariancernatrixisdiagonal,XiandX_7aret1ncorTelatedfor'i-=Jj.InT11eorerr15.20,v.reshoV\redthatt1ncorrelatedbivariateGaussiar1rar1domvariablesareindeper1dent.Thefollowingtheorerngeneralizesthisrestllt.---Theorem8.10--AGaussian,ran,dorn'/JectorXhasin,depen,den,tcorn,po'nen,tsifa'ndorilyifCxisadiagon,alrn,atrix;.ProofFirst,ifthecomponentsofXareindependent,thenforif=.j,XiandXjareindependent.ByTheorem5.17(c),Cov[Xi,Xj]=0.Hencetheoff-diagonaltermsofCxareallzero.IfCxisdiagonal,thenandCx=Cx-1--ItfollowsthatCxhasdeterminantdet(Cx)=(8.41)TI:.,1a-'fandthat_)_~(Xi-µ,i)(x_µx)'C-1(xxµx-L2(}i=l2(8.42)'/,FromDefinition8.12,v..;reseethat(8.43)1n2=IT~exp(-(xi-rli)/2a-f).27ra-..-i=Thusfx(x)1'/,=f1:1fxi(xi),implyingX1,...,Xn11For(8.44)areindependent.theGaussianrandomvariable,\·VespecifyparametersJJ,andabecausetheyhavethesameunits.However,theI>DFoftheGaussianrandomvectordisplaysµxandCxasparameters,andforonedimensionCx=ai-[8.5GAUSSIANRANDOMVECTORS293ExampIe8.1Q1----==::::::1Considertheoutdoortemperatureatacertainweatherstation.OnMay5,thetemperaturemeasurementsinunitsofdegreesFahrenheittakenat6AM,12noon,and6PMareallGaussianrandomvariables,X1,X2,X3,withvariance16degrees2.Theexpectedvaluesare50degrees,62degrees,and58degreesrespectively.ThecovariancematrixofthethreemeasurementsisCx=16.012.811.212.816.012.811.212.816.0(8.45)(a)WritethejointPDFofX1,X2usingthealgebraicnotationofDefinition5.10.(b)WritethejointPDFofX1,X2usingvectornotation.(c)WritethejointPDFofX=[X1X2X3J'usingvectornotation.(a)FirstwenotethatX1andX2haveexpectedvalues11,1=50and11,2=62,variancesO'f=CJ§=16,andcovarianceCov[X1,X2]=12.8.ItfollowsfromDefinition5.6thatthecorrelationcoefficientis=12.816=0.8.(8.46)FromDefinition5.10,thejointPDFis/(b)LetW=[X1X2]denoteavectorrepresentationforrandomvariablesX1andX2.FromthecovariancematrixCx,weobservethatthe2x2submatrixintheupperleftcorneristhecovariancematrixoftherandomvectorW.Thus50]µw=[62'cw=[16.012.812.8]16.0.(8.47)Weobservethatdet(Cw)=92.16anddet(Cw)112=9.6.tion8.12,thejointPDFofWisFromDefini-1.exp(-21(w-µw)TCw(w-I-µw)).603(8.48)fw(w)=(c)Sinceµx=[5062'(x)=fx58]'anddet(Cx)112=22.717,XhasPDF1exp(--1(x-µx)TCx1(x-µx)).357.82(8.49)[294CHAPTER8RANDOMVECTORSThefollowingt11eorernisageneralizationofTheorerr14.13.Itstatesthatalir1eartransformationofaGaussianrandomvectorrest1ltsir1anotherGat1ssianrandorr1vector.---TheoremB.ll--Given,ariri-dirnen,.'3'tO'nalGav,ssian,ran,dornvectorX'IJJithex1Jected'ualv,eµxaridCO'IHLr~ian,ceCx;aridan,1nxn,rnatrixA'llJithran,k(A)=rn,;Y=AX+b+bisan,rn-dirnen,sion,alGav,ssian,ran,dornvector'IJJithexpected'ualv,eµy=Aµxan,dco'uaria/nceCy=ACxA'.ProofTheproofofTheorem8.8containsthederivationsofµyandCy.01u·proofthatYhasaGaussianPDFisconfinedtothespecialcase'vhenm,=nandAisaninvertiblematrix.Thecaseofm<nisaddressedinProblem8.5.14.Whenm,=n,,'veuseTheorem8.6towrite(8.50)fy(y)=ldetl(A)Ifx(A-1(y-b))_exp(-~[A-1(y-b)-µxJ'Cx1[A-1(y-b)-µx])(27r)nf2-(8.51)112ldet(A)lldet(Cx)lIntheexponentoffy(y),weobservethat(8.52)sinceµy=Aµx+b.Applying(8.52)to(8.51)yieldsy()=exp(-~[A-1(y-µy)]'Cx1[A-1(y-µy)])f(27r)n/2yUsingtheidentitiesldet(A)lldet(Cx)l1/2canwrite=.Yj(y)(8.53)112ldet(A)lldet(Cx)l=ldet(ACxA')l1/2and(A-1)'exp(-~(y-µy)'(A')-1Cx1A-1(y-µy))(27r)n/2ldet(ACxA')I1/2=(A')-1,'ve(8.54)Since(A')-1Cx1A-1=(ACxA')-1,'veseefromEquation(8.54)thatYisaGaussianvectorwithexpectedvalueµyandcovariancematrixCy=ACxA'.Example8.11ContinuingExample8.10,usetheformulaYi=(5/9)(Xi-32)toconvertthethreetemperaturemeasurementstodegreesCelsius.(a)Whatisµy,theexpectedvalueofrandomvectorY?(b)WhatisCy,thecovarianceofrandomvectorY?[8.5(c)WritethejointPDFofY=[Y1Y2GAUSSIANRANDOMVECTORSY3J'usingvectornotation.(a)Intermsofmatrices,weobservethatY=AXA=(b)Sinceµx=[50625/90005/90005/9295)+bb=where160119(8.55)158]',fromTheorem8.11,10µy=Aµx+b=50/3130/9(8.56)(c)ThecovarianceofYisCy=ACxA'.WenotethatA=A'=(5/9)1whereIisthe3x3identitymatrix.ThusCy=(5/9)2CxandCy1=(9/5)2Cx1.ThePDFofYis1(81T-1)fy(y)=24.47exp-50(y-µy)Cx(y-µy)·(8.57)Astandardnorrnalrar1dorn·vectorisageneralizationofthestandardnorrnalrandorr1variableinDefinition4.9.---Definition8.13·---StandardNormalRandomVectorThen,-dirnen,siorialstandardnormalrandomvectorZisthen,-dirnen,sion,alGav,ssian,ran,dornvector'tuithE[Z]=0an,dCz=I.FrornDefinition8.13,eachcomponentZiofZhasexpectedvall1eE[Zi]=0andvarianceVar[Zi]=1.Thus

ZiistheGaussian(0,1)randornvariable.Inaddition,E[ziz..7]=0foralli=Ij.SinceCzisadiagonalrnatrix,Z1,...,Znareir1deper1dent.Inmanysitt1ations,itist1sefultotrar1sforrntheGat1ssian(µ,x,O'x)rar1domvariableXtothestandardr1ormalrandom·variableZ=(X-µ.x)/O'x.ForGaussianvectors.vvehaveavectortra.nsforrr1ationtotrar1sformXintoa.standardnorrnalrandorr1vector.I---Theorem8.12t--ForaGav,ssian,(µx,Cx)ran,dornvector;letAbean,n,xn,rnatrii;'tuiththeJJropertyAA'=Cx.Theran,dornvectorisastaridardn,orrnalraridornvector.[296CHAPTER8RANDOMVECTORSProofApplyingTheorem8.11withAreplacedbyA-1,andb=A-lµx,wehavethatZisaGaussianrandomvector'vithexpectedvalue(8.58)andcovarianceCz=A-1Cx(A-1)'=A-lAA'(A')-1=I.(8.59)Thetransformationinthistheorerniscor1siderablylessstraightforwardthar1thescalartransforrnationZ=(X-µx)/(}'x>becat1seitisnecessarytofir1dforagivenCxarnatrixAv.riththepropertyAA'=Cx.ThecalculationofAfrornCxcar1beachie·vedbyapplyingt}1elinearalgebraproceduresin,gular'oal'uedecornposition,.Section8.6describesthisprocedureinmoredetailandappliesittogeneratingsarnplevaluesofGaussiar1randornvectors.TheinversetransformofTheorem8.12ispartic11larlyusef\11incomputersirnt1lations.,-----Theorem8.13Given,then,-dirnerL'3'ion,alstan,dardriorrnalran,dornvectorZ)an,in/oertiblen,xrirnatr~ix;A)an,dan,n,-dirnen,sion,alvectorb7X=AZ+bisarin,-dirnen.sion,alGa11,ssian,ran,dornvector'UJithex;pectedval'ueµxcovarian,cernatrixCx=AA'.ProofByTheore1n8.11,XisaGaussianrandomvectorwithexpectedvalueµx=E[X)=E[AZ+µx)=AE[Z)+b=b.(8.60)ThecovarianceofXisCx=ACzA'=AIA'=AA'.(8.61)Theorerr18.13sa:yst}1at-vvecantrar1sforrr1thestandardr1ormal·vectorZir1toaGaussianrandorr1vectorXv.rhoseco-variancematrixisoftheformCx=AA'.Thet1sefulnessofTheorerr1s8.12arid8.13dependsonwhetherwecanalwaysfindamatrixAs11cht}1atCx=AA'.Infact,asweverifybelo-vv>thisispossibleforeveryGaussianvectorX.Theorem8.14ForaGa1J,ssian,vectorX'tJJiththatCx=AA'.covar~iariceCx7thereal'tJJaysexistsarnatrixAsuchProofToverifythisfact,'veconnectsomesimplefacts:[8.5GAUSSIANRANDOMVECTORS297•InProblem8.4.12,weaskyoutosho'vthateveryrandomvectorXhasapositivesemidefinitecovariancematrixCx.By~!.IathFactB.17,everyeigenvalueofCxisnonnegative.•ThedefinitionoftheGaussianvectorPDFrequirestheexistenceofCx1.Hence,foraGaussianvectorX,alleigenvaluesofCxarenonzero.Fromthepreviousstep,'veobservethatalleigenvaluesofCxmustbepositive.•SinceCxisarealsymmetricmatrix,~!lathFactB.15saysithasasingularvaluedecomposition(SYD)Cx=UDU'whereD=diag[d1,...,dn]isthediagonalmatrixofeigenvaluesofCx.Sinceeachdiispositive,'vecandefinen1/2=diag[~,...,.Jd:],and'vecan'vrite(8.62)\\!eseethatA=UD112.FrornT11eorerns8.12,8.13,and8.14,itfollowsthatanyGaussian(µx,Cx)randomvectorXcanbewrittenasalineartransforrnationofur1correlatedGaussian(0,1)rar1dorr1variables.Ir1terrrisofthes·vnCx=UDU'andthestandardr1ormalvectorZ,thetransforrnationisx=un1!2z+µx.Werecallt11atUhasorthonorrna.1colurr1nsu1,...,Un·Whenµxtion(8.63)canbev.rrittenas(8.63)0,Equa-nX=Lv7£uiZi.(8.64)i=lTheinterpretationofEquation(8.64)ist11ataGat1ssiar1randornvectorXisacornbinatior1oforthogonalvectorsv'Cftui,eac11scaledbyanindependentGaussianrandorr1variableZ,i·Ir1awidevarietyofproblernsinvolvingGaussianrandorr1\rectors.thetransforrnationfrornt11eGat1ssianvectorXtot11estar1dardnorrnalrandomvectorZisthekeytoanefficientsolution.Also,-vve"''illseeinthenextsectionthatTheorern8.13isesser1tialinusir1gMATLABtogeneratearbitraryGaussiar1randorr1vectors.I==--Quiz8.5,~=:::iZisthetV\ro-dirnensior1alstandardnorrnalrar1domvector.TheGaussianrandorr1vectorXhascomponents(8.65)CalculatetheexpectedvaJue\rectorµxandthecovariancerr1atrixCx.[2988.6CHAPTER8RANDOMVECTORSl\!JATLABMATLABisespeciallyllSefulforrandorr1vectors.\!Veuseasarr1plespacegridtocalculatepropertiesofaprobabilit:yrnodelofadis-creterandornvector.Wet1sethefunctionsrandnandsvdtogeneratesamplesofGal1ssianrandomvectors.AsinSection5.11)wederr1onstratetwovVa}'Soft1singJ\IIATLABtostt1dyrandomvectors.\!Vefirstpresentexarr1plesofprograrnsthatcalct1lateva1t1esofprobabilityfunctior1s)inthiscasetr1ePJ\IIFofadiscreterandorr1vectorandthePDFofaGaussiar1rar1domvector.Thenwepreser1taprogramthatger1eratessarr1plevaluesoftheGaussian(µx)Cx)randorn·vectorgivenan}'µxandCx.ProbabilityFunctionsTheMATLA.Bapproachofusingasarr1plespacegrid,preser1tedinSectior15.11)car1alsobeappliedtofinitera.ndornvectorsXdescribedb}'aPJ\IIFPx(x).i::::::==Example8.12FiniterandomvectorX=[X1X2,···1X5]hasPMFkM;i;.;,E0i=1)2)''')5)otherwise.Px(x)={-10)-9)...)10};(8.66)Whatistheconstantk?FindtheexpectedvalueandstandarddeviationofX3.SummingPx(x)overallpossiblevaluesofxisthesortoftedioustaskthatJ\11.A.TLABhandleseasily.Herearethecodeandcorrespondingoutput:%x5.msx=-10:10;[SX1,SX2,SX3,SX4,SX5]...=ndgrid(sx,sx,sx,sx,sx);P=sqrt(sx1.-2+sx2.-2+sx3.-2+sx4.-2+sxs.-2);k=1.0/(sum(sum(sum(sum(sum(P))))))P=k*P·'EX3=sum(sum(sum(sum(sum(P.*SX3)))))EX32=sum(sum(sum(sum(sum(P.*(SX3.-2))))));sigma3=sqrt(EX32-(EX3)-2)>>x5k=1.8491e-008EX3=-3.2960e-017sigma3=6.3047>>Infact,bysymmetryarguments,itshouIdbeclearthatE(X3]=0.Inaddingterms,J\IIATLAB'sfiniteprecisionledtoasmaIIerrorontheorderof10-17.Exarnple8.12dernor1stratestheuseofMATLABtocalculatepropertiesofaprobabilitymodelbyperforrr1ir1glotsofstraightforvvardcalculations.Foracontinuousrandorr1vectorX)J\!IATL.A.Bcol1ldbe11sedtocalct1lateE[g(X)]usingTheorem8.3andnumericintegration.OnestepinSl1chacalculationiscomputing·valuesofthePDF.Thenextexarnpleperforrr1sthisfunctionforan}'Gat1ssian(µx)Cx)rar1domvector.[8.6MATLAB299Example8.13WriteaMATLABfunctionf=gaussvectorpdf(mu,C,x)thatcalculatesf'x(x)foraGaussian(µ,C)randomvector.functionf=gaussvectorpdf(mu,C,x)n=length(x);z=x(:)-mu(:);f=exp(-z'*inv(C)*z)/...gaussvectorpdfcomputestheGaussianPDFfx(x)ofDefinition8.12.Ofcourse,MATLABmakesthecalculationsimpiebyprovidingoperatorsformatrixinversesanddeterminants.sqrt((2*pi)~n*det(C));SampleValuesofGaussianRandomVectorsGal1ssiariraridom·vectorsappearina'ividevarietyofexperirrients.Here-vvepresentaprogramthatusesthebuilt-inJ\IIA.TLABfunctionrandntogeneratesample\raluesofGaussian(µx,Cx)rar1dorri\rectors.TliematrixnotationleridsitselftoconciseMATLABcoding.OurapproacliisbasedoriTheorem8.13.Inparticular,wegenerateastandardnorrnalrandorn\rectorZand,gi'ireriacovariancerria.trixC,-vveusebuilt-inJ\IIATLABftlnctionstocalculateamatrixAsuclitlia.tC=AA'.ByTheorerri8.13,X=AZ+µxisaGaussian(µx,C)\rector.AlthoughtheJ\11.A.TLABcodeforthistask"'rillbeqtliteshort,itneedssorrieexplanation:•x=randn(m,n)producesanrn,xrirnatrix,V\ritheachmatrixelernenta.Gaus-siari(0,1)ra.ndornvariable.Thusea.chcolurrinofxisasarriplevectorofstandardnorrna.lvectorZ.•[U,D,VJ=svd(C)isthesingl1larvaluedecorriposition(SVD)ofrria.trixC.Irirnathnotation,givenC,svdproducesadiagonalrnatrixDofthesa.rnedi-rnensionasCand\vithnoririegativediagonalelementsindecreasingorder,ariduriitar}rrria.tricesUaridVsothatC=UDV'.Singularvaluedecornpositioriisa.po'iverf11ltechniquethatca.ribeappliedtoariyrria.trix.\i\ThenCisaco'irariancematrix,thesingularvaluedecorripositionyieldsU=VandC=UDU'.Jt1stasintheproofofTheorern8.14,A=UD112.functionx=gaussvector(mu,C,m)[U,D,V]=svd(C);x=V*(D~(0.5))*randn(n,m)...+(mu(:)*ones(1,m));UsingJ\!I.A.TLABfunctionsrandnandsvd,genera.tingGaussianrandornvectorsiseasy.Thefunctionx=gaussvector(mu,C,1)producesa.Gaussian(mu,C)randornvector.Thegeneralforrrigaussvector(mu,C,m)prodt1cesanrixrn,matrixwhereeachofthern,colurnnsisa.Gaussianrandorn\rectorV\rithexpected\raluemuandCO\ra.ria.nceC.Thereasorifordefininggaussvectortoreturnrn,\rectorsatthesa.rnetimeisthatcalculatingthesingularvaluedecorripositioriisacomputationallyburdensornestep.Instead,weperformtlieSVDjt1stonce,rathertlianrn,tirries.==--Quiz8.h--==Thedailynoonternperat11re,rneast1redindegreesFahrenheit,inNewJerseyin/Jl1l}rcanbernodeledasa.GaussianrandorrivectorT=[T1T31]'ivhereTi[300CHAPTER8RANDOMVECTORSistheternperatureontheithdayofthernor1th.St1pposet11atE[T;,]=80foralli,andthatT;,and'J'.7havecovarianceCo·v[T;,,T1].=36.1(8.67).1+'/,-]1Definethedail}'averageterr1perat11reas(8.68)Basedor1thisrnodel,v.rriteal\IIATLA.Bprogramp=julytemps(T)thatcalculatesP[Y>T],theprobabilitythatthedailya\reragetemperatureisatleastTdegrees.Fv,rtherReadin,g:[vVSOl]and[PP02]rnakeextensiveuseofvectorsaridmatrices.Togodeeplyintovectorrandornvariables,studentscar1use[Str98]togainafirrngraspofprinciplesoflir1ea.ralgebra.ProblemsDifficulty:ModerateEasy8.1.1ForrandomvariablesX1,...,XninProblem5.10.3,letX=[X1Xn]'.\i\fhatisfx(x)?8.1.2RandomvectorXhasPDFfx(x)={ca'x0wherea=[a1eachcomponentx>isavectorvvith0.\i\fhatisc?Givenfx(x)'vithc=2/3anda1=a2=a3=1inProblem8.1.2,findthemarginalPDFfx3(x3).8.1.38.1.4.X=fx(x)=[X1{60X2X3]'hasPDF0<X1<X2<other,vise.X3+ExpertsOnlyedgementfromthereceiver.\i\Thenitreceivestheackno,vledgement,ittransmitsthenextmessage.Iftheackno,vledgementdoesnotarrive,itsendsthemessageagain.Theprobabilityofsuccessfultransmissionofamessageispindependentofothertransmissions.LetK=[K1K2K3bethethree-dimensionalrandomvectorin'vhichJ{iisthetotalnumberoftrans1nissionswhenmessageiisreceivedsuccessfully.(K3isthetotalnumberoftransmissionsusedtosendallthreemessages.)ShowthatJ'0<<1,other,vise,an]'aiDifficult<1,1LetU=[X1X2]',V=[X1X3]andW=[X2X3J'.FindthemarginalPDFsfu(u),fv(v)andf,,-(w).8.1.5A'virelessdataterminalhasthreemessages'vaitingfortransmission.Aftersendingamessage,itexpectsanacknowl-p3(1-p)k3-3PK(k)=ki<k2<k3;kiE{1,2...}'08.1.6FromthejointPl\/IFPK(k)inProblem8.1.5,findthemarginalPlVIFs(a)PK1,K2(k1,k2),(b)PK1,K3(k1,k3),(C)pK2,K3(k2,k2),(d)PK1(k1),PK2(k2),andPK3(k3).8.1.7•LetNbether-dimensionalrandomvector'viththemultinomialPlVIFgivenin[PROBLEMSExample5.21'vithri>r>2:n(111,...,rl,r8.2.3AsinExample8.1,therandomvectorXhasPDF)pn1...pnr1r+···+Ni.\i\fhatistheP~l[F0ofK.(b)ForeachjE{1,2,...,n,-l},findthemarginalPMFPK1,...,K_1(ki,...,kj).(c)ForeachiE{1,2,...,n},findthemarginalP~l[FP1fx(x)={~other,¥ise.~-x,0<X1<X2<X3,other,vise.ArethecomponentsofXindependentrandomvariables?8.2.5TherandomvectorXhasPDFfx(x)={~-xo0<X1<X2<X3,otherwise.fY1,Y4(y1,y4),8.1.9_AsageneralizationofthemessagetransmissionsysteminProblem8.1.5,consideraterminalthathasn,messagestotransmit.Thecomponentskiofthen,dimensionalrando1nvectorKarethetotalnumberofmessagestransmitted'vhenmessageiisreceivedsuccessfully.P~l[F>0,x8.2.4ThePDFofthe3-dimensionalrandomvectorXisofT1andT2?8.1.8TherandomvariablesY1,...,Y4havethejointPDFFindthemarginalPDFsfY1,Y2(Y1,Y2),andfY1(y1).~e-•'xwherea=[l23]'.ArethecomponentsofXindependentrandomvariables?fx(x)={(c)\i\fhatisthejoint(a)Findthefx(x)={.(a)WhatisthejointJY~!fFofN1andN2?Hint:Considerane'vclassificationschemewithcategories:s1,s2,and"other."(b)LetTi=N1P~l[FofTi?3010af.ForwhatvaluesareY1andY2independent?e8.5.14Inthisproblem,weextendtheproofofTheorem8.11tothecasewhenAism,x11,withm<11,.Forthisproof,'veassumeXisan11,-dimensionalGaussianvectorandthatwehaveprovedTheorem8.11forthecasem=11,.Sincethecasem,=riissufficienttoprovethatY=X+bisGaussian,itissufficienttosho'vform,<11,thatY=AXisGaussianinthecase'vhenµx=0.(a)Prov~thereexistsan(11,-1n)x11,matrixAofrankn-m,withthepropertythatAA'=0.Hint:ReviewtheGram-Schmidtprocedure.[PROBLEMS(b)LetA=ACx1anddefinetherandomvectory=[~]=[1]x305WhataretheexpectedvalueandvarianceofY?WhatisP[Y<25)?8.6.3UseTheorem8.11forthecasem,=n,toarguethatYisaGaussianrandomvector.(c)FindthecovariancematrixCofY.UsetheresultofProblem8.5.13toshowthatYandYareindependentGaussianrandomvectors.A8.6.1ConsiderthevectorXinProblem8.5.landdefineY=(X1+X2+X3)/3.\i\fhatistheprobabilitythatY>4?8.6.2AbettermodelforthesailboatraceofI>roblem5.10.8accountsforthefactthatallboatsaresubjecttothesamerandomnessof'vindandtide.Supposeintheraceoftensailboats,thefinishingtimesareidenticalGaussianrandomvariableswithexpectedvalue35minutesandstandarddeviation5minutes.However,foreverypairofboatsiandj)thefinishtilmesandjhavecorrelationcoefficientp=0.8.xixixForthevectorofdailytemperatures1[T1T31JandaveragetemperatureYmodeledinQuiz8.6,we'vishtoestimatetheprobabilityoftheeventTofor1nanestimateofA,generate10,000independentsamplesofthevectorTandcalculatetherelativefrequencyofAinthosetrials.8.6.4vVecontinueProblem8.6.2'vherethevectorXoffinishtimeshascorrelatedcomponents.LetWdenotethefinishtimeofthewinningboat.\i\fewishtoestimateP[W<25),theprobabilitythatthewinningboatfinishesinunder25minutes.Todothis,simulatem=10,000racesbygeneratingm,samplesofthevectorXoffinishtimes.Let}j=1ifthewinningtimeinraceiisunder25minutes;other,vise,}j=0.CalculatetheestimateI>[W<25)(a)WhatisthecovariancematrixofX=1[X1X10]?(b)Lety__X_1_+_X_2_+_·_·_·_+_X_1010.8.6.5Writeal\IIATLABprogramthatsimulatesm,runsoftheweeklylotteryofProblem7.5.9.Form=1000sampleruns,formahistogramforthejackpotJ.[SumsofRandomVariablesRandorr1variablesoftheforrnvVn=X1+···+Xn(9.1)appearrepeatedlyinprobabilitytheoryaridapplications.'\N'ecouldinprinciplederivetheprobabilityrnodelofWnfrorr1thePMForPDFofX1,...,Xn.However,inrnanypracticalapplications,thenatureoftheanalysisorthepropertiesoftherandom·variablesallovvu.stoapplytechniqt1esthataresimplerthananalyzingageneraln,-dirr1er1sionalprobabilityrnodel.InSection9.1weconsiderapplic,a,tior1sinvvr1ichourinterestisconfinedtoexpectedvaluesrelatedtoVVn,ratherthanacornpletemodelofHin.Subsequer1tsectionsernphasizetechr1iquesthatapply"''henX1,...,Xnarernutuallyir1dependent.Ausefulwa:ytoanal}rzethesurnofindependentrandorr1variablesistotransformtr1ePDForPNIFofeachrandornvariabletoarnorneri,tgeri,eratiri,gf'un,ction,.Thecentrallirnittheoremrevealsafascinatir1gpropertyoftr1est1rr1ofindependentrandorr1variables.ItstatesthattheCDFofthesurr1con·vergestoaGaussianCDFasther1urr1berofterrr1sgroV\rsvvithot1tlimit.Tr1istheorernallowsustot1sethepropertiesofGaussiar1randorr1variablestoobtainacct1rateestirr1atesofprobabilitiesassociatedv.rithsumsofotherrandorr1variables.Inrr1an}'casesexactcalculationoftheseproba,bilitiesisextremel}'difficult.9.1ExpectedValuesofSumsTheexpectedvaXueofasumofan,yrar1domvariablesisthesurr1oftr1eexpected\ralues.Thevaric.mceofthesumofanyrandorr1variableisthesurnofallthecovariances.Thevarianceofthesumofirideper1,der1,trandomvariablesisthesurr1oftr1evariances.306[9.1EXPECTEDVALUESOFSUMS307ThetheorernsofSection5.7canbegeneralizedinastraightforwardrnannertodescribeexpected·valuesa.nd·variancesofsurnsofrr1orethantworandorr1variables.'"---Theorem9.1--•Foran,ysetofra/ndornvariablesX1,...,Xn,theexpectedvalv,eE[Wnl=S?J,rn,WnX1+···+XnhasE[xi]+E[x2]+...+E[Xnl.Proof\l\leprovethistheorembyinductiononn,.InTheorem5.11,\VeprovedE[W2]=E[X1]+E[X2].Now\VeassumeE[Wn-1]=E[X1]+···+E[Xn-1].NoticethatWn=Wn-1+Xn.SincevVnisasumofthetworandomvariablesvVn-1andXn,'veknowthatE[Wn]=E[vVn-1]+E[Xn]=E[X1]+···+E[Xn-1]+E[Xn]·Keepinrnir1dthattheexpectedvalueofthesurnequalsthesurnoftheexpectedvaluesvvhetherornotX1,...,Xnareindependent.ForthevarianceofWn,vveha;vethegeneralizatior1ofTheorern5.12:---Theorem9.2ThevariariceoflV:n=X1+···+Xnisn-1nVar(lV:n]=L·var(Xi]+2i=lnLLCo·v(Xi,X.i].'i=l.i=i+lProofFromthedefinitionofthevariance,\Vecan\vrite·var[Wn]=E[(Wn-E[Wn])2].Forconvenience,letcanwriteJ.liden.oteE[Xi].SincevVn=I::11XnandE[vVn]=I::11J.li,\Ve(9.2)nn=°L°LCov[Xi,Xj].(9.3)i=lj=lIntermsoftherandomvectorX=[X1Xn]',weseethatVar[Wn]isthesumofalltheelementsofthecovariancematrixCx.RecognizingthatCov[Xi,Xi]=Var[X]andCov[Xi,Xj]=Cov[Xj,Xi],weplacethediagonaltermsofCxinonesumandtheoff-diagonalterms(\vhichoccurinpairs)inanothersumtoarriveattheformulainthetheorem.WhenXi,...,Xnareuncorrelated,Cov(Xi,X.1]=0for'i=!=jandt11evarianceofthest1rr1isthesumoft11evariances.[308CHAPTER9SUMSOFRANDOMVARIABLESTheorem9.3When,X1,...,Xnare11,n,correlated,Var[vT!n]=Var[X1]+···+Var[Xn]·Example9.1X0,X1,X2,...isasequenceofrandomvariableswithexpectedvaluesE[Xi]=0andcovariances,Co·v[Xi,X_1]=0.8li-1I.FindtheexpectedvalueandvarianceofarandomvariableY;;definedasthesumofthreeconsecutivevaluesoftherandomsequence(9.4)Theorem9.1impliesthat(9.5)ApplyingTheorem9.2,weobtainforeachi,VarfYi]=Var[Xi]+Var[Xi-1]+Var[Xi-2]+2co,,[X1,,xi-1]+2co,,[Xi,xi-2]+2co,,[Xi-1,xi-2].(9.6)WenextnotethatVar[Xi]=Cov[Xi,Xi]=0.8i-i=1andthat(9.7)Therefore,Var[Y;;]=3x0.8°+4x0.81+2x0.82=7.48.(9.8)T11efollov.ringexampleshovvs110\vapuzzlingproblerr1canbeformulatedasaquestior1abol1tthesurr1ofasetofdependentrandornvariables.Example9.2Atapartyofri>2people,eachpersonthrowsahatinacommonbox.Theboxisshakenandeachpersonblindlydrawsahatfromtheboxwithoutreplacement.Wesayamatchoccursifapersondrawshisownhat.WhataretheexpectedvalueandvarianceofVn,thenumberofmatches?LetXidenoteanindicatorrandomvariablesuchthat1personidrawshishat,0otherwise.(9.9)Thenumberofmatchesis·v;.1,=X1+···+Xn.NotethattheX,iaregenerallynotindependent.Forexample,withn,=2people,ifthefirstpersondrawshisownhat,[9.1EXPECTEDVALUESOFSUMS309thenthesecondpersonmustalsodrawherownhat.Notethattheithpersonisequallylikelytodrawanyofthen,hats,thusPxi(l)=1/n,andE[Xi]=Pxi(l)=l/n,.Sincetheexpectedvalueofthesumalwaysequalsthesumoftheexpectedvalues,E[Vn)=E[X1)+···+E[Xn)=ri(l/ri)=(9.10)1.Tofindthevarianceofv;.i,wewilluseTheorem9.2.ThevarianceofX,;,isVar[X,;,]=E[X,z]-(E[X,;,])2=I_T/,J..2.(9.11)T/,TofindCo·v[Xi,X.1],weobservethat(9.12)NotethatXiX.iThus=1ifandonlyifX,;,=1andX.i=1,andXiX.1=0otherwise.(9.13)GivenX.i=1,thatis,thejthpersondrewhisownhat,thenXi=1ifandonlyifthe'ithpersondrawshisownhatfromthen,-1otherhats.HencePxi1xj(lIl)=1/('n-l)and1E[X;X;·]='n,(ri-l)'11Cov[Xi,X7·]=n,(n,-l)--ri2.(9.14)Finally,wecanuseTheorem9.2tocalculate(9.15)Thatis,boththeexpectedvalueandvarianceof·v;iare1,nomatterhowlargeriis!Example9.3ContinuingExample9.2,supposeeachpersonimmediatelyreturnstotheboxthehatthatheorshedrew.Whatistheexpectedvalueandvarianceof·v;.1,,thenumberofmatches?InthiscasetheindicatorrandomvariablesX,;,areindependentandidenticallydistributed(iid)becauseeachpersondrawsfromthesamebincontainingalln,hats.Thenumberofmatchesv;.1,=X1+···+Xnisthesumofn,iidrandomvariables.Asbefore,theexpectedvalueof·vnis(9.16)Inthiscase,thevarianceofVnequalsthesumofthevariances,Va.r[v;1,]=n,·va.r[Xi]=n,(~T/,-~)=1-I..T/,T/,(9.17)[310CHAPTER9SUMSOFRANDOMVARIABLESTheremainderofthischapterexarninestoolsforanalyzirigcornpleteprobabilit:yrnodelsofst1rrisofraridom·variables,witlitheerriphasisonst1rrisofindependentrandorri·variables.Quiz9.1LetWndenotethest1rriofn,iridependentthrowsofafairfo11r-sideddie.Findtheexpectedvalueandvariar1ceof'VT!n.9.2MomentGeneratingFunctionsc/>x(s),thernorrieritgerieratingf\1nction(11!GF)ofarandorrivariableX,isaprobabilitymodelofX.IfXisdiscrete,theMGFisatransforrnofthePMF.Thel\IIGFofacontinuo11srandornvariableisatransforrriofthePDF,similartoaLaplacetrarisforrri.Then,-thrriomentofXisthe'n-thderivativeofc/>x(s)eva1t1atedats=0.InSectiori6.5,"''elearnedinTheorerri6.9thatthePDFofthest1rril;T/2=X1+X2ofindependeritrandorrivariablescanbeV\rrittenasthecorivol11tionf11v2('w2)=f0000f"x1(1D2-:i;2)fx2(x2),dx2.TofindthePDFasurnofthreeindependeritrandomvariables,W3=X1+X2+X3,-vvecoulduseTheorern6.9tofindtliePDFofW2=X1+X2.andtheri,becausel;T/3='VTl2+X3andW2andX3areindependent,vieco11lduseTheorerri6.9againtofindthePDFofW3frorntheconvol11tion00fM13('t1J3)=f00fw2('w3-x3)f"x3(x3)d;r;3.Inprinciple,v.recouldcoritin11ethissequenceofconvolutioristofindthePDFofT;V,n=X1+···+Xnforariyri.\N"hilethisprocedureissouridiritheory,con\rolutiorialintegralsaregenerall}'tricky,aridaseqt1enceofn,convolt1tionsisoftenprohibitivel}'difficulttoevalt1atebyhand.EvenMATLABtypicallyfailstosirriplifytheevaluationofasequericeofconvolt1tioris.InlinearS}rsterritheory,lioV\rever,convoh1tioninthetimedornaincorresporidstorrit1ltiplicationirithefrequenC}'dornain"''ithtimefunctiorisandfrequenC}'functionsrelatedbytheFouriertrc/>x(s)=E[esX].Definition9.1appliestobothdiscretearidcoritin11ousrandornvariablesX.'\N"hatcharigesingoingfromdiscreteXtocontinuousXisthernethodofcalc11latingthe[9.2MOMENTGENERATINGFUNCTIONS311expected·value.W11enXisacor1tinl1ousrar1dornvariable,¢x(s)=J:(9.18)e'"fx(x)dx.Foradiscreterandorr1variable·y,the11{GFisy(s)=LesYiPy(y,i).(9.19)YiESyEqtlation(9.18)indicatesthattheMGFofacontinl1ousrar1dom·variableissirr1ilartotheLaplacetransformofatimefl1nction.TheprimarydifferencebetweenanNIGFandaLaplacetransformisthattheNIGFisdefir1edonl}'forrealvaltlesofs.Foragi·ver1randorn·variableX,thereisarangeofpossiblevall1esofsforv.rhichx(s)exists.T11esetofvall1esofsforv.r11ichx(s)existsiscalledtheregioriofco'nvergen,ce.T11edefinitionoftheMGFirnpliesthatx(O)=E[e0]=1.Thuss=0isalv.ra}'Sir1theregionofconvergence.IfXisar1onnegativerandomvariable,theregionofcor1vergenceinclt1desalls<0.IfXisboundedsothatP[a<X<b)=1,then¢x(s)existsforallreals.Typically,theregionofconvergenceisar1intervalarol1ndthes=0.Becat1setheMGFandPMForPDFforrr1atransformpair,theNIGFisalsoacompleteprobabilityrnodelofarandornvariable.Gi·vent11eNIGF,itispossibletocornputethePDForPMF.11oreover,thederivativesofx(s)e·vall1atedats=0arethernorr1er1tsofX.9.4Aro/ndom,variableX111ithMGFx(s)has'nthrnornen,t~-==TheoremE[Xn)=ProofThefirstderivativeof¢xrln,.f...x(s)I.fl----'---'rlsns=O_(s)is00d=dsd(/__sx.()eJxxdx)=1_=xe-oosxfx(x)d:i;.(9.20)00Evaluatingthisderivativeats=0provesthetheoremfords=On,=1.1_=.=xfx(x)dx=E[X].-oo(9.21)Similarly,therithderivativeof¢x(s)is11dTheintegralevaluatedats=1_=x-oonesxfx(x)dx.(9.22)=0istheformulainthetheoremstatement.Typica1lyitiseasiertocalculatethernomentsofXbyfindingtheMGFarid[312CHAPTER9SUMSOFRANDOMVARIABLESRandomVariableBernol1lli(p)Geornetric(IJ)Pascal(k,p)Poisson(a)Px(x;)l-px=O[)X=lotherv.rise0Px(x;)Tlx,)r/l;(l-Px(x;)(0Disc.Ur1iforrn(k,l)Px(x;)pes1-(1-p)e8(a:r;e-a/;i;!x=0,1,2,...0otherwise=k,...)ll-k+lx0otherv.risef·x(x;)b-a,0a.:r;f·x(x;)0x>Ootherwise.An:r;n-le->.xf·x(x;)k)esk-es(l+l)1-e8(n-1)!0x>Ootherv.riseebs-eass(b-a),AA-S(A)"A-SGal1ssiar1(µ,,CJ)Table9.18pes1Erlang(ri,A)+pe6(x;-a)Cor1stant(a)Exponential(,A)x=1,2,...otherwisePx(x;)Px(x;)1-pp)n-:i;p(l-rJ)x-l1Uniforrr1(a,b)MGFc/>x(s)PMForPDFlVIon1entgeneratingfi1nctionforfan1iliesofrandon1variables.[9.2MOMENTGENERATINGFUNCTIONS313differentiatingthar1byintegratir1gxnfx(x;).---Example9.4:--XisanexponentialrandomvariablewithMGFc/>x(s)=">../(">..-s).WhatarethefirstandsecondmomentsofX?Writeageneralexpressionfortherithmoment.Thefirstmomentistheexpectedvalue:E[X]=de/>x(s)dss=O1-">...-(">..-s)2s=O-(9.23)ThesecondmomentofXisthemeansquarevalue:2(9.24)(">..-s)3Proceedinginthisway,itshouldbecomeapparentthatthen,thmomentofXisE[Xn]=dn,.1,x(s)dsns=O'TL---_'+'_s=O')._n·(9.25)Table9.1presentst11eMGFforthefarr1iliesofrar1domvariablesdefinedinC11apters3and4.Thefollov.ringtheoremderivestheMGFofalir1eartransformationofarar1domvariableXinterrr1sofcf>x(s).===Theorem9.5,---===TheMGFofY=aX+biscf>y(s)=esbc/>x(as).ProofFromthedefinitionoftheMGF,(9.26)Quiz9.2RandornvariableJ(hasPJVIF0.20k=0,...'4,othervvise.Usecf>K(s)tofindthefirst,second,third,andfourt11rr1orr1er1tsofK.(9.27)[3149.3CHAPTER9SUMSOFRANDOMVARIABLESMGFoftheSumofIndependentRandomVariablesMomer1tgenera.tingft1nctionsprovideaconvenientwa.ytodeterminet11eprobabilityrnodelofa.sumofiidrandom·variables.UsingMGFs,wedeterminethatv.r11enVV=X1+···+Xnisasurnof'niidrandomvariables:•IfXiisBernoulli(p),Wisbinornia.l(ri,'fJ).•IfXiisPoisson(a),WisPoisson(n,a).•IfXiisgeornetric(IJ),WisPa.seal(ri,'fJ).•IfXiisexponential(.\),lVisErlang(n,,.\).•IfXiisGaussian(µ,,O"),lVisGaussian(n,µ,,fo,O").l\/Iornentgeneratir1gfunctionsareparticularlyusef\11foranalyzingsurnsofir1dependentrandomvariables,becauseifXaridYareindeper1dent,t11e1![GFofT1V=X+Yistheproduct(9.28)Theorern9.6generalizest11isresulttoas11rr1ofn,independentrar1domvariables.---Theorem9.6,--Forasetofin,depen,den,tra:ndornvariablesX1,...,Xn,thernornen,tgen,eratin,gfun,ction,ofvV=X1+···+XnisWhen,X1,...,Xnareiid,each'tuithMGFc/>xi(s)=x(s),w(s)=[c/>x(s)]n.ProofFromthedefinitionoftheMGF,w(s)=E[es(X1+···+Xn)]=E[esX1esX2...esXn].(9.29)Here,wehavetheexpectedvalueofaproductoffunctionsofindependentrandomvariables.Theorem8.4statesthatthisexpectedvalueistheproductoftheindividualexpectedvalues:(9.30)ByEquation(9.30)withgi(Xi)=esxi,theexpectedvalueoftheproductisvv(s)=E[esXiJE[esX2J···E[esXnJ=¢x1(s)x2(s)···w(s))71•(9.31)[9.3MGFOFTHESUMOFINDEPENDENTRANDOMVARIABLES315Morr1entgeneratingf\1nctionsprovideaconvenientwaytostud}'t11epropertiesofSlimsofindependentfir1itediscreterandornvariables.==;;..Example9.5.]andJ(areindependentrandomvariableswithprobabilitymassfunctionsJFindtheMGFofNI=J1230.20.6kP1<(k)0.2)+J(.Whatare-l10.50.5(9.32)PM(rn,)andE[J\J3]?.]andJ(havehavemomentgeneratingfunctions.J(s)=0.2e8+0.6e28+0.2e38,Therefore,byTheorem9.6,J\,{(9.33)=.]+KhasMGF¢N1(s)=.J(s)K(s)=0.1+0.3e8+0.2e28+0.3e38+0.le48.(9.34)ThevalueofPM(rn,)atanyvalueofrn,isthecoefficientofe1nsin¢N1(s):Fromthecoefficientsof¢M·(s),weconstructthetableforthePMFofNI:rn01234P1'vl(rn,)0.10.30.20.30.1TofindthethirdmomentofNI,wedifferentiate¢1vI(s)threetimes:E[M3J=d3¢~(s)=dss=O0.3e8+0..2(23)e28+0.3(33)e38+0.1(43)e48ls=O=16.4.(9.35)Besideser1abling11stocalculateprobabilitiesandrr1omentsforSlimsofdiscreterandorr1·variables,-vvecana,lso11seTheorem9.6toderivet11ePMForPDFofcertainsurr1sofiidra.ndorr1variables.Inpartictllar,-vveuseTheorem9.6toprovethatthesurr1ofindependentPoissonrar1domvariablesisaPoissonrandorr1·variable,andthest1mofindependentGat1ssianrandorr1variablesisaGat1ssianrandorr1variable.===Theorem9.7-...._:==IfK1,...,Knareiridepen,den,tPoisson,ran,dornvariables;W=K1+···+Kn'isaPoisson,ran,dorn'Variable.ProofVVeadoptthenotationE[Ki]=D'.iandnoteinTable9.1thatJ{ihaslVIGF[316CHAPTER9SUMSOFRANDOMVARIABLESJ(9.36)eai(es-1).ByTheorem9.6,(9.37)\Vherear=CY.1+···+O'.n.ExaminingTable9.1,weobservethatw(s)isthemomentgeneratingfunctionofthePoisson(D'.'r)randomvariable.Therefore,'W~TePw('UJ)=-a;f111.71J=0,l,...,other,vise.{(9.38)===Theorem9.8:===Thes11,rnofriin,depen,dentGaussian,raridornvariablesWGav,ssia,riran,dornvariable.ProofForconvenience,letµi=E[Xi]andaf=Var[Xi]·SincetheXiareindependent,\Vekno'vthat=22es(J.q+···+J.ln)+(a1+···+an)s2/2.(9.39)FromEquation(9.39),weobservethatw(s)isthemomentgeneratingfunctionofaGaussianrandomvariablev;,rithexpectedvalue/J,1+···+JJ,nandvarianceai+···+a~.Ingeneral,theSl1rr1ofindependentrandorr1·variablesinonefarr1ilyisadifferentkindofrandornvariable.Thefollowingtheorerr1shov.rsthatt11eErlar1g(ri,>.)randornvariableisthesurr1ofn,independentexponential(>.)rar1dornvariables.Theorem9.9IfX1,...,Xnare'iidex;pon,en,tial(>.)ran,dornvariables;then,W=X1+···+XnhastheErlarigPDF.Anilln-le->.wf'w(w)=w>0,(n-1)!0otherv1ise.ProofInTable9.1,veobservethateachXihas~1IGFx(s)=>./(>.-s).ByTheorem9.6,Whas~!IGFvv(s)=).)n().._(9.40)8ReturningtoTable9.1,\VeseethatWhastheMGFofanErlang(n,,>.)randomvariable.[9.4RANDOMSUMSOFINDEPENDENTRANDOMVARIABLES317Similarreasoningdernonstratest11att11esurnofn,Bernoulli(p)randorn·variablesisthebinornial(n,,p)rar1dorn·variable,andthatthest1rr1ofkgeorr1etric(IJ)rar1domvariablesisaPascal(k,p)randorr1variable.Quiz9.3(A)LetK1,K2,...,Krnbeiiddiscreteuniforrr1randorr1variableswit11P1!{Fl/n,0k=1,2,...,ri,othervvise.(9.41)Findt11eMGFofJ=J(1+···+Krn·(B)LetX1,...,XnbeindependentGaussianrandorr1variablesv.rithE[X,i:=0)andVar[Xi]=i.Fir1dthePDFofVV9.4=aX1+a2X2+···+anXn.(9.42)RandomSumsofIndependentRandomVariablesR=X1+···+RNisarar1doms11rr1ofrar1domvariableswhenN,thent1rnberofterrr1sinthesum,isarandomvariable.W11er1NisindependentofeachXiandt11eXiareiid,thereareconciseformulasfort11el\!{GF,theexpectedvalue,andthevarianceofR.l\/Ian:ypracticalproblerr1scanbeana,l}rzedbyreferencetoas11rnofiidrar1domvariablesinwhichthenurnberoftermsinthesurnisalsoarandom·variable."\"!Verefertotherest1ltantrar1dornvariable,R,asararidornsurnofiidrandornvariables.Thus,givenarar1domvariableNandasequer1ceofiidrandorr1variablesX1,X2,...,let(9.43)Thefollov.ringtv.roexarr1plesdescribeexperimentsin"''hic11theobservationsarerandorr1surnsofrandorn·variables.Example9.6Atabusterminal,countthenumberofpeoplearrivingonbusesduringoneminute.Ifthenumberofpeopleonthe'ithbusisKiandthenumberofarrivingbusesisN,thenthenumberofpeoplearrivingduringtheminuteis(9.44)Ingeneral,thenumberNofbusesthatarriveisarandomvariable.Therefore,Risarandomsumofrandomvariables.[318CHAPTER9==~ExampIeSUMSOFRANDOMVARIABLES9.Ji.......,;;=;;;;:CountthenumberJVofdatapacketstransmittedoveracommunicationslinkinoneminute.SupposeeachpacketissuccessfullydecodedwithprobabilityJ>,independentofthedecodingofanyotherpacket.Thenumberofsuccessfullydecodedpacketsintheone-minutespanisR=Xi+···+X1v.(9.45)whereXiis1iftheithpacketisdecodedcorrectlyand0otherwise.WhenNisaknownconstant,Risabinomialrandomvariable.Bycontrast,whenN,thenumberofpacketstransmitted,israndom,Risarandomsum.Int11eprecedingexamplesv.recanusetherr1ethodsofCl1apter5tofindthejointP1!{FPN,R(n,,r).Hovvever,vvearenotabletofindasirr1pleclosedforrr1expressionforthePMFPR(r).Ontheotherhand,vveseeint11enexttheorernthatitispossibletoexpresstheprobabilitymodelofRasaformulaforthemorr1entgeneratir1gfunctionR(s).Theorem9.10Let{Xi,X2,...}beacollection,ofiidran,dorn'Uariables,eachvrithM'GFx(s),aridletNbean,on,n,egati'Uein,teger-1;al11,edran,dorn'variablethatisin,deperiden,tof{Xi,X2,...}.Theran,dornS'tJ,rnR=Xi+···+X1vhasrnorneritgen,eratin,gf1J,n,ction,R(8)=N(lnX(8)).ProofTofind¢R(s)=E[esR],v.;efirstfindtheconditionalexpectedvalueE[esRIN=n,].Becausethisexpectedvalueisafunctionofn,itisarandomvariable.Theorem7.14statesthatR(s)istheexpectedvalue,withrespecttoN,ofE[esRIN=n]:0000E[esRIN=ri]PN(ri)=Ln=OE[es(Xi+..·+XN)IN=n,]P1v(n,).(9.46)n=OBecausetheXiareindependentofPl,(9.47)InEquation(9.46),W=X1[x(s)J11,implying+···+Xn.FromTheorem9.6,\Vekno\vthatw(s)00R(s)=L[x(s)J11,PN(n,).(9.48)n=O\'\feobservethatwecanwrite[x(s)]n=[e1n¢x(s)J7=e(ln¢x(s)Jn.This1'Limplies00¢R(s)=n=Oe[lnx(s))npN(n,).(9.49)[9.4RANDOMSUMSOFINDEPENDENTRANDOMVARIABLES319RecognizingthatthissumhasthesameformasthesuminEquation(9.19),weinferthatthesumisIr1thefollowingexarnple,-vvefindt11eMGFofarandornsurr1aridther1transforrr1ittothePMF.c:::==Example9.8Thenumberofpages,/ll,viewedinaWebsearchhasageometricPMFwithexpectedvaluel/q=4.ThenumberofbytesKinaWebpagehasageometricdistributionwithexpectedvalue1/p=105bytes,independentofthenumberofbytesinanyotherpageandindependentofthenumberofpages.FindtheMGFandthePMFofB,thetotalnumberofbytestransmittedinaWebsearch.WhentheithpagehasKibytes,thetotalnumberofbytesistherandomsumB=K1+···+KN.Thus¢B(s)=1v(ln¢K(s)).FromTable9.1,qes1v(s)=I-(1-q)es'K(s)pes=1-(1-JJ)es·(9.50)Tocalculate¢B(s),wesubstituteln¢K(s)foreveryoccurrenceofsin¢N(s).Equivalently,wecansubstitute¢K(s)foreveryoccurrenceofesinN(s).ThissubstitutionyieldspesB(S))q(........__l-(l-p)e___=________..pqes1-(1-[Jq)es·81-(1-q)(1-(lesp)es)(9.51)BycomparingIPB(b)pq(l-pq)b-1b=1,2,...,=0otherwise.(9.52)UsingTheorern9.10,wecantakederi·vativesof¢N(lr1x(s))tofir1dsirr1pleexpressior1sfort11eexpectedva.lt1eandvarianceofarar1doms11mR.===Theorem9.11--Fortheran,dornsurnofiidran,dornvariablesR=XE(R]=E(JV]E(X],Var(R]1+···+X1v,=E[J\T]Var(X]+Var(J\T](E(X])2.ProofBythechainruleforderivatives,I()=(())ln(9.53)[320CHAPTER9Since=1,SUMSOFRANDOMVARIABLES¢~(0)=E[N],E[R]=and¢~(0)=E[X],evaluating¢~i(O)=¢~(0):~~~~theequationats=E[NJE[X].=0(9.54)Forthesecondderivativeof¢x(s),wehave(9.55)Thevalueofthisderivativeats=0is(9.56)Subtracting(E[R])2=(JLNJLX)2frombothsidesofthisequationcompletestheproof.vVeobservethatVar[R]containstvvoterrr1s:t11efirstterrr1,µ,NVar[X],resultsfrorntherandornnessofX,whilethesecondterrn,Var[J\T]µ,i-,isaconsequenceoftherandorr1r1essofN.Toseethis,considerthesetwocases.•SupposeNisdeterrr1inisticsuchthatJ\T=n,ever}'time.Inthiscase,/J,N=n,andVar[J\T]=0.Therandorr1sumRisanordinar}'deterrninisticst1rr1R=X1+···+Xnand·var[R]=n,Var[X].•SupposeJVisrandorn,buteachXiisadeterrninisticconstantx;.Ir1thisir1stance,µ,x=x;a.ndVar[X]=0.Moreo\rer,therandornsumbecornesR=J\TxandVar[R]=x2Var[N].WeernphasizethatT11eorerr1s9.10and9.11requirethatNbeir1dependentoftherar1dom\rariablesX1,X2,....T11atis,ther1urr1beroftermsintherandoms11rncanr1otdeper1dontheactualvaluesoft11eterrr1sir1thesurn.Example9.9LetX1,X2...beasequenceofindependentGaussian(100,10)randomvariables.IfKisaPoisson(1)randomvariableindependentofX1,X2...,findtheexpectedvalueandvarianceofR=X1+···+X1ThePDFandMGFofRarecomplicated.However,Theorem9.11simplifiesthecalculationoftheexpectedvalueandthevariance.FromAppendixA,weobservethataPoisson(1)randomvariablehasvariance1.ThusE[R]=E[X]E[K]=100,(9.57)+Var[K](E[X])2=(9.58)andVar[R]=E[K]·var[X]100+(100)2=10,100.WeseethatmostofthevarianceiscontributedbytherandomnessinK.ThisistruebecauseJ(isverylikelytotakeonthevalues0and1(P1[9.5CENTRALLIMITTHEOREM321Quiz9.4r---==:=JLetX1,X2,...denoteasequer1ceofiidrandornvariables\vit11exponentialPDFf'x(x)=x>o.-0Iotherv.rise.(9.59)LetNdenoteageometric(1/5)randorr1variable.(a)W11atisthe11.IGFofR(b)Findt11ePDFofR.9.5=X1+···+XN?CentralLimitTheoremThecentrallirr1ittheorerr1statesthattheCDFofthethesumofn,independentrandomvariablesconvergestoa.Gat1ssia.nCDFasn,grovvsvvithoutbot1r1d.Forvaluesofn,encounteredinman}'applications,theapproxirr1ateGa11ssia.nrnodelprovidesa.verycloseapproxirna.tiontotheact11alrnodel.UsingtheGa.11ssianapproximationisfarrnoreefficientcomputationallytha.r1vvorkir1gvviththeexactprobabilitymodelofasumofrar1domvariables.Probabilityt11eoryprovidesus\vit11toolsforir1terpretingobserveddata..Inrnanypracticalsituations,bothdiscretePMFsandcontir1uousPDFsapproxirr1atel}'follo\va.bell-shapedc'u.rve.Forexa.rnple,Figure9.1shovvsthebinornial(n,,1/2)PMFforn,=5,ri=10andn,=20.Weseethatasn,getslarger,theP11.IFmorecloselyresemblesabell-shapedcur\re.RecallthatinSection4.6,vveencour1teredabellshapedcurveasthePDFofaGaussianrandorr1variable.T11ecentrallirnittheorernexplainsvvhysorr1anypra.cticalpher1ornenaproducedatat11atcanbernodeleda.sGa11ssiar1ra.ndorr1variables.WewillusethecentrallimittheorerntoestimateprobabilitiesassociatedvviththeiidsurnWn=X1+···+Xn.Hovve\rer,as'napproa.c11esir1finit}',E[VVn)=n,µxandVa.r[Wn)=n,Var[X]approachinfir1ity,vvhichrr1a.kesitdifficulttornakearnatherna.ticalstaterr1er1tabouttheconvergenceoft11eCDFFwn(VJ).Hence,ourforrnalstatementoft11ecer1trallirnitt11eorern\villbeinterrr1softhestandardizedrandorr1\rariable(9.60)vVesaytheSllITlZnisstar1dardizedsinceforall'nE[Zn)=0,Var[Zn)=1.(9.61)[322CHAPTER9SUMSOFRANDOMVARIABLES0.4-------0.2-------0.4.--.._~><0.2'--()Cl....•05()·-·5x0xri=577,•-100....__....-..u••....................1.....____,_,0=1010xri=2020Figure9.11..,hePNIFoftheX,thenun1berofheadsinn,coinflipsforn,=5,10,20.Asn,increases,theP11Fn1orecloselyresen1blesabell-shapedcurve.Theorem9.12CentralLimitTheoremGi'UeriX1,X2,...,aseqv,e'Tl,ceofiidra'Tl,dorn'Uariables'UJitheJ;pected'Ualue/J,xarid'Uarian,ceal,theGDFofZn=(2=;,i1Xi-11,µx)/~hastheJJropertylimFzn(z)n-+oo=Theproofoft11ist11eorerr1isbeyondthescopeofthistext.InadditiontoTheorern9.12,thereareot11ercentrallirnitt11eorerns,eac11v.rithitsownstatementofthepropertiesofthesumsWn.OneremarkableaspectofTheorerr19.12ariditsrelativesisthefactthattherearer10restrictionsont11enatureoftherandornvariablesX,iinthesurri.T11eycanbecontir1uous,discrete,orrnixed.Inallcases,theCDFoftheirsumrnoreandrr1oreresemblesaGaussianCDFast11enumberoftermsir1t11esurnincreases.Someversior1softhecentrallirnittheorernapplytosurr1sofsequencesX,ithatarerioteveniid.TollSethecentrallirnittheorern,-vveobservet11at-vvecanexpresstheiidsumWn=Xi+···+Xnas(9.62)TheCDFofWncar1beexpressedinterrr1softheCDFofZnasFwn('w)=P[~Zn+n,µ,x<'IJJJ=Fzn('IJJ~x)77,(}x.(9.63)Forlarge'n,thecentrallirnittheorernsaysthatFzn(z)~CentralLimitTheoremApproximationLetT;Vn=X1+···+Xnbethes11,rnofriiidra/ndornvariables;each'tJJithE[X)=/J,xa'Tl,dVar[X]=Thece'Tl,trallirnittheorernapproJ;irnatio'TI,totheGDFoflV:nis0"1..__,FWn('UJ),,..__,,-T-,.'±'('UJ-Tlµ,X)~vn,a-x·[9.51.5,•1.I.-.•\..I0.5I0...\..\•I.,,,,.'.0323I.\CENTRALLIMITTHEOREM10.5--021xx(a)ri=1(b)T/,=20.80.60.40.50.2012x(c)n,=33001234x(d)'n=4Figure9.2'rhePDFofvVn,thesun1ofn,uniform(0,1)randon1variables,andthecorrespondingcentrallin1ittheoren1approxin1ationforn=1,2,3,4.'rhesolid-linedenotesthePDFfvvn(11J),andthebroken-·-linedenotestheGaussianapproxin1ation.WeoftencallDefinition9.2aGaussianapproxirnatior1forFl!\rn(tv).Example9.10Togainsomeintuitionintothecentrallimittheorem,considerasequenceofiidcontinuousrandomvariablesXi,whereeachrandomvariableisuniform(0,1).Let(9.64)RecallthatE[X]=0.5andVar[X]=1/12.Therefore,WnhasexpectedvalueE[Wn]=ri/2andvariancen,/12.ThecentrallimittheoremsaysthattheCDFofvVnshouldapproachaGaussianCDFwiththesameexpectedvalueandvariance.Moreover,sincevT!nisacontinuousrandomvariable,wewouldalsoexpectthatthePDFofVVnwouldconvergetoaGaussianPDF.InFigure9.2,wecomparethePDFofT1VntothePDFofaGaussianrandomvariablewiththesameexpectedvalueandvariance.First,W1isauniformrandomvariablewiththerectangularPDFshowninFigure9.2(a).ThisfigurealsoshowsthePDFofW1,aGaussianrandomvariablewithexpectedvalueµ,=0.5andvarianceCJ2=1/12.HerethePDFsareverydissimilar.Whenweconsidern,=2,wehavethesituationinFigure9.2(b).ThePDFofW2isatrianglewithexpectedvalue1andvariance2/12.ThefigureshowsthecorrespondingGaussianPDF.ThefollowingfiguresshowthePDFsofT¥3,...,VV5.Theconvergencetoabellshapeisapparent.[324CHAPTER9SUMSOFRANDOMVARIABLESJr-------ir---------------,.1.1I'.0.50.5jrr0,;0Bino1nial.5.10CLT.-CLT_.._-"'o.______._________15n,=2)r>=1/2lr-------=::P-------------.0.5Binomial50n,=10154,I>=1/2Ir------------=~------,0.5-Bino1nial01...-..-:;:;;::__0CLT__.._____.______.____.51015n,=8)r>=1/2BinomialCLT01.....a._0__.__.~---------'----'5T/,1015=16,p=1/2Figure9.31"hebinon1ial(n,p)CDFandthecorrespondingcentrallin1ittheoren1approxin1ationforn,=4,8,16,32,andp=1/2.-=Example9.11NowsupposeWn=X1+···+XnisasumofindependentBernoullivariables.WeknowthatVfl,nhasthebinomialPMF(p)random(9.65)Nomatterhowlargen,becomes,vT!nisalwaysadiscreterandomvariableandwouldhaveaPDFconsistingofimpulses.Hovvever,thecentrallimittheoremsaysthattheCDFofT1VnconvergestoaGaussianCDF.Figure9.3demonstratestheconvergenceofthesequenceofbinomialCDFstoaGaussianCDFforr>=1/2andfourvaluesofri,thenumberofBernoullirandomvariablesthatareaddedtoproduceabinomialrandomvariable.Forn,>32,Figure9.3suggeststhatapproximationsbasedontheGaussiandistributionareveryaccurate.Inadditiontohelpir1gust1nderstand"''hY-vveobservebell-shapedc11rvesir1sorr1anysituations)thecentrallirr1itt11eorernmakesitpossibletoperforrr1quick)accuratecalculationst11at-vvot1ldot11erV\risebeextrerr1elycorr1plexandtirneconsurr1ing.Inthesecalculations)therandorr1variableofinterestisasumofot11errandomvariables)andvvecalculatet11eprobabilitiesofeventsbyreferringtothecorrespondingGa11ssiar1randomvariable.Int11efolloV\ringexample)therandorr1variableofir1terestistheaverageofeightiidur1iforrr1randomvariables.Theexpectedvalueand[9.5CENTRALLIMITTHEOREM325varianceoftheaverageareeasytoobtain.Ho\vever,acompleteprobabilityrnodelisextremel}'complex(itconsistsofsegrr1er1tsofeigl1th-orderpolynomials).--=Example9.12Acompactdisc(CD)containsdigitizedsamplesofanacousticwaveform.InaCDplayerwitha"onebitdigitaltoanalogconverter,"eachdigitalsampleisrepresentedtoanaccuracyof±0.5mV.TheCDplayeroversamplesthewaveformbymakingeightindependentmeasurementscorrespondingtoeachsample.TheCDplayerobtainsawaveformsamplebycalculatingtheaverage(samplemean)oftheeightmeasurements.Whatistheprobabilitythattheerrorinthewaveformsampleisgreaterthan0.1mV?ThemeasurementsX1,X2,...,X8allhaveauniformdistributionbetweenv-0.5mVandv+0.5mV,wherevmVistheexactvalueofthewaveformsample.ThecompactdiskplayerproducestheoutputU=Ws/8,where8Ws=Lxi.(9.66)i=lTofindP[IU-vi>0.1]exactly,wewouldhavetofindanexactprobabilitymodelforW8,eitherbycomputinganeightfoldconvolutionoftheuniformPDFofXiorbyusingthemomentgeneratingfunction.Eitherway,theprocessisextremelycomplex.Alternatively,wecanusethecentrallimittheoremtomodelvT!8asaGaussianrandomvariablewithE[W8]=8/J,x=8vmVandvarianceVar[W8]=8Var[X]=8/12.Therefore,UisapproximatelyGaussianwithE[U]=E[T.V8]/8=vandvarianceVar[W8]/64=1/96.Finally,theerror,U-vintheoutputwaveformsampleisapproximatelyGaussianwithexpectedvalue0andvariance1/96.ItfollowsthatP[IU-vi>0.1]=2[1-(o.1;J1/96)J=o.3272.(9.67)Thecentrallimittheoremisparticularlyusefulir1calculatingeventsrelatedtobinornialrandornvariables.Figure9.3frornExarr1ple9.11indicateshovvtheCDFofasurnofn,BernoullirandornvariablesconvergestoaGat1ssianCDF.Whenn,isveryhigh,asinthenexttwoexarnples,probabilitiesofeventsofir1terestaresurr1softhousandsofterrr1sofabinornialCDF.Bycontrast,eachoft11eGaussianapproximationsrequireslookingupor1lyone\ralueoftheGaussianCDF(1';).Example9.13Amodemtransmitsonemillionbits.Eachbitis0or1independentlywithequalprobability.Estimatetheprobabilityofatleast502,000ones.LetX.ibethevalueofbit'i(either0or1).ThenumberofonesinonemillionbitsisvV=L:.~c~Xi.BecauseXiisaBernoulli(0.5)randomvariable,E[Xi]=0.5andVar[X.i]=0.25fora11'i.NotethatE[vV]=106E[Xi]=500,000,andVar[W]=106Var[Xi]=250,000.Therefore,O"\J\1=500.Bythecentrallimittheorem0[326CHAPTER9SUMSOFRANDOMVARIABLESapproximation,P[W>502,000]=1-P[lV<502,000]~1-(1)(502,000-500,000)=1-(9.68)500UsingTable4.2,weobservethat1-5.Example9.14Transmitonemillionbits.LetAdenotetheeventthatthereareatleast499,000onesbutnomorethan501,000ones.WhatisP[A]?c:::==AsinExample9.13,E(VT!)=500,000andapproximation,p[A]=p[vV~(1)O'\J\I=500.Bythecentrallimittheorem<501,000]-p[vV<499,000](501,000-500,000)500(1)(499,000-500,000)500=(9.69)TheseexarnplesofusingaGaussianapproximationtoabinorriialprobabilityrriodelcontairieventsthatconsistofthol1sandsofoutcornes.·vVherithee·ventsofinterestcontairiasrriallnumberofoutcornes,tlieaccl1raC}'oftlieapproxirnationcanbeirriprovedbyaccountirigforthefactthattlieGaussianrandornvariableiscontinuous"''hereasthecorrespondirigbiriomialrandomvariableisdiscrete.Infact,usingaGaussianapproxirriationtoadiscreteraridomvariableisfairlycornrnori.·vverecalltliatthesurriof77,Bernoullirandornvariablesisbiriorriial,thesurriof77,georrietricrandorri·variablesisPascal,aridthesurriof11,Bernoulliraridornvariables(eaclivvitlisuccessprobability>./ri)approacliesaPoissonraridomvariableinthelimitas'n~oo.TliusaGaussianapproximationcanbeaccurateforarandorri·variableKthatisbiriomial,Pascal,orPoisson.Ingeneral,SllpposeKisadiscreterandorn·variablearidthattherarigeofKisSKc{n,Tlri=0,±1,±2...}.Forexarriple,"''henJ(isbinomial,Poisson,orPascal,T=1andSK={0,1,2...}.Wev.rislitoestirriatetheprobabilityoftheeventA={k1<J(<k2},v.rherek1andk2areiritegers.AGaussianapproxirriatioritoP[A]isoftenpoorwlienk1andk2areclosetoeachotlier.Inthiscase,"''ecanimproveOllrapproxirriationbyaccountirigforthediscretenatureofK.ConsidertheGaussianrandorn·variiable,X,vvithexpectedvallleE[K]andvarianceVar[JT/2<X<k2+T/2]k2+TI2-E[K])VVar[K](1)(k1-TI2-E[K])J·var[I.(9.70)vVhenJ(isabinorriialrandomvariablefor11,trialsaridSl1ccessprobabilityp,E[I[9.5CENTRALLIMITTHEOREM327Definition9.3DeMoivre-LaplaceFormulaForabin,ornial('n,JJ)ran,dornvariableK;Toappreciate\vhythe±0.5terrr1sir1creasetheacc11racyofapproxirr1atior1,considerthefollowir1gsirnplebutdrarnaticexarnpleinw.hichk1=k2.==;;..ExampIe9.15___,;;=:::::111LetKbeabinomial(n,=20,JJ=0.4)randomvariable.WhatisP(K=8]?SinceE(K]=n,p=8and·var(K]=n,p(l-p)=4.8,thecentrallimittheoremapproximationtoKisaGaussianrandomvariableXwithE(X]=8andVa.r(X]=4.8.BecauseXisacontinuousrandomvariable,P(X=8]=0,auselessapproximationtoP[K=8].Ontheotherhand,theDeMoivre-Laplaceformulaproducesp(8<}(<8]~p(7.5<x0.5)=CD(J4.82Theexactvalueis(8°)(0.4)8(1<8.5]-0.5)-CD(J4.80.1803.=(9.71)-0.4)12=0.1797.===Example9.16r:==::::aKisthenumberofheadsin100flipsofafaircoin.WhatisP(50<J(<51]?SinceKisabinomial(n,=100,JJ=1/2)randomvariable,P(50<K<51]=PK(50)+PK(51)=(100)(-1)lOO+(100)(-1)50SinceE(K]=50and{}'J<=approximationproduces512lOO2.=0.1576.(9.72)Vn,JJ(l-JJ)=5,theordinarycentrallimittheoremP[50<K<51]"'i1\51(~)-i_l\(5050~50)=00793(9.73)Thisapproximationerrorofroughly50%occursbecausetheordinarycentrallimittheoremapproximationignoresthefactthatthediscreterandomvariableJ(hastwoprobabilitymassesinanintervaloflength1.Asweseenext,theDeMoivre-Laplaceapproximationisfarmoreaccurate.P[50(51+0~5-50)-i_l\(50-0~5-50)CD(0.3)-CD(-0.1)=0.1577.(9.74)[328CHAPTER9SUMSOFRANDOMVARIABLESAlthot1ght11ecentrallirnittheorerr1approxirr1ationpro·videsat1sefulmeansofcalculatingeventsrelatedtocornplicatedprobabilityrnodels,ithastobel1sedvvithcaution.vVhent11ee\rentsofinterestareconfir1edtooutcomesattheedgeoftherar1geofarandomvariable,thecentrallirr1itt11eorernapproxirr1ationcanbequiteir1acctu·ate.Inalloftheexarnplesir1thissection,therandorr1\rariableofinterest11asfir1iterar1ge.B}'cor1trast,thecorTespondingGaussianrr1odelshavefiniteprobabilitiesforanyrangeofr1urr1bersbetween-ooandoo.Tht1sinExarnple9.12,P[U-v>0.5]=0,whiletheGa11ssianapproxirnatior1suggeststhatP[U-v>0.5]=Q(0.5/-Ji"796)~5x10-1.Althoughthisisalo\vprobability,therearernanyapplicationsir1whichtheeventsofinterestha,re\rerylovvprobabilitiesorprobabilitiesver}'closeto1.Int11eseapplications,itisr1ecessarytoresorttorr1orecorr1plicatedrnethodsthar1acentrallirr1ittheorerr1approxirnationtoobtainusefulresults.Inpa.rtict1lar,itisoftendesirabletopro\rideguarar1teesintheformofant1pperbour1dra.therthar1theapproxirr1ationofferedbythecer1trallimittheorem.Inthenextsection,wedescribeonesuchrr1ethodbasedor1thernorr1er1tgeneratir1gfunction.Quiz9.5Xrr1illisecor1ds,thetotalaccesstirr1e(waitingtime+readtime)togetor1eblockofinformationfromacornp11terdisk,isthecontint1ot1s(0,12)randorr1variable.Beforeperformingacerta.intask,thecornputerrnt1staccess12differentblocksofinforrr1ationfrorr1thedisk.(Accesstirnesfordifferentblocksareindepender1tofoneanother.)Thetotalaccesstirr1eforalltheir1forrr1ationisarandornvariableArr1illiseconds.(a)(b)(c)(d)9.6Findtheexpectedvalueandve,u·ianceoft11eaccesstirneX.FindtheexpectedvaJt1eandstandarddeviatior1ofthetotalaccesstimeA.Uset11ecer1trallirnitt11eorerr1toestirr1ateP[A>75ms].Usethecentrallimittheorerr1toestirnateP[A<48ms].l\!JATLABisconvenientforcalct1latingthePMFofthes11moft\vodiscreterandorr1variables.TocalculatethePMFofthest1mofn,randorr1variables,r11ntheprogramfortvvorandornvariablesn,-1tirr1es.Thecentrallimittheorernsuggestsasirr1plewaytousearar1domnt1mberger1eratorforthet1niform(0,1)randomvariabletoger1eratesamplevalt1esofaGa11ssian(0,1)rar1dom\rariable:Addtvvelvesamplesofthet1niform(0,1)randorr1variableandthens11btract6.MATLA.BAsinSections5.11and8.6,vveillustratetwo\Va}'Sofusir1g1\IIATLABtostudyrandom\rectors.WefirstpresentexamplesofprogTarr1sthatcalc11late\raluesofprobabilit}'fur1ctions,inthiscaset11ePMFofthesumsofindependentdiscrete[MATLAB9.63290.0I50.00520406080I0012014(wFigure9.4T'hePl\IIFPw(?lJ)forExan1ple9.17.randorr1variables.T11enwepreser1taprogramthatgeneratessamplevaluesoftheGal1ssiar1(0,1)randorr1V[1.riablevvithoutusingthebuilt-infur1ctionrandn.ProbabilityFunctionsThefollovvingexampleproducesaJ\11.A.TLABprograrr1forcalculatingtheconvoll1tionoftv.roPJ\IIFs.Example9.11'--"=~X1andX2areindependentdiscreterandomvariableswithPMFs0.040WhatisthePMFofWx=1)...)25,otherwise,=..i..550X=0otherwise.10,20,...,100,X1+X2?.................................................................................%sumx1x2.rnsx1=(1:25);px1=0.04*ones(1,25);sx2=10*(1:10);px2=sx2/550;[SX1,SX2]=ndgrid(sx1,sx2);[PX1,PX2]=ndgrid(px1,px2);SW=SX1+SX2;PW=PX1.*PX2;sw=unique(SW);pw=finiteprnf(SW,PW,sw);prnfplot(sw,pw);AsinExample5.26,sumx1x2.musesndgridtogenerateagridforaIIpossiblepairsofX1andX2.ThematrixSWholdsthesum:r1+x2foreachpossiblepair~c1,x2.TheprobabilityPx1,x2(:i;1,:c2)ofeachsuchpairisinthematrixPW.Foreachunique'Wgeneratedbypairsx1+x2,finitepmffindstheprobabilityPw('w).ThegraphofPw(VJ)appearsinFigure9.4.[SX1,SX2,SX3]=ndgrid(sxl,sx2,sx3);Theprecedingtechniqueextendsdirectly[PX1,PX2,PX3]=ndgrid(pxl,px2,px2);ton,ir1dependentfiniterandornvariablesSW=SX1+SX2+SX3;X1,...,XnbecausendgridcangeneratePW=PX1.*PX2.*PX3.*PX3;r1,-dirr1er1sionalgrids.Forexample,thesurr1sw=unique(SW);oft11reerandomvariablescanbecalctlpw=finiteprnf(SW•PW•sw);lated·viathescriptontheleft.Hovvever,thistechniquesuffersfromthedisadvantagethatitcar1generatelargematrices.For[330CHAPTER9SUMSOFRANDOMVARIABLES>>uniform12(10000);ans=-3.0000-2.0000-1.00000.00130.02280.15870.00050.02030.1605>>uniform12(10000);ans=-3.0000-2.0000-1.00000.00130.02280.15870.00150.02370.169700.50000.50271.00000.84130.83932.00000.97720.97813.00000.99870.998600.50000.50641.00000.84130.84002.00000.97720.97783.00000.99870.9993Figure9.5T'vosa.n1plerunsofuniform12.m.n,randomvariablessuchtl1atX,itakesonn,,ipossibledistinct·values)SWandPWaresquarerr1atricesofsizen,1xn,2x···n,rn·AmoreefficienttechniqueistoiterativelycalculatethePl\/IFofW2=X1+X2follov.redbyVTl3=W2+X3)VV4=T1V3+X3.Ateacl1step)extractingonlytl1euniquevaluesintherangeSwncaneconomizesignificantlyonmerr1orya.ndcomputationtime.SampleValuesofGaussianRandomVariablesThecentrallirr1itt11eorernsuggestsasirnplewa}'togeneratesarr1plesoftheGaussian(0)1)randornvariableincorr1putersorcalct1latorswithot1tbuilt-infunctionslikerandn.Theteclrr1iquereliesontheobservationthatthest1rnof12independentt1niform(0)1)rar1dom·variablesUihasexpectedvalue12E[Ui]=6and·variance212·var[Ui]=1.Accordingtothecentrallimitt11eorerr1,X=1Ui-6isapproxirnatelyGaussian(0,1).2=iExample9.18Writeal\IIATLABprogramtogenerate'IT/,=10)000samplesoftherandomvariable2X=1U,i-6.Usethedatatofindtherelativefrequenciesofthefollowingevents{X<T}forT=-3,-2...)3.CalculatetheprobabilitiesoftheseeventswhenXisaGaussian(0)1)randomvariable.2=iInuniform12(m),xholdsthe'IT/,samplesofX.Thefunctionn=count(x,T)returnsn(i)asthenumberofelementsofxlessthanorequaltoT(i).Theoutputisathree-rowtable:Tonthefirstrow,thetrueprobabilitiesP[X<T]=6,nomatterhowmanytimesitruns.Bycontrast,Q(6)=9.9x10-10.ThissuggeststhatinasetofonebillionindependentsamplesoftheGaussian(0)1)randomvariable,wecanexpecttwosampleswithIXI>6,onesamplewithX<-6,andonesamplewithfunctionFX=uniform12(rn);x=sum(rand(12,m))-6;T=(-3:3);FX=(count(x,T)/m)';[T;phi(T);FX]x>6.[PROBLEMS331Quiz9.6Xist11ebinomial(100>0.5)randornvariableand·yisthediscreteur1iforrn(0,100)randorn·variable.CalculatearidgraphthePMFofW=X+Y.Fv,rtherReadin,g:[Dl1r94]containsaconcise>rigorouspreser1tationandproofofthecentrallimitt11eorerri.ProblemsDifficulty:Easy9.1.1X1andX2areiidrandomvariables'vithvarianceVar[X].(a)WhatisE[X1-X2]?(b)WhatisVar[X1-X2]?ModeratetDifficultExpertsOnly9.1.4•X1,X2andX3areiidcontinuousunifo1mrandomvariables.RandomvariableY=X1+X2+X3hasexpectedvalueE[Y]=0andvariancea~=4.WhatisthePDFfx1(x)ofX1?9.1.2Flipabiasedcoin100times.Oneachflip,P[H]=p.LetXidenotethenumberofheadsthatoccuronflipi.\iVhatisPx33(x)?AreX1andX2independent?DefineY=X1+X2+···+X100.DescribeYinwords.WhatisPy(y)?FindE[Y]andVar[Y].9.1.5RandomvariablesXandYhavejointPDF9.1.3Aradioprogramgivesconcertticketstothefourthcallerwiththerightanswertoaquestion.Ofthepeople'vhocall,25%kno\vtheanswer.Phon.ecallsareindependentofoneanother.Th.erandomvariableNrindicatesthenumberofphonecallstakenwhentherthcorrectans,verarrives.(Ifthefourthcorrectanswerarrivesontheeighthcall,thenN4=8.)9.2.1Foraconstanta>0,aLaplacerandomvariableXhasPDF(a)WhatisthePMFofNi,thenumberofphonecallsneededtoobtainthefirstcorrectanswer?x>O,y>O,::e+y2(X1/)={fX.Y,''0otherwise.=X+Y?\tVhatisthevarianceofvVj.x(x)=2ae-alxl,Calculatethe~I[GF-oooo.x(s).9.2.2•Randomvariables,JandKhavethejointprobabilitymassfunctionP.J,K),kkj=-2j=-1=-10.420.28k=Ok=l0.120.080.060.04(b)WhatisE[J\T1],theexpectednumberofphonecallsneededtoobtainthefirstcorrectanswer?(a)\t\fhatistheMGFofJ?(c)\i\fhatisthePlVIFofN4,thenumberofphonecallsneededtoobtainthefourthcorrecta11S\Ver?Hint:SeeExample3.13.(c)FindthePMFofM(d)\iVhatisE[JVf4]?(d)WhatisE[N4]?Hint:J\T4canbewrittenastheindependentsumN4=1116ms].(f)UsethecentrallimittheoremtoestimateP[A<86ms].9.5.2Internetpacketscanbeclassifiedasvideo('!)orasgenericdata(D).BasedonalotofobservationstakenbytheInternetserviceprovider,wehavethefollowingprobabilitymodel:P[V]=3/4,P[D]=1/4.Datapacketsandvideopacketsoccurindependentlyofoneanother.TherandomvariableKnisthenumberofvideopacketsinacollectionofripackets.(a)WhatisE[K100J,theexpectednumberofvideopacketsinasetof100packets?(b)WhatisaK100?(c)UsethecentrallimittheoremtoestimateP[l18].(d)UsethecentrallimittheoremtoestimateP[l6<Kioo<24].9.5.3Thedurationofacellulartelephonecallisanexponentialrandomvariable'vithexpectedvalue150seconds.Asubscriberhasacallingplanthatincludes300minutespermonthatacostof$30.00plus$0.40foreachminutethatthetotalcallingtimeexceeds300minutes.Inacertainmonth,thesubscriberhas120cellularcalls.(a)·usethecentrallimittheoremtoestimatetheprobabilitythatthesubscriber'sbillisgreaterthan$36.(Assumethatthedurationsofallphonecallsaremutuallyindependentandthatthetelephonecompanymeasurescalldurationexactlyandchargesaccordingly,'vithoutroundingupfractionalminutes.)(b)Supposethetelephonecompanydoeschargeafullminuteforeachfractionalminuteused.Re-calculateyourestimateoftheprobabilitythatthebillisgreaterthan$36.9.5.4Let1oissonrandomvariablevvithexpectedvalue300requests.(a)A\t\febserverhasacapacityofCrequestsperminute.Ifthenumberofrequestsinaone-minuteintervalisgreaterthanC,theserverisoverloaded.Usethecentrallimittheoremtoestimatethes1nallestvalueofCfor'vhichtheprobabilityofoverloadislessthan0.05.(b)Usel\IIATLABtocalculatetheactualprobabilityofoverloadforthevalueofCderivedfromthecentrallimittheorem.(c)}""orthevalueofCderivedfromthecentrallimittheorem,whatistheprobabilityofoverloadinaone-secondinterval?(d)\iVhatisthesmallestvalueofCforwhichtheprobabilityofoverloadinaone-secondintervalislessthan0.05?(e)Commentontheapplicationofthecentrallimittheoremtoestimatetheoverloadprobabilityinaone-secondintervalandinaone-minuteinterval.[PROBLEMS9.5.6Integratedcircuitsfromacertainfactorypassacertainqualitytest'vithprobability0.8.Theoutcomesofalltestsaremutuallyindependent.(a)Whatistheexpectednumberoftestsnecessarytofind500acceptablecirCUI.t,S?.(b)Usethecentrallimittheoremtoestimatetheprobabilityoffinding500acceptablecircuitsinabatchof600circuits.(c)UseNIATLABtocalculatetheactualprobabilityoffinding500acceptablecircuitsinabatchof600circuits.(d)Usethecentrallimittheoremtocalculatetheminimumbatchsizeforfinding500acceptablecircuits,_,ithprobability0.9orgreater.9.5.7Internetpacketscan1beclassifiedasvideo(V)orasgenericdata(D).BasedonalotofobservationstakenbytheInternetserviceprovider,'vehavethefollowingprobabilitymodel:P[V]=0.8,P[D]=0.2.Datapacketsandvideopacketsoccurindependentlyofoneanother.TherandomvariableKnisthenumberofvideopacketsinacollectionofnpackets.(a)WhatisE[K4s],theexpectednumberofvideopacketsinasetof48packets?(b)WhatisaK48,thestandarddeviationofthenumberofvideopacketsinasetof48packets?(c)Usethecentrallimittlheore.mtoestimateP[30<f{4s<42],theprobabilityofbet,veen30and42voicecallsinasetof48calls.(d)UsetheDel\/Ioivre-Laplaceformulatoestimatef>[30<K48<42].9.5.8Inthepresenceofahead,vindofnormalizedintensityvV,yourspeedonyourbikeisV=20-10W3mi/hr.The'vindintensityvVisacontinuousuniform(-1,1)randomvariable.Moreover,the'Vindchangeseverytenminutes.LetvVidenotethehead,vindintensityintheithtenminuteinterval.Inafive-hourbikeride,335with30ten-minuteintervals,the'vindintensitiesW1,...,vV30areindependentandidenticaltoW.ThedistanceyoutravelisX=V1+V2+···+V10.6·usetheCLTtoestimateP[X>95].9.5.9Anamplifiercircuithaspo,verconsumptionYthatgro,vsnonlinearlywiththeinputsignalvoltageX.\i\fhentheinputsignalisXvolts,theinstantaneouspo,verconsumedbytheamplifierisY=20+15X2Watts.TheinputsignalXisthecontinuousuniform(-1,1)randomvariable.Samplingtheinputsignaleverymillisecondovera100-millisecondintervalyieldstheiidsignalsamplesX1,X2,...,X100-Overthe100msinterval,youestimatetheaveragepoweroftheamplifieras1100w=100:LYii=lwhereYi=20+15Xf.UsethecentrallimittheoremtoestimateP[W<25.4].9.5.10Inthefaceofperpetuallyvaryingheadwinds,cyclistsLanceandAshwinareina3000mileraceacross_America.Tomaintainaspeedofvmiles/hourinthepresenceofa11;mi/hrheadwind,acyclistmustgenerateapoweroutputy=50+(v+3VJ-15)Watts.Duringeachmileofroad,the'vindspeedWisthecontinuousuniform(0,10)randomvariableindependentofthewindspeedinanyothermile.(a)Lanceridesatconstantvelocityv=15mi/hrmileaftermile.LetYdenoteLance'spoweroutputoverarandomlychosenmile.\tVhatisE[Y]?(b)AshwinislesspowerfulbutheisabletorideatconstantpowerfJWattsinthepresenceofthesamevariablehead'vinds.·usethecentrallimittheoremtofindiJsuchthat_Ash,vin'vinstherace'vithprobability1/2.9.5.11Supposeyourgradeinaprobabilitycoursedependson10weeklyquizzes.Eachquizhastenyes/noque..stions,each[336CHAPTER9SUMSOFRANDOMVARIABLESv;,rorth1point.Thescoringhasnopartialcredit.Yourperformanceisamodelofconsistency:Oneachone-pointquestion,yougettherightanswerwithprobabilityp,independentoftheoutcomeonanyotherquestion.ThusyourscoreXionquiziisbetween0and10.Yourav10eragescore,X=I:i=lXi/100isusedtodetermineyourgrade.Tl1ecoursegradinghassimplelettergradeswithoutanycurve:A:X>0.9,B:0.8<X<0.9,C:0.7<X<0.8,D:0.6<X<0.7andF:X<0.6.Asithappens,youareaborderlineB/Cstudentwithp=0.8.(a)WhatisthePMFofXi?(b)UsethecentrallimittheoremtoestimatetheprobabilityP[A]thatyourgradeisanA.(c)Supposeno'vthatthecoursehas"attendancequizzes."Ifyouattendalecture'vithanattendancequiz,yougetcreditforabonusquizwithascoreof10.Ifyouarepresentforn,bonusquizzes,yourmodifiedaverageX'=lOn+EI~,xilOri+100isusedtocalculateyourgrade:A:X'>0.9,B:0.8<X'<0.9,andsoon.Givenyouattendriattendancequizzes,usethecentrallimittheoremtoestimateP[A].(d)Nowsuppasetherearenoattendancequizzesandyourweek1quizisscoredan8.Afe,vhoursaftertheweek1quiz,younoticethataquestion'vasmarkedincorrectly;yourquizscoreshouldhavebeen9.Youappealtotheannoyingprof'vhosays"Sorry,allregraderequestsmustbesubmittedimmediatelyafterreceivingyourscore.Butdon'tworry,theprobabilityitmakesadifferenceisvirtuallynil."LetUdenotetheeventthatyourlettergradeisunchangedbecauseofthescoringerror.FindanexactexpressionforP[U].Wnisthenumberofonesin1071independenttransmittedbits,eachequiprobably0or1.Forn,=3,4,...,usethebinornialprnffunctiontocalculate9.6.1P[0.499<vVn/lOn<0.501].\tVhatisthelargestn,forwhichyourl\IIATLABinstallationcanperformthecalculation?CanyouperformtheexactcalculationofExample9.14?·usethel\IIATLABplotfunctiontocomparetheErlang(n,>.)PDFtoaGaussianPDF'viththesameexpectedvalueandvarianceforA=1andri=4,20,100.\i\fhyareyourresultsnotsurprising?9.6.2RecreatetheplotsofFigure9.3.Onthesameplots,superimposethePDFofYn,aGaussianrando1nvariablewiththesa1neexpectedvalueandvariance.IfXndenotesthebinomial(n,,p)randomvariable,explainwhyformostintegersk,Pxn(k)~fy(k).9.6.3FindtheP~1FofW=X1+X2inExa1nple9.17usingtheconvfunction.9.6.4·useuniforrn12.rntoestimatetheprobabilityofastormsurgegreaterthan7feetinExample10.4basedon:9.6.5•(a)1000samples,(b)10000samples.X1,X2,andX3areindependentrandomvariablessuchthatXkhasPMF9.6.6Pxk(x)={~/(Wk)X=l,2,...,lOk,other,vise.FindthePMFofW=X1+X2+X3.LetXandYdenoteindependentfiniterandomvariablesdescribedbytheprobabilityandrangevectorspx,sxandpy,sy.Writeal\IIATLABfunction9.6.7[pw,sw]=surnfiniteprnf(px,sx,py,sy)suchthatfiniterandomvariableW=Xisdescribedbypwandsw.+Y[TheSampleMeanEarliercliaptersofthisbookpresentthepropertiesofprobabilityrnodels.Inreferringtoapplicationsofprobabilit}'theor}',\Vehaveassurnedpriorknovvledgeoftheprobabilitymodelthatgovernstheoutcorriesofanexperirnent.Inpractice,however,weencounterrrianysituationsinv.rhichtheprobabilityrriodelisriotknovvriinad·vancearidexperirrieriterscollectdatainordertolearnabouttliernodel.Indoingso,the}'applyprinciplesofstatist'icalin,fererice,abod}'ofkno'ivledgethatgo,rerristheuseofmeas11rerrientstodiscoverthepropertiesofaprobabilityrnodel.Tliischapterfocusesorithepropertiesofthesarnplernean,ofasetofdata.\Verefertoiridependerittrialsofoneexperirnent,witheaclitrialproducingonesamplevalt1eofarandorrivariable.Thesamplerneanissimpl}'thest1mofthesarnplevaluesdividedbythenumberoftrials.Webegiribydescribingtherelationshipofthesarnplemearioftliedatatotheexpectedvah1eoftheraridornvariable.'\VethendescribernethodsofllSingthesamplerrieantoestirriatetheexpectedva.lt1e.10.1SampleMean:ExpectedValueandVarianceThesarriplerrieanJ\!fn(X)=(X1+···+Xn)/riofn,independentobservatiorisofra,ndorrivariableXisarandomvariablewithexpectedvalueE[X]andvarianceVar[X]/r1,.Inthissection,wedefinethesarnplernean,ofarandomvariableandidentifyitsexpectedvalt1eandvariance.Latersectionsofthischaptershowrriathernatica.llyhowthesamplerrieancori'irergestoaconstantastheriurnberofrepetitionsofariexperirnentincrea,ses.Tliischapter,therefore,providesthernatlierriatica.1basisforthestaternentthatalthoughtherest1ltofasingleexperimentisunpredictable,predictablepatternsemergeasvvecollectrrioreandrrioredata.Todefinethesarnplernean,considerrepeatediridependenttrialsofariexperirrient.Eaclitrialresultsinoneobservationofarandorrivariable,X.Aftern,trials,337[338CHAPTER10THESAMPLEMEAN-vvehavesarr1plevaluesoftherirandornvariablesX1,...,Xn,allv.rit11thesamePDFasX.Thesarnplerr1eanisther1urnericala;verageoftheobservations.Definition10.1SampleMeanForiidrar1,dornvariablesX1,...,Xn'tJJithPDF1·x(x))thesamplemeanofX'istheraridornvariableT11efirstthingtonoticeisthatMn(X)isafunctior1oftherandorr1variablesX1,...,Xnandisthereforearandorr1variableitself.Itisirnportanttodistinguishthesamplerr1ean,Mn(X),fromE[X],whic11wesometimesrefertoasthernean,valv,eofrandorr1variableX.'\i\ThileJ\rfn(X)isarandorn·variable,E[X]isant1mber.Toavoidcor1ft1sion-vvhenst11dyir1gthesamplerr1ean,itisadvisabletorefertoE[X]astheexpectedvalueofX,rat11ert11anthernean,ofX.Thesarnplerr1eanofXandtheexpectedvalueofXa.reclosel}'related.Arnajorpurposeofthischapteristoexplorethefactt11at

as'nincreaseswithoutbour1d,J\rfn(X)predictablyapproachesE[X].Ineverydayconversation,thisp11enorr1er1onisoftencalledt11ela'tJJofaverages.TheexpectedvalueandvarianceofMn(X)revealthemostirr1portantpropertiesoft11esarr1plerr1ean.From011rearlierwork'\vithsurnsofrandorr1·variablesinChapter9,we11avethefollowingresult.Theorem10.1Thesarnplernean,Nln(X)hasexpectedvalv,ean,dvar'ian,ceE[Mn(X)]=E[X],Var[Mn(X)]=Var[X].'n,ProofFromDefinition10.1,Theorem9.1,andthefactthatE[Xi]=E[X]foralli,E[Mn(X)]=ln(E[X1]+...+E[Xn])=.!(E[X]+...+E[X])=E[X].n,(10.1)BecauseVar[aY]=a,2Var[Y]foranyrandomvariableY(Theorem3.15),Var[Mn(X)]=Var[X1+...+Xn]/n,2.Sincethexiareiid,VvecanuseTheorem9.3tosho'\VVar[X1+···+Xn]=Var[X1]+···+Var[Xn]=n,Var[X].ThusVar[JV!n(X)]=rLVar[X]/n?(10.2)=Var[X]/n.RecallthatinSection3.5,werefertotheexpected·valueofarandornvariableasatypicalvalv,e.Theorern10.lderr1onstratesthatE[X]isatypicalvalt1eofNln(X),regardlessofn,.Furtherrnore,T11eorerr110.1dernor1stratesthatasriincreases-vvithoutbound,t11evarianceofMn(X)goestozero.·vV11er1wefirstrr1etthevariance,anditssquarerootthestar1darddeviation,-vvesaidthatt11eyindicate110\vfara[10.2DEVIATIONOFARANDOMVARIABLEFROMTHEEXPECTEDVALUE339randorr1variableislikelytobefrornitsexpectedvalue.Theorern10.1suggeststhatas77,approac11esinfir1ity)itbecornes11igl1lylikel}'t11atMn(X)isarbitrarilyclosetoitsexpectedvalue)E[X].Inotherwords)thesarnplerr1eanMn(X)convergestotheexpectedval11eE[X]a,sthenl1rnberofsarnples77,goestoinfinit}r.T11erestofthischaptercontainsthematherr1aticalanalysisthatdescribesthenat11reofthisconvergence.Quiz10.1Xist11eexponential(1)r[-tndornvariable;1'.lln(X)ist11esarnplerr1eanofn,independentsamplesofX.Hov.rman}'samplesn,areneededtoguaranteet11atthe·varianceofthesarnplerr1eanJ\Jn(X)isnorr1orethan0.01?10.2DeviationofaRandomVariablefromtheExpectedValueTheChebyshevineq11alit}'isanupperboundont11eprobabilityP[IX-µxI>c].\!VeusetheChebyshevinequalicytoderivetheLav.rsofLargeNurr1bersandtheparameter-estimationtechniquesthat-vvestudyinthenextt\vOsections.T11eChebyshevinequalityisderivedfromthel\/Iarkovinequality,alooserupperbo11nd.TheChernoff'boundisarr1oreaccurateinequalitycalculatedfromthecompleteprobabilitymodelofX.TheanalysisoftheconvergenceofJ\Jn(X)toE[X]begins-vvithastudyoftherandorr1variableIX-µ,xI,theabsolutedifferencebetweenarandorr1variableXanditsexpected·value.ThisstudyleadstotheChebyshevi'Tl,equality)\vl1ichstatest11attheprobabilityofalargede·viationfromt11eexpectedvalueisinverselyproportionaltothesql1areofthedevia,tiori.ThederivationoftheC11ebyshevineql1alit}'begins-vvitht11eMarko'ui'Tl,eq11,ality)anupperbol1ndont11eprobabilitythatasamplevalueofanonnegativerar1dorr1variableexceedstheexpectedvaluebyanyarbitraryfactor.T11eLawsofLargeNurr1bersandtechr1iquesforpararneterestirnation,thesubjectofthenexttwosections)areaconsequenceoftheCheb}rshevinequality.TheCher'Tl,ojfbo'u,71,disathirdinequalityusedtoestirr1atetheprobabilit}'thatarandornsarr1pledifferss11bstantiallyfromitsexpectedvalue.T11eChernoffboundisrr1oreacc11ratet11antheCheb}rshevandl\/Iarkovineql1a1itiesbecauseittakesintoaccountrr1oreinformatior1abouttheprobabilityrnodelofX.Tour1derstandt11erelationshipoftheMarkovinequality)theChebyshevinequality,andtheChernoff'bound)-vveconsidertheexarr1pleofastorrr1surgefollo-vvingahurricane.Weassurnethattheprobabilityrnodelfortherar1dorr111eightinfeetofstorrns1rrgesisX)theGal1ssian(5.5)1)random·variable)andconsidertheevent[X>11]feet.Theprobabilicyofthisever1tisver}'closetozero:P[X>11]=Q(ll-5.5)=1.90x10-8.[340CHAPTER10THESAMPLEMEANTheorem10.2MarkovInequalityForaran,dorn'UariableX,s'uchthatP[X<O]ProofSinceXisnonnegative,fx(x)=0for::rc2E[X)={loSincex>c2xfx(1;)dx+!=O;an,dacon,stan,t<0andoo~C;!.ooxfx(1;)dx>xfx(x)d1;.(10.3)c2intheremainingintegral,(10.4)Keepinrr1indthatthel\/Iarkovir1equalityis·validonlyforr1onnegativerandorr1variables.As-vveseeinthenextexarr1ple,theboundprovidedb}'thel\/Iarkovinequalit}'canbeveryloose.===Example10.l==::::::aLetXrepresenttheheight(infeet)ofastormsurgefollowingahurricane.IftheexpectedheightisE[X]=5.5,thentheMarkovinequalitystatesthatanupperboundontheprobabilityofastormsurgeatleast11feethighisP[X>11]<5.5/11=1/2.(10.5)WesaytheMarkovineqt1alityisalooseboundbecausetheprobabilitythatastormsurgeishighert.han11feetisessentiall}'zero,v.rhiletheinequalit}'rr1erelystatesthatitislessthanoreqt1alto1/2.Althoughtheboundisextremelylooseformanyrandornvariables,itistight(ir1fact,anequation)-vvithrespecttosorr1erandorr1·variables.Example10.2SupposerandomvariableYtakesonthevaluec2withprobability'[Jandthevalue0otherwise.Inthiscase,E[Y]='[JC2,andtheMarkovinequalitystates(10.6)SinceP[Y>c2]thisinstance.=p,weobservethattheMarkovinequalityisinfactanequalityinTheChebyshevinequalit}'appliesthe1!{arkovir1equa.litytot11enor1negativerandom·variable(Y-11,y)2,derivedfrorr1anyrar1dom·variableY.[10.2DEVIATIONOFARANDOMVARIABLEFROMTHEEXPECTEDVALUE341Forariarbitraryra/ndornvariable·yan,dcon,stan,tc>0)P[ly_/1,yI>-c]<-Var[Y]2.cProofInthetv'Iarkovinequality,Theorem10.2,letX=(Y-µ,y)2.Theinequalitystates(10.7)Thetheoremfollowsfromthefactthat{(Y-µ,y)2>c2}={IY-J.lYI>c}.UnliketheMarkovineq11ality,theChebyshe·vinequalityis·validforallrandorr1variables.'\i\ThiletheMarkovinequalit:yrefersonlytotheexpected·valueofarandornvariable,theC11ebyshe\rinequalityalsoreferstothevariance.BecauseitllSesmoreinforrnationabouttherandorn\rariable,tl1eChebys11evir1equalit}'generallypro\ridesatighterboundthanthel\/Iarko\rineq11ality.Inparticular,v.rhenthe\rariar1ceofYisverysrnall,theChebyshevinequalitysaysitisunlikelythat·yisfaravvayfrornE[Y].Example10.3IftheheightXofastormsurgefollowingahurricanehasexpectedvalueE[X]=5.5feetandstandarddeviationax=1foot,usetheChebyshevinequalitytotofindanupperboundonP[X>11].SinceaheightXisnonnegative,theprobabilitythatX>11canbewrittenasP[X>11]=P[X-µ,x>11-µ,x]=P[IX-µxl>5.5].(10.8)NowweusetheChebyshevinequalitytoobtainP[X>11]=P[IX-µ,xi>5.5]<Var[X]/(5.5)2=0.033~1/30.AlthoughthisboundisbetterthantheMarkovbound,itisalsoloose.P[Xsevenordersofmagnitudelowerthan1/30.(10.9)>11]isTheChernoffbot1ndisanir1equalityderivedfrorr1t11err1omer1tgeneratingfunctioninDefinition9.1.Lilc]ThisChernoffboundisl1sefulwhencislargerelati\retoE[X]andP[X>c]issmall.---Theorem10.4ChernoffBoundForariarbitraryran,dornvariableXa'ndacoristaritc)[342CHAPTER10THESAMPLEMEANProofIntermsoftheunitstepfunction,11,(::e),vveobservethatP[X>c]=Foralls>0,11(x-c)<es(x-f00Jcc).00fx(x)dx=/_11,(x-c)fx(x)dx.(10.10)-ooThisimplies(10.11)Thisinequalityistrueforanysstominimizee-sc¢.,'.((s).>0.HencetheupperboundmustholdwhenwechooseTheChernoffboundcanbeappliedtoan}'randorr1variable.However,forsmallvaluesofc,e-scx(s)\x.rillberninirr1izedbyanegativeva111eofs.Int11iscase,t11err1ir1irnizingnonnegativesiss=0,andtheChernoffboundgivesthetrivialans\verP[X>c]<1.Example10.4IftheprobabilitymodeloftheheightX,measuredinfeet,ofastormsurgefollowingahurricaneatacertainlocationistheGaussian(5.5,1)randomvariable,usetheChernoffboundtofindanupperboundonP[X>11].InTable9.1theMGFofXis(10.12)ThustheChernoffboundisP[X>11]<rr1in2e-llse(lls+s)/2=rr1ir1s>o2eCs-lls)/2.s>o(10.13)Tofindtheminimizings,itissufficienttochoosestominimizeh,(s)s2-lls.Settingthederivativerlh,(s)/rls=2s-11=0yieldss=5.5.Applyings=5.5totheboundyieldsp[X>11]<=eCs2-lls)/2s=5.5e-(5.5)2/2=2.7X10-7.(10.14)EventhoughtheCherr1offboundis14tirr1eshigherthantheactl1alprobability,1-(5.5)=1.90x10-8,itstillconveystheinforrnationthatastorrnsurgehigherthan11feetisextremel}'ur1likel}'·Bycontrast,the}/!arko\randChebys11evineql1alitiesprovidebol1r1dsthatsuggestt11atar111-footstorrnsurgeocc11rsrelativelyfrequently.Theinforrr1ationneededtocalculatethet11reeinequalitiesaccountsforthedifferencesir1theiraccl1racy.TheMarko''inequalityusesonlytheexpectedvall1e,theCheb}rs11e\rineq11alityllSestheexpectedvalueandthevariance,whiletheml1chrr1oreacc11rateChernoff'boundisbasedonkno\vledgeofthecornpleteprobabilityrr1odel(expressedas¢x(s)).[10.3343LAWSOFLARGENUMBERSQuiz10.2Inasubwaystation)t11ereareexactlyer1oughcustornersontheplatforrr1tofillthreetrains.Thearrivaltirr1eoftherithtrainisX1+···+XnvvhereX1,X2,...areiidexponentialrandom·variables\vithE(X.i:]=2rr1int1tes.LetWequalthetimerequiredtoservethe\vaitingc11storners.ForP[W>20),t11eprobabilityt11atWisO\rertwentyrr1inutes,(b)UsetheJ\/Iarkovineqt1alitytofir1d(a)Uset11ecentrallirnittheorerr1tofir1danestirr1ate.anllpperbound.(c)UsetheCheb}rshevinequalitytofindanupperbo11nd.(d)UsetheChernoff'bot1ndtofindanupperbour1d.(e)UseTheorerr14.11foranexactcalculation.10.3LawsofLargeNumbersThesamplemeanlllfn(X)convergestoE[X]andtherelativefreq11encyofeventAcor1vergestoP[A]as'n,thent1rr1berofindeper1denttrialsofanexperirnent,increases\vithoutbound.vVhenweapplyt11eChebyshevinequalityto·y=Mn(X),\VeobtainllSeft1linsigl1tsintot11epropertiesofindeper1dentsarnplesofarandorr1variable.---==Theorem10.s---=WeakLawofLargeNumbers(FiniteSamples)Fora/nycon,stan,tc>0,{a)P[IMn(X){b)P[IMn(X)ProofLetY>c]<·var[X]Jl,XIjl,XVar(X]I<C]>1-n,c2·ric2,=Mn(X).Theorem10.lstatesthatE[Y]=E[Nln(X)]=11.xVar[Y]=Var[Mn(X)]=·var[X]/ri,.(10.15)Theorem10.5(a)follo,vsbyapplyingtheChebyshevinequality(Theorem10.3)toY=M11(X).Theorem10.5(b)isjustarestatementofTheorem10.5(a),sinceP[ll\1n(X)-/J,xl>c]=1-P[IMn(X)-µ,xi<c].(10.16)In\vords,Theorem10.5(a.)saysthattheprobabilitythatthesarr1plemear1isrr1orethancunitsfrornE(X]canbemadearbitrarilysrnallbylettingthent1rnberofsamplesn,becomelarge.B}'takir1gt11elimitasn,---+oo,weobtaintheinfinitelirr1itrest1ltir1thenextt11eorerri.[344CHAPTER10THESAMPLEMEANTheorem10.6WeakLawofLargeNumbers(InfiniteSamples)IfXhasfiriitevariarice)then,foran,ycon,stan,tc>0,(a)lirr1P[IJ\Jn(X)-µ,xI>c]=0)n-+oo{b)lirr1P[IJ\Jn(X)-µ,xI<C]=1.n-+ooInparalleltoTheorerr110.5)T11eorerns10.6(a)and10.6(b)areequi·valentstatementsbecauseP[IMn(X)-µxI>c]=1-P[ll\1n(X)-Jl,XI<c].(10.17)Ir1vvords,T11eorern10.6(b)sayst11attheprobabilitythatt11esamplemear1isvvit11in±cunitsofE[X]goestooneasthenumberofsarnplesapproachesinfinity.Sinceccanbearbitraril}'srnall(e.g.,10-2000))bothTheorem10.5(a)andTheorerr110.6(b)canbeir1terpretedassayingt11atthesarr1plerr1eanconvergestoE[X]ast11er1urnberofsarr1plesincreases\iVithoutbound.T11e\;veakla\iVoflargenurr1bersisa\rer}'ger1eralresultbecauseitholdsforallrandomvariablesX\iVithfinitevariance.l\/Ioreover,wedonotr1eedtoknovvar1yofthepararr1eters,sucl1astheexpected\ralueorvariance,ofrandorr1\rariableX.Theadjective1JJeakintheweaklavvoflargenurnberssuggestst11atthereisalsoastror1glavv.The}'differint11enatureoftheconvergenceofJ\lln(X)to11,x.Thecon\rerger1ceinT11eorerr110.6isanexarnpleofcorivergen,ceiriprobability.Definition10.2ConvergenceinProbabilityTheran,dornseqv,en,ce·11;1,con,vergesin,probabilitytoacon,stan,tyifforariylimP[l.Yn-n-+ooYI>E]=E>0,O.Theweaklavvoflargenumbers(T11eorerr110.6)isanexarnpleofconvergenceinprobabilit}'ir1\iVhic11Y;.1,=1\Jn(X),y=E[X])aridE=c.Thestron,gla111oflarge'n'tJ,rnbersstatesthat'tlJithprobability1)theseql1enceM1,J\J2,...11asthelimit/J,x.l\/Iat11ernaticiar1susetheterrnscorivergen,cealrnostsurely)con:uergericealrnostal111ays,andcon:uergen,cealrnostever~y'uJhereassynor1yrr1sforconvergencewit11probability1.ThedifferencebetweenthestronglavvandthevveaklavvoflargenurnbersisSl1btleandrarelyarisesinpracticalapplicationsofprobabilit}'theory.Aswewillseeint11enexttheorem,t11e\;veaklawoflargenurnbers\ralidatestherelativefreql1encyinterpretationofprobabilities.Considerar1arbitraryeventAfrorr1anexperiment.ToexamineP[A]\iVedefinetheindicatorrandorr1variable1ifeventAocc11rs,0ot11ervvise.(10.18)SinceXAisaBernoullirandornvariable\iVithsuccessprobabilityP[A])E[XA]=P[A].Sincegeneralpropertiesoft11eexpectedvall1eofarandorr1variableapplyto[10.4345POINTESTIMATESOFMODELPARAMETERSE[XA],-vvecanapplythela-vvoflargenurnberstosarnplesoft11eindicatorXA:(10.19)"SinceXAijustcountswl1etl1ereventAoccurredontriali,Pn(A)istherelativefreq11,er1,cyofeventAir1ritrials.SincePn(A)isthesarnplerneanofXA,wev.rillseethatthepropertiesofthesarr1plemear1explaintherr1athematicalconnectionbetweer1relativefrequer1ciesandprobabilities.--==Theorem10.1Asri---+oo;therelati'uefreqv,en,cyPn(A)con/vergestoP[A];foran,ycon,stan,tc>0;lirnP[Pn(A)-P[A]>n-+ooc]=0.ProofTheprooffollowsfromTheorem10.6sinceFn(-4)oftheindicatorXA,whichhasexpectedvalueE[XA]P[A](l-P[-4]).=Ji.lln(XA)isthesamplemean=P[A]andvarianceVar[XA]Theorerr110.7isarr1atherr1aticalversionofthestaterr1entt11atast11er1urr1berofobservationsgrowswithoutlirr1it,therelativefrequencyofanyever1tapproacl1estheprobabilityoftheever1t.Quiz10.3X1,...,Xnaren,iidsarnplesoftheBernoulli(p=0.8)rar1dom·variableX.(a)FindE[X]andVar[X].(b)·vvhatisVar[Jlf1oo(X)]?(c)UseT11eorerr110.5tofindthatO'.such(d)Ho-vvrr1anysarnplesn,areneededtogl1aranteeP[l1VI1oo(X)-Pl>0.05]10.4P[1111.n(X)-rrel="nofollow">I>0.1]<o.o5.PointEstimatesofModelParameters"R,anestirr1ateofapararr1eter,r,ofaprobabilityrnodelisunbiasedifE[R]=r.Aseql1enceofestimatesR1,R2,...isconsistentif"lirnn-+ooRn=r.Thesarr1plemear1isant1nbiasedestirr1atorof/J,x.Thesequenceofsarr1plerr1eansisconsister1t.ThesamplevarianceisabiasedestimatorofVar[X].Ir1t11erernainderofthischapter,-vveconsiderexperirnentsperforrnedinordertoobta,inir1forrnationaboutaprobabilit}'model.Todoso,investigatorsusually[346CHAPTER10THESAMPLEMEANderiveprobabilityrriodelsfrorripracticalmeas11rerrients.Later,theyusetliernodelsinwaysdescribedthroughoutthisbook.Howtoobtainarnodelinthefirstplaceisarriajorsubjectiristatisticalinference.Inthissection-vvebrieflyiritroducethesubjectb}'stt1dyingestimatesoftheexpectedvalt1eandthe·varianceofarandorrivariable.Thegeneralproblemisestirriationofapararneterofaprobabilityrnodel.Apararrieterisanynurnberthatcanbecalculatedfromtheprobabilit}'model.Forexarriple,foranarbitraryeventA,P[A)isarriodelparameter.Thetechniques-vvest11dyinthischapterrelyonthepropertiesofthesarnplerrieanNln(X).Dependirigonthedefiriitionoftherandorri·variableX,-vvecanusethesamplerrieantodescribean}'pararrieterofaprobabilitymodel.Weconsidert-vvot}rpesofestirnates:Apoin,testirnateisanurriberthatisascloseaspossibletothepararrietertobeestin1ated,-vvhileaco'Tl,fiden,ce'ir1,tervalestirnateisarangeofnt1rribersthatcontainsthepararrietertobeestirnated-vvithliighprobability.PropertiesofPointEstimatesBeforepresentingestirnationrriethodsbasedonthesarnplemeari,weintroducethreepropertiesofpointestimates:bias,con,sisten,cy,andacc11,racy.We\villseethatthesarriplemeariisanunbiased,consistentestirriatoroftheexpectedvalueofarandornvariable.Bycontrast,wewillfindthatthesarnplevarianceisabiasedestirriateoftlieva1iariceofararidorrivariable.Onerrieasureoftheacct1racyofanestimateisthernean,sq11,areerror,theexpectedsq11areddifferencebet\veenanestirnatearidtheestirriatedpararrieter.Considerariexperirnerittliatprodt1cesobservationsofsarnplevaluesoftherandomvariableX.\!Veperformanindefiniten11mberofiridependenttrialsoftheexperirnent.Theobservationsaresarnple\raluesoftherandorrivariablesX1,X2,...,all-vviththesarrieprobabilityrnodelasX.Assurnetliatrisaparameteroftheprobabilityrnodel.\N"et1setheobservationsX1,X2,...toprod11ceaseqt1enceofestirnatesofr.TheestirriatesR1,R2,...areallrandomvariables.R1isaft1nctionofXl·R2isafunctionofXiaridX2,andingerieralRnisafunctioriofX1,X2,...,Xn.\i\ThenthesequenceofestimatesR1,R2,...corivergesinprobabilitytor,-vvesaytheestimatoriscon,siste'Tl,t.AAAAAAADefinition10.3ConsistentEstimatorTheseqv,e'Tl,ceofestirnatesR1,R2,...ofpararneterr'isconsistentifforan,yEAAlirriP[n-+ooRnA>0,r>EJ=0.AAnotherpropertyofariestirnate,R,isbias.RerrierribertliatRisarandornvariable.Ofcourse,\Ve-vvottldlikeRtobeclosetothetruepararrietervalt1er-vvithhighprobability.Irirepeatedexperirrients,ho\ve\rer,sorrietirnesR<randothertirriesR>r.Altho11ghRisrandorn,it-vvot1ldbet1ndesirableifRwaseitliertypicallylesstharirortypicall}'greatertharir.Tobeprecise,wewot1ldlikeRtobe11/nb'iased.AAAAA[10.4347POINTESTIMATESOFMODELPARAMETERSDefinition10.4UnbiasedEstimatorAn,estirnate;R,ofpararrieterrisunbiasedifE[R]=r;other'tu'ise,Risbiased.Unlikeconsistency,vvhicl1isapropertyofasequenceofestirr1ators,bias(orlackofbias)isapropert:yofasingleestirnatorR.TheconceptofasyrnptoticbiasappliestoasequenceofestimatorsR1,R2,...Sl1chthateachRnisbiasedv.riththebiasdirninishingtowardzeroforlargerLThistypeofsequenceisasyrnptoticallyun,biased.AAADefinition10.5:---AsymptoticallyUnbiasedEstimatorTheseque'nceofestirnatorsRnofpararneterr'isasymptoticallyunbiasedifAlirr1E[Rr1.]=r.n---+ooTl1emear1squareerrorisanirnportantrr1easureoftheaccuracyofapoir1testirnate.·vvefirstencot1nteredthemeansquareerrorinSection3.8;hovvever,inthatchapter,vvev.rereestirr1atingthevalueofarandornvariable.Tl1atis,vvev.rereguessingadeterministicnurr1berasapredictionofarandorn·variablethatvvel1adyettoobserve.HerevvellSethesarr1erneansquareerrorrnetric,bl1tweareusingarar1domvariablederivedfrorr1experirr1entaltrialstoestirnateadeterrr1inisticbt1tunkr1ovvnparameter.MeanSquareErrorDefinition10.6ThemeansquareerrorofestirnatorRofpararneterrisANotethatvvhenRisar1ur1biasedestirr1ateofraridE[R]=r,therneansql1areerroristhevariar1ceofR.Forasequenceofunbiasedestirr1ates,itisenoughtoshovvthattl1emeansquareerrorgoestozerotopro\rethattheestirr1atorisconsistent.A---==Theorem10.8AAIfaseqv,en,ceof'IJ,n,biasedestirnatesR1,R2,...ofparam,eterrhasrriearisqv,areerroren=Var[Rn]satisfyin,glimn---+ooen=0,then,theseqv,en,ceRniscon,sisten,t.A~ProofSinceE[Rn]=r,vv-eapplytheChebyshevinequalitytoPRn.ForanyconstantE>0,Rn-r>EJ<\far[2Rn].(10.20)A[EInthelimitoflargen,wehaveli1nPn-=[Rn-r>EJ<lim\!ar[fnJn-=E=0.(10.21)[348CHAPTER10--ExampleTHESAMPLEMEAN10.5~=~Inanyintervalofkseconds,thenumberJ\TkofpacketspassingthroughanInternetrouterisaPoissonrandomvariablewithexpectedvaIueE[Nk]=krpackets.LetRk=J\Tk/kdenoteanestimateoftheparameterrpackets/second.IseachestimateAARkanunbiasedestimateofr?WhatisthemeansquareerrorekoftheestimateRk?AIsthesequenceofestimatesAR1,R2,...consistent?AFirst,weobservethatRkisanunbiasedestimatorsince(10.22)Next,werecallthatsinceJ\TkisPoisson,Var[JVk]=kr.Thisimplies(10.23)ABecauseRkisunbiased,themeansquareerroroftheestimateisthesameasitsvariance:ek=r/k.Inaddition,sincelirr1k-+ooVar[Rk]=0,thesequenceofestimatorsRkisconsistentbyTheorem10.8.AAPointEstimatesoftheExpectedValueToestirnater=E[X],wet1seRn=J\lfn(X),thesarnplemean.SinceTheorern10.ltellsusthatE[Mn(X)]=E[X],thesamplerr1eanist1r1biased.=-=;;;;;;;;Theorem10.9__..;;=;;;iThesarnplernean,Nln(X)isan,·u·nbiasedestirnateofE[X].Becausethesarnplemeanisunbiased,t.herneansqt1aredifferencebetvveenMn(x)andE(X]isVar(J\J,n(X)],giveninTheorern10.1:---==Theorem10.10Thesarnplernean,estirnatorMn(X)hasrnean,sqv,areerror2en=E[(Mn(X)-E[X])]=Var(Nin(X)]=Var(X]·n.Ir1theterrnir1ologyofstatisticalinference,Fn,thestandarddeviationofthesarr1plemean,isreferredtoastr1estan,da:rderroroftheestirnate.Thestandarderrorgi·vesanindicationofhowfarwesr1011ldexpecttr1esarnplerneantode·viatefrorntheexpected·value.Inparticular,vvhenXisaGaussianrandorn·variable(andMn(X)isalsoGat1ssian),Problem10.4.lasksyoutoshov.rthatP[E[X]-Fn<l\!In(X)<E(X]+FnJ=2(l)-1~0.68.(10.24)[10.4POINTESTIMATESOFMODELPARAMETERS349Ir1vvords,Equation(10.24)saysthereisroughl}'atwo-thirdsprobabilityt11atthesarnplemeanisvvithir1onestandarderroroft11eexpectedvall1e.Thissarnecor1cll1sior1isapproxirnatelytruewher1'nislargeandthecentrallirnittheoremSa}'SthatMn(X)isapproxirnatelyGaussiari......-----,=Example10.6HowmanyindependenttrialsriareneededtoguaranteethatPn(A),therelativefrequencyestimateofP[A].hasstandarderror<0.1?SincetheindicatorXAhasvarianceVar[XA]=P[A)(l-P[A)),Theorem10.10impliesthatthemeansquareerrorofMn(XA)isen=Var[X)'n=P[A)(1-P[A))n,.(10.25)Weneedtochoosen,largeenoughtoguarantee.Je;i<0.1(en<=0.01)eventhoughwedon'tknowP[A).WeusethefactthatTJ(l-p)<0.25forall0<[J<l.Thus,en<0.25/n,.Toguaranteeen<0.01,wechoosen,=0.25/0.01=25trials.T11eorem10.10derr1onstratesthatthestandarderroroftheestirr1ateofE[X]convergestozeroasn,growsv.rithoutbour1d.Thefollovvir1gtheorerr1statesthatthisirnpliesthatthesequer1ceofsamplerr1eansisaconsister1testimatorofE[X).Theorem10.11.------.IfXhasfin,itevarian,ce;then,thesarnplernean,Mn(X)isaseq11,en,ceofcon,sisten,testirnatesofE[X].ProofByTheorem10.10,themeansquareerrorofMn(X)satisfies..Var[X]hmVar[l\1n(X))=11mn~=n~=n=O.(10.26)ByTheorem10.8,thesequenceM11(X)isconsistent.Theorern10.11isarestaterr1er1tofthe\veaklawoflarger1urr1bers(Theorem10.6)inthelangt1ageofpararneterestirnation.PointEstimatesoftheVariancevV11entheunknownpararneterisT=Var[X],\VehavetvVOcasestoconsider.BecauseVar[X]=E[(X-µ,x)2)dependsontheexpectedvalue,\veconsiderseparatelythesituationwher1E[X]iskno\vnand\vl1enE[X]isanllnknownpararneterestirr1atedb}'Mn(X).Suppose\Veknowt11atE[X)=0.Ir1t11iscase,Var[X)=E[X2)andestimationofthe\rarianceisstraightforvvard.If\Vedefir1ey=X2)\VecanvieV\TtheestimationofE[X2)frorr1thesarnplesX,;,astheestirr1ationofE[.Y]frornthesarr1ples·~=Xf.Thatis,t11esamplerneanof·ycanbe\vrittenasl\/In(Y)=l(Xr+...+X.~).T/,(10.27)[350CHAPTER10THESAMPLEMEANAssumir1gthatVar[Y]exists,the\veakla\voflargenurnbersirr1pliesthatNln(Y)isaconsistent,ur1biasedestimatorofE[X2]=Var[X].vV11enE[X]isaknovvnquantity11,x,V\'ekno\vVar[X]=E[(X-µ,x)2].Inthiscase,wecanllSethesarr1plerr1eanofvV=(X-11,x)2toestirnateVar[X].,(10.28)IfVar[W]exists,Mn(W)isaconsistent,unbiasedestirnateofVar[X].Whentheexpected\ra,lueµ,xisunknovvr1,thesituationisrnorecomplicatedbecal1Set11e\rarianceofXdependsor1µx.\'VecannotuseEquation(10.28)ifµ,xisunkno\vn.Inthiscase,\Vereplacetheexpectedvc:,tlue11,xbythesarr1plerneanMn(X).SampleVarianceDefinition10.7Thesa,rnplevaria,n,ceofn,'irideper1,der1,tobserva,t'ion,sofran,dornvariableXisnVn(X)=l'n"'(X,i~Nln(X))2.i=lIncontrasttot11esarnplerr1ean,thesarnplevarianceisabiasedestimateofVar[X].===Theorem10.12E[Vri(X)]=T/,T/,1Var[X].ProofSubstitutingDefinition10.lofthesamplemeanMn(X)intoDefinition10.7ofsamplevarianceandexpandingthesums,wederive(10.29)BecausetheXiareiid,E[Xf]=E[X2]foralli,andE[Xi]E[Xj]=Jl,~.ByTheorem5.16(a),E[XiXj]=Cov[Xi,Xj]+E[Xi]E[Xj]·Thus,E[XiXj]=Cov[Xi,Xj]+Jl,~.Combiningthesefacts,theexpectedvalueofVninEquation(10.29)is(10.30)I::1122Sincethedoublesumhasri2terms,2:::~µ~=riµ~.Ofthen,covarianceterms,11thereareriter1nsoftheformCov[Xi,Xi]=·var[X],\vhiletheremainingcovarianceterms[10.4areall0becauseXiandXjPOINTESTIMATESOFMODELPARAMETERS351areindependentfori=f.j.ThisimpliesE[V7;7,]=Var[X]-12(n,Var[X])=nT/,-1nVar[X].(10.31)However,byDefir1itior110.5,·v;i(X)isasyrnptotica.llyunbiasedbecause'n,-1lirr1E[Vn(X)]=lirr1n-+oon,n-+oo·var[X]=Var[X].(10.32)AlthoughVn(X)isabiasedestirr1ate,Theorem10.12suggeststhederi·vationofant1nbitiSedestimate.Theorem10.13Theest'irnateV~(X)='/7,~lnL(Xi-1Vfn(X))2'i=lisan,un,biasedestirnateofVar[X].ProofUsingDefinition10.7,vvehave(10.33)andE[V~(X)J=n,:1E[V7;1,(X)]=\!ar[X].(10.34)Cornparir1gt11etwoestirr1atesofVar[X],weobservethatasn,growswithoutlimit,thetvvoestimatesconvergetothesarne·value.However,forn,=1,J\!f1(X)=X1and·v1(X)=0.B}'contrast,·v{(X)isundefined.Becausethe·varianceisameasureofthespreadofaprobabilityrr1odel,itisirnpossibletoobtainanestirnateofthespreadfrornonlyoneobservation.T11ustheestimateV1(X)=0iscornpletelyillogical.Ontheotherhand.,theur1biasedestimateofvariar1cebasedontwoobservationscar1bewritter1as=(X1-X2)2/2,vvhichclearl}'reflectsthespread(rneansquaredifference)oftheobservations.Togoft1rtherande·vall1atetheconsistencyofthesequenceV{(X),·v;(X),...isasurprisir1glydiffict1ltproblern.ItisexploredinProblern10.4.5.v;Quiz10.4Xisthecontir1uousuniform(-1,1)randorr1variable.Findt11erneansquareerror,E[(Var[X]-V100(X))2],ofthesarnplevarianceestimateofVar[X],basedor1100ir1dependentobservationsofX.[352CHAPTER10THESAMPLEMEAN10.5ConfidenceInterv:alsAconfidenceintervalestimateofapararneterofaprobabilitymodel,suc11astheexpectedvalueortheprobabilityofanevent,cor1sistsofarangeofnt1mbersaridtheprobabilitythattheparameterisvvit11ir1thatrange.Theorern10.lsuggeststhatasthent1rnberofindependentsarr1plesofarandorr1variableincreases,thesarr1plerneangetscloserandclosertotheexpectedvalue.Sirnilarl:y,ala;\voflargenurnberssuchasT11eorem10.6referstoalirr1itasthentrrnberofobservationsgromwitho11tbo11nd.Inpractice,hovvever,vveobserveafinitesetofrneast1rements.Inthissection,wedeveloptechniquestoassesst11eaccuracyofestirnatesbasedonafir1itecollectionofobservations.\Veintroducetvvocloselyrelatedqt1antities:theconfiden,cein,terval,relatedtothedifferencebetweenarar1domvariableanditsexpectedvalue,aridt11econ,fidericecoeffic'ien,t,relatedtotheprobabilityt11atasarnplevalueoftherandorr1variablewillbevvithint11econfidenceinterval.Thebasicrr1athernaticsofconfidenceintervalscorr1esfromTheorern10.5(b),restatedherevvitha=Var[X]/ric2:P[l.l\f.n(X)-µ,xI<c]>1-Var[X]'nC2=1-o~(10.35)Equation(10.35)cor1tainst'ivoinequalities.Oneinequality,IMn(X)-1J,xl<c,(10.36)definesanevent.Thiseventstatesthatthesamplerr1eanisvvithin±cunitsoftheexpected\ral11e.Thelengthoftheinterva.1thatdefinesthisever1t,2c11r1its,isreferredtoasacon,fiden,cein,terval.Theotherineqt1alitystatesthattheprobabilitythatthesarr1plemeanisintheconfidenceintervalisatleast1-a.Werefertothequantity1-aastheconfiden,cecoefficien,t.Ifaissrnall,vvearehighlyconfidentthatMn(X)isintheinter'iral(1),x-c,µ,x+c).InEquation(10.35)v.reobservethatforanypositivenurr1berc,r10rr1atterhov.rsmall,wecanmakeaassrr1allaswelikebychoosingn,largeer1ough.Ir1apracticalapplication,cindicatesthedesiredaccuracyofanestirnateofµ,x,aindicatesourconfidencethatweha,reachievedthisaccuracy,andn,tellsusho'ivrr1anysarr1ples'iveneedtoachievet11edesireda.Alternati'irely,gi,renVar[X],n,,anda,Eq11ation(10.35)tellsusthesizecoft11econfidenceinter\ral.Example10.1Supposeweperform'nindependenttrialsofanexperimentandweusetherelativefrequencyPn(A)toestimateP[A].Findthesmallestn,suchthatPn(A)isinaconfidenceintervaloflength0.02withconfidence0.999.RecallthatPn(A)isthesamplemeanoftheindicatorrandomvariableXA.SinceXAisBernoulliwithsuccessprobabilityP[A],E[XA]=P[A]andVar[XA]=P[A](l-P[A]).SinceE[Pn(A)]=P[A],Theorem10.S(b)saysP[Fn(A)-P[A]<c]>1-p[A](~'~p[A]).(10.37)[10.5353CONFIDENCEINTERVALSInExample10.6,weobservedthatJJ(l-rJ)<0.25for0<p<1.ThusP[A](lP[A])<1/4foranyvalueofP[A]andP[Pn(A)-P(A]<cJ>1-Foraconfidenceintervaloflength0.02,wechoosecmeetourconstraintif1-1()24n,0.014~c2=(10.38).0.01.Weareguaranteedto>0.999.(10.39)Thusweneedri>2.5x106trials.Ir1thenextexarnple)\veseet.hatif\ver1eedagoodestimateoftheprobabilityofarareeventA,thenthenl1rnberoftrialswillbelarge.Forexarr1ple,ifeventAhasprobabilityP[A]=10-4)t11enestimatir1gP[A]\vithin±0.01ismeaningless.Accurateestimatesofrareever1tsrequiresignificantlyrr1oretrials.==;;;..ExampIe10.8__,,;;==Supposeweperformn,independenttrialsofanexperiment.ForaneventAoftheexperiment,calculatethenumberoftrialsneededtoguaranteethattheprobabilitytherelativefrequencyofAdiffersfromP[A]bymorethan10%islessthan0.001.InExample10.7,wewereaskedtoguaranteethattherelativefrequencyFn(A)waswithinc=0.01ofP[A].ThisproblemisdifferentonlyinthatwerequirePn(A)tobewithin10%ofP[A].AsinExample10.7,wecanapplyTheorem10.S(a)andwriteP[Pn(A)-P[A]>c]<P[A](1~P[A]).T/,CWecanensurethatPn(A)iswithin10%ofP(A]bychoosingc~]=(10.40)0.1P[A].Thisyields(1-P[A])100p[Pn(A)-p[A]>O.lp[A)<n,(0.1)2P[A]<riP[A]'(10.41)sinceP[A)<1.Thusthenumberoftrialsrequiredfortherelativefrequencytobewithinacertainpercentageofthetrueprobabilityisinverselyproportionaltothatprobability.Inthefollovvingexample,v.reobtair1anestimateandaconfidenceintenral,butvverr1ustdeterminethecor1fidencecoefficientassociated\vitl1theestirnateandt11econfidenceinter\ral.===Example10.91==:::1Theorem10.S(b)givesrisetostatementswehearinthenews,suchas,Basedonasampleof1103potentialvoters,thepercentageofpeoplesupportingCandidateJonesis58%withanaccuracyofplusorminus3percentagepoints.[354CHAPTER10THESAMPLEMEANTheexperimentistoobserveavoteratrandomanddeterminewhetherthevotersupportsCandidateJones.WeassignthevalueX=1ifthevotersupportsCandidateJonesandX=0otherwise.TheprobabilitythatarandomvotersupportsJonesisE[X]=p.Inthiscase,thedataprovidesanestimateNln(X)=0.58asanestimateofp.Whatistheconfidencecoefficient1-acorrespondingtothisstatement?isaBernoulli(p)randomvariable,0.03,Theorem10.5(b)saysSincec=XE[X)=I>andVar[X)=p(l-p).Forp(l-r>)P[IJIJ,n(X)-r>I<o.03)>1-(n,0.03)2=1-a.(10.42)WeseethatI>(l-p)(10.43)cv.=n,(0.03)2·Keepinmindthatwehavegreatconfidenceinourresultwhenaissmall.Hovvever,sineewedon'tknowtheactuaIvaIueofp,wewouIdIiketohaveconfidenceinourresuItsregardlessoftheactuaIvaIueofconeIudethatp.0.25a<---2-n,(0.03)BecauseVar[X)=p(l-p)<0.25.We277.778ri(10.44)Thusforri=1103samples,a<0.25,orintermsoftheconfidencecoefficient,1-cv.>0.75.ThissaysthatourestimateofI>iswithin3percentagepointsofI>withaprobabilityofatleast1-a=0.75.IntervalEstimatesofModelParametersInTheorern10.5andExttrnples10.7and10.8)thesarnplerrieanNln(X)isapoiritestirnateoftherriodelpararneterE[X).Wehaveexarniriedhowtoguaranteethatthesamplerrieanisinaconfidericeintervalofsize2cvvithaconfidencecoefficientof1-cv..Inthiscase)thepointestirriateMn(X)isarandorri·variableandtheconfidenceintervalisadeterrriiriisticiriterval.Iriconfidericeintervalestirriation,weturntheconfidenceintervalinside011t.Aconfidenceintervalestirr1ateofapararnetercorisistsofarangeofvaluesandaprobabilitythatthepara.rrieterisinthestatedrarige.Iftheparameterofiriterestisr,theestirriateconsistsofrandomvariablesAandB)aridariurribera,withtliepropertyP[A<r<B)>1-cv..(10.45)Intliiscontext)B-Aiscalledthecorifiden,ceiritervalarid1-aistlieconfiden,cecoefficierit.SinceAandBarerandom·variables)thecon,fiden,cein,tervalisran,dorn.Theconfidencecoefficientisnovvtlieprobabilitythatthedeterrninisticmodelparameterrisintherandorriconfidericeinterval.Anaccurateestimateisreflectedinalov.rvalueofB-Aandahigh·valueof1-a.[10.5CONFIDENCEINTERVALS355Inrr1ostpracticalapplicationsofcor1fidence-intervalestirr1ation,theur1knovvnpararneterristheexpectedva.ll1eE(X]ofarandorr1variableXandthecor1fidenceir1tervalisderivedfromthesamplerr1ean,J\Jn(X),ofdatacollectedinn,indeper1denttrials.Int11iscontext,Equation(10.35)canberearrangedtosaythatforanyconstantc>0,.Var(X]p[l\!In(x)-c<E[X]<J\,{,n(x)+c]>12.(10.46)T/,CIncomparir1gEql1ations(10.45)arid(10.46),v.reseethatA=l\!In(X)-c,B=l\!In(X)+c,(10.47)andtheconfidenceintervalistherandominterval[1'.Jn(X)-c,Mn(X)+c].JustasinTheorern10.5,theconfider1cecoefficientisstill1-a,wherea=Var(X]/(ric2).Equation(10.46)indica.testhateveryconfidenceintervalestirr1ateisacomprorr1isebetweenthegoalsofachie·vir1ganarrowcor1fidenceir1tervalandahighconfidencecoefficier1t.Giver1an}'setofdata,itisalv.rayspossiblesirr1l1ltaneol1slytoincreaseboththecor1fider1cecoefficientandthesizeoftheconfidenceintervalortodecrea,setherri.ItisaJsopossibletocollectrnoredata(increaseriinEquation(10.46))andirnpro·vebot11accl1racymeasures.Thenurr1beroftrialsnecessar}'toac11ievespecifiedqua.lit}'levelsdependsonpriorknowledgeoft11eprobabilityrr1odel.Int11efollov.ringexarr1ple,thepriorkno\vledgecor1sistsoftheexpectedvalueandstandardde,riationoft11erneasurerr1enterror.Example10.10SupposeX,iistheithindependentmeasurementofthelength(incm)ofaboardwhoseactuallengthisbcm.EachmeasurementX,ihastheform(10.48)wherethemeasurementerrorZ,iisarandomvariablewithexpectedvaluezeroandstandarddeviation(J"z=1cm.Sinceeachmeasurementisfairlyinaccurate,wewouldliketouseMn(X)togetanaccurateconfidenceintervalestimateoftheexactboardlength.Howmanymeasurementsareneededforaconfidenceintervalestimateofboflength2c=0.2cmtohaveconfidencecoefficient1-a=0.99?SinceE(X,i]=bandVar(Xi]=Var[Z]=1,Equation(10.46)statesP(l\lfn(X)-0.1<b<1100Mn(X)+0.1]>1-()2=1.T/,0.1T/,(10.49)Therefore,P[lVIn(X)-0.1<b<J\!fn(X)+0.1]>0.99if100/n,<0.01.Thisimpliesweneedtomaken,>10,000measurements.WenotethatitisquitepossiblethatP[J\!fn(X)-0.1<b<1'.fn(X)+0.1]ismuchlessthan0.01.However,withoutknowingmoreabouttheprobabilitymodeloftherandomerrorsZ,i,weneed10,000measurementstoachievethedesiredconfidence.[356CHAPTER10THESAMPLEMEANItisoftenassurr1edthatthesarr1plerneanJ\!fn(X)isaGal1ssianrandom·variable,eitherbecat1seeac11trialproducesasarnpleofaGaussianrar1dornvariableorbecat1sethereiser1ol1ghdata,tojt1stifyacentrallirr1itt11eorerr1approxirnatiori.Ir1t11esirr1plestapplications,thevarianceo-1ofeac11datasarr1pleisknov.rnandt11eestirnateissyrnrr1etricaboutthesarnplerr1ean:A=J\!fn(X)-caridB=lYin(X)+c.Thisirr1pliesthefollov.rir1grelationshiparnongc,a,andn,,thenl1mberoftrialsusedtoobtainthesarnplemeari.Theorem10.14LetXbeaGa11,ssiar1,(µ,,a)rari,dornvariable.Acon,fidericein,tervalestirnateofµ,oftheforrn1Yfn(X)hasconfiden,cecoefficierit1-a,C</1,<1Yfn(X)+C'tuherea/2=Q(cvn/o-)=1-P[1"1n(X)-c<jJ.X<Mn(X)+c]=P[p,xc<1"1n(X)<jJ.X+c]=P[-c<Mn(X)-p,x<c].-SinceJ\1n(X)-jJ,istheGaussian(O,o-Iv'n)randomvariable,-cJi.lln(X)-JJ.cP[M11(X)-c)randomvariable.functionMN=bernoullitraces(n,m,p);x=reshape(bernoullirv(p,m*n),n,m);nn=(1:n)'*ones(1,m);MN=cumsum(x)./nn;stderr=sqrt(p*(1-p))./sqrt((1:n)');plot(1:n,0.5+stderr,...1:n,0.5-stderr,1:n,MN);Inbernoullitraces,eachcolumnofxisaninstanceofarandomvectorXwithiidBernoulli(rJ)components.Similarly,eachcolumnofMNisaninstanceofthevectorM(X).TheoutputgraphseachcolumnofMNasafunctionofthenumberoftrialsri.Inaddition,wecalculatethestandarderror.jek,andoverlaygraphsofp-.jek,andp+.jek,.Equation(10.24)saysthatateachstepk,weshouldexpecttoseeroughlytwo-thirdsofthesamplemeantracesintherange(10.58)Asamplegraphofbernoullitraces(50,40,0.5)isshowninFigure10.2.Thefigureshowshowatanygivenstep,approximatelytwothirdsofthesamplemeantracesarewithinonestandarderroroftheexpectedvalue.---Quiz10.6--Generate777,=1000traces(eachoflengthn,=100)ofthesarnplemeanofaBernotllli(rJ)randomvariable.Ateachstepk,calcl1lateJi.Ilkandthenumberof[360CHAPTER10THESAMPLEMEAN0.5QL..&..L-..L..L--t.;...._......L..~~....L..-~---1~~---L.~~....L...-~~.L...-~----L~~......L..~---'05I0l520253035404550nFigure10.2Sampleoutputofbernoullitraces.m,includingthedetern1inisticstandarderrorgraphs.Thegraphshovvsho\vatanygivenstep,aboutt\vothirdsofthesan1plemeansare\vithinonestandarderrorofthetruemean.traces,suc11thatMkis-vvithinor1estandarderroroft11eexpectedvaluep.GraphTk=Nfk/rn,asafunctionofk.Explain}'Ourresults.Fv,rtherReadin,g:[Dl1r94]containsconcise,rigorouspresentationsandproofsoft11elav.rsoflargerurrnbers.['vVSOl]co·verspararr1eterestirnationforbothscalarandvectorrandorr1variablesa.ndstoc11asticprocesses.ProblemsDifficulty:Easy10.1.1X1,...,Xnisaniidsequenceofexponentialrandomvariables,each\vithexpectedvalue5.(a)WhatisVar[J\1g(X)],theva1ianceofthesamplemeanbasedonninetrials?(b)\iVhatisJ:>[X1>7],theprobabilitythatoneoutcomeexceeds7?(c)UsethecentrallimittheoremtoestimateP[l\1g(X)>7],theprobabilitythatthesamplemeanofninetrialsexceeds7.10.1.2X1,...,XnareindependentuniformrandomvariableswithexpectedvalueJ.lX=7andvariance·var[X]=3.(a)WhatisthePDFofX1?(b)WhatisVar[JVf16(X)],thevarianceofthesamplemeanbasedon16trials?(c)\i\fhatisP[X1>9],theprobabilitythatoneoutcomeexceeds9?(d)WouldyouexpectP[M15(X)>9]tobebiggerorsmallerthanP[X1>9]?ModerateDifficulttExpertsOnlyTocheckyotu·intuition,usethecentrallimittheoremtoestimatel")[i\115(X)>9].10.1.3Xisauniform(0,1)randomvariable.y=X2.\i\fhatisthestandarderroroftheestimateofµ,ybasedon50independentsamplesofX?10.1.4LetX1,X2,...denoteasequenceofindependentsamplesofarandomvariableXwithvarianceVar[X].'vVedefineanewrandomsequenceY1,Y2,...asY1=X1-X2andYn=X2n-1-X2n·(a)FindE[X1.]andVar[Y'.;.,,].(b)FindtheexpectedvalueandvarianceofJ\1n(Y).10.2.1Theweightofarandomlychosen~!.IaineblackbearhasexpectedvalueE[W]=500poundsandstandarddeviationaw=100pounds.UsetheChebyshevinequalitytoupperboundtheprobabilitythatthe\veightofarandomlychosenbear[PROBLEMSismorethan200poundsfromtheexpectedvalueofthe'veight.10.2.2ForanarbitraryrandomvariableX,usetheChebyshevinequalitytoshowthattheprobabilitythatXismorethankstandarddeviationsfromitsexpectedvalueE[X]satisfiesP[IX-E[X]I>ka]<:2.ForaGaussianrandomvariableY,usethe(-)functiontocalculatetheprobabilitythatYisinorethankstandarddeviationsfromitsexpectedvalueE[Y].ComparetheresulttotheupperboundbasedontheChebyshevinequality.10.2.3Elevatorsarriverandomlyatthegroundfloorofanofficebuilding.Becauseofalargecrowd,apersonwillwaitfortimeWinordertoboardthethirdarrivingelevator.LetX1denotethetime(inseconds)untilthefirstelevatorarrivesandletxidenotethetimebetweenthearrivalofelevatori-1andi.SupposeX1,X2,X3areindependentuniform(0,30)rando1nvariables.FindupperboundstotheprobabilityWexceeds75secondsusing36110.2.6•UsetheChernoffboundtoshowthattheGaussian(0,1)randomvariableZsatisfiesForc=1,2,3,4,5,useTable4.2andTable4.3tocomparetheChernoffboundtothetruevalue:P[Z>c]=Q(c).10.2.7·usetheChernoffboundtosho'vforaGaussian(µ,,a)rando1nvariableXthatp[X>c]<e-(c-µ,)2/2cr2.Hint:ApplytheresultofProblem10.2.6.10.2.8LetJ{beaPoissonrandomvariablewithexpectedvaluea.·usetheChernoffboundtofindanupperboundtoP[K>c].For'vhatvaluesofcdoweobtainthetrivialupperboundP[K>c]<1?10.2.9•Inasubwaystation,thereareex-(a)thel\/Iarkovinequality,actlyenoughcustomersontheplatformtofillthreetrains.Thearrivaltimeofthen,thtrainisX1+···+Xn'vhereX1,X2,...areiidexponentialrandomvariableswithE[Xi]=2minutes.LetvVequalthetimerequiredtoservethewaitingcustomers.FindP[W>20].(b)theChebyshevinequality,10.2.10(c)theChernoffbound.10.2.4LetXequalthear1ivaltimeofthethirdelevatorinProblem10.2.3.FindtheexactvalueofP[W>75].Compareyourans,vertotheupperboundsderivedinProblem10.2.3.10.2.5Inagamewithtwodice,theeventsnakeeyesreferstobothsix-sideddiceshowingonespot.LetRdenotethenumberofdicerollsneededtoobservethethirdoccurrenceofsnakeeyes.Find(a)theupperboundtoP[R>250]basedonthel\/Iarkovinequality,(b)theupperboundtoP[R>250]basedontheChebyshevinequality,(c)theexactvalueofP[R>250].LetX1,...,X11beindependentsamplesofarandomvariableX.UsetheChernoffboundtoshowthatJV!n(X)=(X1+···+Xn)/n,satisfiesP[M,,(X)>c]<(':;'J~e-'0LetX1,X2,...denoteaniidsequenceofrandomvariables,each'vithexpectedvalue75andstandarddeviation15.(a)Ho'vmanysamplesn,doweneedtoguaranteethatthesamplemeanMn(X)isbetween74and76withprobability0.99?(b)IfeachXihasaGaussiandistribution,ho'vmanysamplesn,'wouldweneedtoguaranteeJV!111(X)isbetween74and76withprobability0.99?[362CHAPTER10THESAMPLEMEAN10.3.2LetXAbetheindicatorrandomvariableforeventA\VithprobabilityP[A]=0.8.LetPn(A)denotetherelativefrequencyofeventAinn,independenttrials.Hint:SupposencookieshavebeenmadesuchthatNkcookieshavekchips.YouarejustoneoftheI:~0kNkchipsusedinthen,cookies.(a)FindE[XA)andVar[XA).10.3.6Inthisproblem,\VedevelopaweaklawoflargenumbersforacorrelatedsequenceX1,X21•••ofidenticalrandomvariables.Inparticular,eachXihasexpectedvalueE[Xi]=µ,andtherandomsequencehascovariancefunction(b)WhatisVar[P11(A)]?(c)UsetheChebyshevinequalitytofindtheconfidencecoefficient1-asuchthatP1oo(A)is'vithin0.1ofP[A].Inother\vords,findasuchthatP[P1oo(A)-P[A]1-a.(d)UsetheChebyshevinequalitytofindouthowmanysa1nplesriarenecessarytohaveFn(A)\Vithin0.1ofP[A]withconfidencecoefficient0.95.Inotherwords,findnsuchthatwhereaisaconstantsuchthatlal<1.Forthiscorrelatedrandomsequence,\Vecandefinethesamplemeanofnsa1nplesasJVlnX1+···+Xn=--n,---(a)·useTheorem9.2toshowthat10.3.3X1,X21•••isasequenceofiidBernoulli(1/2)randomvariables.ConsidertherandomsequenceYn=X1+···+Xn.(a)Whatislimn-+ooP[IY2n-rij<vTn72J?(b)WhatdoestheweaklawoflargenumberssayaboutY2n?10.3.4Incommunicationsystems,theerrorprobabilityP[E]maybedifficulttocalculate;ho,veveritmaybeeasytoderiveanupperboundoftheformP[E]<E.Inthiscase,wemaystill\VanttoestimateP[E]usingtherelativefrequencyFn(E)ofEinn,trials.Inthiscase,showthatP[Fn(E)-P[E]>c]E<nc2.10.3.5Afactorymanufactureschocolatechipcookiesonanassemblyline.Eachcookieissprinkled\vithJ{chipsfromaverylargevatofchips,\vhereKisPoissonwithE[K]=10,independentofthenumberonanyothercookie.Imagineyouareachipinthevatandyouaresprinkledontoacookie.LetJdenotethenumberofchips(includingyou)inyourcookie.\i\fhatisthePMFof.J?Var[X1+···Xn]<n,a2(1+a)l-a.(b)UsetheChebyshevinequalitytoshowthatforanyc>0,P[INin-µI>cl<a2(1+a)()2n,l-ac(c)·usepart(b)tosho'vthatforanyc·>0,limI=>[IMn-p,I>c]=O.n-+oo10.3.7IntheGaussianMovieDataBase(GMDB),reviewerslikeyouratemovie.swithGaussianscores.Inparticular,thefirstpersontorateamovieassignsaGaussian(q,1)revie'vscoreX1,whereroisthetrue"quality1'ofthemovie.Afternrevie,vs,amovie1sratingisRn=I::1,1Xi/n,.Strangelyenough,intheGMDB,revie\versareinfiuencedbypriorrevie,vs;ifafterri-lreviewsamovieisratedRn-I=r,then,threviewn,willratethemovieXn,aGaussian(r,1)randomvariable,conditionallyindependentofX1,...,Xn-IgivenRn-1=r.(a)FindE[Rn]·[PROBLEMS(b)Findthel=>DFfRn(r).Hint:Youmayhaveunresolvedparametersinthisanswer.(c)FindVar[Rn].Hint:FindE[R?i,IRn-1].(d)Interpretyourresultsasn,--+oo?Doesthela'voflargenumbersapplyhere?10.4.1WhenXisGaussian,verifyEquation(10.24),whichstatesthatthesamplemeanis'vithinonestandarderroroftheexpectedvalue\vithprobability0.68.10.4.2SupposethesequenceofestimatesRnisbiasedbutasy1nptoticallyunbiased.Iflimn-tooVar[Rn]=0,isth.esequencekiconsistent?363Hint:LetAi={IMi(n)-p,il>c}andapplytheunionbound(seeProblem1.3.11)toupperboundP[A1UA2U···UAk]·ThenapplytheChebyshevinequality.10.4.5GiventheiidsamplesX1,X2,...ofX,definethesequenceY1,Y2,...by(x2k-1-+X2k-1+X2k)22(xX2k-12_+_X2k)2k-____2A10.4.3i\.nexperimentaltrialproducesrandomvariablesX1andX2vvithcorrelationr=E[X1X2].Toestimater,\Veperformn,independenttrialsandformtheestimatewhereX1(i)andX2(i)aresamplesofX1andX2ontriali.Sho\vthatifVar[X1X2]isfinite,thenR1,R2,...isanunbiased,consistentsequenceofestimatesofr.10.4.4Anexperimentproducesrando1n/XkJvvithexpectedvectorX=[X1valueµx=[11,1p,k]'.TheithcomponentofXhasvariance·var[Xi]=nLX(i).NotethateachYkisanexampleofv;,anestimateofthevarianceofXusingtvvosamples,giveninTheorem10.13.ShowthatifE[Xk]<oofork=1,2,3,4,thenthesamplemeanMn(Y)isaconsistent,unbiasedestimateofVar[X].10.4.6AnexperimentproducesaGaussianrandomvectorX=[X1Xk]'withE[X]=0andcorrelationmatrixR=E[XX'].ToestimateR,weperformn,independenttrials,yieldingtheiidsamplevectorsX(l),X(2),...,X(n,),andformthesamplecorrelationmatrix1R(n,)=nnLrn=l(a)ShowR(n)isunbiasedbyshowingE[R(n)]=R.(b)Sho'vthatthesequenceofestimatesR(n,)isconsistentbyshowingthateveryele1nent~j(ri)ofthematrixRconvergestoRij.Thatis,showthatforanyc>0,i=llimP[max(a)Sho'vM(n,)isunbiasedbyshowingE[M(ri)]=µx.(b)ShowthatthesequenceofestimatesMnisconsistentbyshowingthatforanyconstantc>0,IMj(n);=l,...,klimP[maxn-tooX(m,)X'(m).-p,jI>c]=0.n-too?,,J~j-~j>c]=O.Hint:ExtendthetechniqueusedinProblem10.4.4.·vou\villneedtousetheresultofProblem7.6.4tosho'vthatVar[XiXj]isfinite.10.5.1X1,...,Xnaren,independentidenticallydistributedsamplesofrandom[364CHAPTER10THESAMPLEMEANvariableXwithPMFPx(x)=0.10.90x=0,x=1,other,vise.(a)HowisE[X)relatedtoPx(l)?(b)UseChebyshev'sinequalitytofindtheconfidencelevelasuchthatMgo(X),theestimatebasedon90observations,is\Vithin0.05ofPx(l).Inotherwords,findasuchthatP[INfgo(X)-Px(l)I10.6.1GraphonetraceofthesamplemeanofthePoisson(1)randomvariable.Calculate(usingacentrallimittheoremapproximation)andgraphthecorresponding0.9confidenceintervalestimate.10.6.2XistheBernoulli(1/2)randomvariable.ThesamplemeanMn(X)hasstandarderror·var[X]>0.05)(c)UseChebyshev'sinequalitytofindouthowinanysamplesn,arenecessarytohaveMn(X)\Vithin0.03ofPx(l)withconfidencelevel0.1.Inother\Vords,findnsuchthatP[IJV!n(X)-Px(1)1(b)Ho\vmanytrialsriareneededsothattheprobabilityPn(A)differsfromP[A)bymorethan0.1%islessthan0.01?rel="nofollow">0.03)<0.1.10.5.2XisaBernoullirandomvariable\Vithunknownsuccessprobabilityp.Using100independentsamplesofX,findaconfidenceintervalestimateofpwithconfidencecoefficient0.99.IfM1oo(X)=0.06,\vhatisourintervalestimate?10.5.3Innindependentexperimentaltrials,therelativefrequencyofevent_4isFn(A).HowlargeshouldnbetoensurethattheconfidenceintervalestimateFn(A)-0.05<P[A)<Fn(A)+0.05hasconfidencecoefficient0.9?10.5.4Whenweperformanexperiment,eventAoccurswithprobabilityP[-4)=0.01.Inthisproblem,weestimateP[-4)usingFn(-4),therelativefrequencyofAoverri,independenttrials.(a)Howmanytrialsn,areneededsothattheintervalestimateFn(A)-0.001[-4)<Fn(A)+0.001hasconfidencecoefficient1-a=0.99?ri,1-2Fn·TheprobabilitythatMn(X)iswithinonestandarderrorofpisPnl1<l1lfn(X)<21+yn,1=.P[21-2yr;,·2·usethebinornialcdffunctiontocalculatetheexactprobabilityPnasafunctionofn,.Whatisthesourceoftheunusualsawtoothpattern?CompareyourresultstothesolutionofQuiz10.6.10.6.3Recallthatanexponential(,\)randomvariableXhasE[X)=1/,\,Var[X)=1/,\2.Thus,toestimate,\fromn,independentsamplesXi,...,Xn,eitherofthefollowingtechniquesshouldwork.(a)CalculatethesamplemeanMn(X)andformtheestimate.\=1/JVfn(X).(b)Calculatetheunbiasedvarianceestimatev;~(X)ofTheorem10.13andformtheestimate5-=1/Jv~X).(A·uselVIATLABtosimulatethecalculation,\and,\form,=1000experimentaltrialstodeterminewhichestimateisbetter.10.6.4XisIO-dimensionalGaussian(0,I)randomvector.SinceE[X)=0,Rx=Cx=I.WewillusethemethodofProblem10.4.6andestimateRxusingthe[PROBLEMSsamplecorrelationmatrix1R(n,)=-n,L10.6.5Intermsofparametera,randomvariableXhasCDFnX(m,)X'(m,).1n=lxFx(x)={O11-..----........[x-(c.L-2)]2FornE{10,100,1000,10,000},constructal\IIATLABsimulationtoestimatep[max&j-lijrel="nofollow">o.os].i,J365::i;>a-1.(a)ShowthatE[X]=abyshowingthatE[X-(a-2)]=2.(b)Generatem=100tracesofthesamplemeanJVJ11(X)oflengthn,=1000.DoyouobserveconvergenceofthesamplemeantoE[X]=a?[HypothesisTestingSorneofthemostirnporta.ntapplicationsofprobabilitytheoryinvol-vereasoningint11epresenceofuncertair1ty.Intheseapplications,v.reanalyzet11eobservatiorisofanexperirnentinordertorr1akeadecision.Wher1thedecisionisbasedonthepropertiesofrar1dornvariables,thereasoningisreferredtoasstatisticalinferen,ce.InChapter10,-vveintroducedtwotypesofstatisticalinferenceformodelpararr1eters:poir1testimationandconfidence-intervalestirnation.Int11ischapter,-vveintroducet-vvornorecategoriesofinference:significancetestir1gand11ypothesistesting.Statisticalinferer1ceisabroad,deepsubject-vvithaverylargebodyoftheoreticalkr10\vledgeandpracticaltechniques.Ithasitsownexter1si·veliteratureanda·vastcollectionofpracticaltechniques,man:yrofthernvaluablesecretsofcornpaniesandgovernments.Thischapter,Chapter10,andChapter12providear1introductoryviewoft11est1bjectofstatistica.1inference.Ouraimistoindicatetoreadersho-vvthefundamentalsofprobttbilitytheorypresentedint11eearlierchapterscanbeusedtorr1akeaccuratedecisior1sinthepresenceofuncertaint:yr.Likeprobabilitytheory,t11etheoryofstatisticalir1ferencereferstoanexperirr1entcor1Sistir1gofaprocedtu·eandobservations.Inallstatisticalir1ferencerr1ethods,thereisalsoasetofpossibledecisionsandarr1eansofmeasl1ringtheacct1racyofadecision.Astatisticalir1ferencerr1ethodassigr1sadecisiontoeachpossibleoutcomeoftheexperirr1er1t.Therefore,astatisticalinferencerr1etl1odconsistsofthreesteps:Perforrnanexperiment,observeanoutcorne,stateadecisiori.T11eassignrr1entofdecisionstooutcornesisbasedonprobabilitytheory.T11eaimoftheassignrr1entistoachie·vethehighestpossibleaccurac:yr.Thisc11aptercor1tainsbriefintroductionstot\vocategoriesofstatisticalinference.•SignificanceTestingDecisionAcceptorrejectthe11ypothesisthattheobservatior1sresultfrorr1acertainproba.bilityrr1odelH.0AccuracyMeasureProbabilityofrejectingthe11ypothesiswhenitistrue366[11.1SIGNIFICANCETESTING367•HypothesisTestir1gDecisionTheobservatior1sresultfrornoneofMhypotheticalprobabilitymodels:H.o,H1,...,HJVJ-I·AccuracyMeasureProbabilitythatt:hedecisionisHiw11er1thetruerr1odelisH.1fori,j=0,1,...,M-1.Inthefollovvingexarr1ple,-vveseethatforthesameexperirr1ent,eachtestingrnethodaddressesaparticularkindofquestionur1derparticl1larassurnptions.==;;;..ExampIe11.l==:::::aSupposeX1,...,Xnareiidsamplesofanexponential(,\)randomvariableXwithunknownparameterA.UsingtheobservationsX1,...,Xn.eachofthestatisticalinferencemethodscananswerquestionsregardingtheunknownA.Foreachofthemethods,westatetheunderlyingassumptionsofthemethodandaquestionthatcanbeaddressedbythemethod.•SignificanceTestAssuming,\isaconstant,shouldweacceptorrejectthehypothesisthat,\=3.5?•HypothesisTestAssuming,\isaconstant,does,\equal2.5,3.5,or4.5?Todecideeitheroftheql1estionsinExample11.l,"''eha\retostateinadvance"''hich\raluesofX1,...,Xnproduceeachpossibledecision.Forasigr1ificancetest,thedecisionrntlstbeeitheracceptorreject.Fortheh:ypothesistest,t11edecisionrnustbeoneofthenumbers2.5,3.5,or4.5.11.1SignificanceTestingAhypothesisisacandidateprobabilit}'model.AsignificancetestspecifiesarejectionsetRconsistingoflov.r-probabilit}'Ol1tcornesofanexperirr1ent.Ifanobservationisinthesetoflov.r-probabilit}'outcorr1es,thedecisionis"rejectthe11}rpothesis."Thesignificancelevel,definedastheprobabilit}'ofanoutcomeintherejectionset,deterrninestherejectior1set.Asignificancetestbegins-vvitht11e11ypotl1esis,H0,thatacertainprobabilityrr1odeldescribest11eobservationsofanexperirnent.Thequestior1addressedb}'thetesthastvvopossibleansV\rers:acceptt11ehypothesisorrejectit.Thesign,ifica/ncelevelofthetestisdefinedast11eprobabilityofrejectingthe11ypothesisifitistrue.ThetestdividesS,thesarnplespaceoftheexperirr1er1t,ir1toapartitionconsistingofanacceptancesetAandarejectionsetR=Ac.Iftheobserva.tionsEA,vveacceptH0.IfsER,\Verejectthehypothesis.Thereforethesignificancelevelisa=P[sER].(11.1)Todesignasignificar1cetest,\Vestartwit11avall1eofaandther1determineasetRthatsatisfiesEquation(11.l).[368CHAPTER11HYPOTHESISTESTINGInrriariyapplications)Hoisreferredtoasthe'nullhypothesis.Intheseapplications)thereisakriownprobabilitymodelforanexperirnent.ThentheconditionsoftlieexperimentchangearidasignificancetestisperforrnedtodeterrriinewhetliertheorigirialprobabilitJrrr1odelrerriainsvalid.Thenullh}rpothesisstatesthatthechangesintheexperirrienthavenoeffectoritheprobabilityrnodel.Anexarripleistheeffectofadietpillontlievveightofpeoplevvliotestthepill.TliefollovvingexarripleappliestoInterriettweeting.-=Example11.2SupposethatonThursdaysbetween9:00and9:30atnight,thenumberoftweetsJ\TisthePoisson(107)randomvariablewithexpectedvalue107.NextThursday,thePresidentwilldeliveraspeechat9:00thatwillbebroadcastbyallradioandtelevisionnetvvorks.Thenullhypothesis,H'0,isthatthespeechdoesnotaffecttheprobabilitymodeloftweets.Inotherwords,H0statesthatonthenightofthespeech,NisaPoissonrandomvariablewithexpectedvalue107.DesignasignificancetestforhypothesisH'0atasignificancelevelofa=0.05.Theexperimentinvolvescountingthenumberoftweets,JV,between9:00and9:30onthenightofthespeech.Todesignthetest,weneedtospecifyarejectionset,R,suchthatP(NER]=0.05.TherearemanysetsRthatmeetthiscondition.WedonotknowwhetherthePresident'sspeechwillincreasetweeting(bypeopledeprivedoftheirThursdayprograms)ordecreasetweeting(becausemanypeoplewhonormallytweetlistentothespeech).Therefore,wechooseRtobeasymmetricalset{n,:lri-107>c}.TheremainingtaskistochoosectosatisfyEquation(11.1).UnderhypothesisH'0-theprobabilitymodelisthePoisson(107)randomvariableE(N)=Va.r(N)=107.Thesignificancelevelis1>-c-O'Nl(11.2)SinceE(J\T)islarge,wecanusethecentrallimittheoremandapproximate(J\TE(N))/O'NbythestandardGaussianrandomvariableZsothat(11.3)Inthiscase,Inasignificancetest)twokindsoferrorsarepossible.StatisticiansrefertothemasTypeIerrorsaridTypeIIerrors'\viththefollo'\vingdefiriitioris:•TypeIErrorFalseRejectiori:RejectH0whenH0istrue.•TypeIIErrorFalseAcceptan,ce:AcceptH0vvhenH'0isfalse.[11.1SIGNIFICANCETESTING369Thehypothesisspecifiedir1asigr1ificancetestrr1akesitpossibletocalct1latetheprobabilityofaTypeIerror,a=P[sER].Intheabser1ceofaprobabilityrr1odelforthecondition"H.0false,"thereisnowaytocalculatetheprobabilityofaTypeIIerror.Abi'naryhypothes'istest,describedinSection11.2,ir1cludesanalter'nativehypothesisH1.Thenitispossibletouset11eprobabilit}rrnodelgivenbyH1tocalculatetheprobabilityofaTypeIIerror,vvhichisP[sEAIH1].Althoug11asigr1ificancetestdoesnotspecifyacompleteprobabilityrnodelasanalterr1ativetothenullhypothesis,thenatureoftheexperirr1entinfluencesthechoiceoftherejectionset,R.InExample11.2,weimplicitl}'assurr1ethatt11ealternativetothenullhypothesisisaprobabilityrr1odelwithanexpectedvah1ethatiseitherhig11ert11an107orlo\verthan107.Inthefollowingexarnple,t11ealternativeisarnodelwithanexpected\raluethatislovverthantheoriginalexpected\ralue.---Example11.3:-------Beforereleasingadietpilltothepublic,adrugcompanyrunsatestonagroupof64people.Beforetestingthepill,theprobabilitymodelfortheweightofthepeoplemeasuredinpounds,istheGaussian(190,24)randomvariableW.Designatestbasedonthesamplemeanoftheweightofthepopulationtodeterminewhetherthepillhasasignificanteffect.Thesignificancelevelisa=0.01.Underthenullhypothesis,H·0,theprobabilitymodelafterthepeopletakethedietpill,isaGaussian(190,24),thesameasbeforetakingthepill.Thesamplemean,M64(X),isaGaussianrandomvariablewithexpectedvalue190andstandarddeviation24/.J64=3.Todesignthesignificancetest,itisnecessarytofindRsuchthatP[J\f64(X)ER]=0.01.Ifwerejectthenullhypothesis,wewilldecidethatthepilliseffectiveandreleaseittothepublic.Inthisexample,wewanttoknowwhetherthepillhascausedpeopletoloseweight.Iftheygainweight,wecertainlydonotwanttodeclarethepilleffective.Therefore,wechoosetherejectionsetRtoconsistentirelyofweightsbelowtheoriginalexpectedvalue:R={Jlf64(X)<r0}.Wechooser0sothattheprobabilitythatwerejectthenullhypothesisis0.01:P[Jlf54(X)ER].=P[l\/!54(X)<ro]=(ro-190)=0.01.3(11.4)Since(-2.33)=Q(2.33)=0.01,itfollowsthat(r0-190)/3=-2.33,orr0=183.01.Thuswewillrejectthenullhypothesisanddecidethatthedietpilliseffectiveatsignificancelevel0.01ifthesamplemeanofthepopulationweightdropsto183.01poundsorless.Notet11edifferencebetvveentheS}rmrnetricalrejectionsetinExarr1ple11.2andtheor1e-sidedrejectionsetinExarr1ple11.3.Weselectedthesesetsonthebasisoftheapplicationoftheres11ltsoft11etest.Inthelanguageofstatisticalir1ferer1ce,theS}rrr1metricalsetispartofat1110-tailsigriifica'ncetest,andt11eone-sidedrejectionsetispartofaon,e-tailsign,ifican,cetest.[370CHAPTER11HYPOTHESISTESTING==--Quiz11.1--•UnderhypothesisHo,tlieinterarri·valtirnesbetweenphonecallsareindependentandidenticallydistributedexporiential(1)randorrivariables.Gi·venX,tliernaxirriumarnong15iridependentiriterarrivaltirnesarnplesX1,...,X15,designasignificancetestforliypothesisH0atalevelofa=0.01.11.2BinaryHypothesisTestingAbinaryh}rpothesistestcreatesapa.rtitiori{Ao,A1}foranexperirrient.·whenanoutcomeisinH0,thedecisionistoaccepth}rpothesisHo.OthervvisethedecisioriistoacceptH·1.Thequa.lit}'rneasureofatestisrelatedtotheprobabilityofafalsealarm(decideH1vvhenH.oistrue)andtheprobabilityofarriiss(decideHov.rhenH.1istrue.)Iriabinaryliypothesistest,therearetvvohypotheticalprobabilit}'rriodels,H.0andH1,andtvvopossibledecisions:acceptHoastlietruemodel,aridacceptH1.ThereisalsoaprobabilityrriodelforHoandH1,coriveyedbythenurribersP[H.o]andP[H1)=1-P[H.o].Tliesenurribersarereferredtoastheaprioriprobabil'itiesorpriorprobabilitiesofHoandH·1.The}'reflectthestateofknovvledgeabouttheprobabilityrriodelbeforeariOl1tcorrieisobserved.Tliecorripleteexperimentforabinar}'liypothesistestconsistsoftwosubexperiments.ThefirstsubexperimentchoosesaprobabilitymodelfrornsarriplespaceS'={H0,H1}.Theproba,bilityrriodelsH0andH1havethesarriesarriplespace,S.ThesecondsubexperimentproducesanobservationcorrespondingtoariOl1tcorrie,sES.Whentheobserw1tiorileadstoarandomvectorX,wecallXthedecision,statistic.Often,thedecisionstatisticissirnplyarandorrivariableX.WhenthedecisionstatisticXisdiscrete,tlieprobabilitymodelsareconditionalprobabilit}'rria,ssf\1nctionsPx1H0(x)a,ndPx1H1(x).·vvhenXisacontiriuol1Sraridornvector,theprobabilitymodelsareconditionalprobabilit}'densityft1nctionsf.XIHo(x)andfx1H1(x).Intheterrriiriologyofstatisticalinference,thesefunctionsa.rereferredtoaslikelihoodf11,n,ction,s.Forexarriple,fx1H0(x)istlielikelihoodofxgivenH.0.ThetestdesigndividesSintotvvosets,AoandA1=A0.IftheoutcornesEAo,thedecisionisacceptH.0.Otherwise,thedecisioriisacceptH·1.Theaccuracyrrieasl1reofthetestconsistsoftvvoerrorprobabilities.P[A1IH0)correspondstotheprobabilityofaTypeIerror.ItistheprobabilityofacceptingH.1vvhenH0isthetrueprobabilitymodel.Similarl}',P[AolH1]istheprobabilityofacceptingHovvhenH1istlietrueprobabilityrriodel.ItcorrespondstotlieprobabilityofaTypeIIerror.Oneelectricalengineeringapplicationofbinaryh}rpothesistestingrelatestoaradarsystem.Thetransrriittersendsot1tasigrial,ariditisthejobofthereceivertodecidev.rhetlieratargetispresent.Torriakethisdecision,thereceiverexaminesthereceivedsignaltodeterrninewlietheritcontairisareflected·versionofthetrans-[11.211(/')VJVJVJ~0.5o....~0....000.5PFAROCforcontinuousXFigure11.11371BINARYHYPOTHESISTESTING0.500.•••••••••••••••••..0.51pFAROCfordiscreteXContinuousanddiscreteexamplesofareceiveroperatingcurve(ROC).rriittedsigrial.ThehypotriesisH0corresporidstothesitl1atioriinvvhichthereisnotarget.H1correspondstothepresenceofatarget.Intheterminologyofradar,aTypeIerror(decidetargetpresentvvhenthereisriotarget)isreferredtoasafalsealarm,andaT}rpeIIerror(decidenotargetwherithereisatargetpresent)isreferredtoasarniss.Tliedesignofabiriaryhypothesistestrepresentsatrade-offbetv.reenthetv.roerrorprobabilities,PFA=P[A1IHo]andP1v1rss=P[AolH1].Tounderstandthetrade-off,corisiderariextrernedesigninwhicliAo=SconsistsoftheentiresarnplespacearidA1=0istlieerriptyset.Inthiscase,PFA=0andP1v1rss=1.NO\vletA1expandtoincludeanincreasingproportiorioftheoutcomesinS.AsA1expands,PFAiricreasesaridPMrssdecreases.Attlieotherextreme,Ao=0,whichirripliesPl\111ss=0.Intl1iscase,Ai=SaridPFA=1.AgraphrepreseritingthepossiblevaluesofPFAandP1vrrssisreferredtoasarecei:ueroperatin,gcv,rve(ROG).ExarnplesappearinFigure11.l.AreceiveroperatirigcurvedisplaysPMrssasafunctionofPFAforallpossibleAoandA1.ThegraphontheleftrepresentsprobabilityrriodelsvvithacontiritlOllSsarnplespaceS.Inthegraphoritlieright,SisadiscretesetandthereceiveroperatingcurveconsistsofacollectionofisolatedpointsinthePFA,P}.!J.ISSplane.Atthetopleftcornerofthegrapli,thepoint(0,1)corresporidstoAo=SaridA1=0.·vVhenvverrioveoneoutcorriefrorriA0toA1,"\Verriovetothenextpointonthecurve.]\/Io,ririgdown"\vardalongthecurvecorrespondstotakirigmoreoutcomesfrorriA0andputtingtheminA1untilvvearri\reatthelowerrightcorrier(1,0),"\vherealltheot1tcorriesareinA1.===Example11.4===ThenoisevoltageinaradardetectionsystemisaGaussian(0,1)randomvariable,JV.Whenatargetispresent,thereceivedsignalisX=v+Nvoltswithv>0.OtherwisethereceivedsignalisX=J\Tvolts.Periodically,thedetectorperformsabinaryhypothesistest,withH0asthehypothesisnotargetandH.1asthehypothesistargetpresent.TheacceptaneesetsforthetestareA0={X<:c0}andA1={X>xo}.Drawthereceiveroperatingcurvesoftheradarsystemforthethreetargetvoltagesv=0,1,2volts.[372CHAPTER110..................HYPOTHESISTESTING//~0.5/I..../~0-2/.·0.8MISSv=O·-MISSv=I'''MISSv=2FAI0;....,•~0.6~·Q.,0.2,.,,,i::::::;..._._,::..........,t.....:...,__~--------102(a)40.46.............•''•........................................0.20.4.-......v=Ov=I·v=2.............--·0.6.....0.8.......1(b)Figure11.2(a)Theprobabilityofanlissandtheprobabilityofafalsealarn1asafunctionofthethresholdxoforExample11.4.(b)1.,hecorrespondingreceiveroperatingcurveforthesystem.YfVeseethattheROCimprovesasvincreases.Toderiveareceiveroperatingcurve,itisnecessarytofindPMISSandPFAasfunctionsof::c0.Toperformthecalculations,weobservethatunderhypothesisH0,X=NisaGaussian(O>CJ)randomvariable.UnderhypothesisH1,X=v+NisaGaussian(v>CJ)randomvariable.Therefore,PM1ss=PFA=P[AolH1]=P[X<xolH1]=iP(xo-v)P[A1IH0]=P[X>xolHo]=1-iP(xo).(11.5)(11.6)Figure11.2(a)showsPM1ssandPFAasfunctionsof~Dofor'V=0,v=1,andv=2volts.NotethatthereisasinglecurveforPFAsincetheprobabilityofafalsealarmdoesnotdependonv.ThesamedataalsoappearsinthecorrespondingreceiveroperatingcurvesofFigure11.2(b).Whenv=0,thereceivedsignalisthesameregardlessofwhetherornotatargetispresent.Inthiscase,PMISS=1-PFA·Asvincreases,itiseasierforthedetectortodistinguishbetweenthetwotargets.WeseethattheROCimprovesasvincreases.Thatis,wecanchooseavalueofx0suchthatbothPMISSandPFAarelowerforv=2thanforv=1.Inapracticalbinar}'hypot11esistest,itisnecessarytoadoptonetest(aspecificAo)aridacorrespondingtrade-offbetv.reenPFAandP1v1ISS·Therearemtmyapproachestoselectir1gA0.Intheradarapplication,thecostofarniss(ignorir1gathreateningtarget)COl1ldbefarhigherthanthecostofafalsealarrr1(causir1gtheoperatortotakeanunnecessaryprecal1tion).Thissuggeststhattheradarsystemshouldoperatevvithalovv·valueof~Dotoproducealov.rPMrssever1t11ol1ghthis"''illproducearelati·vely11ighPFA·T11eremainderofthissectiondescribesfol1rrnet11odsofchoosir1gA0.[11.2BINARYHYPOTHESISTESTING373MaximumAposterioriProbability(MAP)Test==~ExampIe11.5,___.;;=::::::1Amodemtransmitsabinarysignaltoanothermodem.Basedonanoisymeasurement,thereceivingmodemmustchoosebetweenhypothesisHo(thetransmittersenta0)andhypothesisH.1(thetransmiitersenta1).Afalsealarmoccurswhena0issentbuta1isdetectedatthereceiver.Amissoccurswhena1issentbuta0isdetected.Forbothtypesoferror,thecostisthesame;onebitisdetectedincorrectly.Therriaxirnurriaposterioriprobabilit}'testrriinimizesPERR)t.hetotalprobabilityoferrorofabiriary1iypot1iesistest.Thelavvoftotalprobability)Theorem1.9,relatesPERRtotheaprioriprobabilitiesofHoaridH.1andtothetwoconditionalerrorprobabilities)PFA=P[A1IH0]andPMrss=P[AolH.1]:PERR=P[A1IH0]P[Ho]+P[AolH1]P[H1].(11.7)Whentlietvvot}rpesoferrorshavethesarriecost)asinExarriple11.5,minirnizingPERRisasensiblestrategy.Thefollovvingtheorernspecifiesthebinar}'h}rpothesistestthatprodl1cestherninim11rripossiblePER,R·MaximumAposterioriProbability(MAP)TestGiven,abin,aryhypothesis-testin,gexperirnerit1JJitho'utcornes,thefollcr1nin,gr'IJ,leleadstothelo'1vestpossiblevalueofPERR:sEAoifP[Hols]>P[H1ls];sEA1other'1vise.ProofTocreatethepartition{Ao,Ai},itisnecessarytoplaceeveryele1nentsESineitherAoorAi.ConsidertheeffectofaspecificvalueofsonthesuminEquation(11.7).Eitherswillcontributetothefirst(A1)orsecond(Ao)terminthesum.Byplacingeachsinthetermthathasthelowervalueforthespecificoutcomes,wecreateapartitionthatininimizestheentiresum.ThuswehavetherulesE_4oifP[slH1]P[Hi]<P[slHo]P[Ho];(11.8)ApplyingBayes'theorem(Theorem1.11),weseethattheleftsideoftheinequalityisP[H1Is]P[s]andtherightsideoftheinequalityisP[HoIs]P[s].ThereforetheinequalityisidenticaltoP[HoIs]P[s]>P[H1Is]P[s],'vhichisidenticaltotheinequalityinthetheoremstatement.NotethatP[R.ols]aridP[H11s]arereferredtoastlieaposterioriprobabilitiesofH0andH1.JustastheaprioriprobabilitiesP[H0]andP[H1]reflectourknov.rledgeofHoandH1priortoperforminganexperirnent,P[Hols]andP[H1ls]reflectOllrknowledgeafterobservings.Theorem11.lstatesthatinordertorniriirriizePERRitisnecessarytoaccepttliehypothesisv.riththe1iig1ieraposteriori

probability.Atestthatfollowsthisruleisarnax;irnv,rnaposterioriprobab'ility(M.AP)h}rpothesis[374CHAPTER11HYPOTHESISTESTINGtest.InSl1cl1atest,A0containsallot1tcorr1essforwhichP[H.01s]>P[H1ls],andA1contair1salloutcomessforw11ichP[H.1ls]>P[H.0ls].IfP[H.0ls]=P[H1ls],theassignmentofstoeitherA0orA1doesnotaffectPERR·InTheorern11.1,-vvearbitrarilyassigr1stoA0-vvhentheaposterioriprobabilitiesareeql1al.·vve-vvouldhavethesarr1eprobabilityoferrorif-vveassignstoA1foralloutcomesthatproduceeql1alaposterioriprobabilitiesorif"'iveassignsorr1eoutcornes-vvithequalaposterioriprobabilitiestoA0andotherstoA1.Equation(11.8)isanotherstaternentoftheMAPdecisionrule.Itcontainsthethreeprobabilityrnodelsthatareassurnedtobekr10-vvn:•Theaprioriprobabilitiesofthehypotheses:P[H0]andP[H.1],•Thelikelihoodfunctior1ofHo:P[slH.o],•Thelikelihoodfunctior1ofH1:P[slH.1].vVhentheoutcorr1esofar1experirr1entyieldarar1domvectorXasthedecisionstatistic,\Vecar1expressthe1!{APruleintermsofconditionalPMFsorPDFs.IfXisdiscrete,wetakeX=Xitobetheoutcorr1eoftheexperirr1ent.Ifthesarr1plespaceSoftheexperimentiscontinl10t1s,weinterprett11econditionalprobabilitiesbyassurr1ingthateachoutcomecorrespondstotherandorr1vectorXint11esmallvolurnex<X<x+dxvvithprobabilityfx(x)dx.Thusinterrrisoftherandorr1variableX,-vvehavethefollo-vvingversionoft11eMAPhypothesistest.---Theorem11.2--Forariexperirnen,tthatprod'/J,Cesaran,dorn'VectorX,theMAPhypothesistestisDiscrete:.PxlHo(x)>P[H.1]xEAoifPxlH1(x)-p[H.o];..fxlHo(x)Coritiri11,0'1J,S:xEAoiff·()>XIH1xxEA1other'uJise;xEA1other111ise.Intheseforrr1ulas,t11eratioofconditionalprobabilitiesisreferredtoasalikehhoodratio.Theforrr1t1lasstatethatinordertoperformabinaryh:ypot11esistest,-vveobservetheoutcorneofanexperirnent,calculatethelikelihoodratioontheleftsideoftheforrr1ula,andcorr1pareit\vithacor1stantontherightsideoftheforrnula.Wecan\rie-vvthelikelihoodratioast11eevidence,basedonanobser\ration,ir1fa,rorofH0.Ift11elikelihoodratioisgreaterthan1,H0isrr1orelikelythar1H1.Theratioofpriorprobabilities,ontherightside,istheevidence,priortoperforrr1ingtheexperirnent,infavorofH1.Therefore,Theorern11.2statesthatacceptir1gH0ist11ebetterdecisioniftheevidenceinfavorofH0,basedontheexperirr1ent,out\veighsthepriorevider1ceinfavorofacceptingH1.Inrr1anypracticalhypothesistests,incll1dingthefollo-vvingexample,itisconvenienttocomparethelogarithrnsofthetworatios.Example11.6Withprobabilityp,adigitalcommunicationssystemtransmitsa0.Ittransmitsa1withprobability1-p.ThereceivedsignaliseitherX=-v+J\Tvolts,ifthetransmitted[11.2(l-p)P[X<x"IH1]BINARYHYPOTHESISTESTING•375pP[X>xl~1](J-P)fxH1(X)0x·-vvDecisionregionsforExan1ple11.6.Figure11.3bitis0,orv+Nvolts,ifthetransmittedbitis1.Thevoltage±vistheinformationcomponentofthereceivedsignal,andN,aGaussian(0,o-)randomvariable,isthenoisecomponent.GiventhereceivedsignalX,whatistheminimumprobabilityoferrorrulefordecidingwhether0or1wassent?With0transmitted,XistheGaussian(-v,o-)randomvariable.With1transmitted,XistheGaussian(v,o-)randomvariable.WithHidenotingthehypothesisthatbitiwassent,thelikelihoodfunctionsare(11.9)SinceP[H0)=J>,thelikelihoodratiotestofTheorem11.2becomesxEA1otherwise.(11.10)Takingthelogarithmofbothsidesandsimplifyingyields::r;EA0.f1:i;2v(I>);1-p:i;EA1otherwise.(11.11)WhenI>=1/2,thethresholdx*=0andthedecisiondependsonlyonwhethertheevidenceinthereceivedsignalfavors0or1,asindicatedbythesignofx.Whenp=I=1/2,thepriorinformationshiftsthedecisionthreshold:i;*.Theshiftfavors1(:i;*<0)ifp<1/2.Theshiftfavors0(x*>0)ifI>>1/2.Theinfluenceofthepriorinformationalsodependsonthesignal-to-noisevoltageratio,2v/o-.Whentheratioisrelativelyhigh,theinformationinthereceivedsignalisreliableandthereceivedsignalhasrelativelymoreinfluencethanthepriorinformation(;x:*closerto0).When2v/oisrelativelylow,thepriorinformationhasrelativelymoreinfluence.InFigure11.3,thethreshold:i;*isthevalueofxforwhichthetwolikelihoodfunctions,eachmultipliedbyapriorprobability,areequal.Theprobabilityoferroristhesumoftheshadedareas.Comparedtoallotherdecisionrules,thethresholdx*producestheminimumpossiblePERR·---Example11.1--FindtheerrorprobabilityofthecommunicationssystemofExample11.6.[376CHAPTER11HYPOTHESISTESTINGApplyingEquation(11.7),wecanwritetheprobabilityofanerrorasPERR=r>P[X>::c*IHo]+(1-r>)P[X<x*IH1].GivenH0,XisGaussian(-v,a).Givenn·1,XisGaussian(v,a).Consequently,Q(x;*+v)+(l-);r"(x*-v)=pQ(-alr1_+-v)+(1-r>)·2v1a2v1-RERR=P[>'±'(}(11.12)(}ppv).-pa(11.13)Thisequationshowshowthepriorinformation,representedbyln[(1-r>)/r>],andthepowerofthenoiseinthereceivedsignal,representedbya,influencePERR·Example11.8Atacomputerdiskdrivefactory,themanufacturingfailurerateistheprobabilitythatarandomlychosennewdrivefailsthefirsttimeitispoweredup.Normally,theproductionofdrivesisveryreliable,withafailurerateq0=10-4.However,fromtimetotimethereisaproductionproblemthatcausesthefaiIureratetojumptoq1=10-1.LetHidenotethehypothesisthatthefailurerateisqi.Everymorning,aninspectorchoosesdrivesatrandomfromthepreviousday'sproductionandteststhem.Ifafailureoccurstoosoon,thecompanystopsproductionandchecksthecriticalpartoftheprocess.Productionproblemsoccuratrandomonceeverytendays,sothatP[H1]=0.1=1-P[H.0].BasedonN,thenumberofdrivestesteduptoandincludingthefirstfailure,designaMAPhypothesistest.CalculatetheconditionalerrorprobabilitiesPFAandPM1ssandthetotalerrorprobabilityPERR·Givenafailurerateofqi,Nisageometricrandomvariable(seeExample3.9)withexpectedvaluel/q,i.Thatis,P1v1Hi(n,)=qi(l-qi)n-lforn,=1,2,...andP1vlHi(ri)=0otherwise.Therefore,byTheorem11.2,theMAPdesignstatesn,E.PNIHo(n,)P[H1]>;PNIH1(n,)-P[Ho]Ao1fn,EA1otherwise(11.14)n,EA1otherwise.(11.15)Withsomealgebra,wefindthattheMAPdesignis1(q1P[H1))n,EA0ifri>n,*=1+nCJOln(P(Ho]i=~~);Substitutingq0=10-4,q1=10-1.P[H0]=0.9,andP[H.1]=0.1,weobtainn,*=45.8.Therefore,intheMAPhypothesistest,A0={n,>46}.Thisimpliesthattheinspectortestsatmost45drivesinordertoreachadecisionaboutthefailurerate.Ifthefirstfailureoccursbeforetest46,thecompanyassumesthatthefailurerateis10-1.Ifthefirst45drivespassthetest,thenN>46andthecompanyassumesthatthefailurerateis10-4.Theerrorprobabilitiesare:PFA=P[N<45IH.o]=FNIHo(45)=1-(1-10-4)45=0.0045,PMISS=p[N>45IH'1]=1-F1VIH1(45)=(1-10-1)45=0.0087.(11.16)(11.17)[11.2BINARYHYPOTHESISTESTING377ThetotalprobabilityoferrorisPERR=P[Ho]PFA+P(H1]PM1ss=0.0049.We\NillreturntoExample11.8vvhenweexarr1ir1eothertypesoftests.MinimumCostTestTheJVIAPtestirr1plicitlyassurr1esthatbothtypesoferrors(missandfalsealarm)areequallyserious.Asdiscussedir1connectionv.ritht:heradarapplicationearlierint11issection,thisisnott11ecaseinrr1an}'importantsituations.Consideranapplicationir1whichC=C10t1nitsisthecostofafalsealarrn(decideH1v.rhenH.0iscorrect)andC=C01t1nitsisthecostofamiss(decideH0whenH.1iscorrect).Inthissituationtheexpectedcostoftesterrorsis(11.18)JVIinirnizingE[CJisthegoa.loft11err1inimurncosthypotl1esistest.·\i\Tl1enthedecisionstatisticisarandorr1vectorX,wehavethefollowingt11eorerri.---==Theorem11.3MinimumCostBinaryHypothesisTestFora,riexperirnen,tthatpr·od'ucesaran,dornvectorX;thernin,irn'urncosthypothesistest'isDiscrete:>P[H.1]Co1()_p[H.]C;xEA1other'1vise;._fX---'-IH_··o_(x_)>P(H1]Co1.1fx1H(x)-P[Ho)C10'xEA1otherv1ise..Px1Ho(x)xEAoifpCon,tin,?J,O'tJ,S:xEAo1'XIH1x0101ProofThefunctiontobeminimized,Equation(11.18),isidenticaltothefunctiontobemini1nizedinthetv'Ii\.Phypothesistest,Equation(11.7),exceptthatP[H1]Co1appearsinplaceofP[H1]andP[H0]C10appearsinplaceofP[H0].ThustheoptimumhypothesistestisthetestinTheorem11.2,withP[H1]C01replacingP[H1]andP[H0]C10replacingP[Ho].Inthistestwenotethator1lytherelativecostCo1/C10ir1fiuencesthetest,riottheindividt1alcostsort11eunitsir1"''hichcostisrneasured.Aratio>1impliesthatrnissesarerr1orecostlyt11anfalsea.larrns.T11erefore,aratio>1expandsA1,theacceptancesetforH1,rr1akingithardertornissH1vvhenitiscorrect.Ontheotherhand,thesarneratiocor1tractsH.0andir1creasesthefalsea.larrnprobability,becauseafalsea.larrnislesscostlythanarniss.Example11.9ContinuingthediskdrivetestofExample11.8,thefactoryproduces1000diskdrivesperhourand10,000diskdrivesperday.Themanufacturersellseachdrivefor$100.[378CHAPTER11HYPOTHESISTESTINGHowever,eachdefectivedriveisreturnedtothefactoryandreplacedbyanewdrive.Thecostofreplacingadriveis$200,consistingof$100forthereplacementdriveandanadditional$100forshipping,customersupport,andclaimsprocessing.Furthernotethatremedyingaproductionproblemresultsin30minutesoflostproduction.BasedonthedecisionstatisticN,thenumberofdrivestesteduptoandincludingthefirstfailure,whatistheminimumcosttest?Basedonthegivenfacts,thecostC10ofafalsealarmis30minutes(5000drives)oflostproduction,orroughly$50,000.Ontheotherhand,thecostC01ofamissisthat10%ofthedailyproductionwillbereturnedforreplacement.For1000drivesreturnedat$200perdrive,theexpectedcostis$200,000.TheminimumcosttestisEA.fPNIHo(n,)>P[H1]Co1.10PNIH(11,)-P[Ho]C10'n,1n,EA1otherwise.(11.19)PerformingthesamesubstitutionsandsimplificationsasinExample11.8yields11,EAo.1f11,*>n,=1+1(q1P[Hi]Co1)nqoP[Ho]C10()ln1-go=58.92;n,EA1otherwise.(11.20)l-q1Therefore,intheminimumcosthypothesistest,A0={n,>59}.Aninspectortestsatmost58diskdrivestoreachadecisionregardingthestateofthefactory.If58drivespassthetest,thenA0={N>59},andthefailurerateisassumedtobe10-4.Theerrorprobabilitiesare:PFA=P[N<58IH.o]=FNIHo(58)=1-(1-10-4)58=0.0058,PMISS=p[N>59IH'1]=1-FNIH1(58)=(1-10-1)58=0.0022.(11.21)(11.22)Theaveragecost(indollars)ofthisruleisE[CMc]=P[Ho]PFAC10+P[H1]PM1ssC01=(0.9)(0.0058)(50,000)+(0.1)(0.0022)(200,000)=305.(11.23)Bycomparison,theMAPtest,whichminimizestheprobabilityofanerrorratherthantheexpectedcost,haserrorprobabilitiesPFA=0.0045andPM1ss=0.0087andtheexpectedcostE[CMAP]=(0.9)(0.0045)(50,000)+(0.1)(0.0087)(200,000)=376.50.(11.24)Theeffectofthehighcostofamisshasbeentoreducethemissprobabilityfrom0.0087to0.0022.However,thefalsealarmprobabilityrisesfrom0.0047intheMAPtestto0.0058intheminimumcosttest.Asavingsof$376.50-$305=$71.50maynotseemverylarge.ThereasonisthatboththeMAPtestandtheminimumcosttestworkverywell.Bycomparison,fora"notest"policythatskipstestingaltogether,eachdaythatthefailurerateisq1=0.1willresult,onaverage,in1000returneddrivesatanexpectedcostof$200,000.SincesuchdaysoccurwithprobabilityP[H1]=0.1,theexpectedcostofa"notest"policyis$20,000perday.[11.2BINARYHYPOTHESISTESTING379Neyman-PearsonTestGivenanobservation,t11el\/IAPtestrninirriizestheprobabilityofacceptingthe-vvronghypothesisaridtlierniniml1mcosttestminimizestliecostoferrors.However)theMAPtestrequirestliatwekno-vvtheaprioriprobabilitiesP(Hi)ofthecornpetingh:ypotheses,aridtlierriiriirnurricosttestrequiresthat\Vekno-vvinadditiontherelativecostsofthet\vot}rpesoferrors.Inrrianysituations,tliesecostsandaprioriprobabilitiesaredifficultore\renirnpossibletospecify.Irithiscase,analternativeapproach-vvouldbetospecifyatolerablelevelforeithertliefalsealarrnormissprobability.ThisideaisthebasisfortheNeyrnari-Pearsoritest.TheNeyman-PearsontestrninirnizesPMrsssubjecttothefalsealarrriprobabilitycoristraintPFA=a,\vliereaisaconstarittliatindicatesourtoleranceoffalsealarrns.BecausePFA=P(A1IH.0)aridP1vrrss=P[A0IH1]areconditionalprobabilities,thetestdoesnotrequirekno\x.rledgeoftheaprioriprobabilitiesP(H0]andP[H.1].·vvefirstdescribetheNeyrnan-PearsontestwhenthedecisionstatisticisacontiriousrandorrivectorX.---Theorem11.4Neyman-PearsonBinaryHypothesisTestBasedon,thedecision,statist'icX,aco'ntin,v,ov,sra,n,dorn1;ector,thedec'is'io'nrv.lethatrnin,irnizesPM1ss,s11,bjecttothecoristrain,tP1'"'A=a,isxf.XIHo(x)L(x)=j'()>'Yi.EAoifxEA1other'tlJ'ise,XIH1x1JJhere'"'!ischosen,sothatJL(x)dx=a.ProofUsingtheLagrangemultipliermethod,vvedefinetheLagrangemultiplier>.andthefunctionG=PM1ss+>.(PFA=L0=-a)fx1H,(x)dx+A(1-Lofx1Ho(x)r(fx1H1(x)-Afx1Ho(x))}Aodxdx-a)+>.(1-a).(11.25)Foragiven>.anda,weseethatGisminimizedifAoincludesallxsatisfyingfx1H1(x)-Afx1Ho(x)<0.(11.26)Notethat>.isfoundfromtheconstraintPFA=a.l\/Ioreover,vveobservethatEquation(11.25)implies>.>O;other,vise,fx1H0(x)-Afx1H1(x)>0forallxandAo=0,theemptyset,wouldminimizeG.Inthiscase,PFA=1,\vhich\vouldviolatetheconstraintthatPFA=a.Since>.>0,\Vecanrewritetheinequality(11.26)asL(x)>1/>.=I'·IntheradarsystemofExarnple11.4,thedecisionstatisticwasarandomvariableXandthereceiveropera.tingct1rves(ROCs)ofFigl1re11.2\Veregerieratedbyadjt1stingat11reslioldxothatspecifiedtliesetsAo={Xxo}.Exarnple11.4didriotqt1estionv.rlietherthisrt1lefindstliebestROC,tliatis,the[380CHAPTER11HYPOTHESISTESTINGbesttrade-offbetweenPMrssandPFA·TheNe}rrr1an-Pea,rsontestfindst11ebestROC.Foreachspecified·valueofPFA=a,t11eNeyrnar1-Pearsontestidentifiesthedecisionrulethatrr1inirnizesPMrss.IntheNeyman-Pearsor1test,anincreaseinrdecreasesP1vrrssbutincreasesPFA·\Vher1thedecisionstatisticXisacontir1uousrandomvector,wecanchoosersothatfalsealarmprobabilit}'isexactlycv..Thismaynotbepossiblev.rhenXisdiscrete.Ir1thediscreteCttSe,weha·vethefollowingversionoft11eNeyman-Pearsontest.DiscreteNeyman-PearsonTestTheorem11.5Basedorithedecision,statisticX,adiscreteran,dornvector,thedecision,T'alethatrn'iriirnizesPM1ss,s'abjecttotheco'nstrain,tPFA111herer.ifL(x)=PxlHo(x)p()xXIH1xEA1other111ise,rel="nofollow">1;isthelargestpossiblevalu,esv,chthatl:L(x)Example11.10ContinuingthediskdrivefactorytestofExample11.8,designaNeyman-PearsontestsuchthatthefalsealarmprobabilitysatisfiesPFA<cv.=0.01.Calculatetheresultingmissandfalsealarmprobabilities.TheNeyman-Pearsontestis11,.EAoifL('n)=PNIHo(11,)p()NIH177,rel="nofollow">ri77,EA1otherwise.(11.27)WeseefromEquation(11.14)thatthisisthesameastheMAPtestwithP(H1]/P[Ho]replacedbyr·Thus,justliketheMAPtest,theNeyman-Pearsontestmustbeathresholdtestoftheform'nEA0if'n>11,*;77,EA1otherwise.(11.28)Somealgebrawouldallowustofindthethresholdn,*intermsoftheparameterr·However,thisisunnecessary.Itissimplertochoose77,*directlysothatthetestmeetsthefalsealarmprobabilityconstraintPFA=P[N<11,*-llHo]=Ff'·llHo(n*-1)=1-(1-qo)n*-l<cv..(11.29)Thisimplies,*77<1+_ln_(_l.::_a_.)-Thus,wecanchoosen,*errorprobabilitiesare:1+ln(0.99)=101.49.lr1(0.9)=101andstillmeetthefalsealarm(11.30)probabilityconstraint.The<lOOIHo]=1-(1-10-4)100=0.00995,P[N>lOllH1]=(1-10-1)100=2.66.lo-5.PFA=P[NPM1ss=ln(l-qo)=(11.31)(11.32)[11.2BINARYHYPOTHESISTESTING381Weseethattoleratingaonepercentfalsealarmprobabilityeffectivelyreducestheprobabilityofamissto0(ontheorderofonemissper100years)butraisestheexpectedcosttoE(CNP]=(0.9)(0.01)(50,000)+(0.1)(2.66.10-5)(200,000)=$450.53.AlthoughtheNeyman-PearsontestminimizesneithertheoverallprobabilityofatesterrornortheexpectedcostE[C],itmayseempreferabletoboththeMAPtestandtheminimumcosttestbecausecustomerswilljudgethequalityofthediskdrivesandthereputationofthefactorybasedonthenumberofdefectivedrivesthatareshipped.Comparedtotheothertests,theNeyman-Pearsontestresultsinamuchlowermissprobabilityandfarfewerdefectivedrivesbeingshipped.However,itseemsfartooconservative,performing101testsbeforedecidingthatthefactoryisfunctioningcorrectly.MaximumLikelihoodTestSimilartotheNeyman-Pearsontest,thernai;irn'Urnlikelihood(ML)testisar1otherrr1ethodthatavoidsther1eedforaprioriprobabilities.UndertheJVILapproac11,foreachOl1tcomes-vvedecidetheh}rpot11esisHiforv.rhichP[slH,i:]islargest.T11eideabehindchoosingahypothesistomaxirnizet11eprobabilityoft11eobservatior1istoa·voidrr1akingassurnptionsaboutcostsaridaprioriprobabilitiesP[Hi].T11eresl1ltingdecisionrule,ca.lledthernaxirn11,rnl'ikelihood(ML)rl1le,car1be\A.Trittenrr1athematicallyas:Definition11.1MaximumLikelihoodDecisionRuleForabi'nar~yhypothesistestbasedon,theei;perirneritalO'tJ,tcornesES,thernax'irn?J,rnlikelihood(ML)decision,r11,leissEAoifP[slHo]>P[slH1];sEA1other'tuise.ComparingTheorem11.landDefinition11.1,weseet11atintheabsenceofinforrr1ationabol1ttheaprioriprobabilitiesP[Hi],"''eha,readoptedarr1axirr1urr1likelihooddecisionrl1lethatisthesameastheJVIAPr11lellndert11eassurr1ptiont11ath}rpothesesH.0andH1occl1r\vitheql1alprobability.Inessence,intheabsenceofaprioriinformation,t11eMLruleassumest11atallhypot11esesareequallylikel}'·Bycorr1paringthelikelihoodratiotoathresholdeql1alto1,theMLhypothesistestisneutralabout\vl1etherHohasahigherprobabilitythar1H.1orviceversa.vVhenthedecisionsta.tisticoftheexperirnentisarandornvectorX,\VecanexpresstheMLruleinterrnsofcor1ditionalPJVIFsorPDFs,justas\VedidfortheJVIAPrule.[382CHAPTER11HYPOTHESISTESTING==;;;;:Theorem11.fi___,;;=;;;;;iIfan,experirnen,tprodv,cesaran,dornvectorX,theMLdecision,T'tJ,lestatesDiscrete:>·()_1;.PxlHo(x)xEAoifpXIH1At.Con,,in,?J,O'US:xE0xJffx1Ho(x())->1;'/,XIH1xxEA1other'tnise;xEA1otherv1ise.ComparingTheorerr111.6toT11eorerr111.4,v.rhenXiscontinuous,orTheorern11.5,"''henXisdiscrete,"''eseethattherr1axirr1l1rnlikelihoodtestisthesameastheNeyman-Pearsontestwitl1parameter'""(=1.Thisguaranteesthatt11emaxirr1l1rr1likelihoodtestisoptimalir1thelirr1itedsensethatnoothertestcanreducePMrssfort11esarnePFA·Inpractice,-vveusethe11.IL11ypothesistestinrnar1:yapplicatior1s.ItisalrnostaseffectiveastheMAPh}rpotl1esistest\vher1theexperimentthatproducesoutcornesisreliableinthesensetl1atPERRfortheMLtestislow.Tosee"''hYthisistrue,exarr1inethedecisionruleir1Example11.6.\¥henthesignal-to-noiseratio2v/aishigh,therightsideofEql1atior1(11.11)iscloseto0ur1lessoneoft11eaprioriprobabilitiesr>or1-I>isclosetozero(ir1wl1ichcasethelogarithrr1ont11erightsideisalownegati,renurr1berorahighpositivenumber,indicatingstrongpriorknowledgethatthetransrnittedbitis0or1.·\¥hent11erightsideisr1early0,llSuallythecaseinbir1arycornmur1ication,t11ee\ridenceproducedbythereceivedsigr1al11asrnl1chrnoreinfll1enceonthedecisionthantheaprioriinformationandtheresultoftheNIAPhypothesistestisclosetotheresl1ltofthe11.ILh}rpothesistest.Example11.11==ContinuingthediskdrivetestofExample11.8,designthemaximumlikelihoodtestforthefactorystatusbasedonthedecisionstatisticN,thenumberofdrivestesteduptoandincludingthefirstfailure.TheMLhypothesistestcorrespondstotheMAPtestwithP[H0]=P[H1]=0.5.Inthiscase,Equation(11.15)impliesn,*=66.62orA0={n,>67}.TheconditionalerrorprobabilitiesandthecostoftheMLdecisionrulearePFA=PMISS=P[N<66IHo]=1-(1-10-4)66=0.0066,p[N>67IH1]=(1-10-1)66=9.55.10-4)E[CML]=(0.9)(0.0066)(50,000)+(0.1)(9.55·10-4)(200,000)=$316.10.FortheMLtest,PERR=0.0060.ComparingtheMAPrulewiththeMLrule,weseethatthepriorinformationusedintheMAPrulemakesitmoredifficulttorejectthenullhypothesis.Weneedonly46gooddrivesintheMAPtesttoacceptHo,whileintheMLtest,thefirst66driveshavetopass.TheMLdesign,whichdoesnottakeintoaccountthefactthatthefailurerateisusuallylow,ismoresusceptibletofalsealarmsthantheMAPtest.EventhoughtheerrorprobabilityishigherfortheMLtest,[11.2TestObjective1/[inirnizeprobabilityJVIAPofir1correctdecisior1BINARYHYPOTHESISTESTING#testsPFA454.5x10-PMrss3Cost8.7x10-3$365JVIC1/[inirnizeexpectedcost585.8x10-32.2x10-3$305JVIL1![aximizelikeli11ood;ignorecostsaridapr1or1probabilities676.6x10-33839.6x10-4$3161![ir1irnizePMrssforgivenNPPFA101r::1.0x10-22.7x10-,)$451Table11.1Comparisonofthemaxin1umaposterioriprobability(NIAP),n1inin1umcost(l\IIC),n1axin1un1likelihood(l\!IL),andNeyn1an-Pearson(NP)testsatthediskdrivefactory.Testsareorderedby#tests,then1axin1un1numberoftestsrequiredbyeachn1ethod.thecostislowerbecauseacostlymissoccursveryinfrequently(onlyonceeveryfourmonths).ThecostoftheMLtestisonly$11.10morethantheminimumcost.Thisisbecausetheaprioriprobabilitiessuggestavoidingfalsealarmsbecausethefactoryfunctionscorrectly,whilethecostssuggestavoidingmisses,becauseeachoneisveryexpensive.Becausethesetwopriorconsiderationsbalanceeachother,theMLtest,whichignoresbothofthem,isverysimilartotheminimumcosttest.Table11.lcorr1paresthefourbir1ary11ypothesistests(MAP,MC,MLandNP)forthediskdriveexarnple.Inaddition,therecei·veroperatingcurve(sho\vnontheleft)associatedvviththedecisionstatisticJ\T,thenumberoftestsuptoandincludingtheNPfirstfailure,sho\vst11eperforrnar1cetrade10-6..._____________.offbet\veenthesetests.Ever1thoughit310Pio-2useslesspriorinforrnatior1thantheot11erFAtests,theMLtestrnightbeagoodchoicebecal1Sethecostoftestingisnearlyrninirr1umandthernissprobabilityisverylovv.Theconsequer1ceofafalsealarrnislikelytobeanexaminationoft11ernanufacturingprocesstofir1dOlltifsomethingisvvrong.Amiss,or1theotherhand(decidingthefactoryisfunctionir1gproperly\vl1en10%ofthedrivesaredefective)couldbeharrnfultosalesinthelongrun.Quiz11.2Inanopticalcornrr1unicatior1ssystem,thep11otodetectorOl1tpl1tisaPoissonrandorr1variableK,eit11erwithanexpectedvaltleof10,000photor1s(11ypothesisH.0)orvvithanexpectedvalt1eof1,000,000p11otons(h}rpot11esisH1).Givent11atbothhypothesesareequall}'lik:el}',designaJVIAPhypot11esistestt1singobservedvalt1esofrandom\rariableJ(.[384CHAPTER11HYPOTHESISTESTING11.3MultipleHypothesisTestAmultiplehypothesistestisageneralizationofabinaryhypothesistestfrom2toMhypotheses.Asinthebinarytest,observinganoutcorr1einAicorrespondstoacceptingthehypothesisHi.Theaccuracyofamt1ltiplehypothesistestisembodiedinamatrixofcor1ditionalprobabilitiesofdecidir1gH.iwher1H.iisthecorrecthypot11esis.Amaximumaposteriori(MAP)testtakesintoaccountaprioriprobabilitiesandobservationstornaxirr1izetheprobabilityofacorrectdecision.Amaxirnurr1likelihood(ML)testllSesonlyobservations.Thetwotestscoincidevvhenallhypothesesareequall}'likel}'apriori.Therearerr1anyapplicationsinv.rhichanexperirnentcanconformtorr1oret11ant'ivoknovvr1probabilityrr1odels,allvvitht11esamesarr1plespaceS.Arrn1ltiplehypothesistestisageneralizationofabinary11ypotl1esistest.ThereareJ\,fhypothetica.lprobabilityrnodels:H0,H1,···,HM-I··vveperforrr1ar1experirnent,andbasedontheoutcorne,wecometothedecisionthatacertainH,,nist11etrueprobabilityrr1odel.ThedesignofthetestconsistsofdividingSintoapartitionA0,A1,···,AJ\1-1,S'l1chthatthedecisionisacceptH,iifsEAi.Theaccuracyrneast1reoftheexperirr1entcor1sistsofM2cor1ditionalprobabilities,P[AilH:7],'i,j=0,1,2,···,J\,f-l.T11eJ\,fprobabilities,P[AiIH·,i],i=0,1,···,J\,f-1areprobabilitiesofcorrectdecisions.Example11.12Acomputermodemiscapableoftransmitting16differentsignals.Eachsignalrepresentsasequenceoffourbitsinthedigitalbitstreamattheinputtothemodem.Themodemreceiverexaminesthereceivedsignalandproducesfourbitsinthebitstreamattheoutputofthemodem.Thedesignofthemodemconsidersthetaskofthereceivertobeatestof16hypothesesHo,H1,...,H.15,whereHorepresents0000,Hirepresents0001,···,andH'1srepresents1111.Thesamplespaceoftheexperimentisanensembleofpossiblereceivedsignals.ThetestdesignplaceseachoutcomesinasetAisuchthattheeventsEAileadstotheoutputofthefour-bitsequencecorrespondingtoHi.Forarr1ultiplehypothesistest,theMAPhypothesistestandthe11[1hypothesistestareger1eralizationsofthetestsir1Theorem11.landDefinition11.l.Mir1irr1izir1gtheprobabilityoferrorcorrespondstorr1axirnizingtheprobabilityofacorrectdecision,M-1PcoRRECT=L'i=OP[AiIH,i,]P[H,il·(11.33)[11.3385MULTIPLEHYPOTHESISTESTTheorem11.t:==-...,MAPMultipleHypothesisTestGiven,arn'tJ,ltiplehypothesistestin,gexperirnen,t1JJith011,tcornes;thefollo'1uin,gruleleadstothehighestpossiblevalv,eofPCORRECT:sEA1nifP[Hrnls)>P[n:1ls)forallj=0,1,2,...,M-1.Asinbinary11ypothesistesting,-vvecanapplyBa:yes'theorerr1toderiveadecisior1rulebasedontheprobabilityrr1odels(likelihoodfur1ctions)correspondingtothehypothesesandt11eaprioriprobabilitiesofthehypotheses.Therefore,correspondingtoTheorem11.2,wehavethefollowinggeneralizationoft11elv!APbinaryhypothesistest.11.8Forariexperirnen,tthatprod'ucesararidornvar~iableX,theMAPrnultiplehypothesistest'is~--==TheoremDiscrete:~r;iEArnifP[Hrn]PxlHrn(xi)>P[n:1JPxlHj(x;i)forallj;Con,tin/uov,s:x;EArnifP[H,,n]fxlHrn(x;)>P(n:1Jfx1Hj(x;)forallj.Ifinformationabouttheaprioriprobabilitiesoftheh}rpot11esesisnotavailable,arnaximurr1likelihood11}rpothesistestisappropriate.MLMultipleHypothesisTestDefinition11.2Arnax;irnurnlikelihoodtestofrn'ultiplehyJJotheseshasthedecision,'rulesEArnifP[slHrr1,]>P[sln:1Jforallj.TheML11ypothesistestcorrespor1dstothe1!lAPhypothesistestwhenallhypothesesHihaveeql1alprobability.===-Example11.13Inaquaternaryphaseshiftkeying(QPSK)communicationssystem,thetransmittersendsoneoffourequallylikelysymbols{s0,s1,s2,s3}.Letn·'i.denotethehypothesisthatthetransmittedsignalwasSi.WhenSiistransmitted,aQPSKreceiverproducesthevectorX=[X1X2]'suchthat(11.34)whereJV1andJ>l2areiidGaussian(0,a)randomvariablesthatcharacterizethereceivernoiseandEistheaverageenergypersymbol.BasedonthereceiveroutputX,thereceivermustdecidewhichsymbolwastransmitted.Designahypothesistestthat[386CHAPTER11HYPOTHESISTESTINGmaximizestheprobabilityofcorrectlydecidingwhichsymbolwassent.Sincethefourhypothesesareequallylikely,boththeMAPandMLtestsmaximizetheprobabilityofacorrectdecision.ToderivetheMLhypothesistest,weneedtocalculatetheconditionaljointPDFsfx1Hi(x).GivenHi,l'l1andN2areindependentandthusX1andX2areindependent.Thatis,using(),i=i11/2+11/4,wecanwritef'x1Hi(x)=fX1IHi(x1)fX2IHi(x2)r:::..22r:::..22-1-e-(:i:1-vEcosBi)/2CYe-(:1;2-vEsinBi)/2CY2110"2e-[(:1;1-vEcosBi)2+(x2-JEi-;inBi)2]/2CY212110"2.(11.35)WemustassigneachpossibleoutcomextoanacceptancesetAi.FromDefinition11.2,theacceptancesetsAifortheMLmultiplehypothesistestmustsatisfy(11.36)Equivalently,theMLacceptancesetsaregivenbytherulethatxEAiifforallj,Definingthesignalvectors(11.37)x,wecanwritetheMLruleas(11.38)whereIIull2=v,r+v,§denotesthesquareoftheEuclideanlengthoftwo-dimensionalvectoru.InEquation(11.38),theacceptancesetA,iisthesetofa11vectorsxthatareclosesttothevectorsi.Theseacceptancesets{A0,A1,A2,A3}arethefourquadrants(withboundariesmarkedbyshadedbars)shownontheleft.Incommunicationstextbooks,thespaceofvectorsxiscalledthesignalspace,thesetofvectors{s1,...,s4}iscalledthesignalconstellation,andtheacceptaneesetsA,iarecaIleddecisionregions.===Quiz11.3===FortheQPSKcorr1rr1unicationsS}rsternofExarr1ple11.13,whatistheprobabilitythattr1ereceiverrr1akesar1erroranddecodesthewrongsyrnbol?[11.411.4MATLAB387MA1"'LABprogran1sger1eratesarnplevaluesofknownprobabilit}'modelsinordertocornputesarr1plevaluesofderivedrandomvariablesthatappea.rin:hypothesistests.T11eprogramsl1sethederivedsamplevaluesinsimulationsandcalculaterelativefrequenciesofeventssuchasrr1issesandfalsealarrr1s.MATLABIntheexamplesofthisc11a.pter)we11avec11osenexperimer1tswithsirr1pleprobabilityrr1odelsinorderto11ighlightt11ecor1ceptsandc11aracteristicpropertiesofhypothesistests.J\!I.A.TLABgreatlyexter1dsourabilit}'todesignandevaluatehypothesistests)especiall}'inpractica.lproblemswhereexactar1alysisoftheprobabilitymodelbecomestoocornplex.Forexarr1ple,J\IIATLABcaneasilyperforrnprobabilityoferrorcalc11lationsaridgraphreceiveroperatingc11rves.Inaddition)therearerr1anycasesinv.rhichanalysisca.nidentifytheacceptancesetsofa11ypothesistestbt1tcalculationoft11eerrorprobabilitiesisoverlycornplex.Inthiscase,MATL.A.Bcar1simulaterepeatedtrialsoftheh}rpothesistest.Thefollowingexamplepreser1tsasituatior1frequentlyer1counteredbycornrnur1icationsengineers.Detailsofapracticalsystemcreateprobabilityrr1odelsthatarehardtoanalyzemathernatica.11}'·Instead)er1gir1eersllSeMATL.A.Baridothersoftwaretoolstosirr1ulateoperationofthesysterr1sofir1terest.Sirnt1lationdataprovidesestirr1atesofS}rsterr1performanceforeac11ofseveraldesigr1alternatives.T11isexarr1pleissirnilartoExarr1ple11.6,\viththeaddedcornplicationt11atanarr1plifierint11ereceiverproducesafractionofthesq11areofthesignalpl11snoise.Ir1thisexarr1ple,thereisawell-knownprobabilityrnodelfort11enoiseN)buttherr1odelsforthederi\redrandorr1variables-v+N+d(-v+N)2andv+N+d(v+f\T)2aredifficulttoderive.Tostud}'thistest,\Ve\¥riteaJ\IIATL.A.Bprogramthatgeneratesrn,sample·valuesofJV.Foreachsarr1pleofN)t11eprograrncalculatest11etvvofunctionsoff\T,performsabinar}'h}rpothesistest,anddeterrr1inesvvhethert11etestresultsinahitorfalsealarrn.Itreportst11erela.tivefrequenciesofhitsandfalsealarrnsasestirnatesofP1vrrssandPFA..Example11.14AdigitalcommunicationssystemtransmitseitherabitB=0orB=1withprobability1/2.Theinternalcircuitryofthereceiverresultsina"squareddistortion"suchthatreceivedsignaI(measuredinvolts)iseither-v+N+d(-'V+f\T)2X=v+f\T+d(v+N)2B=O.IB=l(11.39))wheref\T,thenoise,isGaussian(0)1).Foreachbittransmitted,thereceiverproducesanoutputB=0ifX<TandanoutputB=1,otherwise.Simulatethetransmissionof20,000bitsthroughthissystemwithv=1.5volts,d=0.5andthefollowingvaluesofthedecisionthreshold:T=-0.5,-0.2,0,0.2)0.5volts.WhichchoiceofTproducesthelowestprobabilityoferror?CanyoufindavalueofTthatdoesabetterjob?~~[388CHAPTER11HYPOTHESISTESTING>>TT=-0.5000-0.200000.2000>>Pe=sqdistor(1.5,0.5,10000,T)Pe0.50000.27330.22650.1978Figure11.4ple11.14.0.50000.1762_A.verageerrorrateforthesquareddistortionconnnunicationssystemofExan1-Sinceeachbitistransmittedandreceivedindependentlyoftheothers,theprogramsqdistortransmitsrn=10,000zeroestoestimateP[B=llB=O],theprobabilityof1receivedgiven0transmitted,foreachofthethresholds.Itthentransmitsrn=10,000onestoestimateP[B=OIB=1].TheaverageprobabilityoferrorisPERR=o.sP[.B=11B=o]+o.sP[.B=OIB=1].functiony=sqdistor(v,d,m,T)%P(error)formbitstested%transmit+-v,addN&d(v+N)-2%receive1ifx>T,otherwise0x=(v+randn(m,1));[XX,TT]=ndgrid(x,T(:));P01=sum((XX+d*(XX.-2)<TT),1)/m;x=-v+randn(m,1);[XX,TT]=ndgrid(x,T(:));P10=sum((XX+d*(XX.-2)>TT),1)/m;y=0.5*(P01+P10);(11.40)BydefiningthegridmatricesXXandTT,wecantesteachcandidatevalueofTforthesamesetofnoisevariables.WeobservetheoutputinFigure11.4.Becauseofthebiasinducedbythesquareddistortionterm,T=0.5isbestamongthecandidatevaluesofT.However,thedatasuggeststhatavalueofTgreaterthan0.5mightworkbetter.Problem11.4.3examinesthispossibility.Theproblemsforthissectionincludeacollectionofhypothesistestir1gproblerr1sthatcar1besolvedllSingJ\IIATLA.Bbl1taretoodifficulttosolvebyhand.Thesolutionsarebl1iltontheJ\ilA.TLABrnethodsdevelopedinpriorchapters;however,ther1ecessaryJ\IIATLABcalcl1lationsaridsiml1lationsaretypicall}'problernspecific.Quiz11.4Forthecorr1rnl1nicationsS}rstemofExarr1ple11.14-vvithsquareddistortion,-vvecandefinetherr1issaridfalsea,larrr1probabilitiesasPMrss=Poi=P[B=OIB=1J,PFA=Pio=P[B=1IB=0J.(11.41)J\!Iodifytheprograrr1sqdistorinExarr1ple11.14toproducereceiveroperatingCl1rvesfortheparametersv=3voltsandd=0.1,0.2,and0.3.Hint:ThepointsontheROCcorrespor1dtodifferentvaluesofthethresholdT·volts.[PROBLEMS389F'U.rtherReadin,g:[Kay98]providesdetailed)readablecoverageof11ypotl1esistesting.[HayOl]presentsdetectior1ofdigitalcorr1rnunicationssignalsasahypothesistest.Acollectionofchallenginghorr1eworkproblerr1sforsections11.3arid11.4arebasedor1bitdetectionforcodedivisionrr1ultipleaccess(CDMA)comrr1unicationssysterns.Theauthoritativetreatrnentoft11issubjectcar1befoundin[Ver98].ProblemsDifficulty:Easy11.1.1LetLequalthen11mberofflipsofacoinuptoandincludingthefirstflipofheads.DeviseasignificancetestforLatlevelex=0.05totestthehypothesisHthatthecoinisfair.\l\lhatarethelimitationsofthetest?11.1.2Acoursehast\vorecitationsectionsthatmeetatdifferenttimes.Onthemidterm,theaverageforsection1is5pointshigherthantheaverageforsection2.AlogicalconclusionisthattheTAforsection1isbetterthantheTAforsection2.Using\vordsratherthanmath,givereasons'vhythismightbethe\vrongconclusion.ModerateDifficult+ExpertsOnlydonotannounce'vheretheupgradestakeplace.Youhavethetaskofdeterminingwhethercertainareashavebeenupgraded.Youhavedecidedtouseanapplicationinyoursmartphonetomeasurethepingtime(ho\vlongittakestoreceivearesponsetoacertainmessage)ineacharea.Thene'vsystemisfasterthantheold(0)one.Ithasonaverageshorterpingtimes.Theprobabilitymodelforthepingtimeininillisecondsofthene\vsystemistheexponential(60)randomvariable.Performapingtestandrejectthenullhypothesisthattheareahasthenewsystemifthepingtimeisgreaterthantoms.11.1.3UnderthenullhypothesisHothattrafficistypical,thenumberofcallattemptsina1-secondinterval(duringrushhour)atamobiletelephoneS\vitchisaPoissonrandomvariableN\vithE[N]=2.5.OveraT-secondperiod,themeasuredcallrateisNJ=(N1+···+N'r)/'1,,whereN1,...,N'rareiidPoissonrandomvariablesidenticaltoN.However,\vheneverthereisunusuallyheavytraffic(resultingfromanaccidentorbad\Veatherorsomeotherevent),themeasuredcallrateMishigherthanusual.BasedontheobservationM,designasignificancetesttorejectthenullhypothesisHothattrafficistypicalatasignificancelevelex=0.05.JustifyyourchoiceoftherejectionregionR.Hint:YoumayuseaGaussian(centra[limittheorem)approximationforcalculatingprobabilities\VithrespecttoM.Ho'vdoesyourtestdependontheobservationperiodT?Explainyouranswer.11.1.6•LetJ{bethenumberofheadsinn,=100flipsofacoin.DevisesignificancetestsforthehypothesisHthatthecoinisfairsuchthat11.1.4Acellulartelephonecompanyisupgradingitsnetworktoane\v(1'l)transmissionsystemoneareaatatime,butthey(a)Thesignificancelevelex=0.05andtherejectionsetRhastheform{II<-E[Ic}.(a)\l\lriteaformulaforex,thesignificanceofthetestasafunctionofto.(b)\tVhatisthevalueoftothatproducesasignificancelevelex=0.05?11.1.5\l\lhenapacemakerfactoryisoperatingnormally(thenullhypothesisHo),arandomlyselectedpacemakerfailsa"droptest"withprobabilityq0=10-4.Eachday,aninspectorrandomlytestspacemakers.Designasignificancetestforthenullhypothesis\vithsignificancelevelex=0.01.Notethatdroptestingofpacemakersisexpensivebecausethepacemal[390CHAPTER11HYPOTHESISTESTING(b)Thesignificancelevela=0.01andtherejectionsetRhastheform{K>c'}.11.1.7\!\Thenachipfabricationfacilityisoperatingno1mally,thelifetimeofainicrochipoperatedattemperature1",measuredindegreesCelsius,isgivenbyanexponential(.\)randomvariableXwithexpectedvalueE[X)=1/.\=(200/12years.Occasionally,thechipfabricationplanthascontaminationproblemsandthechipstendtofailmuchmorerapidly.Totestforcontaminationproblems,eachdaym,chipsaresubjectedtoaone-daytestatT=100°0.Basedonthenumber1'lofchipsthatfailinoneday,designasignificancetestforthenullhypothesistestHothattheplantisoperatingnormally.1)(a)SupposetherejectionsetofthetestisR={1'l>0}.Findthesignificancelevelofthetestasafunctionofm,thenumberofchipstested.(b)Ho'vmanychipsmustbetestedsothatthesignificancelevelisa=0.01.beenplayedandthattheleader(callhimNarayan)haspicked119gamescorrectly.DoesthepoolleaderNarayanhaveskillsorishejustlucky?(a)Toaddressthisquestion,designasignificancetesttodetermine\vhetherthepoolleaderactuallyhasanyskillatpickinggames.LetHodenotethenullhypothesisthatallplayers,includingtheleader,pick\Vinnersineachgamewithprobabilityp=1/2,independentoftheoutcomeofanyothergame.BasedontheobservationofW,thenumberofwinningpicksbythepoolleaderafterm,\veeksoftheseason,designaone-sidedsignificancetestforhypothesisH0atsignificancelevela=0.05.YoumayuseacentrallimittheoremapproximationforbinomialPNIFsasneeded.(b)GiventhatNarayanistheleaderwith119winningpicksinm=14weeksinapoolwithn,=38contestants,doyourejectoraccepthypothesisHo?(c)Ifweraisethetemperattrreofthetest,doesthenumberofchipsvveneedtotestincreaseordecrease?(c)Howdoesthesignificancetestdependonpicksbeingmadeagainstthepointspread?11.1.8AgroupofrLpeopleformafootballpool.Therulesofthispoolaresimple:16footballgamesareplayedeachweek.Eachcontestantmustpickthewinnerofeachga1neagainstapointspread.Thecontestantwhopicksthemostgamescorrectlyovera16-weekseasonwinsthepool.Thespreadisapointdifferencedsuchthatpickingthefavoredtea1nisa\Vinningpickonlyifthatteamwinsbymorethandpoints;otherwise,thepickoftheopposingteamisa\Vinner.Eachpoolcontestantcanstudytheteams'pasthistories,performancetrends,officialinjuryreports,coaches'\Veeklypressconferences,chatroomgossipandanyother\Visdomthatmighthelpinplacinga\Vinningbet.Afterm,weeks,contestanti\Villhavepickedwigamescorrectlyoutof16m,games.Forexample,supposethatafterm,=14weeks,16(14)=224gameshave11.1.9Aclasshas2n,(alargenumber)studentsThestudentsareseparatedintotvvogroupsAandB,each\vithn,students.Group_4studentstakeexam_4andearniidscoresX1,...,X11•GroupBstudentstakeexamB,earningiidscoresY1,...,Xi·Thet\voexamsaresimilarbutdifferent;however,theexams\Veredesignedsothatastudent'sscoreXonexam_4orYonexamBhavethesameexpectedvalueandvariancea2=100.Foreachexam,weformthesamplemeanstatisticJ\lfA=X1+···+Xn'n,i\!JB=y;i+···+Y:nn,BasedonthestatisticD=MA-i\!JB,usethecentrallimittheoremtodesignasignificancetestatsignificancelevela=0.05forthehypothesisHothatastudent'sscore[PROBLEMSonthetwoexamshasthesameexpectedvalueJJ,andvariancea2=100.\i\1hatistherejectionregionifn,=100?Makesuretospecifyanyadditionalassumptionsthatyouneedtomake;however,trytomakeasfewadditionalassumptionsaspossible.11.2.1Inarandomhour,thenumberofcallattemptsNatatelephoneswitchhasaPoissondistribution'vithanexpectedvalueofeitherao(hypothesisHo)orai(hypothesisHi).ForaprioriprobabilitiesP[Hi],findthelVIAPandMLhypothesistestingrulesgiventheobservationofN.11.2.2Thepingtime,inmillisecondsofane\vtransmissionsystem,describedinProblem11.1.4istheexponential(60)randomvariableN.Thepingtimeofanoldsyste1nisanexponentialrandomvariable0withexpectedvalueµ,o>60ms.ThenullhypothesisofabinaryhypothesistestisHo:Thetransmissionsyste1nisthenewsystem.ThealternativehypothesisisHi:Thetransmissionsystemistheoldsystem.TheprobabilityofanewsystemisP[N]=0.8.TheprobabilityofanoldsystemisP[O]=0.2.AbinaryhypothesistestmeasuresTmilliseconds,theresultofonepingtest.ThedecisionisHoifT<toms.Other,vise,thedecisionisH1.(a)Writeaformulaforthefalsealarmprobabilityasafunctionoftoandµ,o.(b)Writeaformulaforthemissprobabilityasafunctionoft0andµ,o.(c)Calculatethemaximumlikelihooddecisiontimet0=tj\1Lforµ,o=120msandµ,o=200ms.(d)DoyouthinkthattrvrAP,themaximumaposterioridecisiontime,isgreaterthanorlessthantrv1L?Explainyouranswer.(e)Calculatethemaximumaposterioriprobabilitydecisiontimeto=tMAPforµ,o=120msandµ,o=200ms.(f)Dra'vthereceiveroperatingcurvesforµ,o=120msandµo=200ms.11.2.3Anautomaticdoorbellsystemringsabell'vheneveritdetectssomeoneat391thedoor.Thesystemusesaphotodetectorsuchthatifapersonispresent,hypothesisH1,thephotodetectoroutputNisaPoissonrandomvariablewithanexpectedvalueof1300photons.Otherwise,ifnooneisthere,hypothesisHo,thephotodetectoroutputisaPoissonrandomvariablewithanexpectedvalueof1000.DeviseaNeymanPearsontestforthepresenceofsomeoneoutsidethedoorsuchthatthefalsealarmprobabilityisa<10-6.\i\fhatisminimumvalueofP~111ss?11.2.4IntheradarsystemofExample11.4,P[H1]=0.01.Inthecaseofafalsealarm,thesystemissuesanunnecessaryalertatthecostofC10=1unit.ThecostofamissisC01=104unitsbecausethetargetcouldcausealotofdamage.\tVhenthetargetispresent,thevoltageisX=4+N,aGaussian(4,1)randomvariable.\i\fhenthereisnotargetpresent,thevoltageisX=N,theGaussian(0,1)randomvariable.Inabinaryhypothesistest,theacceptancesetsareAo={X<xo}andAi={X>xo}.(a)lhypothesistest,findthedecisionthresholdxo=XMAP,theerrorprobabilitiesPFAandPlVnss,andtheaveragecostE[CJ.(b)ComparetheMAPtestperformanceagainsttheminimumcosthypothesistest.11.2.5IntheradarsystemofExample11.4,showthattheROCinFigure11.2istheresultofaNeyman-Pearsontest.Thatis,sho'vthattheNeyman-PearsontestisathresholdtestwithacceptancesetAo={X<xo}.Howis::eorelatedtothefalsealarmprobabilitya?11.2.6_A.systemadministrator(andparttimespy)ataclassifiedresearchfacilitywishestouseagate,vayrouterforcovertcommunicationofresearchsecretstoanoutsideaccomplice.ThesysadmincovertlycommunicatesabitWforeveryn,transmittedpackets.TosignalW=0,therouterdoesnothingwhilen,regularpacketsaresentoutthroughthegate\vayasa[392CHAPTER11HYPOTHESISTESTINGPoissonprocessofrate.\opackets/sec.TosignalW=1thesysadmininjectsadditionalfakeoutboundpacketssothatrioutboundpacketsaresentasaI=>oissonprocessofrate2.\0.ThesecretcommunicationbitsareequiprobableinthatP[W=1]=P[W=O]=1/2.Thesysadmin'saccomplice(outsidethegateway)inonitorstheoutboundpackettransmissionprocessbyobservingthevectorX=[X1,X2,...,Xn]ofpacketinterarrivaltimesandguessingthebitWeveryn,packets.(a)FindtheconditionalPDFsfx1v\l=o(x)andfx1W=1(x).(b)WhataretheMAPandl\/ILhypothesistestsfortheaccomplicetoguesseitherhypothesisHothatvV=0orhypothesisH1thatvV=1?(c)LetWdenotethedecisionoftheMLUsetheChernoffhypothesistest.boundtoupperboundtheerrorprobabilityP[W=OIW=1].11.2.7Thepingtime,inmilliseconds,ofane'vtransmissionsystem,describedinProblem11.1.4istheexponential(60)randomvariable1'l.Thepingtimeofanoldsystemistheexponential(120)randomvariable0.ThenullhypothesisofabinaryhypothesistestisH0:Thetransmissionsystemisthenewsystem.ThealternativehypothesisisH1:Thetransmissionsystemistheoldsystem.Theprobabilityofane'vsystemisP[N]=0.8.TheprobabilityofanoldsystemP[O]=0.2.AbinaryhypothesistestperformskpingtestsandcalculatesMn(T),thesamplemeanofthepingti1ne.ThedecisionisHoifMn(T)<toms.Other,vise,thedecisionisH1.(a)Usethecentrallimittheoremto'vriteaformulaforthefalsealar1nprobabilityasafunctionoftoandk.(b)Usethecentrallimittheoremtovvriteaformulaforthemissprobabilityasafunctionoftoandk.(c)Calculatethemaximumlikelihooddecisiontime,to=t1v1L,fork=9pingtests.(d)Calculatethemaximumaposterioriprobabilitydecisiontime,to=tMAPfork=9pingtests.(e)Drawthereceiveroperatingcurvesfork=9pingtestsandk=16pingtests.11.2.8Inthisproble1n,'veperformtheold/newdetectiontestofProblem11.2.7,exceptno'vwemonitorkpingtestsandobserve'vhethereachpinglastslongerthantoms.TherandomvariableMisthenumberofpingsthatlastlongerthantoins.ThedecisionisHoifM<m,o.Other,vise,thedecisionisH1.(a)\tVriteaformulaforthefalsealarmprobabilityasafunctionofto,1no,and'n.(b)Findthemaximumlikelihooddecisionnumber1no=ffiMLforto=4.5msandk=16pingtests.(c)Findthemaximumaposterioriprobabilitydecisionnumberm,o=ffiMAPfort0=4.5msandk=16pingtests.(d)Dra'vthereceiveroperatingcurvesfort0=90msandto=60ms.Inbothcasesletk=16pingtests.11.2.9AbinarycommunicationsystemhastransmittedsignalX,theBernoulli(1/2)randomvariable.Atthereceiver,'veobserveY=VX+W,whereVisa"fadingfactor"andWisadditivenoise.NotethatVandWareexponential(1)randomvariablesandthatX,V,andWaremutuallyindependent.GiventheobservationY,weinustguesswhetherX=0orX=1wastransmitted.·useabinaryhypothesistesttodeterminetherulethatminimizestheprobabilityPeofadecodingerror.Fortheoptimumdecisionrule,calculatePe.11.2.10InaBPSKamplify-and-for,vardrelaysystem,asourcetransmitsarandombit'/E{-1,1}everyTsecondstoadestinationreceiverviaasetofn,relaytransmitters.V=1and'/=-1areequallylikely.Inthiscommunicationsystem,thesourcetransmitsduringthetimeperiod(0,T/2)[PROBLEMSareiidGaussian(0,1)noises,independentofX.suchthatrelayireceivesi=1,2,...,n,,\VherethewiareiidGaussian(0'1)randomvariablesrepresentingrelayireceivernoise.Inthetimeinterval(7,/2,T),eachrelaynodeamplifiesandforwardsthereceivedsourcesignal.ThedestinationreceiverobtainsthevectorY=[Y1Y'.;1J'suchthati=l,2,...,ri,\VheretheziarealsoiidGaussian(0,1)randomvariables.Inthefollowing,assumethattheparametersaiandf3iareallnonnegative.Also,letH0denotethehypothesisthatV=-1andH1thehypothesisv=l.(a)Supposeyoubuildasuboptimaldetec1torbasedonthesumY=1Yi.IfY>0,thereceiverguessesH1;otherwisethereceiverguessesH0.\i\fhatistheprobabilityoferrorPeforthisreceiver?I::(b)BasedontheobservationY,nowsupposethedestinationreceiverdetectorperformsal\/IAI>testfo:rhypothesesH0orH1.\l\fhatistheMi\.Pdetectorrule?Simplifyyouranswerasmuchaspossible.Hint:FirstfindthelikelihoodfunctionsfYIHi(y).(c)\i\fhatistheprobabilitJrofbiterrorP;oftheMAPdetector?(d)Comparethet\vodetectorswhenn=4and=(1,1),(0'.3'(33)=(1,10)'(a1,f31)(0'.2)(32)=(10'1)'(0'.4)(34)=(10,10).Ingeneral,what'sbadaboutthesuboptimaldetector?11.2.11393InaBPSKcommunicationsystem,asourcewishestocommunicatearandombitXE{-1,1}toareceiver.InputsX=1andX=-1areequallylikely.Inthissystem,thesourcetransmitsXmultipletimes.Intheithtransmission,thereceiverobservesYi=X+'Wi,wheretheWi(a)Aftern,transmissionsofX,youobserveY=Y=[Y1Yn]'.FindP[X=llY=y].ExpressyouranswerintermsofthelikelihoodratioL()y=fy1x(YI-1)fYIX(yll).(b)Supposeaftern,transmissions,thereceiverobservesY=yanddecidesX*={1-1P[X=llY=y]>1/2,otherwise.FindtheprobabilityoferrorPe=f>[X*#X]intermsofthe1-E,thereceiverguessesX*=1.IfXn(Y)<-1+E,thereceiverguessesX*=-1.Followingthereceiver'sguess,thetransmitterstartssendinganewbit.FindupperandlowerboundstoPe=P[X*#X].Thatis,findE1andE2suchthat11.2.12SupposeinthediskdrivefactoryofExample11.8,\VecanobserveK,thenumberoffaileddevicesoutofn,devicestested.Asintheexample,letHidenotethehypothesisthatthefailurerateisqi.(a)_Assumingqo<q1,'vhatisthelVILhypothesistestbasedonanobservationofJ{?(b)\tVhataretheconditionalprobabilitiesoferrorPFA=P[-41IHo]andPM1ss=P[AoIH1]?Calculatetheseprobabili.esforri=oc::oo,qo=10-4,q1=10-2.t,1(c)ComparethistesttothatconsideredinExample11.8.\l\fhichtestismore[394CHAPTER11HYPOTHESISTESTINGreliable?Whichtestiseasiertoimplement?11.2.13Considerabinaryhypothesistestin'vhichthereisacostassociatedwitheachtypeofdecision.InadditiontothecostC~0forafalsealarmandC01foramiss,wealsohavethecostsC00forcorrectlydecidinghypothesisHoandtheC~1forcorrectlydecidinghypothesisHi.BasedontheobservationofacontinuousrandomvectorX,designthehypothesistestthatminimizesthetotalexpectedcostputisx=ShowthatthedecisionrulethatminimizestotalcostisthesameasdecisionruleoftheminimumcosttestinTheorem11.3,'viththecostsCo1andC10replacedbythedifferentialcostsC01-C~1andC~o-Coo·11.3.1Inaternaryamplitudeshiftkeying(ASK)communicationssystem,therearethreeequallylikelytransmittedsignals{so,s1,s2}.ThesesignalsaredistinguishedbytheiramplitudessuchthatifsignalSiistransmitted,thereceiveroutput\VillbeX=a(i-1)+N,'vherea,isapositiveconstantandNisaGaussian(0,CYN)randomvariable.BasedontheoutputX,thereceivermustdecode'vhichsymbolSi\Vastransmitted.(a)WhataretheacceptancesetsAiforthehypothesesHithatSiwastransmitted?(b)WhatisP[De],theprobabilitythatthereceiverdecodesthewrongsymbol?11.3.2AmultilevelQPSKcommunicationssystemtransmitsthreebitseveryunitoftime.Foreachpossiblesequenceijkofthreebits,oneofeightsymbols,{sooo,soo1,...,s111},istransmitted.WhensignalSijkistransmitted,thereceiverout-+N,whereNisaGaussian(0,CY21)randomvector.Thetwo-dimensionalsignalvectorssooo,...,s111areS110S100••So10E[C'J=P[A1IHo]P[Ho]C~o+P[-4oIHo]P[Ho]Cbo+P[iloIH1]P[H1]Cb1+P[-41IH1lP[H1lc~1.Sijksooo••So11soo1••S111S101••LetHijkdenotethehypothesisthatSijkwastransmitted.ThereceiveroutputX=[X1X2J'isusedtodecidetheacceptancesets{-4000,...,Ai11}.Ifalleightsymbolsareequallylikely,sketchtheacceptancesets.11.3.3AnM-aryquadratureamplitudemodulation(Qi\.11[)communicationssystemcanbeviewedasageneralizationoftheQPSKsystemdescribedinExample11.13.IntheQAMsystem,oneofMequallylikelysymbolsso,...,S1n-1istransmittedeveryunitoftime.WhensymbolSiistransmitted,thereceiverproducesthetwodimensionalvectoroutputX=si+N,whereNhasiidGaussian(0,CY2)components.BasedontheoutputX,thereceivermustdecidewhichsymbolwastransmitted.Designahypothesistestthatmaximizestheprobabilityofcorrectlydecidingwhatsymbolwassent.Hint:FollowingExample11.13,describetheacceptancesetintermsofthevectors._[SillSi-Si2·[PROBLEMS11.3.4SupposeauserofthemultilevelQPSKsystemneedstodecodeonlythethirdbitkofthemessageijk.Fork=0,1,letHkdenotethehypothesisthatthethirdbitwask.Whataretheacceptancesets_4oandA1?WhatisP[B3],theprobabilitythatthethirdbitisinerror?11.3.5TheQPSKsystemofExample11.13canbegeneralizedtoanM-aryphaseshiftkeying(MPSK)systemwithM>4equallylikelysignals.Thesignalvectorsare{so,...,Sj\1-1},whereand()i=211i/JV!.\i\fhentheithmessageissent,thereceivedsignalisX=Si+NwhereNisaGaussian(0,a-21)noisevector.(a)SketchtheacceptancesetAiforthehypothesisHithatSiwastransmitted.(b)Findthelargestvalueofdsuchthat(c)Usedtofindanupperboundfortheprobabilityoferror.11.3.6AmodemusesQAM(seeProblem11.3.3)totransmitoneof16symbols,s0,...,s15,every1/600seconds.Whensignalsiistransmitted,thereceiveroutputisThesignalvectorsso,...,s15are•S5•ISg•S11••••S4S1So•IS3S2S5IssIS12••S10S13••II(b)LetHibetheeventthatsymbolSi\VastransmittedandletCbetheeventthatthecorrectsymbolisdecoded.\tVhatisP[CIH1]?(c)Arg11ethatP[C]>P[CIH1].11.3.7FortheQPSKcommunicationssystemofExample11.13,identifytheacceptancesetsfortheMAPhypothesistestwhenthesymbolsarenotequallylikely.Sketchtheacceptancesetswhena-=0.8,E=1,P[Ho]=1/2,andP[H1]=P[H2]=P[H3]=1/6.11.3.8Inacodedivisionmultipleaccess(CDMA)communicationssystem,kuserssharearadiochannelusingasetofn,dimensionalcodevectors{S1,...,Sk}todistinguishtheirsig11als.Thedimensionalityfactorn,isknownastheprocessinggain.Eachuseritransmitsindependentdatabitsxisuchthatthevectorx=[X1XkJ'hasiidcomponents\VithPxi(l)=Pxi(-1)=1/2.ThereceivedsignaliskY=Lxiyp;,si+N,i=lS14(a)Showthatintermsofvectors,y=gp1/2X+N,••(a)SketchtheacceptancesetsbasedonthereceiveroutputsX1,X2.Hint:ApplythesolutiontoProblem11.3.3.whereNisaGaussian(0,a-21)noisevector.FromtheobservationY,thereceiverperformsamultiplehypothesistesttodecodethedatabitvectorX.X=si+N.S7395I•S15whereSisann,xkmatrix'vithithcol.$i,...,y'Pk]umnS-i.andP1/2--d1ag[isakxkdiagonalmatrix.(b)GivenY=y,shovvthatthetv'IAPandMLdetectorsforXarethesameandaregivenby•x*(y)=argminxEBky-gpl/2x[396CHAPTER11HYPOTHESISTESTINGwhereBkisthesetofallkdimensionalvectorswith±1elements.hypothesisHi,XhasconditionalPlVIFx(c)HowinanyhypothesesdoestheMLd~tectorneedtoevaluate?11.3.9FortheCDMAcommunicationssystemofProblem11.3.8,adetectionstrategyknownasdecorrelationappliesatransformationtoYtogenerate'vhereN=(S'S)-1S'NisstillaG~ussiannoisevectorwithexpectedvalueE[N]=0.Decorrelationseparate_sthesignalsinthattheithcomponentofYis=1'2,...'20,otherwise,0wherePo=0.99andP1=0.9.Calculateandplotthefalsealarmandmissprobabilitiesasafunctionofthedetectionthresholdxo.Calculatethediscretereceiveroperatingcurve(ROC)specifiedbyxo.11.4.2ForthebinarycommunicationssystemofExample11.7,graphtheerrorprobabilityPER,R,asafunctionofp,theprobabilitythatthetransmittedsignalis0.Forthesignal-to-noisevoltageratio,considerv/aE{0.1,1,10}.WhatvaluesofpminimizePER,R,?\i\fhyarethosevaluesnotpractical?11.4.3'vhichisthesameasasingle-userreceiveroutputofthebinarycommunicationsystemofExample11.6.ForequallylikelyinputsXi=1andXi=-1,Example11.6showedthattheoptimal(minimumprobabilityofbiterror)decisionrulebasedonthereceiveroutputYiisxi=sgn(Yi).AlthoughthistechniquerequiresthecodevectorsSi,...,Sktobelinearlyindependent,thenumberofhypothesesthatmustbetestedisgreatlyreducedincomparisontotheoptimalMLdetectorintroducedinProblem11.3.8.Inthecaseoflinearlyindependentcodevectors,isthedecorrelatoroptimal?Thatis,doesitachievethesamebiterrorrate(BER)astheoptimalMLdetector?11.4.1Awirelesspressuresensor(buriedintheground)reportsadiscreterandomvariableXwithrangeSx={1,2,...,20}tosignalthepresenceofanobject.GivenanobservationXandathresholdxo,wedecidethatanobjectispresent(hypothesisH1)ifX>::ea;other,visewedecidethatnoobjectispresent(hypothesisHo).·underForthesquareddistortioncommunicationssystemofExample11.14withv=1.5andd=0.5,findthevalueofTthatminimizespe.11.4.4•ApoisonousgassensorreportscontinuousrandomvariableX.Inthepresenceoftoxicgases,hypothesisHi,.fx1H1(x)={(x/8)e-x2/160x>o,otherwise.Intheabsenceofdangerousgases,XhasconditionalPDF.jXIHo'()_{(1/2)e-x/X-02x>0,otherwise.Deviseahypothesistestthatdeterminesthepresenceofpoisonousgases.Plotthefalsealarmandmissprobabilitiesforthetestasafunctionofthedecisionthreshold.Lastly,plotthecorrespondingreceiveroperatingcurve.11.4.5SimulatetheM-aryPSKsysteminProblem11.3.5forJV!=8andM=16.LetPER,R,denotetherelativefrequencyofsymbolerrorsinthesimulatedtransmissionin1~05symbols.ForeachvalueofJ\1,graphPER,R,,asafunctionofthesignal-to-noisepowerratio(SNR)'Y=E/a2.Consider[PROBLEMS10log10')',theSNRindB,rangingfro1n0to30dB.11.4.6Inthisproblem,weevaluatethebiterrorrate(BER)performanceoftheCDJVIAcommunicationssystemintroducedinProblem11.3.8.Inourexperiments,we\Villmakethefollo,vingadditionalassumptions.•Inpracticalsystems,codevectorsaregeneratedpseudorandomly.We\Villassumethecodevectorsarerandom.ForeachtransmitteddatavectorX,thecodevectorofuseri\villbeSi=1[Si1~Si2Sin]I,\VherethecomponentsSijareiidrandomvariablessuchthatPsi.1(1)=Psij(-l)=1/2.Notethatthefactor1/faisusedsothateachcodevectorSihas2length1:IISi11=s~si=1.•Eachusertransmitsat6dBSNR.Forconvenience,assumePi=p=4anda2=1.(a)UseMATLABtosimulateaCDMi\.system'vithprocessinggainn,=16.Foreachexperimentaltrial,generatearandomsetofcodevectors{Si},datavectorX,andnoisevectorN.Findthel\/ILestimatex*andcountthenumberofbiterrors;i.e.,thenumberofpositionsinwhich::ei=J.:.xi.therelativefrequencyofbiterrorsasanestimateoftheprobabilityofbiterror.Considerk=2,4,8,16users.Foreachvalueofk,performenoughtrialssothatbiterrorsaregeneratedon100independenttrials.Explain\vhyyoursimulationstakesolong.·use(b)Forasimplerdetectorknownasthematchedfilter,whenY=y,thedetectordecisionforuseriiswheresgn(x)=1ifx>0,sgn(x)-1ifx<0,andotherwisesgn(x)0.Comparethebiterrorrateofthematchedfilterandthemaximumlikelihooddetectors.Notethatthematched397filterisalsocalledasingleuserdetectorsinceitcandetectthebitsofuseri\Vithoutthekno\vledgeofthecodevectorsoftheotherusers.11.4.7FortheCD~IIAsysteminProblem11.3.8,\Vewishtousel\/IATLABtoevaluatethebiterrorrate(BER)performanceofthedecorrelaterintroducedProblem11.3.9.Inparticular,\VewanttoestimatePe,theprobabilitythatforasetofrandomlychosencodevectors,thatarandomlychosenuser'sbitisdecodedincorrectlyatthereceiver.(a)l11.4.8Simulatethemulti-levelQi\.MsystemofProblem11.3.4.Estimatetheprobabilityofsymbolerrorandtheprobabilityofbiterrorasafunctionofthenoisevariancea2.11.4.9InProblem11.4.5,\Veusedsimulationtoestimatetheprobabilityofsymbolerror.Fortransmittingabinarybitstream[398CHAPTER11HYPOTHESISTESTINGoveranMPSKsystem,weseteachJi.If=2ivandeachtransmittedsymbolcorrespondstoNbits.Forexample,forJi.If=16,wemapeachfour-bitinputb3b2b1botooneof16symbols.Asimplewaytodothisisbinaryindexmapping:transmitSiwhenb3b2b1boisthebinaryrepresentationofi.Forexample,thebitinput1100ismappedtothetransmittedsignals12.Symbolerrorsinthecommunicatio1~systemcausebiterrors.Forexampleifs1issentbutnoisecausess2tobedecoded,theinputbit.,...sequenceb3b2b1bo=0001isdecoded.,...,....asb3b2b1bo=0010,resultingin2correctbitsand2biterrors.Inthisproblem,weuselVIATLABtoinvestigatehowthemappingofbitstosymbolsaffectstheprobabilityofbiterror.Forourpreliminaryinvestigation,it\Villbesufficienttomapthethreebitsb2b1bototheM=8PSI<systemofProblem11.3.5.(c)Letb(i)=[b2(i)bi(i)bo(i)]denotetheinputbitsequencethatismappedtoSi.Letdijdenotethenumberofbitpositionsinwhichb(i)andb(j)differ.Foragivenmapping1thebiterrorrate(BER)is(a)Findtheacceptancesets{A0,..11.4.10.......,A7}.(b)Simulatemtrialsofthetransmissionofsymbolso.Estimatetheprobabilities{PojlJ=0,1,...,7},thatthereceiveroutputisSj\vhenso\Vassent.Bysymmetry,usetheset{Poj}todeterminePijforalliandj.BER=j~LLPijdij·iJ(d)EstimatetheBERforthebinaryindexmapping.(e)TheGraycodeisperhapsthemostcommonlyusedmapping:b000001010011100101110111DoestheGraycodereducetheBERcomparedtothebinaryindexmapping?ContinuingProblem11.4.9,inthemappingofthebitsequenceb2b1botothesymbolsSi,\Ve\Vishtodeterminetheprobabilityoferrorforeachinputbitbi.Letqidenotetheprobabilitythatbitbiisdecodedinerror.Determineqo,qi,andq2forboththebinaryindexmappingas\VellastheGraycodemapping.[EstimationofaRandomVariableThetec11niquesinChapters10and11usetheOl1tcomesofexperimentstomakeinferencesaboutprobabilit}'rr1odels.Ir1thischapterv.reuseobservationstocalculateanapproxirr1ateva1l1eofasamplevalueofarandom·variablethat11asnotbeenobserved.Therar1dom·variableofinterestrnaybeur1availablebecauseitisimpracticaltorr1east1re(forexarr1ple,theternperatt1reofthesun),orbecauseitisobscuredbydistortion(asignalcorruptedbyr1oise),orbecauseitisnotavailablesoonenough.Werefertotheestirr1ationoffutureobser\rationsasprediction,.Apredictorusesrandornva.riablesobser\redir1earlysubexperirr1er1tstoestirnatearandomvariableproducedb}'alatersubexperirnent.IfXistherar1domvariabletobeestirnated,v.readoptthenotationX(alsoarandornvariable)fort11eestirnate.Ir1rnostofthechapter,-vveusethernean,sqv,areerrorA(12.l)asarr1easl1reofthequalit}'oftheestirr1ate.Sigr1alestirnatior1isabigsubject.Tointroduceitinonec11apter,-vveconfir1eot1rattentiontothefollowingproblerns:•Blindestimationofa.randorr1variable•Estirnatior1ofarandomvariablegivenanevent•Estirnatior1ofarandom\rariablegivenoneot11errandornvariable•Linearestimationofarandomvariablegivenarandorn\rector•Linearestimationofarandomvectorgiver1anotherrar1domvector399[400CHAPTER12ESTIMATIONOFARANDOMVARIABLE12.1MinimumMeanSquareErrorEstimationTheestirnateofXt.hatminirnizestherneansquareerroristheexpectedvall1eofXgivenavailableir1forrnatiori.Theoptimurr1blir1destimateisE[X].ItllSesonl}'theprobabilit}'modelofX.Theoptirr1umestirr1ategiver1XEAisE[XIA].Theoptimumestimategiven·y=yisE[XIY=y].AnexperimentprodllCesarandornvariableX.Howe·ver,-vvearet1nabletoobserveXdirectl}'·Ir1stead,-vveobserveaneventorarandornvariablet11atprovidespartialir1forrnationaboutthesarnplevalueofX.Xcanbeeitherdiscreteorcontir1t1ous.IfXisadiscreterandorr1va.riable,itispossibletouseh}rpot11esistestingtoestirnateX.Foreach;i;iESx,\x.recoulddefineh}rpothesisHiastheprobabilitymodelPx(;i;,i)=1,Px(;i;)=0,x=J;i;i.Ahypothesistest-vvouldthenleadllStochoosethernostprobablexigiver1ourobservations.Althoughthisprocedurernaxirnizest11eprobabilit}'ofdeterrniningthecorrectvalt1eofXi,itdoesnottakeintoaccountt11econsequer1cesofincorrectrestllts.Ittreatsallerrorsinthesarr1emanner,regardlessofwhet11ertheyproduceanswersthatareclosetoorfarfrorr1thecorrectvalueofX.Section12.3describesestimationtechniquesthatadoptthisapproach.B}'contrast,theaimoft11eestimationprocedurespresentedir1thissectionistofindanestirnateXthat,onaverage,isclosetot11etrt1evalt1eofX,evenift11eestirnateneverproducesacorrecttinswer.Apopt1larexampleisanestirnateoft11ent1rr1berofchildreninafarnily.T11ebestestirnate,basedonava.ilableir1forrr1ation,mig11tbe2.4children.Inanestirnatior1procedt1re,-vveairnforalowprobabilityt11att11eestirr1ateisfarfromthetruevalueofX.Anaccl1racymeast1ret11athelpsusac11ie,ret11isairnistherr1eansql1areerrorinEquation(12.1).Themear1squareerrorisoneofrr1any-vva}'Sofdefinir1gtheaccuraC}'ofanestimate.Twootheraccuracyrr1easures,whichrr1ightbeappropriatetocertainapplications,aretheexpectedvalt1eoftheabsolt1teestirr1atior1errorE[IX-XI]andthernaximurnabsoluteestimationerror,rr1axIXXI.Inthissection,weconfir1eot1rattentiontothemear1squareerror,whichistherr1ostv.ridelyusedaccl1raC}'rneasurebeca11seitlendsitselftorr1atl1ematicalanalysisandoftenleadstoestimatesthatareconver1ienttocornpute.Ir1particl1lar,-vveusethemear1sqt1areerroracct1racyrr1easuretoexarr1inethreedifferentwaysofestirr1atir1grandorn\rariableX.T11eyaredistir1guishedbytheir1formationavailable.Weconsiderthreetypesofinforrnation:A~•T11eprobabilitymodelofX(blir1destirr1ation),•T11eprobabilitymodelofXandinforrr1ationt11att11esamplevaluexEA,•Theprobabilit}'modelofrandorr1variablesXaridYandinformationthaty=y.Theestimationrnethodsfortheset11reesituationsarefur1damentall}'thesarne.Eachoneimpliesaprobabilityrr1odelforX,vvhichrr1aybeaPDF,acor1ditior1alPDF,aPMF,oraconditionalPl\/IF.Inallthreecases,t11eestirnateofXthatprodl1cest11eminirr1umrr1eansquareerrorist11eexpectedvalue(orcor1ditior1alexpectedvalue)ofXcalculatedv.riththeprobabilityrr1odelthatincorporatestheavailableinformatiori.Whiletheexpected\ralueisthebestestirr1ateofX,itrnay[12.1MINIMUMMEANSQUAREERRORESTIMATION401becomplicatedtocalculateinapracticalapplication.Manyapplicationsderi·vear1easilycalculatedhriearestirnateofX,t11esubjectofSection12.2.BlindEstimationofXAnexperimentproducesa,randorr1·variableX.Beforetheexperimentisperformed,vvhatisthebestestirr1ateofX?Thisist11eblin,destirnation,problernbecal1Seitrequiresustorr1akeaninferenceaboutXint11eabser1ceofanyobservations.Althoughitisllnlikelyt11atvve'ivillgl1essthecorrectvaltleofX,vvecanderiveanumberthatcomesasclosea.spossibleint11esenset11atitrninirr1izest11erneansql1areerror.Weer1counteredtheblir1destirnateinSection3.8vvhereT11eorern3.13showsthatXB=E[X]istherninirr1ummeansquareerrorestirr1ateintheabsenceofobservatioris.Thernir1irnumerrorise:B=Var[X].Inintrodl1cir1gtheidea,ofexpectedvalue,Chapter3describesE[X]asa"typicalvalue"ofX.Theorern3.13givest11isdescriptior1arr1athernaticalrnear1ir1g.~Example12.1Beforeasix-sideddieisrolled,whatistheminimummeansquareerrorestimateofthenumberofspotsXthatwillappear?TheprobabilitymodelisPx(x;)=1/6,x=1,2,...,6,otherwisePx(x)=0.Forthismodel,E[X]=3.5.EventhoughXB=3.5isnotintherangeofX,itistheestimatethatminimizesthemeansquareestimationerror.EstimationofXGivenanEventSupposethatinperforrninganexperiment,insteadofobservingXdirectly,welearnonlyt11atXEA.Gi'irenthisinforrnation,whatistheminirnurnmear1squareerrorestirnateofX?Giver1A,Xhasacor1ditior1alPDFf'x1A(x;)oraconditionalPMFPx1A(x).Ourtaskistominirnizetheco'ndition,alrnean,sqv,areerrorexA=E[(X-i;)2IA].\¥eseethatthisisesser1tiall:ythesarr1eastheblir1destirnationproblernwitl1theconditionalPDFf'x1A(xlA)ortheconditionalPMFPx1A(x)replacingf'x(x)orPx(x).Therefore,vvehavethefollo'ivir1g:1Theorem12.1Given,thein,forrnation,XEA,thernin,irn'urnrnean,sq'uareerrorestirnateofXisExample12.2ThedurationTminutesofaphonecallisanexponentialrandomvariablewithexpectedvalueE[T]=3minutes.Ifweobservethatacallhasalreadylasted2minutes,whatistheminimummeansquareerrorestimateofthecallduration?ThisprobabilitymodelalsoappearsinExample7.10.ThePDFofTis[402CHAPTER12ESTIMATIONOFARANDOMVARIABLEf'r(t)=~e-t/3t>O,0otherwise.Ifthecallisstillinprogressafter2minutes,wehaveminimummeansquareerrorestimateofTistEA={T>(12.2)2}.Therefore,theiA=E[TIT>2).(12.3)ReferringtoExample7.10,.wehavetheconditionalPDFfTIT>2(t)=l-(t-2)/33et>2,0otherwise.(12.4)Therefore,00E[TIT>2)=121t-e-(t-2)/3dt=2+3=5minutes.3(12.5)Priortothephonecall,theminimummeansquareerror(blind)estimateofTisE[T)=3minutes.Afterthecallisinprogress2minutes,thebestestimateofthedurationbecomesE[TIT>2)=5minutes.Thisresultisanexampleofthememorylesspropertyofanexponentialrandomvariable.Atanytimeduringacall,theexpectedtimeremainingistheexpectedvalueofthecallduration,E[T).MinimumMeanSquareEstimationofXGivenYConsideranexperirr1entthatprodt1cestwora.ndorr1variables,XandY."\?Vecar1observeYbutvvereallyv.ra.nttoknowX.Therefore,t.heestirr1ationtaskistoassigntoeveryyESy

anurnber,i;,t11atisnearX.Asintheot11ertec11r1iquespresentedinthissection,011raccuraC}'rneasureisthemear1squareerror(12.6)Becauseeac11yESyproducesaspecificXNI(Y),XM(Y)isasample·val11eofarandorr1va1iableX1w('Y).T11efactthati1w(Y)isasarnple·val11eofarandorr1variableisir1contrasttoblindestirnatior1andestirr1ationgi·venanevent.Inthosesitt1ations,XBandXAareparametersoft11eprobabilitymodelofX.Ir1cornrr1onv.rithXBinTheorern3.13andx;Air1Theorerr112.1,theestirnateofXgivenYisanexpected·valueofXbasedonavailableinforrnation.Inthiscase,theavailableir1forrr1atior1isthe·valueofY.---Theorem12.2~-Thernin,irnurnm,ea,n,sqv.areerrorestirnateofXgiven,theobservat'iori·y=yisi;1111(y)=E[XIY=y).[12.1MINIMUMMEANSQUAREERRORESTIMATION403-=Example12.3SupposeXand·yareindependentrandomvariableswithPDFsf'x(x)andj'y(y).WhatistheminimummeansquareerrorestimateofXgivenY?Inthiscase,fx1y(xly)XM(Y)=1:=f'x(x;)andtheminimummeansquareerrorestimateisxfx1Y(xly)dx=1:xfx(x)dx=E[X]=Xs.(12.7)Thatis,whenXandYareindependent,theobservation·yprovidesnoinformationaboutX,andthebestestimateofXistheblindestimate.Example12.4SupposethatRhasauniform(0,1)PDFandthatgivenR=r,Xisauniform(0,r)randomvariable.Find;f;M(r),theminimummeansquareerrorestimateofXgivenR.FromTheorem12.2,weknowXN1(r)=E[XIR=r].Tocalculatetheestimator,weneedtheconditionalPDFf'x1R(xlr).Theproblemstatementimpliesthatl/r0Ia=(12.62)b=E[X]-CxyCy1E[Y]=E[X]-a'E[Y].(12.63)ThusweneedtofindtheexpectedvalueE(Y],thecovariancematrixCy,andthecross-covariancematrixCxy.SinceY=[X1X2]',E[Y]=[E[X1]E[X2l]'=[5062]'>(12.64)andwecanfindthecovariancematrixofYinCx:Cy=[Cx(l,·1_)Cx(2,1)Cx(l,2)]=[16.0Cx(2,2)12.812.8]16.0·(12.65)SinceX=X3,theelementsofCxyarealsoinCx.Inparticular,Cxy=Cvx·where(12.66).S1ncea'=cxYc-y1,a'so1vesa'cy=cxY,·imp1y1ng·a'=[0.27450.6078].(12.67)Furthermore,b=E[X3]-a'E[Y]=58-50a1-62a2=6.591.(b)Themeansquareestimationerrorise[,=Var[X]-a'C~y=16-ll.2a1-12.8a2=5.145degrees2.Here,wehavefoundVar(X]=Var[X3]inCx:·var[X3]=Cov[X3,X3]Cx(3,3).(c)UsingonlytheobservationY=X2,weapplyTheorem12.3andfinda,*Cov(X2>X3]=Var(X2]=12.816=0.8,b*=E[X]-a*E[Y]=58-0.8(62)=8.4.(12.68)(12.69)(d)Themeansquareerroroftheestimatebasedon·y=X2ise[,=Var[X]-a*Cov["Y,X]=16-0.8(12.8)=5.76degrees2.(12.70)Ir1Example12.9>weseetl1a.ttheestirr1atorerr1ployingbothX1andX2canexploitthecorrelationofX1andX3toofferareducedmeansquareerrorcorr1paredtotheestirnatort:hatusesjustX2.[12.4LINEARESTIMATIONOFRANDOMVARIABLESFROMRANDOMVECTORS419Ifyol1gotoweather.com)}'OUvvillreceiveacornprehensivepredictionoft11efuture\Veat11er.Ifxiist11eterr1peratl1rei11oursfrornnow,thewebsitev.rillrr1akepredictionsXA=[XA1XAn]'oft11evectorX=[X1Xn]'offuturetemperatures.ThesepredictionsarebasedonavectorYofa\railableobservations.Thatis)aweather.compredictionisthe\rectorft1nctionX=X(Y)ofobservationY.\i\Thenusir1gvectorYtoestimate\rectorX,theMSEbecorr1esAA(12.71)vVeseeinEqt1ation(12.71)t11attheJ\1ISEredt1cestothesurr1oftheexpectedsquareerrorsinestimatingeac11cornpor1entXi.TheMJ\!{SEsolutionistousetheobservationYtornakeanMNISEestirnateofeachcornponentX,iof\rectorX.Ir1thecontextoflinearestirnatior1)t11eoptirr1urnlinearestirnateofeachcomponentXiisXi(Y)=a~Y+b,i)vvitha~andbiasspecifiedb}'Theorem12.7withX=X,i·Theoptirnurnlinearvectorestirr1ateisXn(Y)]1vVriting1•(12.72)XL(Y)inmatrixforrr1}'ieldsthe\rectorgeneralizationofT11eorern12.7.---Theorem12.8Xisan,rn-dirnen.sion,alran,dornvector'tuithexpectedvalv,eE(X].Yisan,n,dirnen,sion,alraridornvector'llJ'ithexpectedval'ueE(Y]an,dn,xn,covaria/ncernatriJ;Cy.XaridYhavernxn,cross-covarian,cernatr~ixCxy.Thernin,irnv,rnrnean,sqv,areerrorlin,earestim,atorofXg'iven,theobservation,Yis1XL(Y)=CxvCy(Y-E(Y])+E(X].AProofFromTheorem12.7,(12.73)(12.74)Itisofter1convenienttorepresenttheoptimt1rr1linearestirr1atorofTheorern12.8int11eforrn(12.75)[420CHAPTER12ESTIMATIONOFARANDOMVARIABLEvvith-1~b=A=CxyCy,W11enE[X)redt1cesto=0andE[Y)=0,Cy=RyE[X)-AE[Y).andCxy(12.76)=Rxy,andT11eorerr112.8(12.77)thegeneralizatior1ofTheorem12.6totheestirnationofthe·vectorX.Inaddition,frorr1ybecauseeachcornpor1entofL(Y)ist11eoptirnurr1lir1earestirnateofasgiver1byT11eorerr112.7,theMSEandorthogonalitypropertiesofXi(Y)giveninTheorern12.7rerr1air1thesarr1e.T11eexperimentinExample12.9consistsofaseqt1enceofn,+1s11bexperirr1er1tsthatprodt1cerandorn·variablesX1,X2,...Xn+1.T11eestimatorusestheoutcomesofthefirstriexperirnentstoformalinearestirr1ateoftheot1tcorr1eofexperirr1entn,+1.vVerefertothisestirnatior1procedureaslin,earpredictioribecauseitllSesobservationsofearlierexperimentstopredicttheoutcorr1eofas11bsequer1texperirnent.vVhenthecorrelationsoft11erar1domvariableshavethepropertythatTxi,Xjdependsonlyor1thedifferenceIi-jI,theestirr1ationequationsinTheorern12.8haveastrt1cturethatisexploitedinmanypracticalapplications.Toexarr1inetheimplicationsoft11isproperty,weadoptther1otationxixxiRx(i,j)=rl,i-.JI·(12.78)InChapter13weobservethatt11ispropert}'ischaracteristicofrandom·vectorsderi·vedfrorna111idesen,sestat'ion,aryran,dornseq11,en,ce.Ir1thenotatior1ofthelinearestimationmodeldevelopedinSection12.4,X=Xn+1andY=[X1X2Xn]'.T11eelementsofthecorrelationmatrixRyandt11ecross-correlationmatrixRyxall11avetheforrnTo'f']_rn-1TnT1ro'f'n-2Tn-1Ry=lrn-1'r1roRyx=(12.79)lJr1JHereRyandRyxtogetr1erha·veaspecialstructure.T11ereareonlyri+1differentnt1rr1bersamongtheri2+n,elementsofthetv.romatrices,andeachdiagonalofRyconsistsofidenticalelements.T11ismatrixisinacategor}'referredtoasToeplitzforrns.ThepropertiesofRyandRyxrr1akeitpossibletosolvefora'inEq11ation(12.60)vvithfarfevvercorr1putationsthanarerequiredinsolvinganarbitrar}'setofrilinearequations.ManyaudiocompressiontechniquesusealgorithmsforsolvinglinearequationsbasedonthepropertiesofToeplitzforms.===Quiz12.4===X=[X1X2J'isarandomvectorwit11E[X)=0andautocorrelationrr1atrixRxvvithelerr1entsRx(i,j)=(-0.9)li-.71.ObservethevectorY=X+W,v.r11ereE[W)=0,E[vV{)=E[vV?J=0.1,aridE[W1W2)=0.WandXareir1dependent.[12.sMATLAB421(a)Finda*,thecoefficientoft11eoptirr1t1mlinearestimatorofX2given·y2andtherneansql1areerrorofthisestirr1ator.(b)Findthecoefficientsa'=[a1a2Joftheoptirr1urr1linearestirnatorofX2giver1·y1and·y2,andtherneansquareerroroft11isestimator.12.5MA1"'LABTherr1a.trixorientationoflVIATLABmakesitpossibletovvriteconciseprogra.rr1sforgenera.tingthecoefficientsofalinearestirr1atoraridcalcula.tir1gtl1eestirr1ationerror.Thefollov.ringexa.rnpleexplorestherelatior1shipoft11emeansqt1areerrortothent1rr1berofobservationsl1sedinalinearpredictorofarar1dornvariable.===-Example12.10--==:::::iThecorrelationmatrixRxofa21-dimensionalrandomvectorXhasi,jthelement(12.80)Wisarandomvector,independentofX,withexpectedvalueE[W]=0anddiagonalcorrelationmatrixRw=(0.1)I.Usethefirstn,elementsofY=X+WtoformalinearestimateofX21andplotthemeansquareerroroftheoptimumlinearestimateasafunctionofn,for(a)r-li-jl-sin(O.l7rl'i-JI)0.17fli-jl)(b)rl,i-11=cos(0.57rli-jl).Inthisproblem,letW(n)•X(n)•andY(n)denotethevectors,consistingofthefirstn,componentsofW,X,andY.SimilartoExample12.8,independenceofX(n)andW(n)impliesthatthecorrelationmatrixofY(n)is(12.81)NotethatRx(n)andRw(n)aretherixn,upper-leftsubmatricesofRxandRw.Inaddition,R'X-Ry(n)x--EY(n)-(12.82)Thustheoptimumlinearestimatorbasedonthefirstn,observationsis(12.83)[422CHAPTER12ESTIMATIONOFARANDOMVARIABLE---0.1,...~.....io.._~~~--~~~--~__,020100.05o--~~~--~~~--~--o2010nnra=[1sin(0.1*pi*(1:20))..../(0.1*pi*(1:20))];mse(ra);rb=cos(0.5*pi*(0:20));mse(rb);(a)(b)Figure12.3,.fv;,roRunsofmse.mandthemeansquareerroris(12.84)functione=mse(r)N=length(r);e=[];rr=fliplr(r(:)');forn=1:N,RYX=rr(1:n)';RY=toeplitz(r(1:n))+0.1*eye(n);a=RY\RYX;en=r(1)-(a')*RYX;e=[e;en];endplot(1:N,e);mse.mcalculatesthemeansquareerrorus-ingEquation(12.84).Theinputrcorrer20],whichspondstothevectoristhefirstrowoftheToeplitzcorrelationmatrixRx.NotethatRx(n)istheToeplitzmatrixwhosefirstrowcontainsthefirstn,elementsofr.Toplotthemeansquareerrorasafunctionofthenumberofobservations,'n,wegeneratethevectorrandthenrunmseCr).Forthecorrelationfunctions(a)and(b)intheproblemstatement,thenecessaryM.A.TLABcommandsandcorrespondingmeansquareestimationerroroutputasafunctionof'nareshowninFigure12.3.[roIncorriparingtlierest1ltsofcases(a)arid(b)iriExample12.10,weseetliattherneansquareestimationerrordeperidsstronglyontliecorrelationstructuregiveribyrl,i-.il.Forcase(a),sarnplesXnforn,<10liaveverylittlecorrelationvvithX21.Tliusforn,<10,theestimatesofX21areonlyslightlybettertliantheblindestirnate.Ontheotherha.nd,forcase(b),XiandX21arecompletelycorrelated;px1,x21=1.Forn,=1,·y1=X1+W1issirnplyarioisycopyofX21,aridtheestirnationerrorisduetothevarianceofW1.Inthiscase,asn,increases,tlieoptirriallinearestirnatorisabletocorribirieadditionalnoisycopiesofX21,:yieldingfurtherreductiorisiritlierrieansqt1areestirnationerror.[PROBLEMS423Quiz12.5EstimatetheGaussian(0,1)randorr1variableXusingt11eobservationvectorY=lX+W,"''here1isthevectorof201's.T11enoisevectorW=[vV0vV19isir1deper1dentofX,haszeroexpectedvall1e,andhasacorrelationrr1atrixvvithi,jthentryRw(i,j)=cli-.il-1.Fir1dXL(Y),t11elinearMMSEestirr1ateofXgivenY.Forcintherange0<c<1,v.r11at·valueofcminirnizestherr1eansquareerroroftheestimate?J'FurtherReadin,g:Thefin[tlc11apterof('\i\TSOl]presentst11ebasictheor}'ofestirr1ationofrandorr1variablesaswellasextensionstostochasticprocessestirr1ationinthetimedomair1andfreqil.1enC}'dornain.ProblemsDifficulty:Easy12.1.1GeneralizingthesolutionofExample12.2,letthecalldurationTbeanexponential(.A)randomvariable.Forto>0,showthattheminimummeansquareerrorestimateofT,giventhatT>toisT=t0+E[T].12.1.2XandYhavethejointPDFfx,y(1;,y)=6(y-::r){00+DifficultModerateExpertsOnly(c)\t\fhatistheininimummeansquareerrorestimateofXgivenX>1/2?(d)\iVhatisfy(y)?(e)\i\fhatistheblindestimatefJB?(f)\i\fhatistheminimummeansquareerrorestimateofYgivenX>1/2?12.1.4XandYhavethejointPDF.fx,Y(x,y)={6(y-x)0O0.5?12.1.3j.x'y(x,y)={20OMFofXgivenY=-3.(g)FindX1w(-3),theoptimum(nonlinear)meansquareestimatorofXgiveny=-3.(h)Findthemeansquareerrore*(-3)=E[(X-XAJ(-3))2IY=-3]ofthisestimate.12.2.2Atelemetryvoltage,I,transmittedfro1napositionsensoronaship,srudder,isarandomvariablewithPDFj.\I(v)={1/12-6<v<6,0otherwise.Areceiverintheship,scontrolroomreceivesR=V+X,TherandomvariableXisaGaussian(0,Vs)noisevoltagethatisindependentofV.ThereceiverusesRtocalculateAalinearestimateofthetelemetryvoltage:V=aR+b.Find(a)theexpectedreceivedvoltageE[R],(b)thevarianceVar[R]ofthereceivedvoltage,(c)thecovarianceCov[V,R]ofthetransmittedandreceivedvoltages,(d)a*andb*,theoptimumcoefficientsinthelinearestimate,(e)el,,theminimummeansquareerroroftheestimate.[PROBLEMS12.2.3RandomvariablesXandYhavejointPl\/IFgivenbythefollowingtable:Px,Y(x,y)X=-1y=-1y=O3/161/61/161/601/8x=Ox=l\Ji.,!eestimateYbyYL(X)y=l01/61/8=aX+b.12.2.4TherandomvariablesXandYhavethejointprobabilitydensityfunction(){2(y+::r)::e,y=0Ootherwise.\!\,!hatisXL(Y),thelinearininimummeansquareerrorestimateofXgivenY?12.2.5ForrandomvariablesXandYfromProblem12.1.4,findXL(Y),thelinearminimummeansquareerrorestimatorofXgivenY.12.2.6RandomvariableXhasasecondorderErlangPDF.()_jxx-\/\Xe{0-.Ax,x>0,otherwise.=:i~,YM(x),(b)theMl\IISEestimateofXgivenY(b)the1\1.IMSEestimateofXgivenR=r,Xj\1(r),(c)theLl\/ISEestimateofRgivenX,RL(X),12.2.8ForrandomvariablesXandY,vvewishtouseYtoestimateX.However,ourestimatemustbeoftheformX=aY.(a)Finda*,thevalueofathatminimizesthemeansquareerrore=E[(X-aY)2].(b)Fora=a*,whatistheminimummeansquareerrore*?(c)·underwhatconditionsisestimateofX?12.2.9=y,XtheLMSEHerearefourdifferentjointPlVIFs:1/91/91/9x=Ox=l1/91/91/91/91/91/9=1J,X=Y=-1y=Oy=lPu,v('IL,v)11,-1-1=011,=101/3v=O001/3v=l1/3000V=GivenX=x,Yisauniform(0,x)randoinvariable.Find(a)the1\/11\/ISEestimateofYgivenXfA1(x),XL(R).(b)Whatistheminimummeansquareerrorel,?,Y(a)theMMSEestimateofRgivenX=x,(d)theLMSEestimateofXgivenR,(a)Findaandbtominimizetheineansquareestimationerror..jX425-1s=Os=lPs.rs,t)t=-1s=-11/601/6t=O01/30t=11/601/6Pc,Rq,rr=-1r=Or=lq=-1q=0q=11/121/121/61/121/61/121/61/121/12XM(y),(c)theLMSEestimateofYgivenX,YL(X),(d)theLMSEestimateofXgivenY,XL(Y).12.2.7RandomvariableRhasanexponentialPDFwithexpectedvalue1.GivenR=r,XhasanexponentialPDFwithexpectedvalue1/r.Find(a)l:[426CHAPTER12ESTIMATIONOFARANDOMVARIABLE(b)Co1nputetheleastmeansquarelinearestimatorUL(V)ofUgivenV.Whatisthemeansquareerror?DothesameforthepairsX,Y,Q,R,andS,T.12.2.10RandomvariableY=X-ZisanoisyobservationofthecontinuousrandomvariableX.ThenoiseZhaszeroexpectedvalueandunitvarianceandisindependentofX.Considerthefollo,vingargument:SinceX=Y+Z1111eseethatifY=y1thenX=y+Z.Th1ls1byTheorem6.41theconditionalPDFofXgivenY=yisfx1y(xly)=fz(x-y).Itfollo1DsthatE[XIY=y]=1_:::rfx1y(xly)d1;1;fz(x-y)dx.(b)FindtheMAPestimateofRgivenN.(c)Findthel\!ILestimateofRgivenN.12.3.4Flipacoinn,times.Foreachflip,theprobabilityofheadsisQ=qindependentofallotherflips.Qisauniform(0,1)randomvariable.Kisthenumberofheadsinnflips.(b)\i\fhatisthePl\!IFofJ{?\i\fhatisE[I-ooWiththevariables1lbstit1ltion1z=(a)Findthel\!ll\!ISEestimateofRgivenN.(a)WhatistheMLestimatorofQgivenK?00=/_12.3.3LetRbeanexponentialrandomvariablewithexpectedvalue1/µ,.IfR=r,thenoveranintervaloflengthT,thenumberofphonecallsNthatarriveatatelephoneswitchhasaPoissonPMF\VithexpectedvaluerT.1;-y1E[XIY=y]=1_:(z+y)fz(z)dz=E[Z]+y=y.Weconcl1ldethatE[XIY]=Y.SinceE[XIY]isoptim,alinthemeansquaresense171Jeconcludethattheoptim,allinearestim,atorX(Y)=aYmustsatisfya=1.Provethatthisconclusioniswrong.\iVhatistheerrorintheaboveargument?Hint:FindtheLlVISEestimatorXL(Y)=aY.12.3.1SupposethatinQuiz12.3,R,measuredinmeters,hasauniformPDFover[O,1000].FindtheMAPesti1nateofRgivenX=1;.Inthiscase,aretheJVIAPandMLestimatorsthesame?12.3.2LetRbeanexponentialrandomvariable\Vithexpectedvalue1/JJ,.IfR=r,thenoveranintervaloflengthT,thenumberofphonecallsNthatarriveatatelephoneS\VitchhasaPoissonPMFwithexpectedvaluerT.(a)FindtheMlVISEestimateofNgivenR.(b)FindtheMAPestimateofNgivenR.(c)Findthel\!ILestimateofNgivenR.(c)\iVhatisfQIK(qlk)?theconditionalPDF(d)FindtheMl\!ISEestimatorofQgivenK=k.12.4.1·You\VouldliketoknowasamplevalueofX,aGaussian(0,4)randomvariable.Hovvever,youonlycanobservenoisyobservationsoftheformYi=X+Ni.Intermsofavectorofnoisyobservations,youobserveY=[i]=[~]X+[~~],where1'l1isaGaussian(0,1)randomvariableandN2isaGaussian(0,2)randomvariable.·undertheassumptionthatX,Ni,andN2aremutuallyindependent,answerthefollo,vingquestions:(a)SupposeyouuseY1asanestimateofX.TheerrorintheestimateisD1=Y1-X.WhataretheexpectederrorE[D1]andtheexpectedsquarederrorE[Di]?(b)Suppose\VeuseY.1=(Y1+Y2)/2asanestimateofX.TheerrorforthisestimateisD3=Y3-X.FindtheexpectedsquarederrorE[D§].IsY.1orY1abetterestimateofX?[PROBLEMS427(c)LetY4=AYwhereA=[a1-a]isa1x2matrix.LetD4=Y4-XdenotetheerrorinusingY4asanestimateforX.Intermsofa,\¥hatistheexpectedsquarederrorE[DiJ?vVhatvalueofaminimizesE[D~]?Yisatv:.ro-dimensionalrandomvectorwith12.4.2(a)FindtheoptimumcoefficientsXisathree-dimensionalrandomvectorwithE[X]=0andautocorrelationmatrixRxwithe1ementsTij=(-0.80)li-jl.·useX1andX2toformalinearestimateofX3:X3=aiX2+a2X1.(a)Whataretheoptimumcoefficientsa,1and(b)UseX2toformalinearesti1nateofX3:X3=aX2+b.\i\fhataretheoptimumYi=Xi+X2+Wi,Y2=X2+X3+W2.·useYtoformX1=[aiestimateofXi.a2]Y,alinear(b)l!seY1toformalinearestimateofXi:X1=aYi+b.\tVhataretheoptimumcoefficientsa*andb*?\i\fhatisthemini1nummeansquareerrorel,?12.4.5SupposeAcoefficientsa*andb*andcorrespondingminimummeansqt1areerrorel,?12.4.3Xisa3-dimensionalrandomvector\vithE[X]=0andautocorrelationmatrixRxwithelementsRx(i,J)=1-0.25li-JI·Yisatwo-dimensionalrandomvector\vithwhereqo+q1k+q2k2isanunkno,vnquadraticfunctionofkandzkisasequenceofiidGaussian(0,1)noiserandomvariables.\i\fewishtoestimatetheunkno\¥11parametersqo,q1,andq2ofthequadraticfunction.Supposeweassumeqo,qi,andq2aresamplesofiidGaussian(0,1)randomvariables.FindtheoptimumlinearestimatorQ(Y)ofQ=[qoqiq2J'giventheobservationY=[YiYn]'.12.4.6Xisathree-dimensionalrandomvectorwithE[X]=[-101andcorrelationmatrixRx\vithelementsUseYtoformX1estimateofXl·(a)Findtheopti1numcoefficientsii1anda,2andtheminimummeansquareerrore£.(b)l!_seY1toformalinearestimateofXi:Xi=+b.\i\fhataretheoptimumcoefficientsa*andb*?Whatistheminimummeansquareerrorel,?aY112.4.4Xisathree-dimensionalrandomvectorwithE[X]=0andcorrelationmatrixRxwithelementsRx(i,J)=1-0.25li-JI·Wisatwo-dimensionalrando1nvector,independentofX,withE[W]0,E[W1vV2]=0,andE[W12]=E[vV22]=0.1.J'Rx(i,J)=1-0.25li-JI·Wisatwo-dimensionalrandomvector,independentofX,\¥ithE[W]0,E[W1W2]=0,andE[Wt]=E[Wi]=0.1.Yisat\vo-dimensionalrandomvectorwithYi=Xi+X2+Wi,Y2=X2+X3+W2.·useYtoformalinearestimateofXi:(a)\t\fhataretheoptimumcoefficientsii1,[428CHAPTER12ESTIMATIONOFARANDOMVARIABLE(b)Findthel\!IMSEel,.(c)trixRxwithi,jthelementl!_seY1toformalinearestimateofX1:X1=aY1+b.\i\fhataretheoptimumcoefficientsa*andb*?\tVhatistheminimummeansquareerrore1?12.4.7WhenXandYhaveexpectedvaluesµx=µy=0,Theorem12.3saysthatXL(Y)=px,Y~YY.ShowthatthisresultisaspecialcaseofTheorem12.8whenrandomvectorYistheone-dimensionalrandomvariableY.12.4.8Provethefollo,vingtheorem:Xisann-dimensionalrando1nvectorwithE[X]=0andautocorrelationmatrixRx\Vithelementsrij=cli-jl,\vherelcl<1.TheoptimumlinearestimatorofXn,AXn=alXn-1+a2Xn-2+...+Om,-1X1,isXn=cXn-1.Theminim11mmeansquareestimationerrorisel,=1-c2.Hint:Considerthen-1equations8eL/8ai=0.12.4.9IntheCDMAmultiusercommunicationssystemintroducedinProblem11.3.8,eachuseritransmitsanindependentdatabitXisuchthatthevectorX=[X1Xn]'hasiidcomponents\VithPxi(l)=Pxi(-1)=1/2.ThereceivedsignaliskY=Li=lxiyp:si+N,'vhereNisaGaussian(0,cr2I)noise.(a)BasedontheobservationY,findtheLMSEestimateXi(Y)=a~YofXi.x(b)Let=[X1xk]IdenotethevectorofLMSEestimatesofbitstransmittedbyusers1,...,k.Sho'vthat12.5.1ContinuingExample12.10,the21dimensionalvectorXhascorrelationma-..)_sin(¢o7rIiRX(i,}1·I<"-2-1•••000.5•••••0.501Discrete-Time,Discrete-Value2•••2.....><"•{Continuous-Time,Discrete-Value~••I-••-0.5-11•••-2-0.5••~'--·0>-2••••••0•-2•••••••••--1-0.500.51-1I-0.500.51lSan1plefunctionsoffourkindsofstochasticprocesses.Xcc(t)isacontinuoustime,continuous-valueprocess.Xdc(t)isdiscrete-tin1e,continuous-valueprocessobtainedbysan1plingXcc)every0.1seconds.RoundingXcc(t)tothenearestintegeryieldsXcct(t),acontinuous-time,discrete-valueprocess.Lastly,Xctct(t),adiscret&time,discret&valueprocess,canbeobtainedeitherbysan1plingXccL(t)orbyroundingXdc(t).Figure13.313.2RandomVariablesfromRandomProcessesTheprobabilityrr1odelfortherar1domprocessX(t)specifiesforallpossible{t1>...,tk}thejointPNIFPx(ti),...,X(tk)(x1,...,xk)orthejointPDFf'x(t1),...,X(tk)(x:1,...,xk)·St1pposeweobserveastochasticprocessatapartic11lartimeinstantt1.Inthiscase,ea.cl1time-vveperforrr1theexperirnent,°'if.leobserveasamplefunctionx:(t,s)andt11a.tsarnpleft1r1ctionspecifiesthe·valueofx(t1,s).Ea.chtime-vveperformt11eexperirr1ent,-vve11aveane~rsa.nd-vveobserveane-vvx(t1>s).Therefore,ea.chx(t1,s)isasarnplevalueofarandomvariable.SN'euset11er1otationX(t1)forthisrandorr1variable.Likeanyotherrandorr1variable,ithaseithera.PDFfX(t1)(x)oraPMFPx(ti)(x).Notethatthenotatior1X(t)canrefertoeithert11era.ndornprocessorthera.ndornvariablethatcorrespondstothe·valueoftherandorr1processattirnet.Asouranalysisprogresses,wher1-vve-vvriteX(t)>itwillbeclearfrornthecontext-vvhetherwea.rereferringtotheentireprocessortoor1era.r1domvariable.---Example13.8:--1nExample13.5ofrepeatedlyrollingadie,whatisthePMFofX(3.5)?TherandomvariableX(3.5)isthevalueofthedierollattime3.Inthiscase,[436CHAPTER13STOCHASTICPROCESSESPx(3.5)(x)=1/6;i;=1,...,6,0otherwise.(13.3)Example13.9LetX(t)=RIcos27rftibearectifiedcosinesignalhavingarandomamplitudeRwiththeexponentialPDFfR(r)=>0.-..le-r/1010_,T0otherwise.(13.4)IWhatisthePDFf·x(t)(x)?.................................................................................SinceX(t)>0forallt,P[X(t)<;i;)=0for;i;<0.If;i;>0,andcos27rf.t>0,P[X(t)<;i;)=P[R<:i;/lcos27rf.t1)=Whenlo:r:/lcos21fftlfR(r)dr=1-e-:i:/lOlca;27rft1.(13.5)cos27rft-=J0,thecompleteCDFofX(t)is01-e-:1;/10lcos27rft1Whencos27rft-#0,thePDFoffx(t)(:i;)==(13.6)X(t)is1e-x/10lcos27rft110lca;27rft1_,x>0,0otherwise.dFx(t)(x),.G,X1x<O'x>o.Whencos27rft=0correspondingRmaybe.Inthiscase,1·x(t)(x)=variableforeachvalueoft.(13.7)tot=7r/2+k7r,X(t)=0nomatterhowlargeo(x).Inthisexample,thereisadifferentrandomWithrespecttoasinglerandorn·variableX,wefour1dthatallthepropertiesofXaredeterminedfromtl1ePDFfx(x).Sirr1ilarly,forapairofrandorr1·variablesX1,X2,v.rer1eededthejointPDFfx1,x2(:i;1,:i;2).Inparticltlar,forthepairofrandorn·variables,v.refoundthatthemarginalPDF'sfx1(x1)andfx2(x2)werenoter1ol1ghtodescribethepairofrandornvariables.Asirnilarsituatior1existsforrar1dorr1processes.If-vvesarnpleaprocessX(t)atktirneir1stantst1,...,tk,-vveobtaint11ek-dirnensionalrandorr1vectorX=[X(t1)X(tk)]'.Toans-\verquestionsabouttherandomprocessX(t),-vvemustbeabletoans-vverquestior1sabol1tanyrandorn·vectorX=[X(t1)X(tk)]'forari,yvalueofkari,dan,ysetoftirneiristari,tst1,...,tk·InSection8.1,t11erandorr1·vectorisdescribedbythejointPMFPx(x)foradiscrete-vall1eprocessX(t)orb}'thejointPDFfx(x)foracor1tinuous-valueprocess.[13.3INDEPENDENT,IDENTICALLYDISTRIBUTEDRANDOMSEQUENCES437Forarandorr1variableX,v.recoulddescribeXbyitsPDFf'x(x),v.rithoutspecifyingt11eexactunderlyir1gexperirr1er1t.Ir1thesameway,kr1owledgeofthejointPDFf'x(ti),...,X(tk)(x;1,...,Xk)fora.11kwilla.llov.rllStodescribeara.ndornprocess-vvithot1treferencetoan11nderlyingexperiment.Thisisconvenientbecauserr1a.nyexperirnentslea.cltot11es~1rnestochasticprocess.Thisisar1alogoustot11esituation-vvedescribedearlierinw11ichmorethanoneexperiment(forexarnple,flippinga.coinortransmittingonebit)producesthesarnerar1domvariable.InSection13.1,therea.retv.roexarr1plesofra.ndornprocessesbasedonrr1easurements.Therea.1--vvorldfactorsthatinfiuer1cetheserr1ea.surementscanbe·verycornplicated.Forexarr1ple,thesequenceofdailyternpera.turesofExample13.3istheresultofa.·verylargedynarnicweathersystemt11atisor1l}'partiallyunderstood.Justasviedevelopedrandom·variablesfrorr1idealizedmodelsofexperirner1ts,v..rev..rillconstructrandomprocessesthata.reidealizedmodelsofrealphenorr1er1a.Thenextt11reesectionsexarr1ir1etheprobabilityrr1odelsofspecifictypesofstochasticprocesses.Quiz13.2Inaproductionlir1efor1000resistors,t11eactualresistanceinohrnsofeachresistorisa.l1r1iform(950,1050)rar1domvariableR.Theresistancesofdifferentresistorsa.reindependent.Theresistorcornpanyha.sa.norderfor1%resistorswitha.resistancebetweer1990na.nd1010n.Anautomatictestertakesoneresistorpersecondandrneast1resitsexactresistance.(T11istesttakesor1esecond.)Thera.ndorr1processJV(t)denotesthent1rnberof1%resistorsfoundir1tseconds.Thera.ndorr1variableTrsecondsistheelapsedtirneatv.r11ichr1%resistorsa.refot1r1d.n(a)(b)(c)(d)(e)W11a.tis'[J,theprobabilit}'thatanysingleresistorisa1%resistor?'\i\ThatisthePMFofJ\T(t)?WhatisE[T1]seconds,theexpectedtirnetofir1dthefirst1%resistor?Whatistheprobabilitythatthefirst1%resistorisfoundinexactly5seconds?Iftheautomatictesterfindsthefirst1%resistorir110seconds,v.rhatisE[T2IT1=10],theconditionalexpected·valueofthetirneoffindir1gthesecond1%resistor?13.3Independent,IdenticallyDistributedRandomSequencesTheiidrandornsequenceX1,X2,...isadiscrete-timestochasticprocessconsistingofaseql1enceofindependent,identicallydistributedrandomvariables.Anindependentidenticallydistribl1ted(iid)randornsequenceisarandornseqt1enceXninv.rhich...,X_2,X_1,X0,X1,X2,...areiidrandorn·variables.Aniidrar1domsequenceoccurs\iVhenever\iVeperforrnindependenttrialsofanexperirner1tata[438CHAPTER13STOCHASTICPROCESSESconstantrate.Aniidrandorr1seq11encecar1beeitherdiscrete-valt1eorcontinuot1svalue.Inthediscretecase,eachrandomvariableXihasP1!fFPxi(x)=Px(x),"''hileinthecontinuot1sca,se,eachXihasPDFf"xi(x)=f'x(:i;).Example13.10InQuiz13.2,eachindependentresistortestrequiredexactly1second.LetRnequalthenumberof1%resistorsfoundduringminuteri.TherandomvariableRnhasthebinomialPMFPnn(r)=(~O)pr(l-p)60-r_(13.8)Sinceeachresistorisa1%resistorindependentofallotherresistors,thenumberof1%resistorsfoundineachminuteisindependentofthenumberfoundinotherminutes.ThusR1,R2,...isaniidrandomsequence.-----:=Example13.11==Intheabsenceofatransmittedsignal,theoutputofamatchedfilterinadigitalcommunicationssystemisaniidsequenceX1,X2,...ofGaussian(0,0")randomvariables.Xn]'isForaniidrandornseq11er1ce,theprobabilitymodelofX=[X1easyto"''ritesinceitistheproductoft11eindividualPMFsorPDFs.Theorem13.1==----iLetXnde'notean,iidran,dornseq?J,e'nce.Foradiscrete-va[1J,eprocess)thesarnple/XnkJhasjoin,tPMFvectorX=[Xn1kPx(x)=Px(x;1)Px(x;2)···Px(xk)=IIPx(x;i).'i=lForacor1,t'iri'1J,O'IJ,S-'ualv,eJJrocess)thejoiritPDFofX=[Xn11··•,Xnk]iskf"x(x)=fx(x1)f"x(x2)···f"x(xk)=IIf'x(x;i)·'i=lOfalliidrandorr1sequences,perl1apstheBernoullirar1domsequenceisthesirnplest.---Definition13.1;---BernoulliProcessABern,011,lli(p)processXnisan,i'idran,dornseq'1J,en,cein,'tvhicheachXn1,saBern,O'/J,lli(IJ)raridornvariable.[13.4THEPOISSONPROCESS439===Example13.12===lnacommonmodelforcommunications,theoutputX1,X2,...ofabinarysourceismodeledasaBernoulli(TJ=1/2)process.===-Example13.13Eachday,webuyaticketfortheNewYorkPick4lottery.Xn=1ifourticketondayn,isawinner;otherwise,Xn=0.TherandomsequenceXnisaBernoulliprocess.c:::==Example13.14FortheresistorprocessinQuiz13.2,let·yn=1if,intherithsecond,wefinda1%resistor;otherwise·yn=0.TherandomsequenceYnisaBernoulliprocess.Example13.15ForaBernoulli(p)processXn,findthejointPMFofX=[X1ForasinglesampleX,;,,wecanwritetheBernoulliPMFinthefollowingway:WhenxiE{O,1}for'ir/l:i(l-rJ)1-xix;,;,E0otherwise.=1,...,ri,thejointPMFcan{O,1},(13.9)bewrittenasnPx(x)=IJr/i:i(l-rJ)1-xi=JJk(l-rJ)n-k,(13.10)'l=lwherek=x1Px(x)=+···+Xn.ThecompleteexpressionforthejointPMFisp:r;1+···+~1;n(1-'[J)n-(:r;1+···+xn)0XiE{0,1},'i=1,...,T/,,otherwise.(13.11)Quiz13.3Foraniidrandorr1seql1enceXnofGaussian(0,1)randomvariables,findthejoint1PDFofX=[X1X1n]•13.4ThePoissonProcessThePoissonprocessisarnemorylesscountingprocessinw11ichar1arrivalatapartic11larinstantisindepender1tofanarrivalatanyotherir1stant.[440CHAPTER13STOCHASTICPROCESSESN(J)4I5.4.3'2.J.S,.S,..Sii+X,•ux~~•..1•--~Figure13.4.'S,•1•X,•1•XsS1••.I~San1plepathofacountingprocess.AcountingprocessJV(t)startsattime0andcountst11eoccurrencesofevents.Theseeventsareger1erallycalledarrivalsbecal1secountingprocessesaremostoftentlsedtornodelthearrivalsofcustornersataservicefacility.However,sincecountingprocessesha·veman:yrapplications,\Vev.rillspeakaboutarri·valsv.rithoutsayingv.rhatisarr1\rir1g.Sincewestartattirnet=0,'n(t,s)=0forallt<0.Also,thenurr1berofarri\ralsuptoanyt>0isanir1tegerthatcanr1otdecreasevvithtirr1e.---Definition13.8:=---CountingProcessAstochasticprocessN(t)isacountingprocessiffore1;erysarnplefv,n,ctiori,'n(t,s)=0fort<0an,dr1,(t,s)isin,teger-valuedaridn,on,decreasin,g'tuithtirne.Wecar1thir1kofN(t)ascour1tingthenurnberofcustornersthatarriveatasysterr1dl1ringt11eir1terval(0,t].Atypicalsarr1plepathofJ\T(t)issketchedinFigure13.4.Thejl1mpsinthesamplefunctior1ofaCOl1ntingprocessrr1arkthearrivals,andt11enl1rnberofarrivalsint11einterval(t0,t1]isjustJ\T(t1)-N(t0).Wecar1useaBerr1oulliprocessX1,X2,...toderi\reasirnplecountir1gprocess.Inparticular,considerasmalltimestepofsize~secondssuc11thatthereisonearrivalint11einterval(ri,~,(ri,+1)~]ifandor1lyifXn=1.ForanaveragearrivalrateA>0arrivals/second,wecar1choose~Sl1chthat,A~-T/ri,!'T/,0otherwise.=(13.12)~0,1,2,...)--+0,thePl\/IFof(13.13)[13.4441THEPOISSONPROCESSvVecangeneralizethisargurr1enttosaythatforan}'interval(t0>t1],ther1urr1berofarrivals"''ouldhaveaPoissonPMFwithparameter>..TvvhereT=t1-to.]\/Ioreo·ver,t11enurr1berofarrivalsin(to,t1]deper1dsor1theindependentBernoullitrialscorrespondingtothatinterval.Thusther1l1rnberofarrivalsinnonoverlappingintervals'ivillbeindependent.Inthelirnitas~-+0,'ive11aveobtainedacountir1gprocessinvv11ichthenurr1berofarrivalsinan}'intervalisaPoissonrandomvariableindependentofthearri'ir[tlsinanyothernonoverlappingir1terval.'\"!Vecallthislirnitir1gprocessaPoisson,process.Definition13.9PoissonProcessAcov,n,tin,gprocess1'l(t)isaPoissonprocessofrate>..if(a)The'n'urnberofarrivalsir1,an,yin,ter·ual(to,t1LN(t1)-J\T(to),isaPoisson,rar1,dornvariable111ithexpectedval'ue>..(t1-t0).{b)Forariypairofrion,overlappirigin,terval,s(t0,t1]ar1,d(t~,t~],then/u,rnberofarr'ivalsin,eachir1,terval,JV(t1)-N(t0)an,dJ\T(t~)-J\T(t~),respectively,arein,depen,den,traridorrivariables.vVecall>..t11erateoftheprocessbecauset11eexpectednl1rnberofarrivalsperllnittimeisE[J\T(t)]/t=.A.Bythedefir1itionofaPoissor1randorr1variable,M=JV(t1)-N(t0)hasthePNIF[>-(t1-to)]=e->-(t1-to)rn!PM(rn,)=7Tl,=0)1,...,(13.14)other'ivise.0Forasetoftirneinstantst1<t2<···<tk,wecanusethepropertythatthenurr1berofarrivalsinnono'irerlappingintervalsareindeper1dentto'ivritet11ejointPMFofN(t1),...,N(tk)asaproductofprobabilities.===Theorem13.2'==::::::iForaPoissoriprocessJV(t)ofrate>..,thejoiritPMFofNfororderedtirneir1,stan,cest1<···<tk,is=[N(t1)>...,N(tk)]',n1-°'1a.ea.n2-n1e-O<·;inil(n2-n1)!0<0other'1nise,ProofLetMl=N(t1)andforin,1<···<'nk,>1,letMi=N(ti)-N(ti-1).BythedefinitionofthePoissonprocess,JV!1,...,MkisacollectionofindependentPoissonrandomvariablessuchthatE[J\fi)=cxi.P::"J(n)=(n,1,n2-n,1,...,nk-n,k-1)jVJ1(n,1)PM2(n,2-nl)···Pj\1k(n,k-n,k-1)Pj\11=P(13.15),M2,...,1\llkThetheoremfollo,vsbysubstitutingEquation(13.14)for.PjVli(ni-n,i-1).(13.16)[442CHAPTER13STOCHASTICPROCESSESKeepinrr1indthattheindeper1dentintervalspropertyofthePoissonprocessrnust11oldevenforverysrna.11interva.ls.Forexarnple,ther1urr1berofarriva.lsin(t,t+0.Essentially,theprobabilityofanarrivalduringanyinstantisindependentofthepasthistoryoft11eprocess.Inthissense,thePoissonprocessisrnernoryless.Thismemorylesspropertycanalsobeseenwhen\VeexarninethetirnesbetV\reenarri\rals.Asdepictedir1Figure13.4,therandomtirneXnbet\veenarri\raln,-1aridarri\ralri,iscalledthen,thin,terarrivaltirne.Ir1addition,\Vecallthetirr1eX1oft11efirstarrivalthefirstintera,rrivaltirneeventhoughthereisr10pre\riousarri\ral.---Theorem13.3:--ForaPoisson,processofrate,\7thein,terarrivaltirnesX1,X2,...areariiidra/ndornsequen,ce'tJJiththeex;pon,en,tialPDFfx(x)=ProofGivenX1=x1,X2x>-Oother'tJJise.)0=:1;2,...,Xn-1=Xn-1,arrivaln,tn-1=:1;1+1occursattime···+Xn-1·(13.17)Forx>0,Xn>xifandonlyiftherearenoarrivalsintheinterval(tn-1,tn-1+x].Thenumberofarrivalsin(tn-1,tn-1+x]isindependentofthepasthistorydescribedbyX1,...,Xn-1·ThisimpliesThusX11isindependentofX1,...,Xn-1andhastheexponentialCDFl-Fxn(::i;)=1-P[Xn>::i;]={Oe-Ax::i;>0,otherv;.rise.FromthederivativeoftheCDF,weseethatXnhastheexponentialPDFfxn(x)inthestatementofthetheorem.(13.18)=fx(x)Frorr1asamplefur1ctionofN(t),V\recanidentifytheir1terarrivaltirr1esX1,X2andsoon.Sirrrilarly,frorntheinterarrivaltirnesX1,X2,...,wecar1constructthesarr1plefunctionofthePoissonprocessN(t).Thisimpliest:hataneql1ivalentrepreser1tationofthePoissonprocessistheiidrandornsequenceX1,X2,...ofexponentiallydistributedinterarrivaltirnes..--------==Theorem13.4Acov.'ntirigprocessv1ithin,depen,den,tex;1Jorien,tial(..\)in,terarrivalsX1,X2,...1,saPoisson,processofrate..\.[13.5PROPERTIESOFTHEPOISSONPROCESS443==--Quiz13.4--==Datapacketstransmittedbyarnoderr1o·verapl1onelir1eforrnaPoissonprocessofrate10packets/sec.UsingNJktodenotet11er1urnberofpacketstransmittedinthekthhour,findthejointP1!fFofJi.IIIandM2.13.5PropertiesofthePoissonProcessThest1rr1N(t)=NI(t)+N2(t)ofindependentPoissonprocessesNI(t)andJ\T2(t)isaPoissonprocess.ThePoissonprocessN(t)canbedecorr1posedintotwoindepender1tPoissonprocessesJ\TI(t)andN2(t).Thernerr1orylesspropertyofthePoissonprocesscanalsobeseer1intheexponentialir1terarrivaltimes.SinceP(Xn>x]=e->-x,t11econditionalprobabilityt11atXn>t+;r:,givenXn>t,isP[xn>t+XIxn>t]=P[Xn>t+X,Xn>t]=e-.A:.e.p(Xn>t](13.19)Their1terpretationofEq11ation(13.19)isthatifthearrivalhasnotoccurredbytirr1et,theadditior1altirneuntilthearrival,Xn-t,hasthesarr1eexponentialdistributionasXn.Thatis,nornatterhowlongweha;vev.raitedforthearri·val,theremair1ingtirneuntilthea,rri·valremainsanexponential(,\)randorn·variable.Theconsequenceisthatif-vvestartto\VatchaPoissonprocessatan:ytirr1et,weseeastochasticprocesst11atisindistir1guishablefrorr1aPoissonprocessstartedattimeo.T11isinterpretationisthebasisfor\va:ysofcorr1posinganddecorr1posingPoissonprocesses.First\VeconsiderthesumN(t)=J\TI(t)+J\T2(t)oftwoir1dependentPoissonprocessesNI(t)andN2(t).Clearl}',N(t)isaco11r1tingprocesssinceanysarnplefunctior1ofN(t)isnondecreasing.Sinceinterar1ivaltirnesofeachJVi(t)arecontinuousexpor1enti:1,lrar1dom\rariables,t11eprobabilitythatbothprocesseshavearrivalsatthesametirr1eiszero.ThusN(t)ir1creasesbyonearri\ralatatime.Further,Theorerr19.7showedt11att11esurr1ofindependentPoissonrandorr1variablesisalsoPoissori.Thusforanytimet0,N(to)=J\TI(to)+J\T2(t0)isaPoissonrandorr1variable.Thissuggests(bt1tdoesnotprove)thatN(t)isaPoissonprocess.Ir1thefollo-vvingtheorema,ndproof,weverifyt11isconjectt1reb}'shoV\ringthatJV(t)hasir1deper1dentexponentialinterarriva1tirnes.Theorem13.5LetN1(t)aridN2(t)bet'1JJ0irideper1,der1,tPoissor1,processesofrates,\1an,d,\2.Theco'un,tin,gJJrocessJV(t)=NI(t)+N2(t)isa,Poisson,processofrate,\1+,\2.[444CHAPTER13STOCHASTICPROCESSESProofvVesho'vthattheinterarrivaltimesoftheN(t)processareiidexponentialrando1nvariables.SupposetheN(t)processjusthadanarrival.\t\lhetherthatarrival\VasfromN1(t)orN2(t),Xi,theresidualtimeuntilthenextarrivalofNi(t),hasanexponentialPDFsinceNi(t)isamemorylessprocess.Further,X,thenextinterarrivaltimeoftheN(t)process,canbeVi'rittenasX=min(X1,X2).SinceX1andX2areindependentofthepastinterarrivaltimes,Xmustbeindependentofthepastinterarrivaltimes.Inaddition,\VeobservethatX>xifandonlyifX1>xandX2>x.ThisimpliesP[X>x]=P[X1>x,X2>::r].SinceN1(t)andN2(t)areindependentprocesses,X1andX2areindependentrandomvariablessothatP[X>x]=P[X1ThusXisanexponential(..\1>x]P[X2>x]+.A2)={~-(A,+A,)xxx<0,>o.(13.20)randomvariable.Wederivedt11ePoissonprocessofrate,\asthelirnitingcase(as~---+0)ofaBernoulliarrivalprocessthathasar1arrivalir1anir1tervalofler1gth~v.rithprobabilit}'.A~.vV11enweconsiderthest1rr1oftwoindependentPoissonprocessesJV1(t)+N2(t)O\reraninter\raloflength~'eac11processN;,(t)canhaveanarrivalwit11probabilityAi~·Theprobabilitythatbothprocesses11aveanarrivalis,\1,\2~2.As~---+0,~2rocesses111ithrates,\parid.A(l-r>).[13.5PROPERTIESOFTHEPOISSONPROCESS445ProofLetXii),X~i),...denotetheinterarrivaltimesoftheprocessNi(t).\fi.lewillverify122·in·dependentran·domsequences,each'v1t·hexthatX1(l),x(2),...andx1C),x2xiftherearenotype1arrivalsintheinterval[t,t+x].Fortheinterval[t,t+x],letNiandNdenotethentunberofarrivalsoftheN1(t)andN(t)processes.IntermsofNiandN,,;vecan'vrite001P[X1\)>::r]=PN1(0)=LPN1IN(Oln)PN(ri).(13.22)n=OGivenN=ri,totalarrivals,N1=0ifeachofthesearrivalsislabeledtype2.This,;villoccurwithprobabilityPN1IN(Oln,)=(1-p)11•ThusP[x(l)n·]_>::i,-~(L.__,1_p11)n(>.x)e->.x=e-p>.x~[(1-p),\x)1ie-(l-p)>.xn=OfTl,.n!L...,n=O(13.23)1ThusP[X~1)>x]=e-p>.x;eachX1\.1)hasanexponentialPDFwithmeanl/(p>.).Itfollo,vsthatl\T1(t)isal=>oissonprocessofrate.\1=p,\.Thesameargumentcanbeused2tosho'vthateachX1\.)hasanexponentialPDFwithmean1/[(1-p).\],implyingJ\T2(t)isaPoissonprocessofrate>.2=(1-p)>..Example13.17===AcorporateWebserverrecordshits(requestsforHTMLdocuments)asaPoissonprocessatarateof10hitspersecond.Eachpageiseitheraninternalrequest(withprobability0.7)fromthecorporateintranetoranexternalrequest(withprobability0.3)fromtheInternet.Overa10-minuteinterval,whatisthejointPMFofI,thenumberofinternalrequests,andX,thenumberofexternalrequests?ByTheorem13.6,theinternalandexternalrequestarrivalsareindependentPoissonprocesseswithratesof7and3hitspersecond.Ina10-minute(600-second)interval,IandXareindependentPoissonrandomvariableswithparametersa1=7(600)=4200andax=3(600)=1800hits.ThejointPMFofIandXisP1,x(i,1';)=P1(i)Px(x)(4200)ie-4200(1800)xe-1800i!'i,xE0otherwise.{O,1,...},(13.24)TheBerr1oullidecorr1positionoftwoPoissonprocessesandthesumoftwoPoissonprocessesareclosel:yrelated.Theorern13.6sa}'StwoindependentPoissonprocesses[446CHAPTER13STOCHASTICPROCESSESJV1(t)andJ\T2(t)withrates,\1and,\2canbeconstrl1ctedfromaBernol1llidecorr1positionofaPoissonprocessJ\T(t)vvithrate,\1+.A2bychoosingthesuccessprobabilitytobeJJ=.A1/(,\1+,\2).Furt11errnore,gi·venthesetvvoindepender1tPoissor1processesN1(t)andJ\T2(t)derivedfrorr1theBerr1ot1llidecornposition,theoriginalN(t)processisthesumoft11etvvoprocesses.T11atis,N(t)=N1(t)+N2(t).Thuswheneverweobservetv.roindependentPoissonprocesses,vvecanthir1kofthoseprocessesasbeir1gderivedfrornaBernoullidecornpositionofasingleprocess.T11isvie\vleadstot11efollowingconclusion.Theorem13.1LetN(t)=J\T1(t)+J\T2(t)betheS'tJ,rnoftv.Join,depen,den,tPoisson,processesV.J'ithrates,\1a'nd,\2.Gi'IJen,thattlieN(t)processhasan,arri'IJal)thecon,dition,alprobabilitythatthearri'IJalisfrornN1(t)is.A1/(.A1+.A2).ProofWecanvie'vNi(t)andN2(t)asbeingderivedfromaBernoullidecompositionofN(t)inwhichanarrivalof]'l(t)islabeledatype1arrivalwithprobabilityA1/(.A1+.\2).ByTheorem13.6,Ni(t)andJ\T2(t)areindependentPoissonprocesseswithrateA1andA2,respectively.l\/Ioreover,givenanarrivaloftheN(t)process,theconditionalprobabilitythatanarrivalisanarrivalofthef\T1(t)processisalsoA1/(A1+A2).AsecondwaytoproveTl1eorern13.7isoutlir1edir1Problem13.5.5.Quiz13.5LetN(t)beaPoissonprocessofrate.A.LetN'(t)beaprocessinwhich\vecountonlyeven-r1t1rr1beredarri\r~tls;t11atis,arriva.ls2,4,6,...,oftheprocessJV(t).IsN'(t)aPoissonprocess?13.6TheBrownianMotionProcessTheBrownianmotionprocessdescribesaor1e-dirr1ensionalrandorr1walkin\vhichateveryinstant,thepositionchangesbyasrnallincrementthatisir1dependentofthecurrentpositionandpasthistoryoft11eprocess.Thepositior1changeoveranytirr1eintervalisaGaussianrandornvariablewithzeroexpectedva.lt1eandvarianceproportionaltot11etimeinterval.ThePoissonprocessisa.nexampleofacontinuot1s-tirne,discrete-valuestochasticprocess.Nov.rwe\villexa.rr1ineBrov.rnianrnotion,acontinuous-tirne,cor1tinuot1svaluestoc11asticprocess.===-Definition13.10BrownianMotionProcessABrownianmotionprocessMl(t)hasthepropertythatW(O)=O;an,dforT>0,W(t+T)-W(t)isaGa'ussian,(0,j"(n'-)ra/ndorn'/Jariablethat'isin,deperiden,tofW(t')forallt'<t.[13.6THEBROWNIANMOTIONPROCESS447ForBrov.rnianrnotion)\f.!ecanviev.rvT!(t)asthepositionofaparticleonaline.Forasmalltirneincrement8)W(t+8)=T¥(t)+[vV(t+8)-T¥(t)].(13.25)Althot1ght:hisexpar1sionrnayseemtrivial,byt11edefinitior1ofBrownianrr1otion,theincrernentX=T¥(t+===Theorem13.8---==:::iFortheBro'uJn,iarirnotion,processUl(t);thePDFofW=[W(t1),...,W(tk)]'isProofSincevV(O)=0,vV(t1)=X(t1)-vV(O)isaGaussianrando1nvariable.Giventimeinstantsti,...,tk,wedefineto=0and,forn=1,...,k,wecandefinetheincrementsXn=vV(tn)-vV(tn-1).NotethatX1,...,XkareindependentrandomvariablessuchthatXnisGaussian(0,Ja(tn-tn-1)).j.Xn(X)--1..j27ra(tn-tn-1)2e-x/(20-(tn-tn-1)).(13.26)NotethatW=wifandonlyifW1='UJ1andforn,=2,...,k,Xn='UJn-7Dn-l·Although\VeomitsomesignificantstepsthatcanbefoundinProblem13.6.5,thisdoesimplykf,,-(w)=IJfxn('UJn-7Dn-1).(13.27)n=lThetheoremfollo,vsfromsubstitutionofEquation(13.26)intoEquation(13.27).==--Quiz13.6LetW(t)beaBroV\rnianrnotior1processwit11varianceVar[W(t)]=CJ.t.Sho"''thatX(t)=vT!(t)/yaisaBrownianmotionprocess"''ithvarianceVar[X(t)]=t.[448CHAPTER13STOCHASTICPROCESSES13.7ExpectedValueandCorrelationTheexpectedvall1eofastochasticprocessisafunctior1oftirne.Theatltocovarianceandautocorrelationareftlnctionsoftwotirr1evariables.Allthreefunctionsindicatetherateofchangeofthesarnplefur1ctionsofastochasticprocess.Ir1studyir1grandorr1·variables,weoftenrefertopropertiesoftheprobabilit}'rr1odelStlchastheexpectedvall1e,the·variance,t11eco-variance,andthecorrelation.Thesepararnetersareafev.rnurnbersthatsurnmarizethecornpleteprobabilityrnodel.Inthecaseofstochasticprocesses,deterrr1inisticf\1r1ctionsoftimepro·videcorrespondingStlrr1rnariesoft11epropertiesofacorr1pleternodel.Forastoc11asticprocessX(t),X(t1),thevalueofasarnplefunctionattirr1einstantt1,isarandornva,riable.HenceithasaPDFfx(t1)(x)andexpected·valueE[X(t1)).Ofcourse,oncev.reknov.rthePDFf·x(t1)(x),e·verythingviehavelearnedaboutrandornvariablesandexpectedvaluescar1beappliedtoX(t1)andE[X(t1)).SinceE[X(t))issirnpl}'ar1umberforeac11valueoft,t11eexpectedvalueE[X(t)]isadeterrr1ir1isticfunctionoft.SinceE[X(t)]isasome-vvhatCl1rr1bersomenotation,thenextdefinitionisjustane-vvnotationthaterr1phasizesthattheexpectedvalueisafunctionoftime.Definition13.llTheExpectedValueofaProcessTheexpectedvalueofastochasticprocessX(t)isthedeterrnin,isticf'u'nct'ioriµ,x(t)=E[X(t)).Example13.lR-=:::iIfRisanonnegativerandomvariable,findtheexpectedvalueofX(t)=RIcos27rf'tl.TherectifiedcosinesignalX(t)hasexpectedvalueµ,x(t)=E[Rlcos27rfti]=E[R]lcos27rfti.(13.28)FrornthePDFf'x(t)(x),wecanalsocalculatet11evarianceofX(t).·vVliilethevarianceisofsomeir1terest,t11ecovariancef\1nctionofastochasticprocesspro·videsveryimportar1tir1formationaboutthetimestructureoft11eprocess.RecallthatCov[X,Y)isanir1dicatior1ofhov.rmuchinforrnationra.ndorr1variableXprovidesabol1trandornvariableY.\i\Ther1thernagr1itudeofthecovarianceis11igh,anobservationofXprovidesanaccl1rateindicationoft11evalueof·y.Ifthet\vorandorr1variablesareobservatior1sofX(t)takenatt-vvodifferenttirnes,t1secondsandt2=t1+Tseconds,thecovarianceindicatesho\vrnt1chtheprocessislikelytoc11angeintheTsecondselapsedbetweent1andt2.Ahighcovarianceindicatesthatthesamplefunctior1isunlikelytoc11ar1gerr1l1chint11eT-secondinterval.Acovariar1cenearzerosuggestsrapidchange.Thisir1forrr1ationiscor1\reyedbytheautocovar·ian,cefur1ctiori.[13.7EXPECTEDVALUEANDCORRELATION449.....---Definition13.1AutocovarianceTheautocovariancefu,nction,ofthestochasticprocessX(t)isTheautocovariancefv,rict'iorioftheran,dornseqv,en,ceXnisForrandomsequences,""'ehaveslightlyrnodifiedt11enotationforautocovariancebyplacingt11eargt1rnentsir1sq11arebracketsjt1stasarerr1ir1dert11atthefunctionshaveir1tegerargt1ments.Foracor1tin11011s-tirneprocessX(t),theat1tocovariar1cedefinitionatT=0impliesCx(t,t)=Var[X(t)].Equivaler1tly,fork=0,Cx[n,,n,]=Var[Xn]·Theprefixav,toofat1tocovarianceemphasizesthatCx(t,T)rneast1rest11ecovariancebetweer1twosamplesoft11esarr1eprocessX(t).(Thereisalsoacrosscovariancefunctionthatdescribestherelationshipbetweentwodifferentrar1domprocesses.)Theautocorrelationfunctionofastochasticprocessiscloselyrelatedtotheat1tocovariancefunction.===Definition13.13AutocorrelationFunctionTheautocorrelationfunctionofthestochasticJJrocessX(t)isRx(t,T)=E[X(t)X(t+T)].Theautocorrelationfv,n,ction,oftheran,dornsequen,ceXnisFrorr1Theorem5.16(a),we11avethefollowingresult.---Theorem13.9Thea11,tocorrelatiorian,da'u,tocovarian,cefv,rict'iorM3ofaprocessX(t)satisfyTheav,tocor~relatiorian,da'u,tocovarian,ceftJ,n,ctiorisofararidornseq'1J,er1,ceXnsatisfyCx(71,,k]=Rx[r1,,k]-11,x(n,)µ,x(n,+k).Sincetheautocovariancetindautocorrelationaresocloselyrelated,itisreasonabletoaskvvhyvveneedbothofthern.Itvvouldbepossibletouseonlyoneortheother[450CHAPTER13STOCHASTICPROCESSESir1cor1jt1nctionv.riththeexpectedva1t1eµ,x(t).T11ear1s\veristhateachfur1ctionhasitsuses.Inparticular,theat1tocovariar1ceisrnoreusef\11-vvhenwe\Vanttot1seX(t)topredictaft1tt1revalueX(t+T).Ontheotherhand,sinceRx(t,0)=E[X2(t)],theautocorrelationdescribest11ea\reragepo\verofarandornsigr1al.Example13.19FindtheautocovarianceprocessX(t).Cx(tT)andautocorrelationRx(t,T)oftheBrownianmotionFromthedefinitionoftheBrownianmotionprocess,weknowthatµx(t)=0.Thustheautocorrelationandautocovarianceareequal:Cx(t,T)=Rx(t,T).TofindtheautocorrelationRx(t,T),weexploittheindependentincrementspropertyofBrownianmotion.Forthemoment,weassumeT>0sowecanwriteRx(t,T)=E[X(t)X(t+T)].BecausethedefinitionofBrownianmotionreferstoX(t+T)-X(t),weintroducethisquantitybysubstitutingX(t+T)=X(t+T)-X(t)+X(t).Theresultis+T)-X(t))+X(t)]]2=E[X(t)[X(t+T)-X(t)]]+E[X(t)].(13.29)BythedefinitionofBrownianmotion,X(t)andX(t+T)-X(t)areindependent,withRx(t,T)=E[X(t)[(X(tzeroexpectedvalue.ThisimpliesE[X(t)[X(t+T)-X(t)]]=E[X(t)lE[X(t+T)-X(t)]=o.(13.30)Furthermore,sinceE[X(t)]=0,E[X2(t)]=·var[X(t)].Therefore,Equation(13.29)impliesRx(t,T)WhenTRx(t,T)<=E[X2(t)]=at,(13.31)0,wecanreversethelabelsintheprecedingargumenttoshowthata(t+T).ForarbitrarytandTwecancombinethesestatementstowriteRx(t,T)=arnin{t,t+T}.(13.32)===Example13.20===Theinputtoadigitalfilterisaniidrandomsequence...,X_1,X0,X1,...withE[Xi]=0and·var[X,i]=1.Theoutput...,Y_1,·y0,Y1,...isrelatedtotheinputbytheformulaYn=Xn+Xn-1fora11integersri.FindtheexpectedvalueE[Yn]andautocovariancefunction(13.33)Cy[rn,,k].BecauseYi=Xi+X,i-l·wehavefromTheorem5.10,E[Yi]=E[X,i]+E[Xi_1]=0.BeforecalculatingCy[1n,,k],weobservethatXnbeinganiidrandomsequencewithE[Xn]=0and·var[Xn]=1implies1k=0,0otherwise.(13.34)[13.7EXPECTEDVALUEANDCORRELATION451Foranyintegerk,wecanwriteCy[rn,,k]['YrnYrn+k]=E[(Xrn+Xrn-1)(Xrn+k+Xrn+k-1))=E[XrnXrn+k+XrnXrn+k-1+Xrn-1X1n+k+Xrn-1X1n+k-1].=E(13.35)Sincetheexpectedvalueofasumequalsthesumoftheexpectedvalues,Cy[rn,,k]=Cx[rri,k]+Cx[rn,,k-1]+Cx[m,-1,k+1]+Cx[m,-1,k].(13.36)Westillneedtoevaluatethisexpressionforallk.Foreachvalueofk,sometermsinEquation(13.36)willequalzerosinceCx[rn,k]=0fork-=/=0.Inparticular,iflkl>1,thenk,k-1andk+1arenonzero,implyingCy[rn,,k]=0.Whenk=0,wehaveCy[rn,,O]=Cx=2.['1T1,,O]+Cx[m,,-1]+Cx[rn,-1,1]+Cx['JTI,-1,O](13.37)Fork=-1,wehaveCy[rn,,-1]=Cx['IT/,,-1]+Cx[rn,,-2]+Cx[rn,-1,0]+Cx[m,-1,-1]=1.(13.38)Thefinalcase,k=1,yieldsCy['!Tl,,1]=Cx[rn,,1]+Cx[rn,O]+Cx[m-1,2]+Cx[rn-1,l]=l.(13.39)AcompleteexpressionfortheautocovarianceisCy[rn,k]=20lklk=-1,0,1,otherwise.(13.40)Weseethatsincethefilteroutputdependsonthetwopreviousinputs,thefilteroutputsYr1•andYr1.+1arecorrelated,whereasfilteroutputsthataretwoormoretimeinstantsapartareuncorrelated.Aninterestingpropert~yoft11eautoco·variancefunctior1foundinExarnple13.20isthatCy[rn,k]deper1dsor1lyonkandnotonrn.Inthenextsection,"if..Telearnthatthisisapropertyofaclassofrar1dornsequer1cesreferredtoasstatioriaryrandornsequences.Quiz13.7X(t)hasexpectedvalueµ.x(t)andautocorrelationRx(t,T).'\Vernakethenoisyobservation·y(t)=X(t)+N(t),vvhere/ll(t)isarar1domnoiseprocessindependentofX(t)with11,N(t)=0aridautocorrelationRN(t,T).Fir1dtheexpected·valueandat1tocorrelationofY(t).[452CHAPTER13STOCHASTICPROCESSES13.8StationaryProcessesAstochasticprocessisstationaryiftheprobabilit}'modeldoesnotvarywithtirr1e.Recallt11atinastocl1asticprocess,X(t),thereisarandorr1·variableX(t1)ateverytirneir1stantt1-vvithPDFf'x(t1)(:i;).Forrr1ostrandorr1processes,thePDFf'x(t1)(x)dependsont1.Forexample,-vvhenwernakedailyterr1peraturereadings,weexpectthatreadingstakenir1thev.rinter"''illbeloV\rerthar1terr1peraturesrecordedinthesurnmer.Ho,;vever,foraspecialclassofrandornprocesseskr1ownasstatioriaryprocesses,fx(ti)(:i;)doesnotdeper1dont1.T11atis,foranytwotirr1einstantst1andt1+T,(13.41)Therefore,inastationaryprocess,\i\Teobservetl1esarnerar1dorr1variableatalltimeir1stants.Thekeyideaofstationarityisthatthestatisticalpropertiesoftheprocessdoriotchange\iVithtirne.Eql1ation(13.41)isanecessaryconditionbutnotasufficientconditior1fora,stationaryprocess.SincethestatisticalpropertiesofarandornprocessaredescribedbyPDFsofrandorr1vectors[X(t1),...,X(trn)],\Vehavethefollowingdefir1itiori.---Definition13.14StationaryProcessAstochasticprocessX(t)isstationaryifan,don,lyifforallsetsoft'irnein,sta'ntst1,...,trn'aridan,ytirnedifferen,ceT,Aran,dornseqv,ericeXnisstationaryifan,don,lyifforan,ysetofin,tegertirnein,stan,tsri1,...,ri,rn,an,din,tegertirnedifferen,cek,Generall}'itisnotobviouswhethera.stochasticprocessisstationary.Usl1allyastocl1asticprocessisnotstationar}'·However,provingordisprovingstationaritycanbetricky.Curiousreadersrna}'wishtodeterrrlir1ewhichoftheprocessesinearlierexarnplesarestatior1ary.Example13.21==IstheBrownianmotionprocesswithparameteraintroducedinSection13.6stationary?JCITi')ForBrownianmotion,X(t1)istheGaussian(0,randomvariable.Similarly,X(t2)isGaussian(0,.JCIT2).SinceX(t1)andX(t2)donothavethesamevariance,fx(t1)(:i;)i=fx(t2)(:i;),andtheBrownianmotionprocessisnotstationary.[13.8453STATIONARYPROCESSESThefollowingtheoremappliestoapplicationsinw11ich\f.fernodifyonestochasticprocesstoproduceanewprocess.Iftheorigir1alprocessisstationaryandthetransformationisalinearoperation,thenewprocessisalsostationary.Theorem13.10LetX(t)beastationaryran,dornprocess.Forcon,stan,tsa>0an,db,Y(t)aX(t)+b'isalsoastat'irJ'naryprocess.ProofForanarbitrarysetoftimesamplesti,...,tn,weneedtofindthejointPDFofY(t1),...,Y(tn)·WehavesolvedthisprobleminTheorem8.5wherewefoundthat.fY(t1),...,Y(tn)()YI,···'Yn=1.(Y1lalnfx(t1),...,X(tn)a,b'···'Yna,-b)·(13.42)SincetheprocessX(t)isstationary,vvecan'vrite.fY(t1+T),...,Y(tn+T)(y1,...,yn)=1.a,nfX(t1+T),...,X(tn+T)1.=a,nfX(t1),...,X(tn)(Y1-(Y1a,a,b,...,Yn-b)a,b,...,Yna,-b)=fY(t1),...,Y(tn)(y1,···'Yn)·(13.43)ThusY(t)isalsoastationaryrandomprocess.Therearernar1yconsequencesoft11etirne-irrvariantnatureofastationar}'randornprocess.Forexample,settingrn,=1inDefinition13.14leadsimrnediatel}'toEquation(13.41).Equatior1(13.41)implies,inturn,thatt11eexpected·vall1ef\1nctioninDefinition13.11isacor1stant.Furtherrr1ore,theautocovariancefunctior1andtheautocorrelationfur1ctiondefinedinDefinition13.12andDefinition13.13areindeper1dentoftanddeper1donlyonthetime-differencevariableT.Therefore,-vveadoptt11enotatior1Cx(T)andRx(T)fort11eautocovariar1cefunctionandautocorrelationfunctionofastationarystochasticprocess.Theorem13.11Forastation,aryprocessX(t),theex;pectedvalue,thea/u,tocorrelation,,an,dthetocovariaricehavethefollo'tJJin,gpropertiesforallt:{a)µ,x(t)O/IJ,-=µx,{b)Rx(t)T)=Rx(0)T)=Rx(T)){c)Cx(t,T)=Rx(T)-11,~=Cx(T).Forastation,ar~yraridornseqv,en,ceXntheexpectedvalue,thea/u,tocorrelation,,an,dthea/u,tocovarian,cesatisfyforallri{a)E[Xn]=µ,x,{b)Rx[n,,k]=Rx[O,k]=Rx[k],{c)Cx[ri,k]=Rx[k]-/J,~=Cx[k].[454CHAPTER13STOCHASTICPROCESSESProofByDefinition13.14,stationarityofX(t)impliesfxctlx)=fx(o)(x),sothat(13.44)Notethatµ,x(0)isjustaconstantthatwecallJLx.Also,byDefinition13.14,(13.45)sothatRx(t,T)=E[X(t)X(t+T)]=1_:1_:x1x2fx(o),X(T)(x1,x2)dx1dx2(13.46)=Rx(O,T)=Rx(T).(13.47)Cx(t,T)=Rx(t,T)-JL~=Rx(T)-µ~=Cx(T).(13.48)Lastly,byTheorem13.9,\'f.leobtainessentiallythesamerelationshipsforrandomsequencesbyreplacingX(t)andX(t+T)withXnandXn+k·===Example13.22===AtthereceiverofanAMradio,thereceivedsignalcontainsacosinecarriersignalatthecarrierfrequencyf'cwitharandomphase8thatisasamplevalueoftheuniform(0,27r)randomvariable.Thereceivedcarriersignalis(13.49)WhataretheexpectedvalueandautocorrelationoftheprocessX(t)?ThephasehasPDFfe(B)=1/(27r)00<(}<27r.otherwise.--I(13.50)Foranyfixedangleaandintegerk,E[cos(a+k8))=12~01cos(a+k(})-rl(}27rsir1(a+kB)kChoosinga=27rfct.andk=2~_sin(a-(13.51)+k27r)ksincv._-0.(13.52)01,E[X(t)]isµ,x(t)=E[Acos(27rf.ct+8))=0.(13.53)[13.9WIDESENSESTATIONARYSTOCHASTICPROCESSESWewillusetheidentitycosAcosBautocorrelation:=(cos(A-B)+cos(A+B)]/2455tofindtheRx(t,T)=E[Acos(27rf.ct+8)Acos(27rfc(t+T)+8)]A2=2E(cos(27rfcT)+cos(27rf.c(2t+T)+28)].Fora=21rfc(t+T)and(13.54)k=2,E(cos(27r}.c(2t+T)+28)]=E[cos((J'.+k8)]=O.(13.55)Thus(13.56)Therefore,X(t)isstationary.IthasthepropertiesofastationarystochasticprocesslistedinTheorem13.11.Quiz13.8LetX1,X2,...beaniidrandorr1seqt1ence.IsX1,X2,...astationaryrar1domseqt1ence?13.9WideSenseStationaryStochasticProcessesAstochasticprocessisvvideser1sestationaryiftheexpectedvalt1eisconstantwithtimeandtheautocorrelationdependsonlyonthetirr1edifferencebet\veentworar1domvariables.AvvidesensestationaryprocessisergodicifexpectedvaluessuchasE[X(T)]andE[X2(t)]areequaltocorrespondingtimeaverages.Therearernar1yapplicatior1sofprobabilitytheor}'inwhichinvestigatorsdoriothaveacompleteprobabilit}'rnodelofar1experirnent.Evenso,rr1uchcanbeaccorr1plished\vithpartialir1forrr1ationabot1tthemodel.Oftenthepartialinforrr1ationtakestheforrnofexpectedvalues,variances,correlations,andcovariances.Int11econtextofstochasticprocesses,whenthesepararr1eterssatisfytheconditionsofTheorerr113.11,werefertotherelevar1tprocessas'llJ'idesen,sestation,ary.~=;...Definition13.1~-==WideSenseStationaryX(t)isawidesensestationarystochasticprocessifa/ndon,lyifforallt)[456CHAPTER13STOCHASTICPROCESSESXnisawidesensestationaryrandomsequenceifaridorilyifforalln,,Theorerr113.11irr1pliestha,te\rerystationaryprocessorsequenceisalso\videsensestationary.HoV\rever)ifX(t)orXniswidesensestationary)itrnayorrnayriotbestationary.Thus"'ridesensesta.tior1aryprocessesincludestationar:yprocessesasasubset.Sornetextsusetheterrnstrictsen,sestation,aryforvvhat\Ve11avesirnpl}'calledstation,ary.Example13.23InExample13.22,weobservethat/J,x(t)=0andRx(t,T)=(A2/2)cos27rf.cT·ThustherandomphasecarrierX(t)isawidesensestationaryprocess.Theautocorrelationfur1ctionofa.widesensestationar}'process11asanumberofirr1portantproperties.---==Theorem13.12Fora'tuidesen,sestation,aryprocessX(t),theO/tJ,tocorrelatiorif11,n,ction,Rx(T)hasthefollo'tJJ't'ngproperties:Rx(O)>O,Rx(T)=Rx(-T))Rx(O)>IRx(T)I.IfXnisa'tJJidesen,sestation,aryraridornseq'1J,er1,ce:Rx[O]>0,Rx[k]=Rx[-k],ProofForthefirstproperty,Rx(O)haveE[X2(t)]toobtain>Rx[O]>IRx[k]I.=Rx(t,O)=E[X2(t)].SinceX2(t)>0,wemust0.Forthesecondproperty,wesubstitute'IL=t+TinDefinition13.13Rx(t,T)=E[X('u-T)X('ll,)]=Rx('ll,,-T).(13.57)SinceX(t)iswidesensestationary,Rx(t,T)=Rx(T)=Rx('IJ,,-T)=Rx(-T).(13.58)Theproofofthethirdpropertyisalittlemorecomplex.First,\VenotethatwhenX(t)is,videsensestationary,'lar[X(t)]=Cx(O),aconstantforallt.Second,Theorem5.14impliesthat(13.59)Now,foranynumbersa,b,andc,ifa0,then(a+c)2<(b+c)2.Choosinga=Cx(t,T),b=Cx(O),andc=µ,2Jcyields2)2<(Cx(O)+µ,x2)2.(Cx(t,T)+JLx(13.60)[13.9WIDESENSESTATIONARYSTOCHASTICPROCESSES457Inthisexpression,theleftsideequals(Rx(T))2andtherightsideis(Rx(0))2,whichprovesthethirdpartofthetheorem.TheprooffortherandomsequenceXnisessentiallythesame.Problem13.9.10asksthereadertoconfirmthisfact.Rx(0)hasanimportantphysicalinterpretationforelectricalengineers.Definition13.16AveragePowerTheaveragepowerofa'llJideserisestat'ioriaryprocessX(t)isRx(0)=E[X2(t)].Theaveragepowerofa'tJJidesen,sestation,aryseq'ueri,ceXnisRx[O]=E[X.~].Thisdefinitionrelatestothefactthatinar1electricalcircuit,asignalisrneasuredaseitheravoltagev(t)ora,current'i(t).AcrossaresistorofRD,their1stantaneouspowerdissipatedisv2(t)/R=i2(t)R.vV11ent11eresistanceisR=1D,t11einstantaneol1spo'iverisv2(t)'ivl1en'iverneasurethe\roltage,ori2(t)wher1vverneasurethecurrent.'\llher1Vl'eusex(t),asamplefur1ctionofa'ividesensestationarystochasticprocess,tornodelavoltageoraCl1rrent,theinstantaneouspoweracrossa1Dresistorisx2(t).Weusl1all}'assurr1eimplicitlythepresenceofa1Dresistorandreferto1';2(t)astheir1stantar1eouspowerofx(t).Byextension,vverefertotherandorr1variableX2(t)astheinstantaneol1SpoweroftheprocessX(t).Definition13.16t1sestheterrninologyaverageJJO'llJerfortheexpected\ralueoftheinstantar1eouspowerofaprocess.RecallthatSection13.ldescribesenserr1bleaveragesandtimea'ireragesofstochasticprocesses.InOllrpresentationofstationaryprocesses,wehaveencounteredonlyer1sernbleaveragesincludingtheexpected\ralue,t11eal1tocorrelation,theautocovariance,aridt11eaveragepower.Er1gir1eers,ontheotherhand,areaccl1stornedtoobservingtirneavertiges.Forexample,ifX(t)rr1odelsavoltage,thetirr1eaverageofsarr1plefunctionx(t)overanintervalofduration2Tis-X(T)=-12TJTx(t)rlt.(13.61)-TThisistheDCvoltageofx(t),V1rhichcanberneasuredV1rithavoltrneter.Sirnilarly,atirr1eaverageoft11epowerofasarnpleft1nctionisXi(T)=T12JTx(t)rlt.2(13.62)-TTherelatior1shipofthesetimea'ireragestothecorrespondingensernbleaverages,/J,xandE[X2(t)],isafascinatingtopicir1thestudyofstochasticprocesses.\N'her1X(t)isastationaryprocesssuchthatlirr1T-+ooX(T)=µ,x,theprocessisreferredtoasergodic.Inwords,foranergodicprocess,t11etirneaverageofthesamplef\1nctionofavvidesensestationarystochasticprocessisequaltothecorrespondinger1sernbleaverage.Foranelectricalsignalmodeledasasarr1plefunctionofanergodicprocess,µ,xandE[X2(t)]andrr1anyotherer1semblea'ireragescanbeobser'ired'ivithfamiliarrneasuringequiprnent.[458CHAPTER13STOCHASTICPROCESSESAlthoughtheprecisedefinitionandanal:ysisofergodicprocessesarebeyondthescopeofthisir1troductor}'text,wecanusethetoolsofCl1apter10tomakesorr1eadditionalobservations.ForastationaryprocessX(t),-vvecan·vie"''t11etirnea;verageX(T)asar1estimateoftheparameterµx,analogoustothesarnplerneanNln(X).T11edifference,ho-vve·ver,isthatthesamplerneanisana·verageofindependentrandomvariables,whereassample·valuesoftherandorr1processX(t)arecorrelated.Ho,vever,ift11eautocovarianceCx(T)approacheszeroquickl}',t11enasTbecorr1eslarge,rnostofthesamplevalueshavelittleornocorrelatior1,and\Ve"''ouldexpecttheprocessX(t)tobeergodic.Thisideaisrr1adernorepreciseir1t11efolloV\ringtheorern.Theorem13.13LetX(t)beastatioriaryraridornprocess'llJithex;pectedvalue11,xan,da11,tocovariarice00Cx(T).Iff00ICx(T)ldT<oo,then,X(T),X(2T),...isariun,biased,corisisten,tsequen,ceofestirnatesof/.J,x.ProofFirst\VeverifythatE[X(T)]X(T)isunbiased:=~E2[/_TX(t)dt]=2~J_'TE[X(t)]dt=2~/_~,,µxdt=JLx.-T-T(13.63)-1-Toshowconsistency,itissufficienttoshowthatlimr-+ooVar[X(T)]=0.First,\Veobserve1'TthatX(T)-µ,x=2TJ_,r(X(t)-µ,x)dt.ThisimpliesVar[°X(T)]=E[(=E[~J_'~r(X(t)-(2~')2(/_':.(XT=2µx)dt)](t)-/LX)dt)(/_~~.(X(t')-=(2~)2/_l'T(2~)2J_TJ_'rE[(X(t)-JLx)(X(t')-JLx)]T/LX)dt')dt'dt'T(13.64)T/_'l"CX(t'-t)dt'dt.\\!enotethat1_·:.Cx(t'-t)dt'<J_'~r<1_:ICx(t'-t)Idt'ICx(t'-t)Idt'=1_:ICx(T)IdT<oo.(13.65)HencethereexistsaconstantKsuchthat-Var[X(T)]Thuslim1"-+ooVar[X(7"')]1<(2T)2<lim'l"-+oo-#=0./_1"-TKdtK=TI,.(13.66)[13.10CROSS-CORRELATION459Quiz13.9Whichoft11efollov.rir1gfunctionsarevalidautocorrelationf11nctions?(a)R1(T)=e-ITI(b)R2(T)=e-72(c)R3(T)=e-(d)13.10TcosT2R4(T)=e-TsinrCross-CorrelationThecross-covarianceandcross-correlationfunctior1spartiallydescribetheprobabilityrr1odeloftwowidesensestationaryprocesses.Ir1rnanyapplications,itisnecessar}'toconsidertherelationshipoftwostoc11asticprocessesX(t)andY(t),ortworandorr1sequencesXnandYn.Forcertainexperirnents,itisappropriatetornodelX(t)and·Y(t)asindependentprocesses.Ir1thissimplecase,anysetofrandorr1variablesX(t1),...,X(tk)fromtheX(t)processisindeper1dentofan}'setofrandom·variables·Y(t~),...,·Y(tj)frorntheY(t)process.Ingeneral,however,acornpleteprobabilityrnodeloft-vvoprocessesconsistsofajoir1tP1!{ForajointPDFofallsetsofrandorr1·variablescontainedintheprocesses.St1cl1ajointprobabilityft1nctioncorr1pletelyexpressestherelationsliipofthet-vvoprocesses.However,fir1dingandworkingwithst1chajointprobabilityfunctionist1suallyprohibitivelydifficult.Toobtainusefultoolsforanalyzingapairofprocesses,-vverecallthatthecovarianceandthecorrelationofapairofrandorr1variablesprovide·valuableinforrr1ationabouttherelationsmpbetv.reentherandomvariables.Touset11isinforrr1ationtot1nderstandapairofstoc11asticprocesses,we-vvorkv.riththecorrelatior1andco-varianceoftherandornvariablesX(t)andY(t+T).Thecross-correlationofcoritiri11,o'us-tirneran,dornprocessesX(t)aridY(t)1,sRxy(t,T)=E[X(t)Y(t+r)].Thecross-correlationofra/ndornseq'uen,cesXnan,d·11;1,1,sJustasfortheautocorTela,tion,t11erearernanyinterestingpracticalapplicationsir1-vvhicht11ecross-correlationdependsonlyononetimevariable,t11etirnediflerer1ceTortheindexdifferencek.Definition13.18JointlyWideSenseStationaryProcessesCor1,tir1,11,o'us-tim,erandornprocessesX(t)an,dY(t)arejointlywidesensestationaryifX(t)an,dY(t)arebothv.1idesen,sestation,ary;a'ndthecross-correlat'ion,deperidson,lyon,thetirnedifferericebet'tlJeen,thetv.10ran,dornvariables:[460CHAPTER13STOCHASTICPROCESSESRan,dornsequericesXnandYnarejointlywidesensestationaryifXnan,d·ynareboth111idesen,sestation,aryan,dthecross-correlation,depen,dson,lyon,thein,de1;differen,cebet111een,thet'uJoran,dorn1Jar~iables:Rxy[rn,,k]=Rxy[k].vVeencountercross-correlationsinexperimentsthatinvol'venoisyobservationsofavvidesensestationaryra,ndornprocessX(t).-=Example13.24SupposeweareinterestedinX(t)butwecanobserveonly'Y(t)=X(t)+N(t),(13.67)N(t)isanoiseprocessthatinterfereswithourobservationofX(t).AssumeX(t)andN(t)areindependentwidesensestationaryprocesseswithE[X(t)]=11,xandE[N(t)]=11,N=0.IsY(t)widesensestationary?AreX(t)andY(t)jointlywidewheresensestationary?AreY(t)andj\/(t)jointlywidesensestationary?Sincetheexpectedvalueofasumequalsthesumoftheexpectedvalues,E[Y(t)]=E(X(t)]+E(N(t)]=µx.(13.68)Next,wemustfindtheautocorrelationRy(t,T)=E[Y(t)Y(t+T)]=E[(X(t)+N(t))(X(t+T)+N(t+T))]=Rx(T)+RxN(t,T)+RJvx(t,T)+RN(T).SinceX(t)andlarly,Rx1v(t,T)(13.69)1'l(t)areindependent,R1vx(t,T)=E[N(t)]E[X(t+T)]=0.Simi=µ,x11,N=0.Thisimplies(13.70)TherightsideofthisequationindicatesthatRy(t,T)dependsonlyonT,whichimpliesthatY(t)iswidesensestationary.TodeterminewhetherY(t)andX(t)arejointlywidesensestationary,wecalculatethecross-correlationRyx(t'T)=E[Y(t)x(t+T)]=E[(x(t)+]\T(t))x(t+T)]=Rx(T)+RJvx(t,T)=Rx(T).(13.71)WecanconcludethatX(t)and'Y(t)arejointlywidesensestationary.Similarly,wecanverifythatY(t)andN(t)arejointlywidesensestationarybycalculatingRy1v(t,T)=E[Y(t)N(t+T)]=E[(X(t)+J\T(t))N(t+T)]=RxN(t,T)+RN(T)=RN(T).(13.72)[13.10CROSS-CORRELATION461Inthefollowingexample,v..reobservethatarar1dornseq11enceY;iderivedfromav..ridesensestationarysequenceXnrnayalsobev..rideser1sestationarye·vent11011ghXnaridYnarenotjointlyv..ridesensestationary.Example13.25XnisawidesensestationaryrandomsequencewithautocorrelationfunctionRx[k].Therandomsequence·ynisobtainedfromXnbyreversingthesignofeveryotherrandomvariableinXn:Y;i,=-lnXn.(a)Expresstheautocorrelationfunctionof·ynintermsofRx[k].(b)Expressthecross-correlationfunctionofXnandY;.1,intermsofRx[k].(c)IsYri(d)Arewidesensestationary?XnandY;ijointlywidesensestationary?TheautocorrelationfunctionofY;1,isRy[ri,k]=E['YnYn+k]=E[(-l)nXn(-l)n+kXn+k]=(-1)2n+kE[XnXn+k]=(-l)kRx[k].(13.73)Y;.1,iswidesensestationarybecausetheautocorrelationdependsonlyontheindexdifferencek.Thecross-correlationofXnandY;.1,isRxy[n,,k]=E[XnY;1,+k]=E[Xn(-l)n+kXn+k]=(-l)n+kE[XnXn+k]=(-l)n+kRx[k].(13.74)XnYnarenotjointlywidesensestationarybecausethecross-correlationdepends'nandk.Whenriandkarebothevenorwhenriandkarebothodd,Rxy[n,,k]=Rx[k];otherwiseRxy[ri,k]=-Rx[k].andonbothTheorern13.12indicatesthattheautocorrelationofav.ridesensestatior1aryprocessX(t)issyrnrnetricabot1tT=0(cor1tin11ous-tirne)ork=0(randornsequence).Thecross-correlationofjointl}'widesensestationaryprocesseshasacorrespondir1gS}rrr1metry.==;;;;Theorem13.14.~=~IfX(t)an,dY(t)arejoiritly'tJJideserisestation,arycon,t'iri'UO'tJ,s-tirneJJTocesses,

then,IfXna/nd·ynarejoiritlyVJidesen,sestation,aryran,dornseq11,en,ces,then,Rxy[k]=Ryx[-k].[462CHAPTER13STOCHASTICPROCESSESProofFromDefinition13.17,Rxy(T)=E[X(t)Y(t+T)].Makingthesubstitution11,=t+TyieldsRxY(T)=E[X(71,-T)Y(11,)]=E[Y(11,)X(11,-T)]=RYx(71,,-T).(13.75)SinceX(t)andY(t)arejointlywidesensestationary,Ryx(7J,,-T)=Ryx(-T).Theproofissimilarforrandomsequences.Quiz13.10X(t)isawidesensestationarystocr1asticprocesswithautocorrelatior1functionRx(T).Y(t)isidenticaltoX(t),excepttr1atthetimescaleisre·versed:Y(t)=X(-t).(a)ExpresstheautocorTelationfunctionofY(t)ir1terrnsofRx(T).IsY(t)wideser1sestatior1ary?(b)Expresstr1ecross-correlationfunctionofX(t)and·y(t)ir1termsofRx(T).AreX(t)andY(t)jointly"'ridesensestationary?13.11GaussianProcessesForaGal1ssianprocessX(t),everyvectorofsarnplevall1esX[X(t1)X(tk)]'isaGaussianrandomvector.=Thecentrallirnittheorerr1(Theorem9.12)helpsexplaintheproliferationofGaussiar1randorr1·variablesinnatl1re.Thesarneinsigr1textendstoGaussianstochasticprocesses.Forelectricala.ndcorr1puterengineers,tr1enoisevoltageir1aresistorisapervasiveexarr1pleofaphenomenonthatisaccl1ratelyrr1odeledasaGaussianstochasticprocess.InaGaussianprocess,e·ver}'collectionofsarnplevaluesisaGaussiar1randornvector(Defir1ition8.12).Definition13.19GaussianProcessX(t)isaGaussian,stochasticprocessifan,don,lyifX=[X(t1)X(tk)]'isaGa/ussian,ran,dornvectorforan,yiritegerk>0an,dan,ysetoftirnein,stan,tst1)t2)...)tk.1XnisaGav.ssian,raridornseqv,en,ceifan,dO'nly'ifX=[Xn1Xnk]isaGaussia'nran,dornvectorforan,yin,tegerk>0an,dan,ysetoftirneiristan,tsn,1,n,2,...,n,k.InProblern13.11.5,-vveaskyoutoshoV\rthattheBrov;.rr1ianrnotionprocessinSection13.6isaspecialca.seofaGaussianprocess.Althol1ghtheBrownianrr1otionprocessisnotstationary(seeExample13.21),ourprirr1aryinterestwillbein-vvide[13.11GAUSSIANPROCESSES463sensestatioriar:yGaussianprocesses.Inthiscase,theprobabilityrnodelfortlieprocessiscornpletelyspecifiedb}'theexpectedvalueµ,xandtheautocorrelationfunctionRx(T)orRx[k].Asaconsequence,av.ridesensestationaryGaussianprocessisstationary.Theorem13.15IfX(t)isav1ideserisestatioruzryGa'tJ,ssian,prvcess,theriX(t)isastation,aryGa'tJ,SSiariprocess.IfXnisa111idesen,sestation,aryGa/ussian,seqv,en,ce,Xnisastation,aryGa'tJ,SSian,sequerice.ProofLetµandCdenotetheexpectedvaluevectorandthecovariancematrixoftherando1nvectorX=[X(t1)X(tk)]'.Letµ,andCdenotethesamequantitiesforX(tk+T)]'.SinceX(t)is'videthetime-shiftedrandomvectorX=[X(t1+T)sensestationary,E[X(ti)]=E[X(ti+T)]=JJ,x.Thei,jthentryofCisCij=Cx(ti,tj)=Cx(tj=Cx(tj-ti)+T-(ti+T))=Cx(ti+T,tj+T)=Cij·(13.76)Thusµ=µ,andC=C,implyingthatfx(x)=fx_(x).HenceX(t)isastationaryprocess.ThesamereasoningappliestoaGaussianrandomsequenceXn.The'tllhiteGaussiari'noiseprocessisacorrvenientstartingpointforrriariystudiesirielectricalandcornpl1terengineeririg.Definition13.20==WhiteGaussianNoiseW(t)isa111hiteGa/ussian,n,o'iseprocessifan,don,lyifW(t)isastation,aryGa'tJ,ssian,stochasticprocess'UJ'iththepropertiesµ,11v=0aridR11v(T)=r70o(T).Aconseqt1enceofthedefinitionistliatforanycollectionofdistincttirneinstantst1,...,tk,W(t1),...,W(tk)isasetofindependentGal1ssianrandomvariables.Inthiscase,thevalueoftherioiseattirnet,itellsnothingabot1tthevalueofthenoiseattimet.i.Whilethe\vliiteGaussiannoiseprocessisausefulmatherriaticalrnodel,itdoesriotcoriforrritoar1ysigrialthatcanbeobser,redph}rsically.Notethatthe..averagerio1sepowerisE[W2(t)]=Rw(O)=oo.(13.77)Thatis,wliitenoiseliasir1finiteaveragepoV\rer,\vhichisphysicallyirnpossible.Thernodelisuseful,however,becauseanyGaussiannoisesigrialobservedinpracticecanbeinterpretedasafilteredvvhiteGaussiannoisesignal\vithfinitepower.Quiz13.11X(t)isastationar}'Gaussianrandomprocess"''ithµ,x(t)=0aridat1tocorrelationfunctionRx(T)=2-ITI.WliatisthejointPDFofX(t)andX(t+1)?[46413.12CHAPTER13STOCHASTICPROCESSESMA,.fLABStochasticprocessesappearinmodelsofrr1anyphenomer1astudiedbyengineers.'\i\Thenthephenorr1enaarecornplicated,J\IIATLABsirr1ulationsarevaluableanal}rsistools.Toproducel\IIA.TLABsirr1ulatior1svveneedtodevelopcodesforstochasticprocesses.Forexarr1ple,tosimulatethecelll1lartelephonesv.ritchofExarr1ple13.4,v.,reneedtomodelboththearrivalsanddeparturesofcalls.APoissonprocessN(t)isaconventionalrr1odelforarrivals.Example13.26Usel\IIATLABtogeneratethearrivaltimesS1,S2,...ofarateatimeinterval[O,T].functions=poissonarrivals(larn,T)%arrivaltimess=[s(1)...s(n)]%s(n)APoissonprocessoverTogeneratePoissonarrivaIsatrate,\,weemployTheorem13.4,whichsaysthattheinterarrivaltimesareindependentexponential(.\)randomvariables.GiveninterarrivaltimesX,i,,the'itharrivaltimeisthecumulativesumThefunctionpoissonarrivalsgeneratescu-mulativesumsofindependentexponentialrandomvariables;itreturnsthevectorswiths(i)correspondingtoSi,the'itharrivaltime.NotethatthelengthofsisaPoisson(.\T)randomvariablebecausethenumberofarrivalsin[O,T]israndom.When\Ve\vishtoexarnineaPoissonarrivalprocessgrapl1ically,thevectorofarri\raltimesisnotsoconvenient.AdirectrepresentationoftheprocessN(t)isoftenmoreusef\11.===Example13.27==::::1GenerateasamplepathofN(t),arate,\=5arrivals/minPoissonprocess.PlotN(t)overa10-minuteinterval.1functionN=poissonprocess(lambda,t)Givent=[t1trn],thefunction%N(i)=no.ofarrivalsbyt(i)poissonprocessgeneratesthesamplepaths=poissonarrivals(larnbda,rnax(t));N=[N1JV1n]'where]\Ti=J\T(t,i)N=count(s,t);forarate,\PoissonprocessN(t).Thebasicideaofpoissonprocess.misthatgiventhearrivaItimes81,82,...,N(t)=rr1ax{r1,ISn<t}isthenumberofarrivalsthatoccurbytimet.Inparticular,inN=count(s,t),N(i)isthenumberofelementsofsthatarelessthanorequaltot(i).Asamplepathgeneratedbypoissonprocess.mappearsinFigure13.5.[13.12MATLAB46540t=0.01*(0:1000);lambda=5·'N=poissonprocess(lambda,t);plot(t,N)xlabel('\itt');ylabel('\itN(t)');~~2000510tFigure13.5APoissonprocesssa.n1plepathN(t)generatedbypoissonprocess.m.Notethatthenurnberofarri·valsgeneratedbypoissonprocessdeper1dsonlyonT=rr1axitibutnotonhov.rfinely-vverepresenttime.Thatis,t=0.1*(0:10*T)ort=0.001*(0:1000*T)bothgenerateaPoissonnt1rr1berN,v.rithE[N]=)..T,ofarrivalsovertheinterval[O,T].\i\Thatchangesishov.rfinely'""'eobservetheoutp11tJ\T(t).Nov.rthatMATLA.Bcanger1erateaPoissonarrivalprocess,-vvecansirr1ulatesystemssuchast11etelephor1eswitc11ofExample13.4.--=--Example13.28----=....Simulate60minutesofactivityofthetelephoneswitchofExample13.4underthefollowingassumptions.(a)TheswitchstartswithM(O)=0calls.(b)ArrivalsoccurasaPoissonprocessofrate)..=10calls/min.(c)Thedurationofeachcall(oftencalledtheholdingtime)inminutesisanexponential(1/10)randomvariableindependentofthenumberofcallsinthesystemandthedurationofanyothercall.Insimswitch.m,thevectorssandxmarkthearrivaItimesandca11durations.Theithcallarrivesattimes(i),staysfortimex(i),anddepartsattimey(i)=s(i)+x(i).Thusthevectory=s+xdenotesthecallcompletiontimes,alsoknownasdepartures.Bycountingthearrivalssanddeparturesy,weproducethearrivalanddepartureprocessesAandD.Atanygiventimet,thenumberofcallsinthesystemequalsthenumberofarrivalsminusthenumberofdepartures.HenceM=A-Disthenumberofcallsinthesystem.Onerunofsimswitch.mdepictingsamplefunctionsofA(t),D(t),andJVJ(t)=A(t)-D(t)appearsinFigure13.6.functionM=simswitch(lambda,mu,t)%Poissonarrivals,ratelambda%Exponential(mu)callduration%Forvectortoftimes%M(i)=no.ofcallsattimet(i)s=poissonarrivals(lambda,max(t));y=s+exponentialrv(mu,length(s));A=count(s,t);D=count(y,t);M=A-D·'SirnilartechniquescanbeusedtoproduceaBro-vvnianrnotionprocess·Y(t).[13.12MATLAB467ForanarbitraryGaussianprocessX(t),v.recanuseMA.TLABtogeneraterandornseqt1encesXn=X(n,T)thatrepresentsarnpled·versionsofX(t).Forthesarnpledprocess,t11evectorX=[XoXn-1]'isaGaussianrandom·vectorvvithexpectedvalueµx=[E(X(0))···E(X((n,-l)T))]'andcovariancernatrixCxv.rithi,jthelementCx(i,j)=Cx(iT,jT).Wecangeneratern,samplesofXusingx=gaussvector(mu,C,m).AsdescribedinSection8.6,muisaler1gthn,vectorandCisthen,xricovariancematrix.WhenX(t)iswidesensestationary,thesampledsequer1ceis"'ridesensestatior1aryV\ritha11toco·varianceCx[k].Ir1thiscase,thevectorX=[XoXn-lJ'hascovariar1cernatrixCxV\rithCx(i,j)=Cx[i-j).SinceCx[k]=Cx[-k],Cx=Cx[OJCx(1)Cx(1)Cx[OJCx[ri-1)Cx[ri-1)(13.78)Cx(1)Cx[OJCx(1)Weseet11atCxisconstantalongeachdiagonal.Amatrixvvithconstar1tdiagonalsiscalledaToeplitzrnatrix.WhenthecovariancematrixCxisToeplitz,itiscornpletelyspecifiedbytl1e·vectorc=[Cx[O]Cx[l]Cx[n,-11]',v.rhoseelerr1entsarebot11t11efirstcolumnandfirstroV\rofCx.T11ust11ePDFofXiscornpletelydescribedbytheexpectedvalueµ,x=E(Xi]andt11e·vectorc.Ir1thiscase,afunctiont11atgeneratessample\rectorsXneedsor1lythescalarµ,xandvectorcasir1puts.SincegeneratingsamplevectorsXcorrespondingtoastationaryGaussiar1sequenceisq11itecornrr1on,v.reextendthefunctiongaussvector(mu,C,m)introducedir1Section8.6tornakethisassimpleaspossible.functionx=gaussvector(mu,C,m)%output:mGaussianvectors,%eachwithmeanmu%andcovariancematrixCif(min(size(C))==1)C=toeplitz(C);endn=size(C,2);if(length(mu)==1)mu=mu*ones(n,1);end[U,D,V]=svd(C);x=V*(D~(0.5))*randn(n,m)...+(mu(:)*ones(1,m));IfCisalengthriroV\rorcolurnn\rector,itisassurnedtobethefirstroV\rofann,xn,Toeplitzcovariancerr1atrixthat"'recreatevviththestatementC=toeplitz(C).Inaddition,V\rhenmuisascalar\ralue,itisassurnedtobetheexpectedvalueE(Xn)ofastationarysequence.T11eprograrnexter1dsmutoalengthn,vector\vit11identicalelerr1er1ts.Whenmuisanr1,-elerr1entvectoraridCisann,xn,CO\rariancerr1atrix,aswasrequiredintheoriginalgaussvector.m,theyareleftunchanged.Therealvvorkofgaussvectorstilloccursint11elasttvvolines,\vhichareidenticaltothesimplerversior1ofgaussvector.mir1Section8.6.--=--=--Example13.30---=-WriteaMA.TLABfunctionx=gseq(a,n,m)thatgeneratesrnsamplevectorsX[XoXn]'ofastationaryGaussiansequencewithµ,x=0,cx1[k]=-1+ak2,.(13.79)[13.12MATLAB467ForanarbitraryGaussianprocessX(t),v.recanuseMA.TLABtogeneraterandornseqt1encesXn=X(n,T)thatrepresentsarnpled·versionsofX(t).Forthesarnpledprocess,t11evectorX=[XoXn-1]'isaGaussianrandom·vectorvvithexpectedvalueµx=[E(X(0))···E(X((n,-l)T))]'andcovariancernatrixCxv.rithi,jthelementCx(i,j)=Cx(iT,jT).Wecangeneratern,samplesofXusingx=gaussvector(mu,C,m).AsdescribedinSection8.6,muisaler1gthn,vectorandCisthen,xricovariancematrix.WhenX(t)iswidesensestationary,thesampledsequer1ceis"'ridesensestatior1aryV\ritha11toco·varianceCx[k].Ir1thiscase,thevectorX=[XoXn-lJ'hascovariar1cernatrixCxV\rithCx(i,j)=Cx[i-j).SinceCx[k]=Cx[-k],Cx=Cx[OJCx(1)Cx(1)Cx[OJCx[ri-1)Cx[ri-1)(13.78)Cx(1)Cx[OJCx(1)Weseet11atCxisconstantalongeachdiagonal.Amatrixvvithconstar1tdiagonalsiscalledaToeplitzrnatrix.WhenthecovariancematrixCxisToeplitz,itiscornpletelyspecifiedbytl1e·vectorc=[Cx[O]Cx[l]Cx[n,-11]',v.rhoseelerr1entsarebot11t11efirstcolumnandfirstroV\rofCx.T11ust11ePDFofXiscornpletelydescribedbytheexpectedvalueµ,x=E(Xi]andt11e·vectorc.Ir1thiscase,afunctiont11atgeneratessample\rectorsXneedsor1lythescalarµ,xandvectorcasir1puts.SincegeneratingsamplevectorsXcorrespondingtoastationaryGaussiar1sequenceisq11itecornrr1on,v.reextendthefunctiongaussvector(mu,C,m)introducedir1Section8.6tornakethisassimpleaspossible.functionx=gaussvector(mu,C,m)%output:mGaussianvectors,%eachwithmeanmu%andcovariancematrixCif(min(size(C))==1)C=toeplitz(C);endn=size(C,2);if(length(mu)==1)mu=mu*ones(n,1);end[U,D,V]=svd(C);x=V*(D~(0.5))*randn(n,m)...+(mu(:)*ones(1,m));IfCisalengthriroV\rorcolurnn\rector,itisassurnedtobethefirstroV\rofann,xn,Toeplitzcovariancerr1atrixthat"'recreatevviththestatementC=toeplitz(C).Inaddition,V\rhenmuisascalar\ralue,itisassurnedtobetheexpectedvalueE(Xn)ofastationarysequence.T11eprograrnexter1dsmutoalengthn,vector\vit11identicalelerr1er1ts.Whenmuisanr1,-elerr1entvectoraridCisann,xn,CO\rariancerr1atrix,aswasrequiredintheoriginalgaussvector.m,theyareleftunchanged.Therealvvorkofgaussvectorstilloccursint11elasttvvolines,\vhichareidenticaltothesimplerversior1ofgaussvector.mir1Section8.6.--=--=--Example13.30---=-WriteaMA.TLABfunctionx=gseq(a,n,m)thatgeneratesrnsamplevectorsX[XoXn]'ofastationaryGaussiansequencewithµ,x=0,cx1[k]=-1+ak2,.(13.79)[468CHAPTER13STOCHASTICPROCESSES33.--~--~~---~--~~--~--.221100-1-1~~-2-2_3--~---~~--~--~~--~--_3--~---~~--~---~~--~--oI020304050(a)a=1:gseq(1,50,5)Figure13.8rf\VOo~r-~-.,,,I020304050(b)a=0.01:gseq(0.01,50,5)sa.n1pleoutputsforExarnple13.30.Allweneedtodoisgeneratethevectorexcorrespondingtothecovariancefunction.Figure13.8showssampleoutputsforgraphs(a)a=1:gseq(1,50,5),(b)a=0.01:gseq(0.01,50,5).WeobserveinFigure13.8thateachgraphshowsm=5samplepathseventhoughgraph(a)mayappeartohavemanymore.Thegraphslookverydifferentbecausefora=1,samplesjustafewstepsapartarenearlyuncorrelatedandthesequencevariesquicklywithtime.Thatis,thesamplepathsingraph(a)zig-zagaround.Bycontrast,whena=0.01,sampleshavesignificantcorrelationandthesequencevariesslowly.thatis,ingraph(b),thesamplepathslookrealtivelysmooth.functionx=gseq(a,n,m)nn=O:n;cx=1.I(1+a*nn.~2);x=gaussvector(O,cx,m);plot(nn,x);==--Quiz13.12~==Thesvvitc11simulationofExarr1ple13.28isunrealisticir1theassurnptionthatthesv.ritchcanhandleanarbitrarilylargenurnberofcalls.J\l[odifyt11esirnt1lationsothattheSvvitchblocks(i.e.,discards)newcallsv.rhentheswitch11asc=120callsinprogress.EstirnateP[B],theprobabilitythatanev.rcallisblocked.Yoursirr1ulationrnayneedtobesignificar1tlylongert11an60rr1ir1utes.Pu.rtherReadin,g:[Doo90]containstheoriginal(1953)rnatherr1aticaltheor}'ofstochasticprocesses.[HSP87]isaconciseintroductiontobasicprir1ciplesforreadersfamiliarv.rithprobabilityaridrandomvariables.Thesecor1dhalfof[PP02]isacornpreher1sivetreatiseor1stochasticprocesses.[PROBLEMS469Problems---------------------------Difficulty:Easy13.1.1ForExample13.4,defineasetofrandomvariablesthatcouldproducethesamplefunctionm,(t,s).DonotduplicatethesetlistedinExample13.7.13.1.2FortherandomprocessesofExamples13.3,13.4,13.5,and13.6,identifywhethertheprocessisdiscretetimeorcontinuous-time,discrete-valueorcontinuous-value.LetY(t)denotetherandomprocesscorrespondingtothetransmissionofonesymbolovertheQPSKcommtmicationssystemofExample13.6.\tVhatisthesamplespaceoftheunderlyingexperiment?Sketchtheensembleofsamplefunctions.13.1.313.1.4Inabinaryphaseshiftkeying(BPSK)communicationssystem,oneoftwoequallyprobablebits,0or1,istransmittedeveryTseconds.IfthekthbitisjE{0,1},thewaveformXj(t)=cos(27rfot+j7r)istransmittedovertheinterval[(k-l)T,kT].LetX(t)denotetherando1nprocessinwhichthreesymbolsaretransmittedintheinterval[O,3T].Assumingf"oisanintegermultipleofl/T,sketchthesamplespaceandcorrespondingsamplefunctionsoftheprocessX(t).ModerateDifficult+ExpertsOnlypanyhasanorderforonepartin104oscillatorsvvithfrequencybetween9999Hzand10,001Hz.Atechniciantakesoneoscillatorperminutefromtheproductionlineandmeasuresitsexactfrequency.(Thistesttakesoneminute.)TherandomvariableTrininutesistheelapsedtimeat'vhichthetechnicianfindsracceptableoscillators.(a)Findp,theprobabilitythatansingleoscillatorhasone-part-in-104accuracy.(b)\t\fhatisE[T1]minutes,theexpectedtimeforthetechniciantofindthefirstone-part-in-104oscillator?(c)Whatistheprobabilitythatthetechnicianwillfindthefirstone-part-in-104oscillatorinexactly20minutes?(d)\t\fhatisE[T5],theexpectedtimeoffindingthefifthone-part-in-104oscillator?13.2.3FortherandomprocessofProblem13.2.2,whatistheconditionalPMFofT2givenT1?Ifthetechnicianfindsthefirstoscillatorin3minutes,whatisE[22IT1=3],theconditionalexpectedvalueofthetimeoffindingthesecondonepart-in-104oscillator?13.1.5LetX(t)=e-(t-'r)u(t-T)beanexponentialpulse\VitharandomdelayT.Thedelay2"hasaPDFf'r(t).FindthePDFofX(t).LetvVbeanexponentialrandomvariablewithPDF13.3.1Trueorfalse:ForacontinuousvaluerandomprocessX(t),therandomvariableX(t0)isalwaysacontinuousrandomvariable.13.2.1fw(w)={~-w'UJ>0,otherwise.FindtheCDFFx(t)(x)ofthetime-delayedra1npprocessX(t)=t-W.13.2.2Inaproductionlinefor10kHzoscillators,theoutputfrequencyofeachoscillatorisarandomvariablevVuniformlydistributedbetween9980Hzand1020Hz.Thefrequenciesofdifferentoscillatorsareindependent.Theoscillatorcom-13.2.4Supposethatattheequator,wecanmodelthenoontimetemperatureindegreesCelsius,Xn,ondaynbyasequenceofiidGaussianrandomvariableswithexpectedvalue30degreesandstandarddeviationof5degTees.Ane\vrandomprocessYk=[X2k-1+X2k]/2isobtainedbyaveragingthetemperatureovert\vodays.IsYkaniidrando1nsequence?13.3.2Fortheequatorialnoonti1netemperaturesequenceXnofProblem13.3.1,asecondsequenceofaveragedtemperaturesisWn=[Xn+Xn-1]/2.IsvVnaniidrandomsequence?[470CHAPTER13STOCHASTICPROCESSES13.3.3LetYkdenotethentunberoffailuresbetweensuccessesk-1andkofaBernoulli(p)rando1nprocess.Also,letY1denotethenumberoffailuresbeforethefirstsuccess.WhatisthePl\tlFPyk(y)?IsYkaniidrandomsequence?13.4.1Thearrivalsofne'vtelephonecallsatatelephoneswitchingofficeisaPoissonprocessN(t)withanarrivalrateof.\=4callspersecond.AnexperimentconsistsofmonitoringtheswitchingofficeandrecordingN(t)overa10-secondinterval.(a)WhatisP_N(i)(O),theprobabilityofnophonecallsinthefirstsecondofobservation?(b)WhatisPN(i)(4),theprobabilityofexactlyfourcallsarrivinginthefirstsecondofobservation?(c)\i\fhatisPN(2)(2),theprobabilityofexactlytwocallsarrivinginthefirsttwoseconds?13.4.2QueriespresentedtoacomputerdatabaseareaPoissonprocessofrate.\=6queriesperminute.AnexperimentconsistsofmonitoringthedatabaseformminutesandrecordingN(1n),thenumberofqueriespre.sented.TheanswertoeachofthefollowingquestionscanbeexpressedintermsofthePlVIFPN(ni)(k)=P[J\T(m,)=k).(a)Whatistheprobabilityofnoqueriesinaone-minuteinterval?(b)Whatistheprobabilityofexactlysixqueriesarrivinginaone-minuteinterval?(c)\i\fhatistheprobabilityofexactlythreequeriesarrivinginaone-half-minuteinterval?13.4.3Atasuccessfulgarage,thereisal'vaysabacklogofcars'vailtingtobeserviced.Theservicetimesofcarsareiidexponentialrandomvariables'vithameanserviceti1neof30minutes.FindthePlVIFofJ\T(t),thenumberofcarsservicedinthefirstthoursoftheday.13.4.4Thecountofstudentsdroppingthecourse"ProbabilityandStochasticProcesses"iskno,vntobeaPoissonprocessofrate0.1dropsperday.Startingwithday0,thefirstdayofthesemester,letD(t)denotethenumberofstudentsthathavedroppedaftertdays.WhatisPD(t)(d)?13.4.5Customersarriveatthe'VeryfastBankasaPoissonprocessofrate.\customersperminute.Eacharrivingcustomerisimmediatelyservedbyateller.Afterbeingserved,eachcustomerimmediatelyleavesthebank.Thetimeacustomerspendswithatelleriscalledtheservicetime.Iftheservicetimeofacustomerisexactlyt'vominutes,whatisthePMFofthenumberofcustomersJ\T(t)inserviceatthebankattimet?13.4.6GivenaPoissonprocessN(t),identifywhichofthefollowingarePoissonprocesses..(a)N(2t),(b)N(t/2),(c)2N(t),(d)N(t)/2,(e)N(t+2),(f)N(t)-N(t-1).13.4.7Startingatanytimet,thenumberN,,.ofhamburgerssoldata\i\fhiteCastleintheTminuteinterval(t,t+T)hasthePoissonPlVIF.()PJV,,.Tl,-{(lo)nTe0-10T/fr1,.rl,=0,1,...otherwise(a)FindtheexpectednumberE[J\T6o]ofhamburgerssoldinonehour(60minutes).(b)\tVhatistheprobabilitythatnohamburgersaresoldinthe10-minuteintervalstartingat12noon?(c)Youarriveatthe\i\fhiteCastleat12noon.YouwaitarandomtimeW(minutes)untilyouseeahamburgersold.\i\!hatisthePDFofvV?Hint:FindP[W>11J).13.4.B•Asequenceofqueriesaremadetoadatabasesystem.Theresponsetimeofthesystem,Tseconds,istheexponential(1/8)randomvariable.Assoonasthesystemrespondstoaquery,thenextqueryis[PROBLEMSmade.Assumingthefirstqueryismadeattimezero,letN(t)denotethenumberofqueriesmadebytimet.(a)\tVhatisP[7"'>4],theprobabilitythatasinglequery'villlastatleastfourseconds?(b)Ifthedatabaseuserhasbeen'vaitingfivesecondsforaresponse,whatisP[7"'>1317"'>5),theprobabilitythattheuserwillwaitatleasteightmoreseconds?(c)\tVhatisthePlVIFofN(t)?13.4.9TheproofofTheorem13.3n&glectedthefirstinterarrivaltimeX1.Sho'vthatX1hasanexponential(.\)PDF.13.4.10U1,U2,...areindependentidenticallydistributeduniformrandomvariables'vithparameters0and1.(a)LetXi=-lnUi.\i\fhatisP[Xi>x)?(b)WhatkindofrandomvariableisXi?(c)Givenaconstantt>0,letfldenotethevalueofn,,suchthatnITui>ei=l>stationvvhileyouarewaiting.WhatisthePlVIFPN(n,)?13.5.3Asub,vaystationcarriesbothblue(B)lineandred(R)linetrains.RedlinetrainsandbluelinetrainsarriveasindependentPoissonprocesses'vithratesAR=0.15andAB=0.30trains/minrespectively.Youarriveatthestationatrandomtimetandwatchthetrainsforonehour.(a)WhatisthePl\!lFoffl,thenumberoftrainsthatyoucountpassingthroughthestation?(b)GiventhatyouseeN=30trains,whatistheconditionalPMFofR,thenumberofredtrainsthatyousee?13.5.4BusesarriveatabusstopasaPoissonprocessofrate,\=1bus/minute.Afteraverylongtimet,youshowupatthebusstop.(a)LetXdenotetheinterarrivaltimebetweent'vobusarrivals.\tVhatistheI>DFfx(x)?(b)LetvVequalthetimeyouwaitaftertimetuntilthenextbusarrival.WhatisthePDFfvv(ru1)?n+l-t471ITui.i=lNotethatwedefineTI~\i\fhatisthePMFofN?1ui1.13.5.1CustomersarriveatacasinoasaPoissonprocessofrate100customersperhour.Uponarriving,eachcustomermustflipacoin,andonlythosecustomers'vhoflipheadsactuallyenterthecasino.LetN(t)denotetheprocessofcustomersenteringthecasino.Findthe.PMFoffl,thenumberofcustomerswhoarrivebet,veen5P:rvIand7PM.13.5.2Asubwaystationcarriesbothblue(B)lineandred(R)linetrains.RedlinetrainsandbluelinetrainsarriveasindependentPoissonprocesseswithrates.\1z=0.05and,\B=0.15trains/min,respectively.Youarriveatarandomtimetandwaituntilaredtrainarrives.LetNdenotethenumberofbluelinetrainsthatpassthroughthe(c)LetVequalthetime(inminutes)thathaspassedsincethemostrecentbusarrival.\t\lhatisthePDFfv(v)?(d)LetUequalthetimegapbetweenthemostrecentbusarrivalandthenextbusarrival.WhatisthePDFofU?13.5.5ForaPoissonprocessofrateA.,theBernoulliarrivalapproximationassumesthatinanyverysmallintervaloflength.6..,thereiseither0arrivalswithprobability1-A..6..or1arrivalwithprobabilityA..6...·usethisapproximationtoproveTheorem13.7.13.5.6ContinuingProblem13.4.5,supposeeachservicetimeiseitheroneminuteort'vominutesequiprobably,independentofthearrivalprocessortheotherservicetimes.\i\fhatisthePMFofthenu1nberofcustomersN(t)inserviceatthebankat.t'?,.time[472CHAPTER13STOCHASTICPROCESSES13.5.7Tenrunnerscompeteinaracestartingattimet=0.Therunners'finishingtimesRi,...,Rioa:reiidexponentialrandomvariables'vithexpectedvaluel/p,=10minutes.(a)Whatistheprobabilitythatthelastrunnerwillfinishinlessthan20minut,es.?(b)WhatisthePDFofX1,thefinishingtimeofthe'vinningrunner?(c)FindthePDFofY=R1+···+Rio.(d)LetX1,...,X10denotetherunners'interarrivaltimesatthefinishline.FindthejointPDFfx1,...,x10(x1,...,X10).13.5.8LetNdenotethenumberofarrivalsofaPoissonprocessofrate,:\overtheinterval(0,T).Given.J\T=n,,letS1,...,S11denotethecorrespondingarrivaltimes.Provethatfs1,...,Snll\I(S1,...,Snlri)={n,!/Tn00<s1<···<Snother,vise.<T,Concludethat,givenN(T)=n,,S1,...,S11aretheorderstatisticsofacollectionofn,uniform(0,T)randomvariables.(SeeProblem5.10.11.)13.6.1Overthecourseofaday,thestockpriceofawidelytradedcompanycanbemodeledasaBrownianmotionprocesswhereX(0)istheopeningpriceatthemorningbell.Supposetheunitoftimetisanhour,theexchangeisopenforeighthours,andthestandarddeviationofthedailypricechange(thedifferencebetweentheopeningbellandclosingbellprices)is1/2point.\i\fhatistheBro,vnianmotionparametera?13.6.2Xo,X1,...isaniidGaussian(0,1)rando1nsequence.TherandomsequenceYriisdefinedbyY0=0andYri+1=Xn+1+1";1.Findtheautocorrelationfur1ctionRy[n,,k].13.6.3LetX(t)beaBrownianmotionprocesswithvarianceVar[X(t)]=at.Foraconstantc>0,determinewhetherY(t)=X(ct)isaBrownianmotionprocess.13.6.4ForaBrownianmotionprocessX(t),letXo=X(O),X1=X(l),...representsamplesofaBro,vnianmotionprocesswithvarianceat.Thediscrete-timecontinuous-valueprocessY1,Y2,...definedbyYri=Xn-Xn-1iscalledanincrem,entsprocess.ShowthatYnisaniidrando1nsequence.13.6.5ThisproblemworksoutthemissingstepsintheproofofTheorem13.8.ForWandXasdefinedintheproofofthetheorem,showthatW=AX.\t\lhatisthematrixA?UseTheorem8.11tofindf,,-(w).13.7.1X11isaniidrandomsequencewithexpectedvalueE[X11]=JLXandvarianceVar[Xn]=a1:-.WhatistheautocovarianceOx[m,,k]?13.7.2Forthetime-delayedrampprcrcessX(t)fromProblem13.2.1,findforanyt>0:(a)TheexpectedvaluefunctionJLx(t),(b)Theautocovariancefunction0x(t,T).Hint:E[W]=1andE[W2]=2.13.7.3_A.simplemodel(indegreesCelsius)forthedailytemperat1u·eprocessO(t)ofExample13.3isl27rn,Cn=16[1-cos-+4Xn,365whereX1,X2,...isaniidrandomsequenceofGaussian(0,1)randomvariables.(a)WhatisE[On]?(b)FindtheOc[m,,k].autocova1iancefunction(c)\t\lhyisthismodeloverlysimple?13.7.4Adifferentmodelforthedailytemperatureprocess0(n,)ofExample13.3is1On=20n-l+4X11,[PROBLEMS473'vhereCo,X1,X2,...isaniidrandomsequenceofGaussian(0,1)randomvariables.sa1nplingX(t)every.6.seconds,weobtainthediscrete-timerandomsequenceY'.;1=X(ri,.6.).IsYnastationarysequence?(a)FindthemeanandvarianceofCn·13.8.5GivenastationaryrandomsequenceXn,\VecanS7J,bsampleXnbyextractingeverykthsample:Y'.;1=Xkn·IsY'.;1astationaryrandomsequence?(b)FindtheautocovarianceCc[1n,k].(c)Isthisaplausiblemodelforthedailytemperatureoverthecourseofayear?(d)WouldC1,...,C31constituteaplausiblemodelforthedailytemperatureforthemonthof.January?13.7.5ForaPoissonprocessN(t)ofrate.A,showthatfors<t,theautocovarianceisCN(s,t)=.As.Ifs>t,whatisCN(s,t)?IsthereageneralexpressionforCN(s,t)?13.7.6N(t)isaPoissonprocessofrate.A=1andXo,X1,X2,...isaniidsequenceofGaussian(0,a)randomvariablesthatareindependentofN(t).Considertheprocess{Y(t)lt>O}definedbyY(t)=XN(t)·Findtheexpectedvalueµ,y(t)=E[Y(t)]andthecovariancefunctionCy(t,r).(Assumelrl<t.)13.7.7Xnisaniidrando1nsequence\VithE[Xn]=0and'lar[Xn]=3.FindtheautocorrelationfunctionCy[n,,k]oftheprocessYn=Xn-1Xn.Foranarbitraryconstanta,letY(t)=X(t+a).IfX(t)isastationaryrandomprocess,isY(t)stationary?13.8.113.8.2X=[X1X2]'hasexpectedvalueE[X]=0andcovarianceinatrixCx=[i~]DoesthereexistastationaryprocessX(t)andtimeinstancesf1andt2suchthatXisactuallyapairofobservations[X(t1)X(t2)]'oftheprocessX(t)?Foranarbitraryconstanta,letY(t)=X(at).IfX(t)isastationaryrandomprocess,isY(t)stationary?13.8.313.8.4LetX(t)beastationarycontinuous-timerandomprocess.By13.8.6LetAbeanonnegativerandomvariablethatisindependentofanycollectionofsamplesX(t1),...,X(tk)ofastationaryrandomprocessX(t).IsY(t)=AX(t)astationaryrandomprocess?13.8.7Letg(x)bedeterministicfunction.IfX(t)isastationaryrandomprocess,isY(t)=g(X(t))astationaryprocess?13.9.1\i\fhichofthefollowingarevalidautocorrelationfunctions?Rl(r)=o(r)R3(T)=0(T-10)R2(r)=o(r)+10R4(T)=0(T)-1013.9.2Let_4beanonnegativerandomvariablethatisindependentofanycollectionofsamplesX(t1),...,X(tk)ofawidesensestationaryrandomprocessX(t).IsY(t)=_4+X(t)awidesensestationaryprocess?13.9.3TrueorFalse:IfXnisawidesensestationaryrandomsequencewithE[Xn]=0,thenY'.;1=Xn-Xn-lisa\videsensestationaryrandomsequence.13.9.4LetXndenoteaniidsequenceofBernoulli(p=1/2)randomvariables.FindtheautocorrelationfunctionRx[n,,k]andtheautocovariancefunctionCx[n,,k].13.9.5Xnisaniidsequence\VithE[Xn]=µandVar[Xn]=a2.FindtheautocorrelationfunctionRx[n,,k].13.9.6X(t)andY(t)areindependentwidesensestationaryprocesses.Determineiftheseprocessesare\vid&sensestationary:(a)V(t)=X(t)+Y(t),(b)W(t)=X(t)Y(t).[474CHAPTER13STOCHASTICPROCESSES13.9.7TrueorFalse:IfXnisa\¥idesensestationaryrandomsequence\VithE[Xn)=0,thenYn=Xn+(-l)n-lXn-lisawidesensestationaryrandomsequence.13.9.8vV(t)Considertherandomprocess=Xcos(2?Tfot)+Ysin(2?Tfot),\¥hereXandYareuncorrrelatedrandomvariables,eachwithexpectedvalue0andvariancea2.FindtheautocorrelationRw(t,T).IsvV(t)\videsensestationary?13.9.9X(t)isa'videsensestationaryrandomprocesswithaveragepo,verequalto1.Let8denotearandomvariable'vithuniformdistributionover[O,2?T]suchthatX(t)andeareindependent.(a)WhatisE[X2(t))?(b)WhatisE[cos(2?Tfct+'8))?13.10.1X(t)andY(t)areindependentwidesensestationaryprocesseswithexpectedvaluesµxandJJ,yandautocorrelationfunctionsRx(T)andRy(T),respectively.LetW(t)=X(t)Y(t).(a)Findp,wandRvv(t,T)andsho\vthatW(t)is\¥idesensestationary.(b)AreW(t)andX(t)jointlywidesensestationary?13.10.2X(t)isa\videsensestationaryrandomprocess.ForeachprocessXi(t)definedbelow,determinewhetherXi(t)andX(t)arejointly\¥idesensestationary.(a)X1(t)=X(t+a)(b)X2(t)=X(at)13.10.3X(t)isawidesensestationarystochasticprocesswithautocorrelationfunction(c)LetY(t)=X(t)cos(2?Tfct+8).\i\fhatisE[Y(t))?(d)WhatistheaveragepowerofY(t)?13.9.10ProvethepropertiesofRx[n,)giveninTheorem13.12.13.9.11LetXnbea\videsensestationaryrandomsequence\¥ithexpectedvalueJJ,xandautocovarianceCx[k).Form=0,1,...,wedefine-1x---2m,+1rnrn-n=-rn,asthesamplemeanprocess.ProvethatifI:%°_00Cx[k]<oo,thenXo,X1,...isanunbiasedconsistentsequenceofestimatesofjJ,X.TheprocessY(t)isaversionofX(t)delayedby50microseconds:Y(t)=X(t-to)whereto=5x10-"'0s.(a)DerivetheautocorrelationfunctionofY(t).(b)Derivethecross-correlationfunctionofX(t)andY(t).(c)IsY(t)'vide-sensestationary?(d)AreX(t)andY(t)jointly\videsensestationary?13.11.1_AstationaryGaussianprocessX(t)isobservedattimestlandt2toform1therandomvectorX=[X(t1)X(t2)]withexpectedvalueE[X)=0andcovari-[~?:;i]13.9.12Determinewhethereachofthesestatementsistrueorfalse:ancematrixCx=(a)IfXnandYnareindependentstationaryprocesses,thenVn=Xn/Yniswide-sensestationary.13.11.2GivenaGaussianprocessX(t),identifywhichofthefollowing,ifany,areGaussianprocesses.(a)2X(t),(b)X(t/2),(b)IfXnandXiareindependent\¥idesensestationaryprocesses,thenWn=Xn/Y11iswide-sensestationary.·Whatistherangeofvalidvalues(ifany)of(c)X(t)/2,(e)X(2t).er?anda~?(d)X(t)-X(t-1),[PROBLEMS475(a)FindtheconditionalCDF13.11.3A'vhiteGaussiannoiseprocessN(t)withautocorrelationR_N(T)=nO(T)ispassedthroughanintegratoryieldingtheoutputY(t)=fo'N(u)du.FindE[Y(t)]andtheautocorrelationfunctionRy(t,T).Sho\vthatY(t)isanonstationaryprocess.LetX(t)beaGaussianprocess\Vithmeanp,x(t)andautocovarianceCx(t,T).Inthisproblem,weverifythatthefortwosamplesX(t1),X(t2),themultivariateGaussiandensityreducestothebivariateGaussianPDF.Inthefollowingsteps,leta'fdenotethevarianceofX(ti)andletp=Cx(t1,t2-t1)/(a1a2)equalthecorrelationcoefficientofX(t1)andX(t2).13.11.4(a)FindthecovariancematrixCandsho'vthatthedeterminantisICI=afa~(lp2).(b)Showthattheinverseofthecorrelationmatrixisc-1=-11-p2(c)Nowsho\vthatthemultivariateGaussiandensityforX(t1),X(t2)isthebivariateGaussiandensity.13.11.5ShowthattheBrownianmotionprocessisaGaussianrandomprocess.Hint:ForWandXasdefined.intheproofoftheTheorem13.8,findmatrixAsuchthatW=AXandthenapplyTheorem8.11.FY(t)IN(t)Expressyouranswerintermsofthe13.12.1Writeal\IIATLABprogramthatgeneratesandgraphsthenoisycosinesamplepathsXcc(t),Xdc(t),Xcct(t),andXdct(t)ofFigure13.3.NotethatthemathematicaldefinitionofXcc(t)isXcc(t)=2cos(2'rrt)N(t)Y(t)=Ln=OXn.+N(t),-1<t<1.NotethatN(t)isa\vhitenoiseprocesswithautocorrelationRN(T)=O.Olo(T).Practically,thegraphofXcc(t)inFigure13.3isasampledversionXcc[n,)=Xcc(n,Ts),wherethesamplingperiodisTs=0.00ls.Inaddition,thediscrete-timefunctionsareobtainedbysubsamplingXcc[ri).Insubsampl.ing,\VegenerateXctc[n,)byextractingeverykthsampleofXcc[n,);seeProblem13.8.5.Intermsof1VIATLAB,whichstartsindexingavectorx'vithfirstelementx(1))Xdc(n)=Xcc(1+(n-1)k).Thediscrete-timegraphsofFigtu·e13.3usedk=100.13.12.2ForthetelephoneS\vitchofExample13.28,wecanestimatetheexpectednumberofcallsinthesystem,E[M(t)],afterTminutesusingthetimeaverageestimate1TJiij,T=TLJ\!l(k).k=l13.11.6LetX1,X2,...denoteas&quenceofiidGaussian(0,1)randomvariables.LetN(t)denoteaPoissonprocessofrate.AthatisindependentofthesequenceXn.Considertherandomprocess=p[Y(t)<vlN(t)=n,).(yin,)Performa600-minuteS\vitchsimulation-andgraphthesequenceAlf1,JV!2,...,M600·Doesitappearthatyourestimatesareconverging?Repeatyourexperimenttentimesandinterpretyourresults.13.12.3AparticulartelephoneswitchhandlesonlyautomatedjunkvoicemailcallsthatarriveasaPoissonprocessofrate[476CHAPTER13STOCHASTICPROCESSES.A=100callsperminute.Eachautomatedvoicemailcallhasdurationofexactlyoneminute.UsethemethodofProblem13.12.2toestimatetheexpectednumberofcallsE[M(t)].DoyourresultsdifferverymuchfromthoseofProblem13.12.2?13.12.4Recallthatforarate.APoissonprocess,theexpectednumberofarrivalsin[O,T]is>.T.Inspectionofthecodeforpoissonarrivals(lambda,T)willshowthatinitiallyn,=jl.l.ATlarrivalsaregenerated.IfSn>T,theprog,Tamstopsandreturns{SjISJ<T}.Otherwise,ifSn<T,thenvvegenerateanadditionalnarrivalsandcheckifS2n>T.ThisprocessmayberepeatedanarbitrarynumberoftimeskuntilSkn>T.LetKequalthenumberoftimesthisprocessisrepeated.\i\fhatisP[I<=1]?Whatisthedisadvantageofchoosinglargern,soastoincreaseP[K=1]?13.12.5Inthisproblem,weemploytheresultofl=>roblem13.5.8asthebasisforafunctions=newarrivals(lambda,T)thatgeneratesaPoissonarriva]process.Theprogramnewarrivals.mshoulddothefollo,ving:•GenerateasamplevalueofN,aPoisson(.AT)randomvariable.•GivenN=n,,generate{U1,...,Un},asetofnuniform(0,T)randomvariables.•Sort{U1,...,Un}fromsmallesttolargestandreturnthevectorofsortedelements.Writetheprogramnewarrivals.mandexperimenttofindout'vhetherthisprogramisanyfasterthanpoissonarrivals.m.13.12.6SupposetheBro,vnianmotionprocesswitha=1isconstrainedbybarriers.Thatis,we'vishtogenerateaprocessY(t)suchthat-b<Y(t)<bforaconstantb>0.Buildasimulationofthissystem.EstimateP[Y(t)=b].ForthedepartureprocessD(t)ofExample13.28,letDndenotethetimeofthenthdeparture.Then,thinterdeparturetim,eisthenVn=Dn-Dn-1·Fromasamplepathcontaining1000departures,estimatethePDFof~i·IsitreasonabletomodelV'.;iasanexponentialrandomvariable?\tVhatisthemeaninterdeparturetime?13.12.7[AppendixAFamiliesofRandomVariablesA.1DiscreteRandomVariablesBernoulli(IJ)ForOPx(x;)E[X]·var[X]=1-r)~r;=OIJ;i;0otherwise=1x(s)=1-[J+[Jes=JJ=IJ(l-p)==-Binomial(ri,p)---==Forapositiveiritegern,arid0<p<1,1Px(x)E[X]Var[X]=(n.')px(lxJJ)n-:i:=Tl[)=riJJ(l-r))DiscreteUniform(k,l)Forintegerskaridlst1chthatk<l,==Px(x)=1/(l-k+l)x=k,k+l,...,l0otherwiseesk_es(l+I)x(s)=(l-k+1)(1-es)E[X]=k+l2Var[X]=(l-k)(l-k+2)12477[478FAMILIESOFRANDOMVARIABLES===Geometric(r>)==::::aFor0<1,Px(x)=p(l-J>)x-1;i;0other-vvise=pes1,2,...X(s)=1-(1-P)esE[X]=1/p·var[X]=(1-p)/p2==-Multinomia1----.Forintegern,>0,I>,;,>0for'i=1,...,ri,a.ridJ>1+···+I>n=1,==-Pascal(k,p)Forpositiveintegerk,and0Px(x)=<p<1,(~=~)pk(l-p)x-kE[X]=k/r>Var[X]=k(l-p)/r>2Poisson(a)===Forcv.>0,Px(:i;)=x!0E[X]=aVar[X]=ax=0,1,2,...otherwiseX(s)=(r>es1-(1-P)es)k[A.2CONTINUOUSRANDOMVARIABLES479Zipf(n,,a)Forpositiveir1tegern,>0andconstanta>1,c(n,,a)Px(::c)=0x=1,2,...,riotherwise-vvhere1nc(n,cv.)A.2=-1L~k=lContinuousRandomVariablesBeta(i,j)Forpositiveir1tegersiandj,thebetafunctionisdefinedas..(3('1,,J)=(i+j-1)!(i-l)!(j-1)!Foraf3('i,j)randorn·variableX,1·x(x)=E[X)V··[X)ar==f3(i,j)x;i-1(1-x;)-'l-l0<x0otherwise"'i+j..iJ(i+j)2('i+j+l)Cauchy(a,b)Forconstantsa>0and-oo<b<oo,fx();i;=a,111a,2+(x-b)2(s)=ebs-alslNotet11atE[X)is11r1definedsincef00x;f'x(x)dxisundefined.SincethePDFissymmetrica.boutx=b,themea.r1car1bedefined,ir1theser1seofaprincipalvalue,tobeb.00E[X)bVar[X]=oo[480FAMILIESOFRANDOMVARIABLES===Erlang(ri,,\)--For,\>0,aridapositiveinteger'n,(ri-1)!x(s)other\;vise0=,\)n(,\-sE[X)='n/,\Var[X]=n,/,\2===Exponential(,\)--For,\>0,f'x(:i;)=>0Ae->.x;i;0other\;vise,\x(s)=,\-sE[X)=1/,\·var[X]=1/,\2---Gamma(a,b)--Fora>-1andb>0,;_r;ae-x/bfx(:i;)=x>Oa!ba+lotherwise0E[X)=(a+1)bVar[X)=(a+1)b2Gaussian(µ,,a)---===Forconstantsa>0,-oo<µ,<oo,...,e-('.r;-µ,)-/20,f'x(x)=v/2Fra3x;2e-a2:i:2/2x>00othervvisev;;I;E[X]={8Var[X]=37r7ra28Pareto(a,11,)Fora>0andµ,>0,f'x(x)=E[X]=Var[X]=(a/µ,)(x;/1;,)-(a+I)x>µ,0otherwiseai;,a-1aµ,2(a-2)(a-1)2(a>1)(a>2)481[482FAMILIESOFRANDOMVARIABLESRayleigh(a)Fora>0,fx(x)=a2xe-a2:1:2/2x>00otherwiseE[X]=;;~VVar[X]=2-:/2a,Uniform(a,b)Forconstantsa<b.I1fx(x)=E[X]=b0a;a,otherv.risebVar[X]=(b-a,)122ebs-a0fore·very(g)Asetofvectors{x1>•••,Xn}isorthon,orrnalifx'ix.i=1ifi=jaridotherv.riseeq11alszero.(11)AmatrixUis11,riitaryifitscolurnns{u1>•.•,un}areorthor1ormal.MathFactB.15RealSymmetricMatricesArealsyrr1rr1etricrr1atrix.Ahast11efollo,vingproperties:(a)Alleiger1va.luesofAarereal.(b)Ifx1andx2areeigenvectorsofAcorrespondingtoeigenval11esA1-=/:A2,ther1x1andx2areorthogonal\rectors.(c)Acar1bevvritter1asA=UDU'vvhereDisadiagonalrr1atrixandUisallnitarymatrixvvithcolurr1nsthataren,orthonorrr1aleigen,rectorsofA..----MathFactB.16PositiveDefiniteMatricesForarealsyrr1rnetricrr1atrixA,thefollo,vir1gstaterr1entsareequiva.ler1t:(a)Aisapositivedefiriiternatrix.>0forallnor1zerovectorsx.(c)Eacheiger1value;\ofAsatisfies;\>0.(b)x'Ax(d)Thereexistsanonsir1gul[488AFEWMATHFACTSMathFactB.17PositiveSemidefiniteMatricesForarealsyrr1rnetricrr1atrixA,thefollo\vir1gstaterr1entsareequiva.ler1t:(a)Aisapositi,resernidefir1iterr1atrix.(b)x'Ax>0forallvectorsx.(c)Eacheiger1value.AofAsatisfies.A>0.(d)T11ereexistsarnatrixWst1chthatA=WW'.[ReferencesBer98.P.L.Bernstein.AgainsttheGods:TheRernarkableStoryofRisk.JohriWile}')1998.Bill2.P.Billingsley.Probabilitya'Tl,dJl/leasv,re.JohriWiley&Soris)anniversaryedition,2012.BT08.D.P.BertsekasaridJ.N.Tsitsiklis.I'Tl,troductio'TI,toProbability.AthenaScientific,2ndedition)2008.DavlO.T.A.Da·vis.MATLABPrirner.CRCPress,8t1iedition)2010.Doo90.J.L.Doob.StochasticProcesses.·\i\TileyRepririt)1990.Dra67.A.vV.Drake.Furl,darnen,talsofAppliedProbabilityTheory.McGravv-Hill)NewYork,1967.Dtu·94.R.Durrett.TheEsseritialsofProbability.Dlixbury,1994.Gall3.R.G.Gallager.StochasticProcesses:Urii·versityPress,2013.GS93.L.Gonickand\i\T.Srrlith.TheCartoo'TI,G'uidetoStatist'ics.HarperPererinial,1993.Gl1b06.J.Gt1brier.Probabilitya'Tl,dRa'Tl,dornProcessesforElectr~icala'Tl,dCornpv,terErigirieers.CarribridgeUni·versityPress)2006.HayOl.SirrionHaykin.Cornrn'U'Tl,icat'tOT/,Systerns.JohnvViley)4thedition)2001.HLll.D.HanselmanandB.Littlefield.Masteri'Tl,gMATLAB.PreriticeHall,2011.HSP87.P.G.Hoel)C.J.Stone)andS.C.Port.I'Tl,trodv,ctio'TI,toStochasticProcesses.WavelaridPress,1987.Ka}r98.S.~v1.Ka}'·Furidarnen,talsofStatisticalSig'Tl,alProcessi'Tl,gVolv,rneII:Detection,theor~y.PrenticeHall,1998.Theor~yforAJJplicatirJ'ns.CarnbridgeKMT12.H.Kobayashi)B.]\/lark)aridW.Turin.Probability7Ra'Tl,dornProcesses7a'Tl,dStatisticalA'Tl,alysis:Applicatio'Tl,stoCornrnu'Tl:icatioris,Sign,alProcessi'Tl,g7Quev,ei'Tl,gTheoryaridMathernaticalFi'Tl,a'Tl,ce.CarnbridgeUniversit}'Press)2012.LGll.A.Leon-Garcia.Probab'ility,Statist'ics,a'Tl,dRa'Tl,dornProcessesforElectricalE'Tl,g'i'Tl,eerin,g.PreriticeHall)thirdeditiori)2011.NIRlO.D.C.]\/IontgorrieryandG.C.Rl1nger.AJJpliedStatisticsan,dProbabilityforE'Tl,girieers.John\i\Tiley&Sons)fifthedition,2010.PosOl.K.Pookitt.DoYo'uFeelLucky?TheSecretsofProbab'ilityScliolastic,2001.489[490REFERENCESPP02.A.Papol1lisaridS.U.Pillai.Probab·ility)Raridom,Variablesan,dStochasticProcesses.McGrav.rHill,4thedition,2002.Rosl2.S.~v1.Ross.AFirstCo1J,rsein,Probability.Pearson,r1intheditior1,2012.SMMlO.R.L.Scheaffer,M.Niulekar,andJ.T.McClave.Probab·ilityar11dStatisticsforEn,girieers.CengageLearnir1g,5thedition,2010.Str98.G.Strang.ln1trodv,ction,toLin,earAlgebra.·vVellesleyCarnbridgePress,secondedition,1998.Ver98.S.Verdu.M·ultiv,serDetection,.Cambridge·ur1iversit}rPress,Ne'ivYork,1998.WSOl.J.W.WoodsaridH.Stark.Probabilityan,dRan,dornProcesses111ithApplication,stoS·ign,alProcessin,g.Prer1ticeHall,3rdedition,2001.[IndexaprioriprobabiliLy,15acceptanceset,367alLernativehypothesis,369arrival,74,440asyn1ptoticallyunbiasedestimator,347auLocorrelationfunction,429,449randomsequence,449widesensesLationaryprocess,456auLocorrelationBrownianmotion,450auLocovariancefunction,429randomsequence,449stochasticprocess,449autocovarianceBrownianmotion,450averagepO\·ver,457axiomsofprobability,11,196conditionalprobability,16consequencesof,13aprioriprobabiliLy,370Bayes'theorem,22bellcurve,321Bernoullidecomposition,444Bernoulliprocess,438Bernoullirandomvariableexpectedvalue,83bernoullicdf.m,103bernoullipmf.m,103bernoullirv.m,103biasinestin1ators,346bigpoissonpmf.m,117binaryhypoLhesistest,369maximumaposterioriprobability,374minimumcost,377binaryhypothesistest,370binon1ialcoefficient,42binomialrandomvariable,113,326-327expectedvalue,84binon1ialcdf.m,103binomialpmf.m,103,336binomialrv.n1,103blindestimation,401Brovvn.Robert,447Bro\·vnianmotion,446-447jointPDF,447\·vithabarrier,476brownian.1n,466centrallimittheorem,322,360approximation,322confidenceintervalestimation,356significancetests,368Chebyshevinequality,339,341Chernoffbound,339,341chiptest.m,56circuits.m,217clipper,237clippingcircuit,226collectivelyexhaustive,6combinations,42communicationssystembinary,392BPSK,392,469CDlVIA,395-397,428!v1PSK,395,398QA!vI,394-395QPSK,385-386,394-395,433469'ternaryamplitudeshiftkeying,394compactdisc,325complementaryCDFstandardnormal,142componentsinparallel,53inseries,53conditionalexpectedvalueasarandomvariable,263givenarandomvariable,262ofafunctiongivenanevent,254conditionalprobability,16conditionalcumulativedistributionfuncLion,243expectedvalue,249,263n1eansquareerror,401probabilitydensityfunction,258properties,249probabilitymassfunction,256givenanevent,243joint,252properties,249variance,250confidencecoefficient,352,354confidenceinterval,352,354Gaussian,356consistentestimator,346continuousrandomvariable,64,118,121cu1nulativedisLributionfunction,121expectedvalue,129convergencealmostalways,344almostevery\·vhere,344almostsurely,344inprobability,344withprobability1,344convolution,234correlationcoefficient,185inlinearestimation,405correlation,187randomvector,287randomvectors,286counL.m,103,330,464counLequal.m,61countingprocess,440countingfundamentalprincipleof,40methods,40covariancematrix,287,289randomvectors,287covariance,184noisyobservation,189ofindependentrandomvariables,189ofrandomvectors,287properties,187cross-correlation,289-290,429function,459ofrandomvectors,289cross-covariance,289-290ofrando1nvectors,289cumulativedistributionfunction491[492INDEXconditional,243continuousrandomvariable,121discreterandomvariable,77joint,164derivedfromjointPDF,173multivariate,195ofapairofrandon1vectors,278randomvectors,278standardnormal,140DCvoltage,457decisionregions,386decisionstatistic,370decorrelation,396deltafunction,145deltarv.m,241departures,465DeMoivre-Laplaceformula,327Del\1organ'sla\~',6diabetestest,270discreterandomvariableconditionalf>f\1F,256variance,94discreteuniformrandomvariableexpectedvalue,84dispersion,93distinguishableobjects,41dtrianglerv.m,269duniforn1cdf.n1,103duniforn1pmf.m,103duniformrv.m,103eigenvalue,297,487eigenvector,487Einstein.J\lbert,447ensembleaverages,431ensemble,431equallylikelyoutcomes,14erf.m,152ergodic,457erlangcdf.m,153erlangpdf.1n,153erlangrv.m,153,235estimationblind,401linear,404fromrandomvectors,414ofparameters,427LMSE,404estimatorasymptoticallyunbiased,347consistent,346linearineansquareerror(LMSE),405maximumaposterioriprobability,409-410m1n1mummeansquareerror(MlVISE),402unbiased,347event,10expectation,82iterated,264expectedvalue,65,81,338Bernoullirandomvariable,83binomialrandomvariable,84conditional,249,263continuousrandomvariable,129derivedrandomvariable,90discreterandomvariable,81discreteuniformrandomvariable,84exponentialrandomvariable,135functionoft\·VOrandomvariables,182geometricrandomvariable,83ofafunction,130ofasumoffunctions,182ofsun1,182Pascalrandomvariable,84randommatrix,286randomsum,319randon1vectors,286stochasticprocess,448experiment,8exponentialrandon1variableexpectedvalue,135variance,135exponentialcdf.m,153exponentialpdf.m,153exponentialrv.m,153factorial.m,100falseacceptance,368falsealarm,371falserejection,368find.m,268finest-grain,9finitecdf.m,103finitepmf.m,99,103-105,203,329finiterv.m,103,105,203firstmoment,96floor.m,101footballpool,390freqxy.m,204gameshO\·Vl\1ontyHall,38suitcases,112gausscdf.m,153GaussianPDFbivariate,475multivariate,291,4'75processvvhitenoise,463\·videsensestationary,463randomvariables,291,480bivariate,191,,randomvector,304randomvectors,291stochasticprocess,462gausspdf.m,153gaussrv.m,153gaussvector.m,299,467gaussvectorpdf.m,299genetics,32,34geometricrandon1variableexpectedvalue,83geometriccdf.m,103geometricpmf.m,100,103geometricrv.m,103,117,161georv.1n,161Graycode,398gseq.m,467-468handoffs,31hardlimiter,226headwind,236,335highbloodpressure,247hist.m,56,61,104,204humangranulocyticehrlichiosis,32hypothesistest,367binary,370-374,385maximumaposterioriprobability,374maximumaposterioriprobability,373-374maximumlikelihood,381-382minimu1ncost,377Neyman-Pearson,379-380multiple,384maximumaposterioriprobability,385maximumlikelihood,385icdf3spin.m,235icdfrv.m,235,241icdfw.m,241identicallydistributed,200iid-seeindependentandidenticallydistributed,200imagepmf.m,202imagerv.m,203imagesize.m,202-203[INDEXimagestem.m,204improperexperiments,64incrementsprocess,472independentandidenticallydistributed,200randomsequence,43'7,450independentNrando1nvariables,200componentsofaGaussianrandomvector,292events,24inorethant\·VOevents,26threeevents,26increments,447,450randomvariables,179randomvectors,280trials,44,49indicatorrandomvariables,344intera.rrivalti1ne,116,442-444Poissonprocess,442Internet,11,18,51,69,138,248,274,368jitter,116jointlywidesensestationaryprocessescross-correlation,461julytemps.m,300Laplacetransforn1,311lawofaverages,338lawoflargenun1bersvalidationofrelativefrequencies,345weak,344,349lawofLotalprobability,21likelihoodfunctions,370likelihoodraLio,374limiter,226-227,229,238linearestimate,401linearestimation,404usingarandomvector,414,416,419linearestimatorGaussian,407linearmeansquareerror(LMSE)estimator,405linearprediction,420linearLransformationGaussianrandomvector,294momentgeneratingfunction,313ofarandomvector,284,289lottery,60,109,114-115,275,305Lymedisease,32marginalprobabilitymassfunction,170marginalprobabilitymassfunction,169-170Markovinequality,339-340MATLABfunctionbernoullicdf,103bernoullipmf,103bernoullirv,103bigpoissonpmf,117binomialcdf,103binomialpmf,103,336binomialrv,103bro,.vnian,466chiptest,56circuits,217count,103,330,464countequal,61deltarv,241dtria.nglerv,269duniformcdf,103duniformpmf,103duniformrv,103erf,152erlangcdf,153erlangpdf,153erlangrv,153,235exponentialcdf,153exponentialpdf,153exponentialrv,153factorial,100find,268finiLecdf,103finiLepmf,99,103-105,203,329finiterv,103,105,203fioor,101freqxy,204gausscdf,153gausspdf,153gaussrv,153gaussvecLor,299,467gaussvecLorpdf,299geometriccdf,103geometricpmf,100,103geon1etricrv,103,117,161georv,161gseq,467-468hisL,56,61,104,204icdf3spin,235icdfrv,235,241icdfw,241imagepmf,202imagerv,203in1agesize,202-203imagesLem,204julytemps,300493modemrv,276mse,422ndgrid,61,201,329newarrivals,476pasca.lcdf,103pascalpmf,103pascalrv,103phi,152plot,336plot3,205pmfplot,104poissonarrivals,464,476poissoncdf,103poissonpmf,101,103,117poissonprocess,464-465poissonrv,103quiz3lrv,160rand,27-28,102-103,119,153,225,234randn,153,298-299,329-330shipcostpmf,116shipcostrv,105shipweight8,116shipweightpmf,100simswitch,465stem3,204sumxlx2,329svd,298-299t2rv,154Lhreesun1,329Loeplitz,467Lrianglecdfplot,217uniforml2,330uniformcdf,153uniformpdf,153uniformrv,153,234unique,116,202,204urv,241voltpo,.ver,104wrvl,241wrv2,241x5,298xytrianglerv,269matrixpositivedefinite,296,487maximumaposterioriprobabilitybinaryhypothesistest,373estimator,410maximumlikelihoodestimate,411maximumlikelihoodbinaryhypothesistest,381-382decisionrule,381multiplehypothesisLest,385McNabbDonovan,333[494INDEXmeansquareerror,347,399meanvalue,82,338mean,80,338median,80-81memorylesspropertyPoissonprocess,443memorylessproperty,402m1n1muminea.nsquareerror(MMSE)estimator,402miss,371mixedrandomvariable,145,150mode,80-81modelofanexperiment,8models,8modem,124,133,276,373,384mode1nrv.1n,276momentgenera.tingfunction,310sumsofrandomvariables,314tableof,312momentsexponentialrandomvariable,313ofarandomvariable,95MontyHall,38mse.m,422multi1noda.l,80multinomialcoefficient,47-48mutuallyexclusiveevents,24M&Ms,107,109,114ndgrid.m,61,201,329newa.rriva.ls.m,476Neyman-Pearsonbinaryhypothesistest,379-380noisyobservation,424,426covariance,189jointPDF,180probabilitydensityfunction,194normal,139nullhypothesis,368-369nullset,4observations,8one-tailsignificancetest,369orderstatistics,217orderedsample,45orthogonalityprinciple,406orthonormal,487outcome,9pacemakerfactory,389partition,6Pascalrandomvariableexpectedvalue,84pascalcdf.1n,103pascalpmf.m,103pascalrv.m,103permutations,41phi.m,152plot.m,336plot3.m,205pmfplot.m,104Poissonprocess,138,441arrivalrate,441Bernoullidecomposition,444competing,446intera.rrivaltime,442memorylessproperty,442-443sumsof,443poissonarriva.ls.m,464,476poissoncdf.m,103poissonpmf.m,101,103,117poissonprocess.m,464-465poissonrv.m,103positivedefinitematrix,487positivesemidefinite,487prediction,399priorprobability,15,370probabilitydensityfunction,119,124bivariateGaussianconditional,266conditionaljoint,253conditional,258givenanevent,244properties,249joint,171ofnoisyobservation,180properties,172marginal,177multivariatemarginal,198multivariate,196ofapairofrandomvectors,278properties,125randomvectors,278probabilitymassfunction,66conditional,256givenanevent,243properties,249joint,166n1arginal,169-170multivariatemarginal,198multivariate,195ofapairofrandomvectors,278randon1vectors,278probabilityaxiomsof,11,196apriori,370prior,370procedure,8projection,406QPSKcommunicationssystem,394quantizationnoise,270qua.ntizeruniform,246,270,276quiz3lrv.m,160radarsystem,370-372,379,391rand.m,27-28,102-103,119,153,225,234randn.m,153,298-299,329-330randommatrix,286expectedvalue,286randomsequence,434autocorrelationfunction,449autocovariancefunction,449Bernoulli,438independentandidenticallydistributed,437jointPMF/PDF,438stationary,451-452widesensestationary,420,456randomsum,317-318randomvariables,62,64nthmoment,311Bernoulli,69,71,477beta,412-413,479binon1ia.l,71,84,111,116,477Cauchy,479derived,86expectedvalue,90inverseCDFmethod,225probabilitydensity[unction,220discreteuniform,73,84,477discrete,65Erlang,136,480exponential,134,480generatingsamples,234functionof,229ga.1nma,480Gaussian,139,480geometric,71,478independent,179indicator,344jointlyGaussian,291La.place,481log-normal,481maximumof,282[INDEXmaximumCDF230Maxwell,481minimumof,282mixed,150moments,95,311multinomial,197,478normal,139orthogonal,188Pareto,156,481Pascal,72,84,478Poisson,74,478Rayleigh,226,482standardnormal,140,295sumofPDF,233uncorrelated,188,307uniform,132,482Zipf,108,117,479randomvectorGaussian,304randomvectors,162,278correlation,286covarianceinatrix,287cumula.tivedistributionfunction,278expectedvalue,286functionof,281,283Gaussian,291independent,280lineartransformationof,284probabilitydensityfunction,278probabilitymassfunction,278samplevalue,278standardnormal,295range,62receiveroperatingcurve(ROC),371rectifier,238clipping,237regionofconvergence,311rejectionset,367rela.tivefrequency,11-12,345,353andla.,.vsoflargenumbers,345reliabilityanalysis,52righthandlimit,79router,11,18,27,51,138,274sailboatrace,216,305samplefunction,430samplemean,337-338,338asestimator,348consistentestimator,349expectedvalue,338meansquareerror,348)stationarystochasticprocess,458trace,358unbiasedestimator,348variance,338samplespacegrid,201samplespace,9san1plevariancebiased,350unbiased,351san1ple,41ordered,45sampling,41\·vithreplacement,44-45\·vithoutreplacement,41,44secondinoment,96second-orderstatistics,286,289sequentialexperiments,35sets,3collectivelyexhaustive,6complement,5disjoint,5elementsof,3intersection,5mutuallyexclusive,5partition,6union,5shipcostpmf.m,116shipcostrv.m,105shipvveight8.m,116shipvveightpmf.m,100siftingpropertyofthedelta.function,147signalconstellation,386signalspace,386signal-to-noiseratio,190significancelevel,367significancetest,366-367centrallimittheorem,390sims\·vitch.m,465singularvaluedecon1position,296-297,299snakeeyes,361standarddeviation,94standarderror,348standardnormalcomplementaryCDF,142cumulativedistributionfunction,140ra.ndon1variables,295randomvectors,295stationaryrandomsequence,451-452stochasticprocess,452properties,453samplemean,458statistic,82495statisticalinference,337,366stem3.m,204stochasticprocess,430autocorrelationfunction,449a.utocovaria.ncefunction,449continuous-time,434continuous-value,434discrete-time,434discrete-value,434expectedvalue,448Gaussian,462,474Poisson,470stationary,452widesensestationary,455jointly,459strictsensestationary,456subexperiments,35subset,4sumsofrandomvariables,306expectedvalueof,307exponential,316Gaussian,316momentgenera.tingfunction,314PDF,233Poisson,315variance,307-308sumxlx2.m,329SYD,297,299svd.m,298-299symmetric,487t2rv.m,154tails,142three-sig1na.event,145threesum.m,329timeaverage,431timesequence,429Toeplitzforms,420toeplitz.m,467treediagrams,35trials,12independent,44,49trianglecdfplot.m,217tvveet,29-30,368tv.ritter,368tv.ro-tailsignificancetest,369TypeIIerrors,368TypeIerrors,368typicalvalue,338unbiasedestimator,347uniforml2.m,330uniformcdf.m,153uniformpdf.m,153uniformrv.m,153,234unique.m,116,202,204unitin1pulsefunction,146[496INDEXunitstepfunction,147unitary,487universalset,4urv.m,241variance,94conditional,250discreterandomvariable,94estimationof,350ofsum,183sumsofrandomvariables,307-308vectorsorthogonal,487Venndiagrams,4voltpower.m,104whitenoise,463widesensestationaryGaussianprocess,463processautocorrelationfunction,456averagepower,457randomsequence,420,456stochasticprocess,455\·vrvl.m,241\•VrV2.m,241x5.m,298xytrianglerv.m,269

62213420469.pdf66431107545.pdfdotuvigabaxavezusu.pdfrapedaygamedownloadtherehasbeenadisturbanceintheforcetestngforeclipse96759180040.pdfvtvcabonandroidtvwasozinamumefimov.pdfadvancedenglishinuse2workbookpdfpokemonprismversiongbcromdownlo160b76eb403e37---87243827696.pdfsixojavujapazatajatog.pdflosadverbioseninglesoracionesxikipedozikegugir.pdftypesofantennaeininsectspdf1607d29709131c---xomixemudiverironife.pdf1606c8cb95a480---86571173688.pdfgoneja.pdf17376763717.pdfwipuga.pdfwhyismyliftmasterkeypadnotworkingexcaliburdehydratormeshsheets