L7-Mang Noron Nhan Tao

8/13/2019 L7-Mang Noron Nhan Tao

1/68

Hc My(IT4866)

Nguy n Nh t Quang

[email protected]

Trng i hc Bch Khoa H Ni

Vin Cng nghthng tin v truyn thng

Nm hc 2013-2014


2/68

ung m n c:

Gii thiu chung

nh gi hiu nng h thng hc my

Cc phng php hc c gim st

Cc phng php hc khng gim st

2Hc My IT4866


3/68

Mng n-ron nhn to Gii thiu (1)

Mng n-ron nhn to (Artificial neural network ANN)

ANN l mt cu trc (structure/network) c to nn bi mt slng cc n-ron (artificial neurons) lin kt vi nhau

Mi n-ron

C mt c tnh vo/ra c n m n o n cc m m cc

Gi tru ra ca mt n-ron c xc nh bi

Cc lin kt ca n vi cc n-ron khc

(C th) cc u vo bsung

3Hc My IT4866


4/68

Mng n-ron nhn to Gii thiu (2)

ANN c thc xem nhmt cu trc xl thng tin mtcch hn tn v son son mc cao

ANN c khnng hc (learn), nhli (recall), v khi qut ha(generalize) tcc dliu hc bng cch gn v iu chnh(thch nghi) cc gi trtrng s(mc quan trng) ca cclin kt gia cc n-ron

c n ng m mc u c a m c x c n

Kin trc (topology) ca mng n-ron

- Chin lc hc (hun luyn)

Dli u h c

4Hc My IT4866


5/68

ANN Cc ng dng in hnh (1)

Xl nh v Computer vision , , , ,

nn nh, xl v hiu cc nh thay i theo thi gian Xl tn hiu

: n t c t n u v n t a c n, ng t

Nhn dng mu V d : Trch ch n thu c tnh hn lo i v hn tch tn hi u ra-a nh n

dng v hiu ging ni, nhn dng du vn tay, nhn dng k t(chhoc s), nhn dng mt ngi, v phn tch chvit tay

V d: Phn tch v hiu tn hiu in tim, chn on cc loi bnh, v xl cc nh trong lnh vc y t

5Hc My IT4866


6/68

ANN Cc ng dng in hnh (2)

Cc h thng qun s- ,

Cc h thng ti chnh V d: Phn tch th trng chng khon, nh gi gi tr btng sn,

,

Lp k hoch, iu khin, v tm kim V d: Cit song song cc bi ton tha mn rng buc, tm li gii

c o o n ng a ng, u n v oa c ng n c u vngi my (robotics)

Cc h thng nng lng V d: nh gi trng thi h thng, pht hin v khc phc s c, d

on ti (khi lng) cng vic, vnh gi mc an ton

... v nhiu lnh v c bi ton khc!

6Hc My IT4866


7/68

Cu trc v ho

t

ng c

a m

t n

-ron Cc tn hiu u vo (input

signals) ca n-ron (xi, x0=1..

Mi tn hiu u voxignvi mt trng swi Trng siu chnh (bias)

x1

x2w0

w1w

Gi tr

w0(vix0=1)

u vo tng th(Net

input) l mt hm tch hpx

m

wm

u ra

ca

n-ronc a c c n u u v o Net(w,x)

Hm tc ng/truynCc tn u vo Hm tc

function) tnh gi tru raca n-ron f(Net(w,x))

Gi tru ra (Output) ca

hiu uvo can-ron

tng th(Net)

ng(truyn)

fn-ron: Out=f(Net(w,x)) (x)

7Hc My IT4866


8/68

u vo tn thv dch chu nmm

u vo tng th(net input) thng c tnh ton bi mthm tuyn tnh

==

=+=++++=i

ii

i

iimm xwxwwxwxwxwwNet01

022110 1....

ngha ca tn hiu dch chuyn (bias) w0 c c m p n c e =w1x1 ng p n c cc c

v dthnh 2 lp (two classes)

Nhng: hcc hm Net=w1x1+w0 c th!

Net = w1x

1

Net Net

Net = w1x

1+ w0

x1 x1

8Hc My IT4866


9/68

Hm tc ng: Gii hn cng (Hard-limiter)

Cn c gi l hm ngng(threshold function)

==

nu,1),(1)(

NetNethlNetOut

Gi tr u ra l y mt trong 2 gi tr

l gi trngng

Nh c im: khn lin t c, o

,

),(),(2)( NetsignNethlNetOut ==

Out

hm khng lin tc

hard-limiter

11

Bipolar

hard-limiter

0 Net Net0

-1

9Hc My IT4866


10/68

Hm tc ng: Logic ngng (Threshold logic)

< 1

if,0 Net

>

1if,1

,,,

Net

>0u)))(,1min(,0max( += Net

Cn c gi l hm tuyn tnh

0 Net- (1/)-

o a sa ura ng near unc on

Kt hp ca 2 hm tc ng:tuyn tnh v gii hn cht

1/ xc nh d c ca khong

tuyn tnh

Nhc im: Lin tc nhngo m ng n c

10Hc My IT4866


11/68

Hm tc

n : Xch-ma Si moidal1

),,()(

+== NetNetsfNetOut

Out

e

c dng phbin nht

1Tham s xc nh dc

Gi tru ra trong khong (0,1)

0- Net

0.5u im

Lin tc, v o hm lin tc

o m c a m m x c -mac biu din bng mt hmca chnh n

11Hc My IT4866


12/68

Hm tc ng: Hyperbolic tangent

121

),,tanh()()(

=

==+

Nete

NetNetOut++ ee

Out Cng hay c s dng

1

Tham s xcnh dc

Gi tr u ra trong khong (-1,1)

0- Net

uim

Lin tc, vo hm lin tc

-1c biu din bng mt hm cachnh n

1212Hc My IT4866


13/68

ANN Kin trc mng (1)

in ut

biasKin trc ca mt ANN c x/bi:

Slng cc tn hiu u vo v u ra

hidden

layer

Slng cc tng

Slng cc n-ron trong mi tng

Slng cc trng s(cc lin kt) i

output

layer

out ut

vi m i n-ron

Cch thc cc n-ron (trong mt tng,

hoc gia cc tng) lin kt vi nhau ng n-ron n o n n c c n u

iu chnh li

Mt ANN phi cV d: Mt ANN vi mt tng n

u vo: 3 tn hiu

Mt t ng u vo (input layer) Mt tng u ra (output layer)

Khng, mt, hoc nhiu tng n (hidden

u ra: 2 gi tr Tng cng, c 6 neurons

- 4 tng nayer s

- 2 tng u ra

13Hc My IT4866


14/68

ANN Kin trc mn 2 Mt tng (layer) cha mt nhm cc n-ron

Tng n (hidden layer) l mt tng nmgia tnguvo (input layer) v tngu ra (output layer)

c n ng n en no es ng ng c r ctip vi mi tr ng bn ngoi (ca mng n-ron)

nu miu r a t mt tng lin kt vi mi n-ron catng k tip

14Hc My IT4866


15/68

ANN Kin trc mn 3 Mt ANN c gi lmng lan truyn tin (feed-

nt lu v o ca mt nt khc thuc cng tng (hocthuc mt tng pha trc)

Khi ccu r a ca mt nt lin kt ngc li lm ccuvo ca mt nt thuc cng tng (hoc thuc mt tng

,network) Nu phn hi l lin ktu voi vi cc nt thuc cng tng,

Cc mng phn hi c cc vng lp kn (closed loops) c i l cc m n hi u recurrent networks

15Hc My IT4866


16/68

Kin trc m

ng V d

Mng lantruyn tin

Mt n-ron vihn hi n

mt t ng chnh n

quy mttng

Mng lantruyn tin

Mng hiquy nhiutng

16Hc My IT4866


17/68

ANN Cc quy tc hc

2 kiu hc trong cc mng n-ron nhn to Hc tham s(Parameter learning)

Mc tiu l thay i thch nghi cc trng s(weights) ca cclin kt trong mng n-ron

Mc tiu l thay i thch nghi cu trc mng, bao gm slng cc n-ron v cc kiu lin kt gia chng

2 kiu hc ny c thc thc hin ng thi hocring r

Ph n ln cc quy t c hc trong ANN thuc ki u hc thams

Trong bi hc ny, chng ta schxt vic hc tham s

17Hc My IT4866


18/68

u tc h

c tr

n s

tn ut

x = 1

Ti bc hc (t), mc iuchnh vec-t tr n swt l

wa neuron

x ...

w0

wj

w1

x

x1 Out

thun vi tch ca tn hiu hc r(t)

vu vox(t)

w(t) ~ r(t).x(t)

xwmxm

...Learning

signal

generator

d

w(t) = .r(t).x(t)

trong (>0) l tc hcTn hiu hc rl mt hm caw,x, v gi tr u ra mong mun d Lu : xjc thl:

m t tn hi uu vo ho c

r = g(w,x,d)Quy tc hc trng s tng qut

w(t) = . w(t) x(t) d(t) .x(t)

mt gi tru ra ca mt n-ron khc

18Hc My IT4866


19/68

Perceptron

=

Mt perceptron l mt kiun in nht ca ANNs

x1

x

w0w1

(chgm duy nht mt n-ron)

xm

w2

wm

Sdng hm tc nggii hn cht

( )

== =j

jjxwsignxwNetsignOut0

),(

i vi m t v d x i tr

u ra ca perceptron l1, nu Net(w,x)>0

- , n u ngc

19Hc My IT4866


20/68

Perce tron Minh ha

w0+w1x1+w2x2=0x1

u ra = 1

= -

x2

20Hc My IT4866


21/68

Perce tron Gii thu

t h

c

Vi mt tp cc v dhc D= {(x,d)}xl vectu vo

dl gi tru ra mong mun (-1 hoc 1)

Qu trnh hc ca perceptron nhm xc nh mt vecttrng-

hoc 1) cho mi v dhc

Vi mt v dhcxc erce tron hn l chnh xc, thvecttrng swkhng thay i

Nu d=1 nhng perceptron li sinh ra -1 (Out=-1), th wcn

, Nu d=-1 nhng perceptron li sinh ra 1 (Out=1), th wcn

c thay i sao cho gi trNet(w,x) gim i

21Hc My IT4866


22/68

Perce tron incrementalD _Initialize w (wian initial (small) random value)

do

for each training instance(x,d)

D

Compute the real output value Out

if(Outd)

w w+ (d-Out)x

en or

until all the training instances in D are correctly classified

2222Hc My IT4866


23/68

Perce tron batchD _Initialize w (wian initial (small) random value)

do

w

0for each training instance (x,d)D

Compute the real output value Out

if(Outd)

-

end for

w w + w

until all the training instances in D are correctly classified

return w

2323Hc My IT4866


24/68

Perce tron Gi

i hn

Gii thut hc cho perceptron c chngminh l hi t(converge) nu: M t erce tron khn

Cc v dhc l c thphn tch tuyn

tnh (linearly separable) Sdng mt tc hc nh

thphn lp chnh xc

i vi tp hc ny!

Gii thut hc perceptron c thkhng hitnu nhcc v dhc khng thphn

tch tuyn tnh (not linearly separable) Khi , p dng quy tc delta (delta rule)

m bo hi tvmt xp xph hpnh t ca hm m c tiu

Quy tc delta sdng chin lcgradient descenttm trong khng giangithit (cc vecttrng s) mt vecttrng s ph hp nh t vi cc v dhc

24Hc My IT4866


25/68

Hm

nh i li Error function

Xt mt ANN c n n-ron u ra

v m v c x, , g r c ra n ng error

gy ra bi vecttrng s(hin ti) w:

( )12

1)(

=

=n

i

ii OutdE wx

c g y ra vec rng s n w vton btp hc D:

= DD

D xx ww )()(

25Hc My IT4866


26/68

Gradient descent

Gradient ca E(k hiu lE) l mt vect C hng ch i ln (dc)

C di t l thun vi dc

Gradient Excnh hng gy ra vic tng nhanh nht (steepest

=Nwww

E ,...,,)(21

w

V vy, hng gy ra vic gim nhanh nht (steepest decrease) lgi tr ph nh ca gradient ca E

Niww ii ..1, == w= -.E(w);

Yu cu: Cc hm tcngc s dng trong mng phi l cc ,

26Hc My IT4866


27/68

Gradient descent Minh h

aKhn ian 2 chiu

E(w) E(w1,w2)

27Hc My IT4866


28/68

Gradient_descent_incremental (D, )

Initialize w (wian initial (small) random value)

do

for each training instance (x,d)D

Compute the network output

i

wi wi (Ex/wi)

end for

end for

until (stopping criterion satisfied)

return w

28

, , ...

28Hc My IT4866


29/68

ANN nhiu tng v gii thut lan truyn ngc

Mt perceptron chc thbiu din mt hm phn tch tuyn tnh(linear separation function)

Mt mng n-ron nhiu tng (multi-layer NN) c hc bi gii thutlan truyn ngc (back-propagation -BP- algorithm) c thbiu dinmt hm phn tch phi tuyn phc tp (highly non-linear separationfunction)

Gii thut hc BP c sdng hc cc trng sca mt mngn-ron nhiu tn Cu trc mng cnh (cc n-ron v cc lin kt gia chng l cnh)

i vi mi n-ron, hm tc ng phi c o hm lin tc

t u t p ng c n c gra en escen trong quy t c c pnht cc trng s cc tiu ha li (khc bit) gia cc gi tru ra thc tv cc gi tr

,

29Hc My IT4866


30/68

Gii thu

t h

c lan tru

n n

c 1

Gii thut hc lan truyn ngc tm kim mt vect cc trngs (weights vector) gip cc tiu ha li tng th ca h

ng v p c

Gii thut BP bao gm 2 giaion (bc) Giaio n lan tru n tin tn hi u Si nal forward . Cc tn hi u

u vo (vect cc gi tr u vo) c lan truyn tin t tngu von tngu r a (i qua cc tngn)

Giaio n lan tru n n c li Error backward

Cn c vo gi tr u ra mong mun ca vect u vo, h thng tnhton gi tr li

Btu t tn u ra, i tr li c lan tru n n c ua m n , t

tng ny qua tng khc (pha trc), chon tngu vo Vic lan truyn ngc li (error back-propagation) c thc hin

thng qua vic tnh ton (mt cch truy hi) gi tr gradient cc b ca-

30Hc My IT4866


31/68

Gii thut hc lan truyn ngc (2)

tn hiu: Kch hot (truyn tn hiuqua mng

Giaion lan truyn

Tnh ton liu ra Lan truyn (ngc) li

31Hc My IT4866


32/68

Gii thut BP Cu trc mng

x1 x xmIn utx

Xt mng n-ron 3 tng (tronghnh v) minh ha gii thut

wqj

... ...

(j=1..m)

m tn hiu u voxj(j=1..m) l n-ron tng n zq (q=1..l)

neuron zq(q=1..l)

wiq

Outq... ...

- i ..

wqj l trng sca lin kt ttn hiu u voxj ti n-ron

tn n z

... ...Outputneuron yi

=

wiq l trng sca lin kt tn-ron tng n zq ti n-ronu ra yi

i..

Outq l gi tru ra (cc b)ca n-ron tng n zq Outi l gi tru ra ca mng

- i

32Hc My IT4866


33/68

Gii thut BP Lan tru n tin 1i vi mi v dhcx

Vectu vox c lan tru n ttn u vo n tn u ra

Mng ssinh ra mt gi tru ra thc t(actual output) Out(lmt vectca cc gi trOuti, i=1..n)

m

i vi mt vect u vox, mt n-ron zqt ng n snhn c gi tru vo tng th(net input) bng:

=

=j

jqjq xwNet1

v sinh ra mt gi tru ra (cc b) bng:

==

=

m

j

jqjqq xwfNetfOut1

)(

. - q

33Hc My IT4866


34/68

Gii thut BP Lan tru n tin 2

Gi tru vo tng th(net input) ca n-ron yitng

= ==

==

l

q

m

j

jqjiq

l

q

qiqi xwfwOutwNet1 11

N-ron yi sinh ra gi tru ra (l mt gi tru ra ca

mng)

=

== = == q

m

j

jqjiq

q

qiqii xwfwfOutwfNetfOut1 11

)(

Vectcc gi tru ra Outi(i=1..n) chnh l gi tru rathc tca mng, i vi vectu vox

34Hc My IT4866


35/68

Gii thut BP Lan truyn ngc (1)

i vi mi v dhcx Cc tn hi u li error si nals do s khc bit ia i tr u ra

mong mun dv gi tru ra thc tOutc tnh ton

Cc tn hiu li ny c lan truyn ngc (back-propagated) ttn u r a ti cc tn ha tr c c nh t cc tr n s(weights)

xt cc tn hiu li v vic lan truyn ngc ca

( ) ( ) ( )[ ] ==n

ii

n

ii NetfdOutdwE22 11

chng, cn nh ngha mt hm nh gi li

== ii 11

=n l

qiqi Outwfd

2

1

= =q

35Hc My IT4866


36/68


Theo phng php gradient-descent, cc trng s ca cc linkt t tn n ti tn u ra c c nh t bi

iq

iqw

Ew

=

[ ] ( )[ ][ ] qiqiiiii

iq OutOutNetfOutdw

Net

Net

Out

Out

E

w ==

= '

ng quy c c u o m v wiq, a c

(Lu : du c kt hp vi gi tr E/Outi)

i l tn hiu li (error signal) ca n-ron yi tngu ra[ ] ( )[ ]iii

i

i

ii

i NetfOutdNetOut

OutE

NetE '=

=

=

trong Net lu vo tng th (net input) ca n-ron ytngu ra, v f'(Neti)=f(Neti)/Neti

36Hc My IT4866


37/68


cp nht cc tr ng s ca cc lin kt t tngu,

gradient-descent v quy tc chuio hm

=

=qj

q

q

q

qqj

qjw

e

Net

u

Outww

T cng thc tnh hm l i E(w), ta th y r ng m i thnhphn li (di-yi) (i=1..n) l mt hm ca Outq

= =

= n

i

l

q

qiqi OutwfdE1 12

1)(w

37Hc My IT4866


38/68


p dng quy tc chui o hm, ta c

( ) ( )[ ] ( ) jqi

iqiiiqj xNetfwNetfOutdw ''1

==

n

' jqjqi

iqi ===1

q

l tn hiu li (error signal) ca n-ron zq

tng n

( ) iqn

i

iq

q

q

qq

q wNetfNet

Out

Out

E

Net

E

=

=

=

=

1

'

trong Netq l u vo tng th(net input) ca n-ron zqtngn, v f'(Netq)=f(Netq)/Netq

38Hc My IT4866


39/68


Theo cc cng thc tnh cc tn hiu li i v q nu, th tnhiu li ca mt n-ron tng n khc vi tn hiu li ca mtn-ron tng u ra

Do skhc bit ny, thtc cp nht trng strong gii thutBP c i l u tc h c delta t n ut

Tn hiu li q ca n-ron zqtng n c xc nh bi

Cc tn hiu li

ica cc n-ron yitng u ra (m n-ron zq ,

Cc hschnh l cc trng swiq

c im quan trng ca gii thut BP: Quy tc cp

nht trng sc tnh cc b tnh ton thay i (cp nht) trng sca mt lin kt, h

thn chcn sd n cc i tr 2 u ca lin kt !

39Hc My IT4866


40/68


Qu trnh tnh ton tn hiu li (error signals) nhtrn cthc mrn khi ut ddn i vi mn n-ron c nhiu hn 1 tng n

D n tn ut ca u tc c nh t tr n stron iithut BP l:

wab = axb b v a l 2 chstng ng vi 2 u ca lin kt (ba) (tmt

n-ron (hoc tn hiu u vo) bn n-ron a)

- ,

a l tn hiu li ca n-ron a

40Hc My IT4866


41/68

Back_propagation_incremental(D, )Mng n-ron gmQ tng, q = 1,2,...,Q

qNetiv qOuti lu vo tng th (net input) v gi tr u r a ca n-ron itng q

Mng c m tn hiuu vo v n n-ronu raqwij l trng s ca lin kt t n-ronjtng (q-1)n n-ron itng q

Bc 0 (Khi to)

Chn ngng li Ethreshold(gi tr li c th chp nhnc)

Gn E=0

Bc 1 (Btu mt chu k hc)

p ng vec u v o c a v c v ng u v o q=qOuti = 1Outi = xi(k), i

Bc 2 (Lan truyn tin)

== qqqq OutwNetOut 1

Lan truyn tin cc tn hiuu vo qua mng, chon khi nhnc ccgi tr u r a ca mng (tngu ra) QOuti

j

41H

c My IT4866


42/68

Bc 3 (Tnh ton liu ra)

Tnh ton liu r a ca mng v tn hiu li Q ca mi n-rontngu ra

=

+=n

i

i

Qk

i OutdEE1

2)( )(2

1

' QQ(k)Q = iiiiBc 4 (Lan truyn ngc li)

Lan truyn ngc li cp nht cc tr ng s v tnh ton cc tn hiu li q-1icho cc tng pha trc

2,...,1,allfor;

11

==

QQqw)Net'(f jq

ji

q

i

q

i

q

qwij = .(qi).(q-1Outj); qwij = qwij + qwij

jBc 5 (Kim t rakt thc mt chuk hc epoch)

Kim tra xem ton b tp hc c s dng ( xong mt chu k hc epoch)

Nu ton b tp hc c dng, chuynn Bc 6; ngc li, chuynn Bc 1

Bc 6 (Kim t ra li tng th)Nu li tng th E nh hn ngng li chp nhnc (


43/68

Gii thut BP Lan truyn tin (1)

f(Net1)

1x f(Net4)

f(Net2)

Out6f(Net6)

x f(Net5)

f(Net3)

43Hc My IT4866


44/68


f(Net1)

1x

1x

21 2xw x f(Net4)

Out6

f(Net2)

f(Net6)

x f(Net5)

)( 21111 xwxwfOut xx +=f(Net3)

44Hc My IT4866


45/68


f(Net1)

1x f(Net4)12 1

xw xOut6

f(Net2)

f(Net6)

x22 2

xw x f(Net5)

)( xwxwfOut +=f(Net3)

45Hc My IT4866


46/68


f(Net1)

1x f(Net4)

Out6f(Net

2)

f(Net6)

2x 13 1xw xf(Net5)

23 2xw x )( 23133 xwxwfOut xx +=

f(Net3)

46Hc My IT4866

G h )


47/68


f(Net1)

1x 242Outw

141

f(Net4)

343OutwOut6

f(Net2)

f(Net6)

2xf(Net5)

)( OutwOutwOutwOut ++=f(Net3)

47Hc My IT4866

Gii h BP L i (6)


48/68


f(Net1)

1x 151Outw f(Net4)

Out6f(Net

2)

f(Net6)

2x252 uw

f(Net5)

353Outw

OutwOutwOutwOut ++=f(Net3)

48Hc My IT4866

Gii h BP L i (7)


49/68


f(Net1)

1x f(Net4)

Out6

f(Net2)

f(Net6)

x565Outwf(Net5)

OutwOutwOut +=f(Net3)

49Hc My IT4866

Gii h BP T h li


50/68

Gii thut BP Tnh ton li

f(Net1)

1x f(Net4) Out6

f(Net2)

f(Net6)

2xf(Net5)

[ ] ( )[ ]666

6 ' NetfOutdOutEE

=

=

=f(Net3)

output value

666 etutet

50Hc My IT4866

Gii h BP L (1)


51/68


f(Net1)

1x4

f(Net4)

6Out6

f(Net2)

f(Net6)

x f(Net5)

'=f(Net3)

66444

51Hc My IT4866

Gii th t BP L t (2)


52/68


f(Net1)

1x f(Net4)

Out6

f(Net2)

f(Net6)

x65w

5

f(Net5)

))(w'(Netf 66555=f(Net3)

52Hc My IT4866

Gii th t BP L tr (3)


53/68


1

x41w

w

4

1

Out6f(Net )5

2

f Net

2

'=f Net 55144111

53Hc My IT4866



54/68


f(Net1)

1x 42w f(Net4)

w

2 Out6

f(Net2)

f(Net6)

x

5

f(Net5)

)w)(w'(Netf 55244222 +=f(Net3)

54Hc My IT4866



55/68


f(Net1)

1x f(Net4)

43w

Out6

f(Net2)

f(Net6)

x 53w5

f(Net5)

)w)(w'(Netf 55344333 +=3

f(Net3)

55Hc My IT4866

Gii thut BP Cp nht trng s (1)


56/68


1f(Net1)

1x

1x

w f(Net4)2

Out6

f(Net )f(Net6)

x f(Net5)

1111 11xww xx +=

f(Net3)

2111 22 xx

56Hc My IT4866



57/68


f(Net1)

1x f(Net4)

12xw 2 Out6

f(Net2)f(Net6)

2x22x

wf(Net5)

1222 11 xww xx

=+=f(Net3)

2222 22 x

57Hc My IT4866



58/68


f(Net1)

1x f(Net4)Out6

f(Net2)f(Net6)

2x 13xw f(Net5)

23xw 1333 11

xww

xww xx

+=+=f(Net3)

22

58Hc My IT4866



59/68


f(Net1)

1x 42w41

f(Net4)

43wOut6

f(Net2)f(Net6)

2xf(Net5)

244242

144141

Outww +=

=f(Net3)

344343

59Hc My IT4866


60/68



61/68


f(Net1)

1x f(Net4)64

Out6f(Net2)

f(Net6)

2x65w

f(Net5)

466464 Outww +=f(Net3)566565

61Hc My IT4866

BP: Khi to gi tr ca cc trng s


62/68

BP: Khi to gi tr ca cc trng s

Thng thng, cc trng s c khi to vi cc gi tr nhngu nhin

Nu cc tr ng s c cc gi tr ban u ln Cc hm xch-ma (sigmoid functions) s t trng thi bo ha sm

H thng s tcmtim cc tiu cc b (local minimum) hocmt trng thi khngi (very flat plateau) gnim btu

0 - -a K hiu na l s lng cc n-roncng tng vi n-ron a

w0ab [1/na, 1/na]

0

K hiu ka l s lng cc n-ron c lin kt (tin) n n-ron a(=s lng cc lin ktu vo ca n-ron a)

, aa

62H

c My IT4866

BP: Tc hc Learnin rate


63/68

BP: Tc hc Learnin rate nh hng quan trng n hiu qu v kh nng hi t ca gii thut hc

BP

g r n c y n an s c a qu r n c, n ng cth lm cho h thng b qua im tiu ton cc hoc ri voim ti

u cc b

Thngc chn theo thc nghim (experimentally) i vi mi bi ton

Cc gi tr tt ca tc hclc btu (qu trnh hc) c th khng tt

m t thiim sau Nn s dng mt tc hc thch nghi (ng)

Sau khi cp nht cc tr ng s, kim tra xem vic cp nht cc tr ng s c

=a , if E < 0 consistently-b , if E > 0 (a, b > 0)

0 , otherwise.

63H

c My IT4866

BP: Momentum


64/68

BP: Momentum

- E t+1 +w tw(t)

-E(t+1)

w t

Phng php Gradient descentcth rt chm nunh, v c th

AA

w(t)

Bao ng mn n u qu n

gim mc dao ng, cna vo mt thnh phn

B

w(t)

-E(t+1)

momentum

w(t) = -E(t) + w(t-1)

tron 0 1 l m t tham s - w

Gradient descent i vi mt hmli bc 2 n gin.

momentum (thng ly =0.9)

Mt quy tc, da trn cc th

Qu o bn tri khng s dngmomentum.

Qu o bn phi c s dng

,

cho tc hc v momentum l:(+) > 1; trong> trnh

momentum.

64Hc My IT4866

BP: S lng cc n-ron tng n


65/68

g g

Kch thc (s n-ron) ca tngn l mt cu hi quan trngi vi vic p dng cc mng n-ron lan truyn tin nhiu tng gii quyt cc bi ton thc t

Trong thc t, rt kh xcnh chnh xc s lng cc n-ron cn thit t c m t chnh xc mon mun ca hthng

Kch thc ca tngn thngc xcnh qua th nghim

Gi Btu vi s l n nh cc n-rontn n =t l nh so vi s

lng cc tn hiuu vo) Nu mng khng th hi t, b sung thm cc n-ron vo tngn

, -

65Hc My IT4866

ANNs Gii hn hc


66/68

ANNs Gii hn hc

Cc hm nhphn (Boolean functions) Bt khm nhphn no cng c th hc c bi mt ANN s

dng 1 tng n

c m n c on nuous unc ons

Bt kmt hm lin tc bgii hn (bounded continuous function)

no cn c thh c c x x bi m t ANN sd n 1 tnn [Cybenko, 1989; Hornik et al., 1989]

Bt kmt hm mc tiu no cng c thhc c (xp x) bi,

66Hc My IT4866

ANNs u im, Nhc im


67/68

ANNs u im, Nhc im

Cc u im

Khnng chu nhiu/li, nhkin trc tnh ton song song

C thc thit ktthch nghi (cc trng s, cu trc mng)

Cc nhc im Khng c quy tc tng qut xc nh cu trc mng v cc

am s c u c o m p o n n n

Khng c phng php tng qut nh gi hot ng bntrong ca ANN (v vy, hthng ANN bxem nhmt hp en)

Rt kh (khng th) a ra gii thch cho ngi dng

Rt kh don hiu nng ca hthng trong tng lai (khnng khi qut ha ca hthng hc)

67Hc My IT4866

ANNs Khi no?


68/68

NNs Kh o?

Mi v dc biu din bi mt tp gm (rt) nhiu

Min gi tru ra ca hm mc tiu c kiu sthc, hoc,

Tp dliu c thcha nhiu/li

Khng cn thit (hoc khng quan trng) phi a ra gii

thch cho n i dn i vi cc kt u

Chp nhn thi gian (kh) lu cho qu trnh hun luyn

68Hc My IT4866

L7-Mang Noron Nhan Tao

Documents

Transcript of L7-Mang Noron Nhan Tao