Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4....
Transcript of Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4....
![Page 1: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/1.jpg)
Wasserstein GAN
![Page 2: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/2.jpg)
Generativni modeli- Predviđamo distribuciju
verovatnoća nad uzoračkim prostorom.
- Centralno mesto u pristupu zauzima nekakva koncepcija distance.
![Page 3: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/3.jpg)
Generative Adversarial Networks
![Page 4: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/4.jpg)
Generative Adversarial Networks
![Page 5: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/5.jpg)
Ključni problemi- Update postaje gori što je diskriminator bolji.- Treniranje GAN modela je kranje nestabilno.
![Page 6: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/6.jpg)
Disjunktni nosači gustina
Šta se dešava ako su nosači disjunktni?
![Page 7: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/7.jpg)
Poklapanje mnogostrukosti
![Page 8: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/8.jpg)
Distribucije čiji se nosači ne poklapaju
![Page 9: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/9.jpg)
Cost funkcije
![Page 10: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/10.jpg)
logD trik
![Page 11: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/11.jpg)
Noisy generatorJedno od rešenja ovog problema je dodavanje normalno raspodeljenog šuma generatoru i podacima. U tom slučaju loss ima stabilne gradijente i konvergira simetričnom izrazu:
Sa druge strane, intenzitet šuma koji je potreban da bi se postiglo stabilno treniranje je krajnje veliki i drastično obara kvalitet generisanih vrednosti.
![Page 12: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/12.jpg)
Alternativna metrika/divergencija
![Page 13: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/13.jpg)
Wasserstein u odnosu na druge divergencije
![Page 14: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/14.jpg)
Wasserstein dual i treniranje
![Page 15: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/15.jpg)
Algoritam
![Page 16: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/16.jpg)
Praktični rezultati- Batchnorm nema nikakvog efekta na proces treniranja- Treniranje postaje praktično neosetljivo na arhitekturu generatora
![Page 17: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/17.jpg)
Praktični rezultati
Diskriminator(kritičar) ima skoro konstantan gradijent
![Page 18: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/18.jpg)
Praktični rezultatiLoss funkcija se može interpretirati na razuman način
![Page 19: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/19.jpg)
Problemi- “Weight clipping” kod diskriminatora.- Spora konvergencija na visokodimenzionim podacima.
![Page 20: Wasserstein GAN - Machine Learningmachinelearning.math.rs/Nesic-WassersteinGAN.pdf · 2018. 4. 4. · 10—7 10—8 Gradient of the generator with the original cost After I epoch](https://reader033.fdokumen.com/reader033/viewer/2022052616/609828838f8df820fa140b98/html5/thumbnails/20.jpg)
Teorija i praksa