Deep Learning Demystified

Demystified: Kullback–Leibler Divergence

A quick primer on Kullback-Leibler Divergence, an important concept to understand in machine learning and information theory

Shannon Entropy

Discrete (countable) random variables (DRVs)

Entropy

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Seth, in the highly unlikely causal chain of events that would end up with you reading this article, I just wanna say, I dig your nerdy giggle.

Kullback–Leibler (KL) Divergence

Image for post
Image for post
for discrete probability distributions
Image for post
Image for post
for continous probability distributions
Image for post
Image for post
BTW — I love cake
Image for post
Image for post

Data Scientist

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store