Dirichlet Distribution | Intuition & Intro | w\ example in TensorFlow Probability
The parameter to the Categorical is a vector of parameters. Can we put a distribution on it? Yes, we can. That's the Dirichlet. Here are the notes: https://raw.githubusercontent.com/Ceyron/machine-learning-and-simulation/main/english/essential_pmf_pdf/dirichlet_intro.pdf
The Parameter vector to the Categorical is of the dimension equal to the number of states of the Categorical. For example, we model the weather as the three states: Cloudy, Rainy or Sunny then we need a parameter (a probability) for each of the states.
There are two requirements on this probability vector: (1) all entries must be chosen from the interval [0, 1] (since they are probabilities), (2) the vector's components have to sum up to one. In this video, we will see that this implies the that the D-dimensional parameter vector is distributed over a (D-1)-dimensional simplex in D dimensions.
The Dirichlet describes a probability density distribution over this simplex. It is parameterized by an alpha-vector with also D-components which we can use to move the probability mass around over the simplex
Here is the website I showed in the video:
https://chart-studio.plotly.com/~david_avakian/14.embed
-------
📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation
📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler
💸 : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim
-------
⚙️ My Gear:
(Below are affiliate links to Amazon. If you decide to purchase the product or something else on Amazon through this link, I earn a small commission.)
- 🎙️ Microphone: Blue Yeti: https://amzn.to/3NU7OAs
- ⌨️ Logitech TKL Mechanical Keyboard: https://amzn.to/3JhEtwp
- 🎨 Gaomon Drawing Tablet (similar to a WACOM Tablet, but cheaper, works flawlessly under Linux): https://amzn.to/37katmf
- 🔌 Laptop Charger: https://amzn.to/3ja0imP
- 💻 My Laptop (generally I like the Dell XPS series): https://amzn.to/38xrABL
- 📱 My Phone: Fairphone 4 (I love the sustainability and repairability aspect of it): https://amzn.to/3Jr4ZmV
If I had to purchase these items again, I would probably change the following:
- 🎙️ Rode NT: https://amzn.to/3NUIGtw
- 💻 Framework Laptop (I do not get a commission here, but I love the vision of Framework. It will definitely be my next Ultrabook): https://frame.work
As an Amazon Associate I earn from qualifying purchases.
-------
Timestamps:
00:00 Introduction
00:33 Restrictions on the Parameter Vector
02:00 Visualizing 2-State Parameter Vector
05:56 Connection to the Beta Distribution
07:03 Visualizing 3-State Parameter Vector
09:25 General D-State Parameter Vector
10:37 Probability Density Function
11:56 Parameters of the Dirichlet
12:24 Plot: Exploring alpha values
15:58 TFP: Creating the Dirichlet Distribution
16:49 TFP: Sampling the Dirichlet
17:16 TFP: Querying the pdf
17:55 TF: Calculating Multivariate Beta Function
18:47 Outro
Видео Dirichlet Distribution | Intuition & Intro | w\ example in TensorFlow Probability канала Machine Learning & Simulation
The Parameter vector to the Categorical is of the dimension equal to the number of states of the Categorical. For example, we model the weather as the three states: Cloudy, Rainy or Sunny then we need a parameter (a probability) for each of the states.
There are two requirements on this probability vector: (1) all entries must be chosen from the interval [0, 1] (since they are probabilities), (2) the vector's components have to sum up to one. In this video, we will see that this implies the that the D-dimensional parameter vector is distributed over a (D-1)-dimensional simplex in D dimensions.
The Dirichlet describes a probability density distribution over this simplex. It is parameterized by an alpha-vector with also D-components which we can use to move the probability mass around over the simplex
Here is the website I showed in the video:
https://chart-studio.plotly.com/~david_avakian/14.embed
-------
📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation
📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler
💸 : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim
-------
⚙️ My Gear:
(Below are affiliate links to Amazon. If you decide to purchase the product or something else on Amazon through this link, I earn a small commission.)
- 🎙️ Microphone: Blue Yeti: https://amzn.to/3NU7OAs
- ⌨️ Logitech TKL Mechanical Keyboard: https://amzn.to/3JhEtwp
- 🎨 Gaomon Drawing Tablet (similar to a WACOM Tablet, but cheaper, works flawlessly under Linux): https://amzn.to/37katmf
- 🔌 Laptop Charger: https://amzn.to/3ja0imP
- 💻 My Laptop (generally I like the Dell XPS series): https://amzn.to/38xrABL
- 📱 My Phone: Fairphone 4 (I love the sustainability and repairability aspect of it): https://amzn.to/3Jr4ZmV
If I had to purchase these items again, I would probably change the following:
- 🎙️ Rode NT: https://amzn.to/3NUIGtw
- 💻 Framework Laptop (I do not get a commission here, but I love the vision of Framework. It will definitely be my next Ultrabook): https://frame.work
As an Amazon Associate I earn from qualifying purchases.
-------
Timestamps:
00:00 Introduction
00:33 Restrictions on the Parameter Vector
02:00 Visualizing 2-State Parameter Vector
05:56 Connection to the Beta Distribution
07:03 Visualizing 3-State Parameter Vector
09:25 General D-State Parameter Vector
10:37 Probability Density Function
11:56 Parameters of the Dirichlet
12:24 Plot: Exploring alpha values
15:58 TFP: Creating the Dirichlet Distribution
16:49 TFP: Sampling the Dirichlet
17:16 TFP: Querying the pdf
17:55 TF: Calculating Multivariate Beta Function
18:47 Outro
Видео Dirichlet Distribution | Intuition & Intro | w\ example in TensorFlow Probability канала Machine Learning & Simulation
Показать
Комментарии отсутствуют
Информация о видео
20 апреля 2021 г. 22:05:07
00:19:14
Другие видео канала
Adjoint Sensitivities over nonlinear equation with JAX Automatic DifferentiationDeriving the Weak Form for Linear Elasticity in Structural MechanicsScientific Python Tutorial Workshop | Part 3 | Scikit-Learn & a bit of TensorFlowWavenumber 128 Kolmogorov Flow | Turbulence SimulationUpdate: Machine Learning & Simulation now has a Patreon PageWavenumber 64 Kolmogorov Flow | Turbulence SimulationBinomial Distribution | Intuition & Introduction | w\ example in TensorFlow ProbabilityPosterior & MAP for the Categorical | Full Derivation | example in TensorFlow ProbabilityFinite Element Method in FEniCS: 1D Transient Heat Diffusion in detailWhat is a Jacobian-Vector product (jvp) in JAX?Simple KS solver in JAXLinear Regression in High Dimensions - Deriving the matrix-valued Least-Squares LossMaximum Likelihood Estimate for the One-Hot Categorical | TensorFlow ProbabilityMaximum A Posteriori Estimate (MAP) for Bernoulli | Derivation & TensorFlow ProbabilitySampling the Multivariate Normal distribution | example in PythonDeformation Gradient | Continuum Mechanics | with simple examplesPosterior for the Bernoulli using the Conjugate Prior | with example in TensorFlow ProbabilityNonlinear System Solve - Pullback/vJp ruleMatrix-Vector product - Pullback/vJp ruleSimple forward-mode AD in Julia using Dual Numbers and Operator Overloading