Maximum Likelihood Estimate by Automatic Differentiation | Bernoulli Distribution

Let's use the capabilities of TensorFlow Probability of automatically computing gradients to solve the MLE optimization problem with a gradient descent optimizer. You can find the notes here: https://raw.githubusercontent.com/Ceyron/machine-learning-and-simulation/main/english/essential_pmf_pdf/bernoulli_maximum_likelihood_estimate_by_automatic_differentiation.pdf

Automatic Differentiation allows for cheap evaluation of derivatives of previously defined algorithmic graphs. We can use this in order to find the Maximum Likelihood Estimate (MLE) by a gradient-based optimization scheme.

Surely, in the case of the Bernoulli, where a simple closed-form solution exists, there is no need for gradient-based optimization. However, using Automatic Differentiation takes away the most complicated part of the MLE - taking the derivative and setting it to zero. Hence, thinking in terms of differentiable programming (and hence Automatic Differentiation) can make hard stuff easy.

-------

📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation

📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler

💸 : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim

-------

⚙️ My Gear:

(Below are affiliate links to Amazon. If you decide to purchase the product or something else on Amazon through this link, I earn a small commission.)

- 🎙️ Microphone: Blue Yeti: https://amzn.to/3NU7OAs
- ⌨️ Logitech TKL Mechanical Keyboard: https://amzn.to/3JhEtwp
- 🎨 Gaomon Drawing Tablet (similar to a WACOM Tablet, but cheaper, works flawlessly under Linux): https://amzn.to/37katmf
- 🔌 Laptop Charger: https://amzn.to/3ja0imP
- 💻 My Laptop (generally I like the Dell XPS series): https://amzn.to/38xrABL
- 📱 My Phone: Fairphone 4 (I love the sustainability and repairability aspect of it): https://amzn.to/3Jr4ZmV

If I had to purchase these items again, I would probably change the following:

- 🎙️ Rode NT: https://amzn.to/3NUIGtw
- 💻 Framework Laptop (I do not get a commission here, but I love the vision of Framework. It will definitely be my next Ultrabook): https://frame.work

As an Amazon Associate I earn from qualifying purchases.

-------

Timestamps:
00:00 Opening
00:17 The Bernoulli Model
00:49 Task of parameter inference
01:14 Log-Likelihood and MLE
02:13 Motivation for Automatic Differentiation
03:27 Reformulation as Minimization
04:20 TFP: Setup
04:41 TFP: Creating a dataset
05:37 TFP: Creating a model distribution with variables
07:09 TFP: Defining the loss
08:17 TFP: The optimization
09:29 TFP: Discussing the result

Видео Maximum Likelihood Estimate by Automatic Differentiation | Bernoulli Distribution канала Machine Learning & Simulation