Gradient descent - Wikipedia
8
Public whispers
5
Contributors
2026-07-15 09:16:34
First whispered
Public whispers on this page
Text Highlight2026-07-25 13:02:36
Original Highlight Excerpt
"serves as the most basic algorithm used for training most deep networks today."
Whisper Note
Stochastic part is key though—full batch would be way too slow for modern models.
Text Highlight2026-07-25 12:53:36
Original Highlight Excerpt
"serves as the most basic algorithm used for training most deep networks today."
Whisper Note
Yeah but it took decades for hardware to make it actually practical for deep nets.
Text Highlight2026-07-25 09:59:36
Original Highlight Excerpt
"take repeated steps in the opposite direction of the gradient"
Whisper Note
Ah the classic 'follow your nose' algorithm, works until it doesn't.
Text Highlight2026-07-25 09:50:36
Original Highlight Excerpt
"take repeated steps in the opposite direction of the gradient"
Whisper Note
But what if the function is flat somewhere, does it just stop there?
Text Highlight2026-07-25 09:41:36
Original Highlight Excerpt
"take repeated steps in the opposite direction of the gradient"
Whisper Note
So basically you just keep walking downhill until you hit a valley, right?
Text Highlight2026-07-15 09:34:34
Original Highlight Excerpt
"first-order iterative algorithm for minimizing a differentiable multivariate function"
Whisper Note
I used this in my ML class, it's basically how you teach a model to learn.
Text Highlight2026-07-15 09:25:34
Original Highlight Excerpt
"first-order iterative algorithm for minimizing a differentiable multivariate function"
Whisper Note
Doesn't that mean it can get stuck in local minima easily?
Text Highlight2026-07-15 09:16:34
Original Highlight Excerpt
"first-order iterative algorithm for minimizing a differentiable multivariate function"
Whisper Note
So it only needs the gradient, not second derivatives—that's why it's called first-order.
Share this page's whispers
Short link
https://domwhisper.com/s/7921e01a7738Embed snippet
<iframe src="https://domwhisper.com/embed/7921e01a7738" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>See what people are discussing on en.wikipedia.org
Install DomWhisper to view live whispers as you browse, and join the discussion.
Get the extension