shen/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2024-11-25 16:22:50 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

add dropout scaling note
Some checks failed
Code tests (Linux) / test (push) Has been cancelled
Details
Code tests (macOS) / test (push) Has been cancelled
Details
Test PyTorch 2.0 and 2.5 / test (2.0.1) (push) Has been cancelled
Details
Test PyTorch 2.0 and 2.5 / test (2.5.0) (push) Has been cancelled
Details
Code tests (Windows) / test (push) Has been cancelled
Details
Check hyperlinks / test (push) Has been cancelled
Details
Spell Check / spellcheck (push) Has been cancelled
Details
PEP8 Style checks / flake8 (push) Has been cancelled
Details

Browse Source
This commit is contained in:
rasbt 2024-11-06 05:52:47 -06:00
parent 9ce0be333b
commit 1183fd7837
No known key found for this signature in database
GPG Key ID: 3C6E5C7C075611DB
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
ch03/01_main-chapter-code/ch03.ipynb
View File

@ -1485,7 +1485,8 @@
"id": "5a575458-a6da-4e54-8688-83e155f2de06",
"metadata": {},
"source": [
"- If we apply a dropout rate of 0.5 (50%), the non-dropped values will be scaled accordingly by a factor of 1/0.5 = 2."
"- If we apply a dropout rate of 0.5 (50%), the non-dropped values will be scaled accordingly by a factor of 1/0.5 = 2\n",
"- The scaling is calculated by the formula 1 / (1 - `dropout_rate`)"
]
},
{