Skip to content

Commit

Permalink
Updating a tutorial to include Ghost Clipping DP-SGD (#667)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: #667

We update an existing tutorial on fine-tuning a language model, but now also with GC DP-SGD.

Differential Revision: D61411180
  • Loading branch information
EnayatUllah authored and facebook-github-bot committed Aug 22, 2024
1 parent 4823344 commit 0e4e25e
Show file tree
Hide file tree
Showing 3 changed files with 115 additions and 41 deletions.
139 changes: 103 additions & 36 deletions tutorials/building_text_classifier.ipynb

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions website/pages/tutorials/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,13 @@ class TutorialHome extends React.Component {
Efficient Per-Sample Gradient Computation for More Layers in Opacus
</a>
</li>
<li>
<a
href="https://pytorch.org/blog/clipping-in-opacus/"
target="_blank">
Enabling Fast Gradient Clipping and Ghost Clipping in Opacus
</a>
</li>
</ol>
<h4>Videos*</h4>
<p>* Note that Opacus API has changed over time and some of the code samples and demos in the videos may not work. The concepts presented in the videos though are concrete and still valid.</p>
Expand Down
10 changes: 5 additions & 5 deletions website/tutorials.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{
"Using Opacus": [
{
"id": "building_image_classifier",
"title": "Building image classifier with Differential Privacy"
"id": "building_text_classifier",
"title": "Building text classifier with Fast Gradient Clipping DP-SGD"
},
{
"id": "building_text_classifier",
"title": "Building text classifier with Differential Privacy"
"id": "building_image_classifier",
"title": "Building image classifier with Differential Privacy"
},
{
"id": "building_lstm_name_classifier",
Expand All @@ -27,6 +27,6 @@
{
"id": "ddp_tutorial",
"title": "Training on multiple GPUs with DistributedDataParallel"
}
}
]
}

0 comments on commit 0e4e25e

Please sign in to comment.