Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Conver layer_norm_hook to a PyTorch hook #121

Merged
merged 1 commit into from
Nov 1, 2023
Merged

Conver layer_norm_hook to a PyTorch hook #121

merged 1 commit into from
Nov 1, 2023

Conversation

cbalioglu
Copy link
Contributor

This PR converts the parameter-based layer_norm_hook in TransformerEncoder and TransformerDecoder to a PyTorch hook (i.e. RemovableHandle for more flexibility (e.g. for early-exit fine-tuning work of SysML)

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 1, 2023
@cbalioglu cbalioglu merged commit 9ffbea3 into main Nov 1, 2023
18 checks passed
@cbalioglu cbalioglu deleted the hook branch November 1, 2023 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cbalioglu @facebook-github-bot