Skip to content

Instantly share code, notes, and snippets.

@kyunghyuncho

kyunghyuncho/label_smoothing_coeff.py

Created August 9, 2023 21:56
Show Gist options
  • Save kyunghyuncho/0ac974039a1a0a31f443190317169047 to your computer and use it in GitHub Desktop.
Save kyunghyuncho/0ac974039a1a0a31f443190317169047 to your computer and use it in GitHub Desktop.
Download ZIP
how to turn the target smoothed probability into label_smoothing coefficient in pytorch
Raw
label_smoothing_coeff.py
import numpy
'''
(1./n_classes) * (1.-label_smoothing) + 1. * label_smoothing = target_prob
1./n_classes - label_smoothing * (1./n_classes - 1.) = target_prob
label_smoothing * (1. - 1./n_classes) = target_prob - 1./n_classes
label_smoothing * (n_classes - 1.) / n_classes = target_prob - 1./n_classes
label_smoothing = (target_prob * n_classes - 1.) /n_classes * n_classes / (n_classes - 1.)
label_smoothing = (target_prob * n_classes - 1.) / (n_classes - 1.)
'''
target_prob = 0.9
n_classes = 50_000
label_smoothing = 1. - (target_prob * n_classes - 1.) / (n_classes - 1.)
print(label_smoothing)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment