Skip to content

Instantly share code, notes, and snippets.

@HarshSingh16

HarshSingh16/Tokens.py

Created February 8, 2019 18:55
Show Gist options
  • Save HarshSingh16/0722ac13318618ebbccbbd6861f71387 to your computer and use it in GitHub Desktop.
Save HarshSingh16/0722ac13318618ebbccbbd6861f71387 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Tokens.py
#Adding tokens to our dictionary
Tokens=["<PAD>","<SOS>","<EOS>","<OUT>"]
for token in Tokens:
dict_integer[token]=len(dict_integer)+1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment