Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-3166: add comments and one-sided dropout #3168

Merged
merged 2 commits into from
Mar 28, 2023
Merged

GH-3166: add comments and one-sided dropout #3168

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b432e3f
GH-3166: add comments and one-sided dropout
alanakbik Mar 28, 2023
c8525b4
Black formatting
alanakbik Mar 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 16 additions & 10 deletions flair/embeddings/transformer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -620,23 +620,29 @@ def __gather_flair_tokens(self, sentences: List[Sentence]) -> Tuple[List[List[To
return sentence_tokens, offsets, lengths

def _expand_sentence_with_context(self, sentence) -> Tuple[List[Token], int]:
expand_context = self.context_length > 0 and (
not self.training or random.randint(1, 100) > (self.context_dropout * 100)
)

# fields to store left and right context
left_context = []
right_context = []

# expand context only if context_length is set
expand_context = self.context_length > 0

if expand_context:
left_context = sentence.left_context(self.context_length, self.respect_document_boundaries)
right_context = sentence.right_context(self.context_length, self.respect_document_boundaries)
# if context_dropout is set, randomly deactivate left context during training
if not self.training or random.randint(1, 100) > (self.context_dropout * 100):
left_context = sentence.left_context(self.context_length, self.respect_document_boundaries)

if self.use_context_separator:
left_context = left_context + [Token(SENTENCE_BOUNDARY_TAG)]
right_context = [Token(SENTENCE_BOUNDARY_TAG)] + right_context
# if context_dropout is set, randomly deactivate right context during training
if not self.training or random.randint(1, 100) > (self.context_dropout * 100):
right_context = sentence.right_context(self.context_length, self.respect_document_boundaries)

expanded_sentence = left_context + sentence.tokens + right_context
# if use_context_separator is set, add a [FLERT] token
if self.use_context_separator and self.context_length > 0:
left_context = left_context + [Token(SENTENCE_BOUNDARY_TAG)]
right_context = [Token(SENTENCE_BOUNDARY_TAG)] + right_context

# return expanded sentence and context length information
expanded_sentence = left_context + sentence.tokens + right_context
context_length = len(left_context)
return expanded_sentence, context_length

Expand Down