Skip to content

Commit

Permalink
Do not reset RegexLogitsProcessor._fsm_state (outlines-dev#760)
Browse files Browse the repository at this point in the history
  • Loading branch information
saattrupdan committed Mar 25, 2024
1 parent c744e25 commit 803439a
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions outlines/integrations/vllm.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,7 @@ def __call__(self, input_ids: List[int], scores: torch.Tensor) -> torch.Tensor:

# Initialize the FSM state dictionary if the input_ids are empty, as this means
# that the input_ids are the first tokens of the sequence.
if len(input_ids) == 0:
self._fsm_state = defaultdict(int)
else:
if len(input_ids) > 0:
last_token = input_ids[-1]
last_seq_id = hash(tuple(input_ids[:-1]))
self._fsm_state[seq_id] = self.fsm.get_next_state(
Expand Down

0 comments on commit 803439a

Please sign in to comment.