In function apply_top_p_filtering, there is the following code:
if len(sorted_indices_to_remove) > len(sorted_indices) - min_tokens_to_keep:
sorted_indices_to_remove = sorted_indices_to_remove[
:len(sorted_indices) - min_tokens_to_keep
]
It remove indices with probabilities from big to small. The correct way is removes from small to big:
if len(sorted_indices_to_remove) > len(sorted_indices) - min_tokens_to_keep:
sorted_indices_to_remove = sorted_indices_to_remove[
-(len(sorted_indices) - min_tokens_to_keep):
]
In function
apply_top_p_filtering, there is the following code:It remove indices with probabilities from big to small. The correct way is removes from small to big: