Noticed while reviewing PR #289 that Vishu’s Hyena implementation passes in the padding_idx as an integer to nn.Embedding, but the other models wrap it in a tensor. It turns out the former is correct per the Pytorch API: https://docs.pytorch.org/docs/stable/generated/torch.nn.Embedding.html. Minor fix, but one that should be made for consistency.
Noticed while reviewing PR #289 that Vishu’s Hyena implementation passes in the
padding_idxas an integer tonn.Embedding, but the other models wrap it in a tensor. It turns out the former is correct per the Pytorch API: https://docs.pytorch.org/docs/stable/generated/torch.nn.Embedding.html. Minor fix, but one that should be made for consistency.