Append new token to prefix tokens. Args: xs (torch.Tensor): The prefix token x (int): The new token to append Returns: torch.Tensor: New tensor contains: xs + [x] with xs.dtype and xs.device
(xs: torch.Tensor, x: int)
| 134 | |
| 135 | @staticmethod |
| 136 | def append_token(xs: torch.Tensor, x: int) -> torch.Tensor: |
| 137 | """Append new token to prefix tokens. |
| 138 | |
| 139 | Args: |
| 140 | xs (torch.Tensor): The prefix token |
| 141 | x (int): The new token to append |
| 142 | |
| 143 | Returns: |
| 144 | torch.Tensor: New tensor contains: xs + [x] with xs.dtype and xs.device |
| 145 | |
| 146 | """ |
| 147 | x = torch.tensor([x], dtype=xs.dtype, device=xs.device) |
| 148 | return torch.cat((xs, x)) |
| 149 | |
| 150 | def score_full( |
| 151 | self, hyp: Hypothesis, x: torch.Tensor |
no outgoing calls
no test coverage detected