Method
__init__
(self, dim_in: int, dim_out: int, approximate: str = "none", bias: bool = True)
Source from the content-addressed store, hash-verified
| 74 | """ |
| 75 | |
| 76 | def __init__(self, dim_in: int, dim_out: int, approximate: str = "none", bias: bool = True): |
| 77 | super().__init__() |
| 78 | self.proj = nn.Linear(dim_in, dim_out, bias=bias) |
| 79 | self.approximate = approximate |
| 80 | |
| 81 | def gelu(self, gate: torch.Tensor) -> torch.Tensor: |
| 82 | if gate.device.type == "mps" and is_torch_version("<", "2.0.0"): |
Tested by
no test coverage detected