| 1020 | {0, 0}}; |
| 1021 | |
| 1022 | void init_transforms(nb::module_& m) { |
| 1023 | nb::class_<PyCustomFunction>( |
| 1024 | m, |
| 1025 | "custom_function", |
| 1026 | nb::type_slots(py_custom_function_slots), |
| 1027 | R"pbdoc( |
| 1028 | Set up a function for custom gradient and vmap definitions. |
| 1029 | |
| 1030 | This class is meant to be used as a function decorator. Instances are |
| 1031 | callables that behave identically to the wrapped function. However, when |
| 1032 | a function transformation is used (e.g. computing gradients using |
| 1033 | :func:`value_and_grad`) then the functions defined via |
| 1034 | :meth:`custom_function.vjp`, :meth:`custom_function.jvp` and |
| 1035 | :meth:`custom_function.vmap` are used instead of the default transformation. |
| 1036 | |
| 1037 | Note, all custom transformations are optional. Undefined transformations |
| 1038 | fall back to the default behaviour. |
| 1039 | |
| 1040 | Example: |
| 1041 | |
| 1042 | .. code-block:: python |
| 1043 | |
| 1044 | import mlx.core as mx |
| 1045 | |
| 1046 | @mx.custom_function |
| 1047 | def f(x, y): |
| 1048 | return mx.sin(x) * y |
| 1049 | |
| 1050 | @f.vjp |
| 1051 | def f_vjp(primals, cotangent, output): |
| 1052 | x, y = primals |
| 1053 | return cotan * mx.cos(x) * y, cotan * mx.sin(x) |
| 1054 | |
| 1055 | @f.jvp |
| 1056 | def f_jvp(primals, tangents): |
| 1057 | x, y = primals |
| 1058 | dx, dy = tangents |
| 1059 | return dx * mx.cos(x) * y + dy * mx.sin(x) |
| 1060 | |
| 1061 | @f.vmap |
| 1062 | def f_vmap(inputs, axes): |
| 1063 | x, y = inputs |
| 1064 | ax, ay = axes |
| 1065 | if ay != ax and ax is not None: |
| 1066 | y = y.swapaxes(ay, ax) |
| 1067 | return mx.sin(x) * y, (ax or ay) |
| 1068 | |
| 1069 | All ``custom_function`` instances behave as pure functions. Namely, any |
| 1070 | variables captured will be treated as constants and no gradients will be |
| 1071 | computed with respect to the captured arrays. For instance: |
| 1072 | |
| 1073 | .. code-block:: python |
| 1074 | |
| 1075 | import mlx.core as mx |
| 1076 | |
| 1077 | def g(x, y): |
| 1078 | @mx.custom_function |
| 1079 | def f(x): |
no test coverage detected