MCPcopy Create free account

hub / github.com/fla-org/flash-linear-attention / functions

Functions2,989 in github.com/fla-org/flash-linear-attention

↓ 1,014 callersFunctionassert_close
(prefix, ref, tri, ratio, warning=False, err_atol=1e-6)
fla/utils/_testing.py:28
↓ 337 callersFunctionexp2
(x)
fla/ops/utils/op.py:20
↓ 281 callersMethodbackward
(ctx, dloss)
fla/modules/grpo.py:265
↓ 248 callersFunctionexp
(x)
fla/ops/utils/op.py:18
↓ 149 callersFunctionprepare_chunk_indices
( cu_seqlens: torch.LongTensor, chunk_size: int, cu_seqlens_cpu: torch.LongTensor | None = None, )
fla/ops/utils/index.py:156
↓ 145 callersMethodnormalize
(existing: KernelConfigFile | None, kernel_name: str)
scripts/utils/autotune_export.py:124
↓ 111 callersMethodregister
Register a backend.
fla/ops/backends/__init__.py:110
↓ 95 callersFunctioncheck_shared_mem
(arch: str = "none", tensor_idx: int = 0)
fla/utils/_device.py:184
↓ 91 callersFunctionfused_attnres
r""" Apply AttnRes residual aggregation. AttnRes normalizes each residual source with RMSNorm, scores it against `query`, applies softmax ove
fla/ops/attnres/fused.py:503
↓ 42 callersFunction_get_stride
Get the row stride for viewing a tensor as 2D (num_rows, D) where D = shape[-1]. Returns stride(-2) if the tensor is at least 2D, or 0 for 1D ten
fla/modules/activations.py:27
↓ 36 callersMethodfrom_legacy_cache
( cls, past_key_values: tuple[dict[str, Any], ...] | None = None, seen_tokens: int = 0
fla/models/utils.py:449
↓ 36 callersMethodreset_parameters
(self)
fla/modules/layernorm.py:1114
↓ 35 callersFunctionrun_test_generation
A foundational test for K/V cache-based generation.
tests/models/test_modeling_base.py:73
↓ 32 callersFunctioncausal_conv1d
A causal 1D convolution implementation that powers Mamba/Mamba2 and DeltaNet architectures. When a residual connection is provided, this imp
fla/modules/conv/causal_conv1d.py:17
↓ 32 callersMethodget_seq_length
(self, cache_position=None)
fla/models/utils.py:143
↓ 32 callersFunctionrun_test_model_forward_backward
A foundational test for the forward and backward passes of a model.
tests/models/test_modeling_base.py:32
↓ 31 callersFunctionprepare_chunk_offsets
( cu_seqlens: torch.LongTensor, chunk_size: int, )
fla/ops/utils/index.py:168
↓ 31 callersMethodupdate
( self, *, recurrent_state: torch.Tensor | tuple[torch.Tensor, ...] | None = None,
fla/models/utils.py:42
↓ 30 callersFunctionchunk_global_cumsum
( s: torch.Tensor, reverse: bool = False, cu_seqlens: torch.Tensor | None = None, scale: float
fla/ops/utils/cumsum.py:396
↓ 30 callersFunctionpad_input
Args: hidden_states ([total_tokens, ...]): where total_tokens denotes the number of tokens in selected in attention_mask.
fla/layers/utils.py:181
↓ 28 callersMethodbackward
(ctx, do, *args)
benchmarks/cp/test_gdn_with_cp.py:115
↓ 28 callersFunctionchunk_local_cumsum
( g: torch.Tensor, chunk_size: int, reverse: bool = False, scale: float = None, cu_seqlens
fla/ops/utils/cumsum.py:433
↓ 27 callersMethodrandint
( self, low: int, high: int, batch_size: int = 1024, g: torch.Generato
legacy/training/flame/data.py:121
↓ 26 callersFunctiongather
Gather operation that works when tl.gather is not supported. This is a fallback implementation that returns None. Just to mak
fla/ops/utils/op.py:70
↓ 26 callersFunctionget_layer_cache
(module, past_key_values)
fla/layers/utils.py:212
↓ 26 callersFunctionupdate_layer_cache
(module, past_key_values, **kwargs)
fla/layers/utils.py:219
↓ 24 callersFunction_ensure_inner_contiguous
Make the tensor inner-contiguous if it isn't already.
fla/modules/activations.py:70
↓ 24 callersFunction_get_stride
(x: torch.Tensor)
fla/modules/backends/triton_ascend/activations.py:23
↓ 23 callersFunctionchunk_gated_delta_rule
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/gated_delta_rule/chunk.py:380
↓ 22 callersFunctionparallel_wall_attn
r"""Wall parallel attention (training / prefill forward + backward). Scores use a per-channel multiplicative decay: with :math:`P = \mathrm{cumsu
fla/ops/wall_attn/parallel.py:919
↓ 20 callersFunctionl2norm_bwd
( y: torch.Tensor, rstd: torch.Tensor, dy: torch.Tensor, eps: float = 1e-6, )
fla/modules/l2norm.py:190
↓ 20 callersFunctionl2norm_fwd
( x: torch.Tensor, eps: float = 1e-6, output_dtype: torch.dtype | None = None, )
fla/modules/l2norm.py:138
↓ 20 callersFunctionlog
(x)
fla/ops/utils/op.py:22
↓ 20 callersMethodrun
(self, *args, **kwargs)
fla/ops/utils/cache.py:374
↓ 19 callersFunctionchunk_kda
r""" Args: q (torch.Tensor): queries of shape ``[B, T, H, K]``. k (torch.Tensor): keys of shape ``[B, T, H
fla/ops/kda/chunk.py:178
↓ 19 callersFunctionget_unpad_data
Retrieves indexing data required to repad unpadded (ragged) tensors. Args: attention_mask (`torch.Tensor`): Boolean or i
fla/layers/utils.py:80
↓ 19 callersFunctionqkvo_all2ll
(x, is_qkv=True, group=None, async_op=False)
benchmarks/cp/test_gdn_with_cp.py:120
↓ 18 callersFunction_alloc_output
Allocate the output: a fresh contiguous buffer, or ``empty_like`` otherwise. ``empty_like`` keeps the input's memory format only when it is dense
fla/modules/activations.py:77
↓ 18 callersMethodget_seq_length
(self, layer_idx: int | None = 0, cache_position=None)
fla/models/utils.py:423
↓ 18 callersFunctionmaybe_wait
(handle)
benchmarks/cp/test_gdn_with_cp.py:345
↓ 17 callersFunctionlog_decay
A valid small Wall log-decay gate (`g <= 0`), as `logsigmoid` produces in practice.
tests/ops/test_wall_attn.py:37
↓ 17 callersFunctionregister_op
(config: OpConfig)
benchmarks/ops/registry.py:166
↓ 16 callersFunctionfused_recurrent_gated_delta_rule
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/gated_delta_rule/fused_recurrent.py:307
↓ 16 callersFunctionfused_recurrent_simple_gla
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/simple_gla/fused_recurrent.py:15
↓ 15 callersFunction_is_inner_contiguous
Check if a tensor can be safely viewed as 2D (num_rows, D) with row stride = stride(-2). This holds when stride(-1) == 1 and all dimensions above
fla/modules/activations.py:39
↓ 15 callersFunctionnaive_parallel_attn
Reference PyTorch implementation of parallel attention that returns both output and max_logits. Args: q: [B, T, HQ, D] k: [B
fla/ops/attn/naive.py:12
↓ 15 callersFunctionparallel_attn
r""" Args: q (torch.Tensor): queries of shape `[B, T, HQ, K]`. k (torch.Tensor): keys of shape `[B, T, H,
fla/ops/attn/parallel.py:771
↓ 15 callersFunctionprint_rank0
(*args, **kwargs)
benchmarks/cp/benchmark_kda_cp8_vs_cp2tp.py:62
↓ 15 callersFunctiontoken_shift
Token-shift operation implemented with Triton kernels. Args: x: Input tensor of shape [B, T, D] (or [1, T, D] when `cu_seqlens` is s
fla/modules/token_shift.py:532
↓ 15 callersFunctionunpad_input
Unpads query, key, and values tensors, using a single dimension for all tokens even though they belong to different batches. Arguments:
fla/layers/utils.py:106
↓ 14 callersFunction_ensure_inner_contiguous
(x: torch.Tensor)
fla/modules/backends/triton_ascend/activations.py:51
↓ 14 callersFunction_run
(cmd: list[str], *, cwd: Path, env: dict[str, str] | None = None)
tests/test_split_package_release.py:26
↓ 14 callersFunctionfn
(new_value=None)
tests/test_utils.py:80
↓ 14 callersFunctionprepare_lens_from_mask
(mask: torch.BoolTensor)
fla/ops/utils/index.py:46
↓ 13 callersFunctionfused_recurrent_kda
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/kda/fused_recurrent.py:336
↓ 13 callersFunctiontanh
(x)
fla/ops/utils/op.py:26
↓ 12 callersMethod__init__
( self, head_dim: int, dot_dim: int = None, bias: bool | None = False, )
fla/modules/feature_map.py:77
↓ 12 callersFunctionget_multiprocessor_count
(tensor_idx: int = 0)
fla/utils/_device.py:82
↓ 12 callersFunctionrun_cp_test_with_spawn
Run CP test using torch.multiprocessing.spawn. This allows running the test directly with pytest.
tests/context_parallel/test_cp_conv.py:228
↓ 11 callersFunctionfused_recurrent_gla
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/gla/fused_recurrent.py:15
↓ 11 callersFunctionmake_inputs
Return ``n`` grad-requiring (B, T, D) tensors; non-contiguous ones are halves of a chunked buffer.
tests/modules/test_activation.py:37
↓ 10 callersFunction_alloc_output
(x: torch.Tensor, contiguous: bool = False)
fla/modules/backends/triton_ascend/activations.py:57
↓ 10 callersFunction_rand_inputs
Well-conditioned GDN-2 inputs. q/k drawn raw (kernels L2-normalize via use_qk_l2norm_in_kernel, or the caller normalizes for the reference).
tests/ops/test_gdn2.py:50
↓ 10 callersFunctionchunk_gated_delta_rule_fwd_h
( k: torch.Tensor, w: torch.Tensor, u: torch.Tensor, g: torch.Tensor | None = None, gk: to
fla/ops/common/chunk_delta_h.py:600
↓ 10 callersFunctionprepare_lens
(cu_seqlens: torch.LongTensor)
fla/ops/utils/index.py:41
↓ 10 callersFunctionrun_cp_test_with_spawn
Run CP test using torch.multiprocessing.spawn. This allows running the test directly with pytest.
tests/context_parallel/test_cp_gdn.py:286
↓ 10 callersMethodstep
( self, hidden_states: torch.Tensor, angle_state: torch.Tensor, ssm_state: tor
fla/layers/mamba3.py:341
↓ 9 callersFunctioncausal_conv1d_ref_torch
x: (batch, dim, seqlen) weight: (dim, width) bias: (dim,) initial_state: (batch, dim, width - 1) final_states_out: (batch, dim, w
tests/modules/test_conv.py:31
↓ 9 callersFunctionchunk_simple_gla
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/simple_gla/chunk.py:238
↓ 9 callersFunctioncompare
(x, y, prefix="")
benchmarks/cp/test_gdn_with_cp.py:157
↓ 9 callersFunctionfused_recurrent_retention
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, scale: float | None = None, initial_s
fla/ops/retention/fused_recurrent.py:15
↓ 9 callersFunctionparallel_parallax
r""" Causal Parallax (parameterized local linear attention) with autograd, backed by Triton kernels. See `fla.ops.parallax.naive.naive_paralla
fla/ops/parallax/parallel.py:751
↓ 9 callersFunctionrun_cp_test_with_spawn
Run CP test using torch.multiprocessing.spawn. This allows running the test directly with pytest.
tests/context_parallel/test_cp_kda.py:412
↓ 9 callersFunctionsolve_tril
Compute the inverse of the matrix I + A A should be strictly lower triangular, i.e., A.triu() == 0. Args: A (torch.Tensor):
fla/ops/utils/solve_tril.py:349
↓ 9 callersMethodupdate
( self, recurrent_state: tuple[torch.Tensor] | None = None, attn_state: tuple[torch.Te
fla/models/utils.py:385
↓ 8 callersFunction_make_attn_state
(start: int, length: int, *, batch_size: int = 1, num_heads: int = 2, head_dim: int = 3)
tests/models/test_cache.py:16
↓ 8 callersFunctionall_gather
(x, group=None)
benchmarks/cp/test_gdn_with_cp.py:190
↓ 8 callersFunctionchunk_fwd_h
( k: torch.Tensor, v: torch.Tensor, g: torch.Tensor | None = None, g_gamma: torch.Tensor | Non
fla/ops/common/chunk_h.py:294
↓ 8 callersFunctionchunk_gdn2
r""" Chunkwise forward for Gated DeltaNet 2 (GDN-2). The matrix-state recurrence on ``S in R^{K x V}`` is S_t = (I - k_t (b_t * k_t)
fla/ops/gdn2/chunk.py:195
↓ 8 callersFunctionfused_recurrent_gdn2
r""" Token-by-token forward for GDN-2 (Gated DeltaNet 2). Inference-only. The recurrence on the matrix state ``S in R^{K x V}`` is S
fla/ops/gdn2/fused_recurrent.py:351
↓ 8 callersFunctionparallel_moba
r"""Flash-attn based MoBA implementation. MoBA (Mixture of Block Attention, https://arxiv.org/abs/2502.13189) makes each query token attend o
fla/ops/moba/parallel.py:296
↓ 8 callersMethodupdate
(self, **kwargs)
tests/layers/test_layer_cache_layer_idx.py:39
↓ 7 callersMethod__init__
( self, hidden_size: int, elementwise_affine: bool = True, bias: bool = False,
fla/modules/layernorm.py:1089
↓ 7 callersMethod__init__
( self, hidden_size: int, elementwise_affine: bool = True, eps: float = 1e-5,
fla/modules/fused_norm_gate.py:993
↓ 7 callersMethod_init_yoco_linear
(module: nn.Linear)
fla/models/yoco/modeling_yoco.py:320
↓ 7 callersFunctionattn_decoding_one_step
r""" Args: q (torch.Tensor): query of shape `[1, B, HQ, K]`. k (torch.Tensor): keys of shape `[1, T, H, K]
fla/ops/attn/decoding.py:119
↓ 7 callersFunctionbuild_cp_context
Build a CP context for the given cu_seqlens and process group. Args: cu_seqlens: Cumulative sequence lengths tensor (before partition).
fla/ops/cp/context.py:155
↓ 7 callersFunctioncausal_conv1d_update
( x: torch.Tensor, cache: torch.Tensor, residual: torch.Tensor | None = None, weight: torch.Te
fla/modules/conv/triton/ops.py:298
↓ 7 callersFunctionchunk_gla
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/gla/chunk.py:1341
↓ 7 callersFunctionfn
()
benchmarks/cp/test_gdn_with_cp.py:781
↓ 7 callersFunctionfused_recurrent_gsa
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/gsa/fused_recurrent.py:433
↓ 7 callersMethodis_available
(cls)
fla/ops/backends/__init__.py:60
↓ 7 callersFunctionnaive_recurrent_kda
r""" Args: q (torch.Tensor): Queries of shape ``[B, T, H, K]``. k (torch.Tensor): Keys of shape ``[B, T, H
fla/ops/kda/naive.py:12
↓ 7 callersFunctionnaive_wall_attn
r"""Eager Wall forward (exact logits): per pair :math:`(i,j)`, :math:`s_{ij} = \sum_n q_{in} k_{jn} \exp_2(P_{in}-P_{jn}) \cdot \mathrm{scale}\cdo
fla/ops/wall_attn/naive.py:17
↓ 7 callersFunctionrun_cp_test_with_spawn
Run CP test using torch.multiprocessing.spawn.
tests/context_parallel/test_cp_token_shift.py:172
↓ 6 callersFunctionall_gather
(x, group=None)
benchmarks/cp/benchmark_kda_cp8_vs_cp2tp.py:72
↓ 6 callersFunctionmake_tensor_descriptor
( base, shape, strides, block_shape, _builder=None, )
fla/ops/utils/op.py:94
↓ 6 callersFunctionnaive_gdn_gate
Torch reference implementation for GDN gate computation. Computes: ``g = -A_log.exp() * softplus(g + dt_bias)`` Args: g (torch.
fla/ops/gated_delta_rule/gate.py:20
↓ 6 callersFunctionnaive_kda_gate
Torch reference implementation for KDA gate computation. Computes: g = -A_log.exp().unsqueeze(-1) * softplus(g + dt_bias.view(g.shape[-2:]))
fla/ops/kda/gate.py:26
↓ 6 callersFunctionnaive_recurrent_gdn2
Token-by-token reference forward pass for GDN-2. Args: q: queries of shape [B, T, H, K]. k: keys of shape [B, T, H, K]. v
fla/ops/gdn2/naive.py:23
next →1–100 of 2,989, ranked by callers