MCPcopy Create free account

hub / github.com/fla-org/flash-linear-attention / functions

Functions2,989 in github.com/fla-org/flash-linear-attention

↓ 2 callersFunctionnaive_mesa_net_exact
(q, k, v, g, lamb, beta, h_kk_init=None, h_kv_init=None)
fla/ops/mesa_net/naive.py:44
↓ 2 callersFunctionnaive_nsa
r""" Args: q (torch.Tensor): queries of shape `[B, T, HQ, K]`. k (torch.Tensor): keys of shape `[B, T, H,
fla/ops/nsa/naive.py:12
↓ 2 callersFunctionnaive_path_attn
Reference PyTorch implementation of path attention. Args: q: [B, T, HQ, D] k: [B, T, H, D] v: [B, T, H, D] w
fla/ops/path_attn/naive.py:13
↓ 2 callersFunctionnaive_recurrent_gated_delta_product
(q, k, v, g, beta, scale, cu_seqlens, initial_state=None, output_final
fla/ops/gated_delta_product/naive.py:11
↓ 2 callersFunctionnaive_recurrent_gla
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, gk: torch.Tensor, initial_state: torc
fla/ops/gla/naive.py:15
↓ 2 callersFunctionnaive_recurrent_hgrn
( x: torch.Tensor, g: torch.Tensor, initial_state: torch.Tensor | None = None, output_final_st
fla/ops/hgrn/naive.py:11
↓ 2 callersFunctionnaive_recurrent_simple_gla
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g: torch.Tensor, scale: float | None
fla/ops/simple_gla/naive.py:61
↓ 2 callersFunctionpack_sequence_fwdbwd
( x: torch.Tensor, cu_seqlens: torch.Tensor, padding_side: str, )
fla/ops/utils/pack.py:63
↓ 2 callersFunctionpad_tensor_by_size
Padding x tensor with `pad_size` on the seq_len dim (dim=1) Assumes that we only have tensors of either size 4 or 3
fla/layers/mamba2.py:53
↓ 2 callersFunctionparallax_decode_one_step
r""" Single-token Parallax decode (one query per sequence over the cached KV). Optimized for the `Sq == 1` autoregressive step: the query is
fla/ops/parallax/decode.py:276
↓ 2 callersFunctionparallel_based
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, scale: float | None = None, use_norm:
fla/ops/based/parallel.py:392
↓ 2 callersFunctionparallel_deltaformer_bwd_qk
( q: torch.Tensor, k: torch.Tensor, u: torch.Tensor, lse: torch.Tensor, grad_v: torch.Tens
fla/ops/deltaformer/parallel.py:71
↓ 2 callersFunctionparallel_deltaformer_bwd_u_chunk
( q: torch.Tensor, k: torch.Tensor, lse: torch.Tensor, grad_v: torch.Tensor, fa_scale: flo
fla/ops/deltaformer/parallel.py:49
↓ 2 callersFunctionparallel_deltaformer_chunk_fwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, u: torch.Tensor, qk_scale: float,
fla/ops/deltaformer/parallel.py:31
↓ 2 callersFunctionparallel_path_attn
r""" Args: q (torch.Tensor): queries of shape `[B, T, HQ, K]` k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/path_attn/parallel.py:225
↓ 2 callersFunctionparallel_retention
r""" Args: q (torch.Tensor): queries of shape `[B, T, H, K]`. k (torch.Tensor): keys of shape `[B, T, H, K
fla/ops/retention/parallel.py:13
↓ 2 callersFunctionpatch_fp32_grad_linear_forward
(model: torch.nn.Module)
benchmarks/cp/test_gdn_with_cp.py:260
↓ 2 callersFunctionpowglu
(x: torch.Tensor, y: torch.Tensor, power: float = 3.0)
fla/modules/activations.py:1185
↓ 2 callersFunctionpowglu_fwd
(x: torch.Tensor, y: torch.Tensor, power: float = 3.0, output_contiguous: bool = False)
fla/modules/activations.py:1062
↓ 2 callersFunctionpowglu_fwdbwd
( x: torch.Tensor, y: torch.Tensor, g: torch.Tensor, power: float = 3.0, use_weight: bool
fla/modules/activations.py:1083
↓ 2 callersFunctionpowglu_linear
( x: torch.Tensor, y: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, power: float
fla/modules/activations.py:1189
↓ 2 callersFunctionpowglu_ref
(x: torch.Tensor, y: torch.Tensor, power: float = 3.0)
tests/modules/test_activation.py:29
↓ 2 callersFunctionprepare_cu_seqlens_from_lens
( lens: torch.LongTensor, dtype: torch.dtype | None = torch.int32, )
fla/ops/utils/index.py:51
↓ 2 callersFunctionprepare_layer_and_inputs
(builder, hidden_states)
tests/layers/test_layer_cache_layer_idx.py:53
↓ 2 callersFunctionprepare_position_ids
(cu_seqlens: torch.LongTensor, cu_seqlens_cpu: torch.LongTensor | None = None)
fla/ops/utils/index.py:138
↓ 2 callersFunctionprepare_wy_repr_bwd
( k: torch.Tensor, v: torch.Tensor, beta: torch.Tensor, gk: torch.Tensor, A: torch.Tensor,
fla/ops/kda/wy_fast.py:297
↓ 2 callersFunctionprepare_wy_repr_fwd
( ag: torch.Tensor, v: torch.Tensor, A_ak: torch.Tensor, A_ab: torch.Tensor, cu_seqlens: t
fla/ops/generalized_delta_rule/dplr/wy_fast_fwd.py:252
↓ 2 callersFunctionprofile_func
(fn, path)
benchmarks/cp/test_gdn_with_cp.py:648
↓ 2 callersFunctionread_config_file
Read a config file, bypassing the in-process cache in ALWAYS mode.
fla/ops/utils/cache.py:264
↓ 2 callersFunctionrecompute_w_u_fwd
( k: torch.Tensor, v: torch.Tensor, beta: torch.Tensor, A: torch.Tensor, g: torch.Tensor |
fla/ops/gated_delta_rule/wy_fast.py:227
↓ 2 callersFunctionrecompute_w_u_fwd
( k: torch.Tensor, v: torch.Tensor, beta: torch.Tensor, A: torch.Tensor, gv: torch.Tensor
fla/ops/gated_oja_rule/wy_fast.py:200
↓ 2 callersFunctionrecompute_w_u_fwd
( k: torch.Tensor, v: torch.Tensor, beta: torch.Tensor, g_cumsum: torch.Tensor, A: torch.T
fla/ops/comba/wy_fast.py:339
↓ 2 callersFunctionrecompute_w_u_fwd
( k: torch.Tensor, v: torch.Tensor, beta: torch.Tensor, A: torch.Tensor, cu_seqlens: torch
fla/ops/delta_rule/wy_fast.py:217
↓ 2 callersFunctionrecompute_w_u_fwd_gdn2
Produce GDN-2 WY auxiliaries from the solved chunk inverse ``A``. Returns ``(w, u, qg, kg)`` with shapes matching the inputs. ``qg`` is produ
fla/ops/gdn2/wy_fast.py:170
↓ 2 callersFunctionrecurrence_iplr_delta_rule_ref
( q, k, v, a, b, initial_state: torch.Tensor | None = None, output_final_state: bo
tests/ops/test_iplr_delta.py:86
↓ 2 callersFunctionrecurrent_dplr_delta_rule_ref
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, a: torch.Tensor, b: torch.Tensor,
tests/ops/test_dplr_delta.py:19
↓ 2 callersFunctionref_prepare_position_ids
(cu_seqlens)
tests/ops/utils/test_index.py:28
↓ 2 callersFunctionref_prepare_split_cu_seqlens
(batch_size, seq_len, split_size, cu_seqlens=None, dtype=torch.int32)
tests/ops/utils/test_index.py:60
↓ 2 callersFunctionrms_norm_swish_gate_linear
( x, g: torch.Tensor, norm_weight: torch.Tensor, norm_bias: torch.Tensor, linear_weight: t
fla/modules/fused_norm_gate.py:898
↓ 2 callersFunctionrmsnorm_fn
(x, weight, bias, z=None, eps=1e-6, group_size=None, norm_before_gate=True)
fla/modules/layernorm_gated.py:466
↓ 2 callersFunctionrotary_embedding_fwdbwd
Args: x: [B, T, H, D]. cos: [TR, R / 2] sin: [TR, R / 2] seqlen_offsets: integer or integer tensor of size [N]
fla/modules/rotary.py:142
↓ 2 callersFunctionrun
(state_v_first: bool)
tests/ops/test_retention.py:345
↓ 2 callersFunctionrun_unified_benchmark
Run the unified runner from a temp directory.
scripts/run_benchmark_compare.py:147
↓ 2 callersFunctionrwkv_mix_fwd
(x, x_prev, x_k)
fla/ops/rwkv7/channel_mixing.py:111
↓ 2 callersFunctionrwkv_relu_and_square_fwd
Triton implementation of RWKV's ReLU and square operation Args: x: Input tensor Returns: Tensor after ReLU and square ope
fla/ops/rwkv7/channel_mixing.py:148
↓ 2 callersFunctionsegment_sum
More stable segment sum calculation. Uses cumulative sums and masking instead of direct subtractions.
fla/layers/mamba2.py:85
↓ 2 callersFunctionserialize_preserved_entry
(entry: dict[str, object])
scripts/utils/autotune_export.py:88
↓ 2 callersFunctionsigmoidglu_fwd
(x: torch.Tensor, y: torch.Tensor, output_contiguous: bool = False)
fla/modules/activations.py:839
↓ 2 callersFunctionsigmoidglu_fwdbwd
( x: torch.Tensor, y: torch.Tensor, g: torch.Tensor, use_weight: bool = False, output_cont
fla/modules/activations.py:859
↓ 2 callersFunctionsizeof_fmt
(num, suffix='B')
benchmarks/benchmark_generation.py:18
↓ 2 callersFunctionsoftmax_bwd
( p: torch.Tensor, dp: torch.Tensor, dtype: torch.dtype | None = torch.float, )
fla/ops/utils/softmax.py:92
↓ 2 callersFunctionsoftmax_fwd
( x: torch.Tensor, dtype: torch.dtype | None = torch.float, )
fla/ops/utils/softmax.py:72
↓ 2 callersMethodstep
( self, x: torch.Tensor, residual: torch.Tensor | None, cache: torch.Tensor |
fla/modules/conv/short_conv.py:201
↓ 2 callersFunctionsum_norm
(x)
fla/layers/delta_net.py:32
↓ 2 callersFunctionswiglu_fwd
(x: torch.Tensor, y: torch.Tensor, output_contiguous: bool = False)
fla/modules/activations.py:640
↓ 2 callersFunctionswiglu_fwd_npu
(x: torch.Tensor, y: torch.Tensor, output_contiguous: bool = False)
fla/modules/backends/triton_ascend/activations.py:451
↓ 2 callersFunctionswiglu_fwdbwd
( x: torch.Tensor, y: torch.Tensor, g: torch.Tensor, use_weight: bool = False, output_cont
fla/modules/activations.py:661
↓ 2 callersFunctionswiglu_fwdbwd_npu
( x: torch.Tensor, y: torch.Tensor, g: torch.Tensor, use_weight: bool = False, output_cont
fla/modules/backends/triton_ascend/activations.py:467
↓ 2 callersFunctionswiglu_linear
(x, y, weight, bias)
fla/modules/activations.py:758
↓ 2 callersFunctiontensor_to_dtensor
( tensor: torch.Tensor, device_mesh: DeviceMesh, current_placement: Placement | list[Placement],
fla/models/mamba2/modeling_mamba2.py:49
↓ 2 callersFunctiontoken_shift_bwd
( dy: torch.Tensor, N: int, T: int, dcache: torch.Tensor | None = None, cu_seqlens: torch.
fla/modules/token_shift.py:450
↓ 2 callersFunctiontoken_shift_fwd
( x: torch.Tensor, cu_seqlens: torch.Tensor | None = None, cache: torch.Tensor | None = None,
fla/modules/token_shift.py:377
↓ 2 callersFunctiontoken_shift_ref
(x)
benchmarks/modules/benchmark_tokenshift.py:15
↓ 2 callersFunctiontorch_addcmul_rwkv7
(hidden_states, delta, xr, xw, xk, xv, xa, xg=None)
fla/ops/rwkv7/fused_addcmul.py:290
↓ 2 callersFunctionunpack_sequence_fwdbwd
( x: torch.Tensor, cu_seqlens: torch.Tensor, padding_side: str, desired_shape: torch.Size, )
fla/ops/utils/pack.py:87
↓ 2 callersMethodupdate
Args: recurrent_state (`torch.Tensor`): The new recurrent state to cache. attn_state (`tuple[torch.Te
fla/models/utils.py:220
↓ 2 callersMethoduses_default_config
Return True for modes that may fall back to default_config (FULL, DEFAULT, ALWAYS).
fla/ops/utils/cache.py:46
↓ 2 callersFunctionvenv_python
(venv_dir: Path)
scripts/smoke_test_split_packages.py:30
↓ 2 callersFunctionweight_quant
Per-tensor quantization to 1.58 bits. No grouping is needed for quantization. Args: w: A weight tensor with shape [d, k]. Retur
fla/modules/fused_bitlinear.py:47
↓ 1 callersMethod__init__
Initializes the BitLinear layer. Args: in_features: Size of each input sample. out_features: Size of each ou
fla/modules/fused_bitlinear.py:563
↓ 1 callersMethod__init__
If group_size is not None, we do GroupNorm with each group having group_size elements. group_size=None is equivalent to group_size=hidden_size
fla/modules/layernorm_gated.py:507
↓ 1 callersMethod__init__
( self, hidden_size: int, hidden_ratio: int | None = None, intermediate_size:
fla/modules/mlp.py:32
↓ 1 callersMethod__init__
( self, *, sequence_dim: int = 1, use_local_output: bool = False, )
fla/modules/fused_linear_cross_entropy.py:718
↓ 1 callersMethod__init__
( self, block_type: str = 'rodimus', mode: str = 'chunk', hidden_size: int = 1
fla/layers/rodimus.py:64
↓ 1 callersFunction_bench_at_ref
Run benchmarks at a git ref using a temporary worktree. Returns (results_list, machine_info_dict) or (None, None) on failure. Does NOT touch
benchmarks/ops/run.py:503
↓ 1 callersFunction_build_kda_bwd_kernel
( B, H, K, V, BT, BK, BV, hD1, hD2, dtype_str, STATE_V_FIRST, IS_VARLEN=False, num_warps=4
fla/ops/kda/backends/tilelang/chunk_bwd_dqkg.py:35
↓ 1 callersFunction_build_kernel
( B, H, K, V, BT, BK, BV, NK, hD1, hD2, dtype_str, USE_G,
fla/ops/common/backends/tilelang/chunk_bwd.py:21
↓ 1 callersFunction_build_parallel_attn_bwd_kernel
Build the TileLang JIT kernel for parallel attention backward. Note: B (batch size) is baked into the compiled kernel as a compile-time const
fla/ops/common/backends/tilelang/parallel_attn_bwd.py:20
↓ 1 callersFunction_build_parallel_attn_fwd_kernel
Build the TileLang JIT kernel for parallel attention forward. Note: B (batch size) is baked into the compiled kernel as a compile-time consta
fla/ops/common/backends/tilelang/parallel_attn_fwd.py:44
↓ 1 callersFunction_check_passing_vs_whole
( B: int, T: int, H: int, cu_seqlens: list[int] | None, dtype: torch.dtype, split_at:
tests/modules/test_token_shift.py:80
↓ 1 callersFunction_combine_params_log
Update rule for Titans in log space Parameters: - log_theta: log(θ) - log_alpha_complement: log(1-α) - log_eta: log(η) - seq
fla/ops/titans/log_impl.py:95
↓ 1 callersFunction_compare_and_swap
( x, ids, flip, i: tl.constexpr, n_dims: tl.constexpr, )
fla/ops/nsa/utils.py:21
↓ 1 callersMethod_compute_scale
(self, device=None)
fla/modules/rotary.py:406
↓ 1 callersFunction_copy_params
Copy parameters with matching names and shapes from src to dst.
tests/models/test_modeling_mamba.py:75
↓ 1 callersFunction_copy_params
Copy parameters with matching names and shapes from src to dst.
tests/models/test_modeling_mamba2.py:133
↓ 1 callersMethod_correct_dx_for_cp
Correct dx gradients for CP backward pass. Args: dx: Gradient tensor to be corrected, shape [1, T, D] grad_cache: Gra
fla/modules/token_shift_cp.py:91
↓ 1 callersMethod_correct_dx_for_cp
Correct dx gradients for CP backward pass by communicating with next rank. Args: dx: Gradient tensor to be corrected, shape [1, T
fla/modules/conv/cp/ops.py:76
↓ 1 callersFunction_create_yoco_config
( L: int, H: int, D: int, use_l2warp: bool = False, vocab_size: int = 1000, self_decod
tests/models/test_modeling_yoco.py:56
↓ 1 callersMethod_decode_incremental
r"""Non-windowed decode against a *pre-rescaled* KV cache. The cache holds ``(k_tilde, v, P[, gs_cumsum])`` rather than raw ``(k, v, g)``.
fla/layers/wall_attn.py:220
↓ 1 callersMethod_decode_rebuild
Windowed decode: rebuild the rescale from the cached raw suffix each step.
fla/layers/wall_attn.py:295
↓ 1 callersFunction_decode_ref
End-aligned causal reference: query i sits at absolute position Skv-Sq+i.
tests/ops/test_parallax.py:238
↓ 1 callersFunction_do_bench_kw
Triton ``do_bench`` uses warmup/rep in *milliseconds* of timed execution (see Triton docs).
benchmarks/ops/run.py:216
↓ 1 callersFunction_generate_constraints
(num_pack)
fla/ops/utils/softplus.py:57
↓ 1 callersFunction_generate_softplus
(num_pack)
fla/ops/utils/softplus.py:17
↓ 1 callersFunction_generate_softplus2
(num_pack)
fla/ops/utils/softplus.py:38
↓ 1 callersMethod_get_cached_state
Return this layer's cached ``attn_state`` tuple, or ``None`` if empty (prefill).
fla/layers/wall_attn.py:210
↓ 1 callersFunction_get_git_label
Return 'branch[short_sha]', e.g. 'main[abc1234]'.
benchmarks/ops/run.py:168
↓ 1 callersFunction_get_machine_info
()
benchmarks/ops/run.py:184
↓ 1 callersFunction_git_describe
()
benchmarks/benchmark_training_throughput.py:36
↓ 1 callersFunction_has_non_standard_layout
QKV-style views (stride_t != D) break triton-ascend masked kernels.
fla/modules/conv/triton/ops.py:25
← previousnext →401–500 of 2,989, ranked by callers