MCPcopy Create free account

hub / github.com/fla-org/flash-linear-attention / functions

Functions2,989 in github.com/fla-org/flash-linear-attention

↓ 1 callersFunctionintra_chunk_preprocess_bwd_fn
(q, k, w, w2, beta, dq, dk, dA_local, dw1,
fla/ops/path_attn/intra_chunk_preprocess_bwd.py:121
↓ 1 callersFunctionintra_chunk_preprocess_bwd_prepare_fn
(q, k, v, w, beta, g_cumsum, A, L, D, do, scale, return_h=True, cu_seqlens=None,
fla/ops/path_attn/intra_chunk_preprocess_bwd_prepare.py:153
↓ 1 callersFunctionintra_chunk_preprocess_fwd_fn
(q, k, v, w, beta, g_cumsum, A, scale, BT, cu_seqlens, chunk_indices: torch.
fla/ops/path_attn/intra_chunk_preprocess_fwd.py:132
↓ 1 callersFunctionintracard_fwd_h
( k: torch.Tensor, w: torch.Tensor, u: torch.Tensor, g: torch.Tensor | None = None, gk: to
fla/ops/common/intracard_cp.py:435
↓ 1 callersFunctionintracard_merge
Merge sub-sequence states using pre-computed parameters. All CPU-side preparation (cumsum, offset lists) is done in the caller using pure Pyt
fla/ops/common/intracard_cp.py:294
↓ 1 callersFunctionintracard_pre_scan
( kg: torch.Tensor, w: torch.Tensor, u: torch.Tensor, g: torch.Tensor | None, gk: torch.Te
fla/ops/common/intracard_cp.py:253
↓ 1 callersFunctionis_backend_class
Check if an AST node is a class inheriting from BaseBackend.
scripts/find_dependent_tests.py:69
↓ 1 callersFunctionis_header_line
(line: str)
scripts/check_header.py:56
↓ 1 callersFunctionis_power_of_2
(n)
fla/modules/feature_map.py:38
↓ 1 callersFunctionk_update_bwd
( grad_out: torch.Tensor, k: torch.Tensor, a: torch.Tensor, ka: torch.Tensor, cu_seqlens:
fla/ops/rwkv7/fused_k_update.py:267
↓ 1 callersFunctionk_update_fwd
( k: torch.Tensor, a: torch.Tensor, ka: torch.Tensor, cu_seqlens: torch.Tensor | None = None,
fla/ops/rwkv7/fused_k_update.py:215
↓ 1 callersFunctionkda_gate_fwd
( g: torch.Tensor, A_log: torch.Tensor, dt_bias: torch.Tensor | None = None, lower_bound: floa
fla/ops/kda/gate.py:210
↓ 1 callersMethodkeys_fuzzy_match
(cached_key: Any, requested_key: Any)
fla/ops/utils/cache.py:142
↓ 1 callersFunctionl2norm
( x: torch.Tensor, eps: float = 1e-6, output_dtype: torch.dtype | None = None, )
fla/modules/l2norm.py:266
↓ 1 callersFunctionlayer_norm_bwd
( dy: torch.Tensor, x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, eps: float,
fla/modules/fused_bitlinear.py:319
↓ 1 callersFunctionlayer_norm_bwd
( dy: torch.Tensor, x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, eps: float,
fla/modules/layernorm_gated.py:303
↓ 1 callersFunctionlayer_norm_bwd_npu
( dy: torch.Tensor, x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, mean: torch
fla/modules/backends/triton_ascend/layernorm.py:232
↓ 1 callersFunctionlayer_norm_fwd
( x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, eps: float, z: torch.Tensor =
fla/modules/layernorm_gated.py:118
↓ 1 callersFunctionlayer_norm_fwd_npu
( x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, eps: float = 1e-5, residual:
fla/modules/backends/triton_ascend/layernorm.py:175
↓ 1 callersFunctionlayer_norm_fwd_quant
( x: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, eps: float, residual: torch.T
fla/modules/fused_bitlinear.py:143
↓ 1 callersFunctionlayer_norm_gated
( x: torch.Tensor, g: torch.Tensor, weight: torch.Tensor, bias: torch.Tensor, activation:
fla/modules/fused_norm_gate.py:821
↓ 1 callersFunctionlayer_norm_swish_gate_linear
( x: torch.Tensor, g: torch.Tensor, norm_weight: torch.Tensor, norm_bias: torch.Tensor, li
fla/modules/fused_norm_gate.py:871
↓ 1 callersFunctionlayernorm_fn
(x, weight, bias, z=None, eps=1e-6, group_size=None, norm_before_gate=True, is_rms_norm=False)
fla/modules/layernorm_gated.py:462
↓ 1 callersFunctionleaky_relu
(x)
fla/ops/utils/matmul.py:158
↓ 1 callersFunctionlevel_lut
(BT, device)
fla/ops/log_linear_attn/chunk.py:1476
↓ 1 callersFunctionlist_autotune_cache_files
(triton_cache_dir: Path)
scripts/extract_triton_autotune_cache.py:55
↓ 1 callersFunctionload_balancing_loss_func
r""" Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch. See Switch Transformer (https://huggingface.co
fla/models/mom/modeling_mom.py:41
↓ 1 callersFunctionload_cached_config
Load cached best config for a kernel from FLA configs directory. This function loads the cached best configuration for a given kernel name
fla/ops/utils/cache.py:271
↓ 1 callersFunctionload_config_file
(config_file: Path)
fla/ops/utils/cache.py:255
↓ 1 callersFunctionload_kernel_config_file_uncached
Read a kernel config file directly from disk. Extract runs in the same process as optional cache generation. During generation, FLA may cache
scripts/utils/autotune_export.py:95
↓ 1 callersMethodload_state_dict
(self, state_dict)
legacy/training/flame/data.py:154
↓ 1 callersMethodload_state_dict
Override the load_state_dict method to handle migration from version 1 to version 2. Handles hierarchical keys like 'model.layers.0.a
fla/models/rwkv7/modeling_rwkv7.py:313
↓ 1 callersFunctionlogsigmoid
(x: torch.Tensor, temperature: float = 1.)
fla/modules/activations.py:327
↓ 1 callersFunctionlogsigmoid_bwd
( x: torch.Tensor, dy: torch.Tensor, temperature: float = 1., output_contiguous: bool = False,
fla/modules/activations.py:287
↓ 1 callersFunctionlogsigmoid_bwd_npu
( x: torch.Tensor, dy: torch.Tensor, temperature: float = 1., output_contiguous: bool = False,
fla/modules/backends/triton_ascend/activations.py:398
↓ 1 callersFunctionlogsigmoid_fwd
(x: torch.Tensor, temperature: float = 1., output_contiguous: bool = False)
fla/modules/activations.py:269
↓ 1 callersFunctionlogsigmoid_fwd_npu
(x: torch.Tensor, temperature: float = 1., output_contiguous: bool = False)
fla/modules/backends/triton_ascend/activations.py:381
↓ 1 callersFunctionlogsumexp_fwd
( x, scale: float | None = None, softcapping: float | None = None, dtype: torch.dtype | None =
fla/modules/fused_linear_cross_entropy.py:69
↓ 1 callersMethodlookup_fuzzy
(self, key: AutotuneKey)
fla/ops/utils/cache.py:245
↓ 1 callersFunctionmain
()
legacy/training/run.py:19
↓ 1 callersFunctionmain
()
scripts/smoke_test_split_packages.py:143
↓ 1 callersFunctionmain
()
scripts/extract_triton_autotune_cache.py:81
↓ 1 callersFunctionmain
()
scripts/run_benchmark_compare.py:310
↓ 1 callersFunctionmain
()
scripts/check_header.py:118
↓ 1 callersFunctionmain
Build split packages and copy to target directory.
scripts/build_packages.py:303
↓ 1 callersFunctionmain
()
evals/ppl.py:173
↓ 1 callersFunctionmain
()
benchmarks/ops/run.py:560
↓ 1 callersFunctionmain
()
benchmarks/cp/test_gdn_with_cp.py:903
↓ 1 callersFunctionmain
()
benchmarks/cp/benchmark_kda_cp8_vs_cp2tp.py:439
↓ 1 callersFunctionmake_header
(end_year: int)
scripts/check_header.py:38
↓ 1 callersFunctionmap_triton_backend_to_torch_device
()
fla/utils/_device.py:104
↓ 1 callersFunctionmasks
(BT, device)
fla/ops/log_linear_attn/chunk.py:1484
↓ 1 callersFunctionmaybe_build
(builder)
tests/layers/test_layer_cache_layer_idx.py:44
↓ 1 callersMethodmaybe_load_cached_config
(self, key: AutotuneKey)
fla/ops/utils/cache.py:380
↓ 1 callersFunctionmean_pooling_bwd
( do: torch.Tensor, batch_size: int, seq_len: int, chunk_size: int, cu_seqlens: torch.Long
fla/ops/utils/pooling.py:137
↓ 1 callersFunctionmean_pooling_fwd
( x: torch.Tensor, chunk_size: int, cu_seqlens: torch.LongTensor | None = None, chunk_indices:
fla/ops/utils/pooling.py:110
↓ 1 callersMethodmerge
(self, result: AutotuneResult)
scripts/utils/autotune_export.py:148
↓ 1 callersFunctionnaive_attnres
r""" Apply AttnRes residual aggregation. AttnRes normalizes each residual source with RMSNorm, scores it against `query`, applies softmax ove
fla/ops/attnres/naive.py:17
↓ 1 callersFunctionnaive_causal_attention_bhtd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, )
fla/ops/deltaformer/naive.py:42
↓ 1 callersFunctionnaive_chunk_kda
r""" Args: q (torch.Tensor): Queries of shape ``[B, T, H, K]``. k (torch.Tensor): Keys of shape ``[B, T, H
fla/ops/kda/naive.py:69
↓ 1 callersFunctionnaive_chunk_linear_attn
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, scale: float | None = None, normalize
fla/ops/linear_attn/naive.py:47
↓ 1 callersFunctionnaive_deltaformer_attn_head_first
Naive reference implementation of DeltaFormer attention for head-first format. Two-stage process: 1. Computes u[i] = v[i] - beta[i] * su
fla/ops/deltaformer/naive.py:58
↓ 1 callersFunctionnaive_mesa_net_decoding_one_step
(q, k, v, g, lamb, beta, prev_h_kk, prev_h_kv, max_CG_iteration=30)
fla/ops/mesa_net/naive.py:12
↓ 1 callersFunctionnaive_parallel_based
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, scale: float | None = None, use_norm:
fla/ops/based/naive.py:12
↓ 1 callersFunctionnaive_parallel_simple_gla
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g: torch.Tensor, scale: float | None
fla/ops/simple_gla/naive.py:97
↓ 1 callersFunctionnorm_residual
(x, weight, bias, eps)
fla/ops/ttt/fused_chunk.py:719
↓ 1 callersFunctionnorm_residual
(x, weight, bias, eps)
fla/ops/ttt/chunk.py:1364
↓ 1 callersFunctionpack_sequence
( x: torch.Tensor, cu_seqlens: torch.Tensor, padding_side: str = 'left', )
fla/ops/utils/pack.py:189
↓ 1 callersMethodpad_for_conv
(self, cu_seqlens, cu_q, cu_k, cu_v)
fla/layers/mom.py:763
↓ 1 callersFunctionparallel_attn_bwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, o: torch.Tensor, g_cumsum: torch.Tens
fla/ops/attn/parallel.py:595
↓ 1 callersFunctionparallel_attn_bwd_tilelang
( q, k, v, o, g_cumsum, lse, do, sink_bias=None, scale=None, window_size=None, chunk_size=128, cu_
fla/ops/common/backends/tilelang/parallel_attn_bwd.py:237
↓ 1 callersFunctionparallel_attn_fwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g_cumsum: torch.Tensor, sink_bias: to
fla/ops/attn/parallel.py:509
↓ 1 callersFunctionparallel_attn_fwd_tilelang
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g_cumsum: torch.Tensor | None, sink_b
fla/ops/common/backends/tilelang/parallel_attn_fwd.py:243
↓ 1 callersFunctionparallel_deltaformer_kernel
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, u: torch.Tensor, w: torch.Tensor,
fla/ops/deltaformer/parallel.py:103
↓ 1 callersFunctionparallel_nsa_block_mask
( block_indices: torch.LongTensor, block_counts: torch.LongTensor | int, cu_seqlens: torch.LongTen
fla/ops/nsa/parallel.py:601
↓ 1 callersFunctionparallel_nsa_bwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, o: torch.Tensor, lse: torch.Tensor,
fla/ops/nsa/parallel.py:628
↓ 1 callersFunctionparallel_nsa_compression
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, block_size: int = 64, scale: float =
fla/ops/nsa/compression.py:530
↓ 1 callersFunctionparallel_nsa_compression_bwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, o: torch.Tensor, lse: torch.Tensor,
fla/ops/nsa/compression.py:378
↓ 1 callersFunctionparallel_nsa_compression_fwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, block_size: int, scale: float, cu
fla/ops/nsa/compression.py:325
↓ 1 callersFunctionparallel_nsa_fwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, block_indices: torch.LongTensor, bloc
fla/ops/nsa/parallel.py:547
↓ 1 callersFunctionparallel_nsa_topk
( q: torch.Tensor, k: torch.Tensor, lse: torch.Tensor, block_counts: torch.LongTensor | int,
fla/ops/nsa/parallel.py:500
↓ 1 callersFunctionparallel_parallax_bwd
Parallax backward (Triton). Returns grads matching `q, r, k, v`.
fla/ops/parallax/parallel.py:674
↓ 1 callersFunctionparallel_parallax_fwd
Parallax forward (Triton). `(B, T, HQ, D)` / packed `(1, T_total, HQ, D)` inputs. Returns `(o, barv, d1, bart, m)`: `o`/`barv` in the input dtype
fla/ops/parallax/parallel.py:645
↓ 1 callersFunctionparallel_path_bwd_dkv_fn
( q, k, v, g_cumsum, do, dv, dg_cumsum, hc_whole, scale, L, D, cu_seqlens, S, BT, BS, chun
fla/ops/path_attn/parallel_path_bwd_inter_dkv.py:146
↓ 1 callersFunctionparallel_path_bwd_dq_fn
( q, k, v, g_cumsum, do, dg_cumsum, hc_whole, scale, L, D, cu_seql
fla/ops/path_attn/parallel_path_bwd_inter_dqh.py:142
↓ 1 callersFunctionparallel_path_bwd_intra_chunk_fn
( q, k, v, g_cumsum, w1, w2, dq, dk, dv, dg_cumsum, dw1, dw2, do, scale, L, D, cu_seqlens,
fla/ops/path_attn/parallel_path_bwd_intra.py:152
↓ 1 callersFunctionparallel_path_fwd_fn
( q, k, v, o, g_cumsum, w1, w2, scale, L, M, cu_seqlens, BT,
fla/ops/path_attn/parallel_path_fwd.py:150
↓ 1 callersFunctionparallel_rebased
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, eps: float = 1e-5, use_scale: bool =
fla/ops/rebased/parallel.py:443
↓ 1 callersFunctionparallel_simple_gla_bwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g: torch.Tensor, do: torch.Tensor,
fla/ops/simple_gla/parallel.py:553
↓ 1 callersFunctionparallel_simple_gla_bwd_kernel_dkv
( i_t, i_k, i_v, q, k, v, g, do, dk, dv, dg, scale, T,
fla/ops/simple_gla/parallel.py:257
↓ 1 callersFunctionparallel_simple_gla_bwd_kernel_dq
( i_t, i_k, i_v, q, k, v, g, do, dq, dg, scale, T, H: tl.c
fla/ops/simple_gla/parallel.py:166
↓ 1 callersFunctionparallel_simple_gla_fwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g: torch.Tensor, scale: float, ou
fla/ops/simple_gla/parallel.py:482
↓ 1 callersFunctionparallel_wall_attn_bwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, o: torch.Tensor, g_cumsum: torch.Tens
fla/ops/wall_attn/parallel.py:667
↓ 1 callersFunctionparallel_wall_attn_fwd
( q: torch.Tensor, k: torch.Tensor, v: torch.Tensor, g_cumsum: torch.Tensor, sink_bias: to
fla/ops/wall_attn/parallel.py:574
↓ 1 callersFunctionprepare_cu_seqlens_from_mask
( mask: torch.BoolTensor, dtype: torch.dtype | None = torch.int32, )
fla/ops/utils/index.py:59
↓ 1 callersFunctionprepare_k_cache_fn
(k, w1, w2, cu_seqlens, BS, use_cache=False, chunk_indices: torch.LongTensor | None = None)
fla/ops/path_attn/prepare_k_cache.py:59
↓ 1 callersFunctionprepare_maxlens
(cu_seqlens: torch.LongTensor)
fla/modules/token_shift.py:373
↓ 1 callersFunctionprepare_moba_chunks
r"""Split a packed variable-length batch into fixed-size MoBA chunks and emit the bookkeeping needed by `parallel_moba`. Each sample in `cu_s
fla/ops/moba/parallel.py:28
↓ 1 callersFunctionprepare_subseq_cu_seqlens
Insert subseq split points into original cu_seqlens. Optimized: uses pure Python loops instead of torch tensor operations for the small index
fla/ops/common/intracard_cp.py:183
↓ 1 callersFunctionprepare_wy_repr_bwd
( k: torch.Tensor, v: torch.Tensor, beta: torch.Tensor, A: torch.Tensor, dw: torch.Tensor,
fla/ops/gated_delta_rule/wy_fast.py:269
← previousnext →801–900 of 2,989, ranked by callers