MCPcopy Create free account

hub / github.com/fla-org/flash-linear-attention / types & classes

Types & classes403 in github.com/fla-org/flash-linear-attention

↓ 51 callersClassShortConvolution
Short convolution layer for efficient causal convolution operations. This class implements a depthwise 1D convolution with causal padding, de
fla/modules/conv/short_conv.py:24
↓ 48 callersClassRMSNorm
fla/modules/layernorm.py:1087
↓ 42 callersClassFusedCrossEntropyLoss
fla/modules/fused_cross_entropy.py:376
↓ 40 callersClassFusedLinearCrossEntropyLoss
fla/modules/fused_linear_cross_entropy.py:622
↓ 32 callersClassTensorSpec
Specification for generating a single benchmark input tensor. Args: shape_fn: (B, T, H, D, **kw) -> tuple of ints requires_
benchmarks/ops/registry.py:91
↓ 23 callersClassAttention
fla/layers/attn.py:37
↓ 20 callersClassFusedRMSNormGated
fla/modules/fused_norm_gate.py:992
↓ 17 callersClassOpConfig
Registry entry describing how to benchmark a single op. Args: name (str): Display and registry name, such as `chunk_gla`.
benchmarks/ops/registry.py:112
↓ 14 callersClassRotaryEmbedding
The rotary position embeddings from RoFormer_ (Su et. al). A crucial insight from the method is that the query and keys are transformed b
fla/modules/rotary.py:319
↓ 10 callersClassGroupNorm
fla/modules/layernorm.py:1023
↓ 7 callersClassFLACache
A cache used for storing hidden states produced by flash linear attention models. It stores the recurrent state of each layer; the exact sta
fla/models/utils.py:355
↓ 6 callersClassLoRA
fla/layers/rwkv6.py:211
↓ 6 callersClassRMSNormGated
fla/modules/layernorm_gated.py:505
↓ 5 callersClassDDLerpLinear
fla/layers/rwkv6.py:333
↓ 5 callersClassGatedDeltaNet
Gated Delta Networks (GDN) layer implementation. Reference: `Gated Delta Networks: Improving Mamba2 with Delta Rule <https://arxiv.org/abs/2
fla/layers/gated_deltanet.py:29
↓ 5 callersClassT2RFeatureMap
r""" Simple linear mapping feature map as in `Finetuning Pretrained Transformers into RNNs <https://arxiv.org/abs/2103.13076>`_
fla/modules/feature_map.py:70
↓ 4 callersClassBackendRegistry
Per-operation backend registry.
fla/ops/backends/__init__.py:95
↓ 4 callersClassFusedBitLinear
A custom linear layer that applies quantization on both activations and weights. This is primarily for training; kernel optimization is neede
fla/modules/fused_bitlinear.py:612
↓ 4 callersClassKimiDeltaAttention
Kimi Delta Attention (KDA) layer implementation. Reference: `Kimi Linear: An Expressive, Efficient Attention Architecture <https://arxiv.org
fla/layers/kda.py:28
↓ 4 callersClassLayerNorm
fla/modules/layernorm.py:970
↓ 3 callersClassFusedKLDivLoss
fla/modules/fused_kl_div.py:317
↓ 3 callersClassGroupNormRef
fla/modules/layernorm.py:124
↓ 3 callersClassHadamardFeatureMap
fla/modules/feature_map.py:123
↓ 3 callersClassHedgehogFeatureMap
r""" Hedgehog feature map as introduced in `The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry <https://arxiv.org
fla/modules/feature_map.py:42
↓ 3 callersClassLerpLinear
fla/layers/rwkv6.py:298
↓ 3 callersClassMamba
Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]
fla/layers/mamba.py:46
↓ 3 callersClassYOCORotaryEmbedding
fla/layers/yoco.py:33
↓ 2 callersClassABCAttention
fla/layers/abc.py:26
↓ 2 callersClassCache
fla/models/utils.py:550
↓ 2 callersClassComba
The layer implementaion for [Comba: Improving Bilinear RNNs with Closed-loop Control](https://arxiv.org/abs/2506.02475). Similar to Mamba2 a
fla/layers/comba.py:29
↓ 2 callersClassDPFPFeatureMap
r""" Deterministic Parameter-Free Projection (DPFP) feature map in `Linear Transformers Are Secretly Fast Weight Programmers <https://arxiv.or
fla/modules/feature_map.py:101
↓ 2 callersClassDeltaNet
r""" The layer implementaion for [Parallelizing Linear Transformers with the Delta Rule over Sequence Length](https://arxiv.org/abs/2406.06484).
fla/layers/delta_net.py:36
↓ 2 callersClassFLACPContext
FLA Context Parallel Context - Operator-level context management.
fla/ops/cp/context.py:23
↓ 2 callersClassFLALayer
fla/models/utils.py:30
↓ 2 callersClassGatedDeltaProduct
Generalized version of GatedDoubleDeltaNet that supports arbitrary number of householder transformations.
fla/layers/gated_deltaproduct.py:30
↓ 2 callersClassGatedLinearAttention
r""" The layer implementaion for [Gated Linear Attention Transformers with Hardware-Efficient Training](https://arxiv.org/abs/2312.06635). # noqa
fla/layers/gla.py:28
↓ 2 callersClassGatedSlotAttention
fla/layers/gsa.py:30
↓ 2 callersClassHGRN2Attention
fla/layers/hgrn2.py:31
↓ 2 callersClassHGRNAttention
fla/layers/hgrn.py:29
↓ 2 callersClassLightNetAttention
fla/layers/lightnet.py:30
↓ 2 callersClassLinearAttention
fla/layers/linear_attn.py:26
↓ 2 callersClassLogLinearMamba2
Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]
fla/layers/log_linear_mamba2.py:272
↓ 2 callersClassMamba2
Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]
fla/layers/mamba2.py:105
↓ 2 callersClassMamba2Config
This is the configuration class to store the configuration of a [`Mamba2Model`]. It is used to instantiate a MAMBA2 model according to the sp
fla/models/mamba2/configuration_mamba2.py:27
↓ 2 callersClassMamba2ForCausalLM
fla/models/mamba2/modeling_mamba2.py:333
↓ 2 callersClassMamba3Config
Configuration for the FLA port of Mamba-3. Mamba-3 differs from Mamba-2 in that it removes the 1D causal convolution, adds per-head bias
fla/models/mamba3/configuration_mamba3.py:13
↓ 2 callersClassMamba3ForCausalLM
fla/models/mamba3/modeling_mamba3.py:249
↓ 2 callersClassMesaNet
The layer implementaion for [MesaNet: Sequence Modeling by Locally Optimal Test-Time Training]. # noqa Args: hidden_size (int, Opti
fla/layers/mesa_net.py:28
↓ 2 callersClassMomAttention
The layer implementaion for [MoM: Linear Sequence Modeling with Mixture-of-Memories](https://arxiv.org/abs/2502.13685).
fla/layers/mom.py:280
↓ 2 callersClassMultiScaleRetention
r""" The layer implementaion for [Retentive Network: A Successor to Transformer for Large Language Models](https://arxiv.org/pdf/2307.08621.pdf).
fla/layers/multiscale_retention.py:29
↓ 2 callersClassRWKV6Attention
fla/layers/rwkv6.py:30
↓ 2 callersClassRWKV7Attention
fla/layers/rwkv7.py:32
↓ 2 callersClassRaven
fla/layers/raven.py:42
↓ 2 callersClassReLUFeatureMap
fla/modules/feature_map.py:252
↓ 2 callersClassRodimusAttention
fla/layers/rodimus.py:63
↓ 2 callersClassSwishFeatureMap
fla/modules/feature_map.py:285
↓ 2 callersClassTileLangBackend
fla/ops/common/backends/tilelang/__init__.py:21
↓ 1 callersClassABCBlock
fla/models/abc/modeling_abc.py:41
↓ 1 callersClassABCModel
fla/models/abc/modeling_abc.py:233
↓ 1 callersClassBitAttention
fla/layers/bitattn.py:38
↓ 1 callersClassBitNetBlock
fla/models/bitnet/modeling_bitnet.py:83
↓ 1 callersClassBitNetMLP
fla/models/bitnet/modeling_bitnet.py:42
↓ 1 callersClassBitNetModel
fla/models/bitnet/modeling_bitnet.py:267
↓ 1 callersClassCachedAutotuner
A modified autotuner that loads best config from FLA's config directory. This class extends Triton's Autotuner but overrides the run method
fla/ops/utils/cache.py:337
↓ 1 callersClassCombaBlock
fla/models/comba/modeling_comba.py:42
↓ 1 callersClassCombaModel
fla/models/comba/modeling_comba.py:244
↓ 1 callersClassDataCollatorForLanguageModeling
Data collator used for language modeling. Args: tokenizer ([`PreTrainedTokenizer`] or [`PreTrainedTokenizerFast`]): The
legacy/training/flame/data.py:165
↓ 1 callersClassDeltaFormerAttention
r""" The layer implementation for DeltaFormer, [Understanding Transformer from the Perspective of Associative Memory] (https://arxiv.org/p
fla/layers/deltaformer.py:27
↓ 1 callersClassDeltaFormerBlock
fla/models/deltaformer/modeling_deltaformer.py:39
↓ 1 callersClassDeltaFormerModel
fla/models/deltaformer/modeling_deltaformer.py:183
↓ 1 callersClassDeltaNetBlock
fla/models/delta_net/modeling_delta_net.py:41
↓ 1 callersClassDeltaNetModel
fla/models/delta_net/modeling_delta_net.py:233
↓ 1 callersClassDependencyFinder
scripts/find_dependent_tests.py:141
↓ 1 callersClassDummyCache
tests/layers/test_layer_cache_layer_idx.py:37
↓ 1 callersClassFLACacheGenerator
Runs FLA kernels to populate a Triton autotune cache for a given head_dim.
scripts/utils/autotune_generate.py:20
↓ 1 callersClassFlashKDABackend
Copyright (c) 2026 Moonshot AI Fused CUTLASS forward (replaces the multi-kernel Triton path). https://github.com/MoonshotAI/FlashKDA Ena
fla/ops/kda/backends/flashkda.py:24
↓ 1 callersClassForgettingAttention
fla/layers/forgetting_attn.py:30
↓ 1 callersClassForgettingTransformerBlock
fla/models/forgetting_transformer/modeling_forgetting_transformer.py:41
↓ 1 callersClassForgettingTransformerModel
fla/models/forgetting_transformer/modeling_forgetting_transformer.py:226
↓ 1 callersClassFusedLayerNormGated
fla/modules/fused_norm_gate.py:925
↓ 1 callersClassGLABlock
fla/models/gla/modeling_gla.py:42
↓ 1 callersClassGLAModel
fla/models/gla/modeling_gla.py:235
↓ 1 callersClassGSABlock
fla/models/gsa/modeling_gsa.py:42
↓ 1 callersClassGSAModel
fla/models/gsa/modeling_gsa.py:236
↓ 1 callersClassGatedDeltaNet2
r""" Gated DeltaNet 2 (GDN-2) layer. GDN-2 extends KDA's scalar-beta erase gate to two independent channel-wise gates: an erase gate ``b`
fla/layers/gdn2.py:35
↓ 1 callersClassGatedDeltaNetBlock
fla/models/gated_deltanet/modeling_gated_deltanet.py:42
↓ 1 callersClassGatedDeltaNetModel
fla/models/gated_deltanet/modeling_gated_deltanet.py:245
↓ 1 callersClassGatedDeltaProductBlock
fla/models/gated_deltaproduct/modeling_gated_deltaproduct.py:42
↓ 1 callersClassGatedDeltaProductConfig
fla/models/gated_deltaproduct/configuration_gated_deltaproduct.py:13
↓ 1 callersClassGatedDeltaProductModel
fla/models/gated_deltaproduct/modeling_gated_deltaproduct.py:232
↓ 1 callersClassGatedMLP
fla/modules/mlp.py:30
↓ 1 callersClassGroupNormLinear
fla/modules/layernorm.py:1301
↓ 1 callersClassHGRN2Block
fla/models/hgrn2/modeling_hgrn2.py:42
↓ 1 callersClassHGRN2Model
fla/models/hgrn2/modeling_hgrn2.py:230
↓ 1 callersClassHGRNBlock
fla/models/hgrn/modeling_hgrn.py:42
↓ 1 callersClassHGRNModel
fla/models/hgrn/modeling_hgrn.py:229
↓ 1 callersClassIntraCardCPBackend
Intra-card context parallel backend for chunk_gated_delta_rule_fwd_h.
fla/ops/common/backends/intracard.py:29
↓ 1 callersClassKDABlock
fla/models/kda/modeling_kda.py:42
↓ 1 callersClassKDAModel
fla/models/kda/modeling_kda.py:259
↓ 1 callersClassKDATileLangBackend
fla/ops/kda/backends/tilelang/__init__.py:17
next →1–100 of 403, ranked by callers