Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/fla-org/flash-linear-attention
/ types & classes
Types & classes
403 in github.com/fla-org/flash-linear-attention
⨍
Functions
2,989
◇
Types & classes
403
↳
Endpoints
219
↓ 51 callers
Class
ShortConvolution
Short convolution layer for efficient causal convolution operations. This class implements a depthwise 1D convolution with causal padding, de
fla/modules/conv/short_conv.py:24
↓ 48 callers
Class
RMSNorm
fla/modules/layernorm.py:1087
↓ 42 callers
Class
FusedCrossEntropyLoss
fla/modules/fused_cross_entropy.py:376
↓ 40 callers
Class
FusedLinearCrossEntropyLoss
fla/modules/fused_linear_cross_entropy.py:622
↓ 32 callers
Class
TensorSpec
Specification for generating a single benchmark input tensor. Args: shape_fn: (B, T, H, D, **kw) -> tuple of ints requires_
benchmarks/ops/registry.py:91
↓ 23 callers
Class
Attention
fla/layers/attn.py:37
↓ 20 callers
Class
FusedRMSNormGated
fla/modules/fused_norm_gate.py:992
↓ 17 callers
Class
OpConfig
Registry entry describing how to benchmark a single op. Args: name (str): Display and registry name, such as `chunk_gla`.
benchmarks/ops/registry.py:112
↓ 14 callers
Class
RotaryEmbedding
The rotary position embeddings from RoFormer_ (Su et. al). A crucial insight from the method is that the query and keys are transformed b
fla/modules/rotary.py:319
↓ 10 callers
Class
GroupNorm
fla/modules/layernorm.py:1023
↓ 7 callers
Class
FLACache
A cache used for storing hidden states produced by flash linear attention models. It stores the recurrent state of each layer; the exact sta
fla/models/utils.py:355
↓ 6 callers
Class
LoRA
fla/layers/rwkv6.py:211
↓ 6 callers
Class
RMSNormGated
fla/modules/layernorm_gated.py:505
↓ 5 callers
Class
DDLerpLinear
fla/layers/rwkv6.py:333
↓ 5 callers
Class
GatedDeltaNet
Gated Delta Networks (GDN) layer implementation. Reference: `Gated Delta Networks: Improving Mamba2 with Delta Rule <https://arxiv.org/abs/2
fla/layers/gated_deltanet.py:29
↓ 5 callers
Class
T2RFeatureMap
r""" Simple linear mapping feature map as in `Finetuning Pretrained Transformers into RNNs <https://arxiv.org/abs/2103.13076>`_
fla/modules/feature_map.py:70
↓ 4 callers
Class
BackendRegistry
Per-operation backend registry.
fla/ops/backends/__init__.py:95
↓ 4 callers
Class
FusedBitLinear
A custom linear layer that applies quantization on both activations and weights. This is primarily for training; kernel optimization is neede
fla/modules/fused_bitlinear.py:612
↓ 4 callers
Class
KimiDeltaAttention
Kimi Delta Attention (KDA) layer implementation. Reference: `Kimi Linear: An Expressive, Efficient Attention Architecture <https://arxiv.org
fla/layers/kda.py:28
↓ 4 callers
Class
LayerNorm
fla/modules/layernorm.py:970
↓ 3 callers
Class
FusedKLDivLoss
fla/modules/fused_kl_div.py:317
↓ 3 callers
Class
GroupNormRef
fla/modules/layernorm.py:124
↓ 3 callers
Class
HadamardFeatureMap
fla/modules/feature_map.py:123
↓ 3 callers
Class
HedgehogFeatureMap
r""" Hedgehog feature map as introduced in `The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry <https://arxiv.org
fla/modules/feature_map.py:42
↓ 3 callers
Class
LerpLinear
fla/layers/rwkv6.py:298
↓ 3 callers
Class
Mamba
Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]
fla/layers/mamba.py:46
↓ 3 callers
Class
YOCORotaryEmbedding
fla/layers/yoco.py:33
↓ 2 callers
Class
ABCAttention
fla/layers/abc.py:26
↓ 2 callers
Class
Cache
fla/models/utils.py:550
↓ 2 callers
Class
Comba
The layer implementaion for [Comba: Improving Bilinear RNNs with Closed-loop Control](https://arxiv.org/abs/2506.02475). Similar to Mamba2 a
fla/layers/comba.py:29
↓ 2 callers
Class
DPFPFeatureMap
r""" Deterministic Parameter-Free Projection (DPFP) feature map in `Linear Transformers Are Secretly Fast Weight Programmers <https://arxiv.or
fla/modules/feature_map.py:101
↓ 2 callers
Class
DeltaNet
r""" The layer implementaion for [Parallelizing Linear Transformers with the Delta Rule over Sequence Length](https://arxiv.org/abs/2406.06484).
fla/layers/delta_net.py:36
↓ 2 callers
Class
FLACPContext
FLA Context Parallel Context - Operator-level context management.
fla/ops/cp/context.py:23
↓ 2 callers
Class
FLALayer
fla/models/utils.py:30
↓ 2 callers
Class
GatedDeltaProduct
Generalized version of GatedDoubleDeltaNet that supports arbitrary number of householder transformations.
fla/layers/gated_deltaproduct.py:30
↓ 2 callers
Class
GatedLinearAttention
r""" The layer implementaion for [Gated Linear Attention Transformers with Hardware-Efficient Training](https://arxiv.org/abs/2312.06635). # noqa
fla/layers/gla.py:28
↓ 2 callers
Class
GatedSlotAttention
fla/layers/gsa.py:30
↓ 2 callers
Class
HGRN2Attention
fla/layers/hgrn2.py:31
↓ 2 callers
Class
HGRNAttention
fla/layers/hgrn.py:29
↓ 2 callers
Class
LightNetAttention
fla/layers/lightnet.py:30
↓ 2 callers
Class
LinearAttention
fla/layers/linear_attn.py:26
↓ 2 callers
Class
LogLinearMamba2
Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]
fla/layers/log_linear_mamba2.py:272
↓ 2 callers
Class
Mamba2
Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1]
fla/layers/mamba2.py:105
↓ 2 callers
Class
Mamba2Config
This is the configuration class to store the configuration of a [`Mamba2Model`]. It is used to instantiate a MAMBA2 model according to the sp
fla/models/mamba2/configuration_mamba2.py:27
↓ 2 callers
Class
Mamba2ForCausalLM
fla/models/mamba2/modeling_mamba2.py:333
↓ 2 callers
Class
Mamba3Config
Configuration for the FLA port of Mamba-3. Mamba-3 differs from Mamba-2 in that it removes the 1D causal convolution, adds per-head bias
fla/models/mamba3/configuration_mamba3.py:13
↓ 2 callers
Class
Mamba3ForCausalLM
fla/models/mamba3/modeling_mamba3.py:249
↓ 2 callers
Class
MesaNet
The layer implementaion for [MesaNet: Sequence Modeling by Locally Optimal Test-Time Training]. # noqa Args: hidden_size (int, Opti
fla/layers/mesa_net.py:28
↓ 2 callers
Class
MomAttention
The layer implementaion for [MoM: Linear Sequence Modeling with Mixture-of-Memories](https://arxiv.org/abs/2502.13685).
fla/layers/mom.py:280
↓ 2 callers
Class
MultiScaleRetention
r""" The layer implementaion for [Retentive Network: A Successor to Transformer for Large Language Models](https://arxiv.org/pdf/2307.08621.pdf).
fla/layers/multiscale_retention.py:29
↓ 2 callers
Class
RWKV6Attention
fla/layers/rwkv6.py:30
↓ 2 callers
Class
RWKV7Attention
fla/layers/rwkv7.py:32
↓ 2 callers
Class
Raven
fla/layers/raven.py:42
↓ 2 callers
Class
ReLUFeatureMap
fla/modules/feature_map.py:252
↓ 2 callers
Class
RodimusAttention
fla/layers/rodimus.py:63
↓ 2 callers
Class
SwishFeatureMap
fla/modules/feature_map.py:285
↓ 2 callers
Class
TileLangBackend
fla/ops/common/backends/tilelang/__init__.py:21
↓ 1 callers
Class
ABCBlock
fla/models/abc/modeling_abc.py:41
↓ 1 callers
Class
ABCModel
fla/models/abc/modeling_abc.py:233
↓ 1 callers
Class
BitAttention
fla/layers/bitattn.py:38
↓ 1 callers
Class
BitNetBlock
fla/models/bitnet/modeling_bitnet.py:83
↓ 1 callers
Class
BitNetMLP
fla/models/bitnet/modeling_bitnet.py:42
↓ 1 callers
Class
BitNetModel
fla/models/bitnet/modeling_bitnet.py:267
↓ 1 callers
Class
CachedAutotuner
A modified autotuner that loads best config from FLA's config directory. This class extends Triton's Autotuner but overrides the run method
fla/ops/utils/cache.py:337
↓ 1 callers
Class
CombaBlock
fla/models/comba/modeling_comba.py:42
↓ 1 callers
Class
CombaModel
fla/models/comba/modeling_comba.py:244
↓ 1 callers
Class
DataCollatorForLanguageModeling
Data collator used for language modeling. Args: tokenizer ([`PreTrainedTokenizer`] or [`PreTrainedTokenizerFast`]): The
legacy/training/flame/data.py:165
↓ 1 callers
Class
DeltaFormerAttention
r""" The layer implementation for DeltaFormer, [Understanding Transformer from the Perspective of Associative Memory] (https://arxiv.org/p
fla/layers/deltaformer.py:27
↓ 1 callers
Class
DeltaFormerBlock
fla/models/deltaformer/modeling_deltaformer.py:39
↓ 1 callers
Class
DeltaFormerModel
fla/models/deltaformer/modeling_deltaformer.py:183
↓ 1 callers
Class
DeltaNetBlock
fla/models/delta_net/modeling_delta_net.py:41
↓ 1 callers
Class
DeltaNetModel
fla/models/delta_net/modeling_delta_net.py:233
↓ 1 callers
Class
DependencyFinder
scripts/find_dependent_tests.py:141
↓ 1 callers
Class
DummyCache
tests/layers/test_layer_cache_layer_idx.py:37
↓ 1 callers
Class
FLACacheGenerator
Runs FLA kernels to populate a Triton autotune cache for a given head_dim.
scripts/utils/autotune_generate.py:20
↓ 1 callers
Class
FlashKDABackend
Copyright (c) 2026 Moonshot AI Fused CUTLASS forward (replaces the multi-kernel Triton path). https://github.com/MoonshotAI/FlashKDA Ena
fla/ops/kda/backends/flashkda.py:24
↓ 1 callers
Class
ForgettingAttention
fla/layers/forgetting_attn.py:30
↓ 1 callers
Class
ForgettingTransformerBlock
fla/models/forgetting_transformer/modeling_forgetting_transformer.py:41
↓ 1 callers
Class
ForgettingTransformerModel
fla/models/forgetting_transformer/modeling_forgetting_transformer.py:226
↓ 1 callers
Class
FusedLayerNormGated
fla/modules/fused_norm_gate.py:925
↓ 1 callers
Class
GLABlock
fla/models/gla/modeling_gla.py:42
↓ 1 callers
Class
GLAModel
fla/models/gla/modeling_gla.py:235
↓ 1 callers
Class
GSABlock
fla/models/gsa/modeling_gsa.py:42
↓ 1 callers
Class
GSAModel
fla/models/gsa/modeling_gsa.py:236
↓ 1 callers
Class
GatedDeltaNet2
r""" Gated DeltaNet 2 (GDN-2) layer. GDN-2 extends KDA's scalar-beta erase gate to two independent channel-wise gates: an erase gate ``b`
fla/layers/gdn2.py:35
↓ 1 callers
Class
GatedDeltaNetBlock
fla/models/gated_deltanet/modeling_gated_deltanet.py:42
↓ 1 callers
Class
GatedDeltaNetModel
fla/models/gated_deltanet/modeling_gated_deltanet.py:245
↓ 1 callers
Class
GatedDeltaProductBlock
fla/models/gated_deltaproduct/modeling_gated_deltaproduct.py:42
↓ 1 callers
Class
GatedDeltaProductConfig
fla/models/gated_deltaproduct/configuration_gated_deltaproduct.py:13
↓ 1 callers
Class
GatedDeltaProductModel
fla/models/gated_deltaproduct/modeling_gated_deltaproduct.py:232
↓ 1 callers
Class
GatedMLP
fla/modules/mlp.py:30
↓ 1 callers
Class
GroupNormLinear
fla/modules/layernorm.py:1301
↓ 1 callers
Class
HGRN2Block
fla/models/hgrn2/modeling_hgrn2.py:42
↓ 1 callers
Class
HGRN2Model
fla/models/hgrn2/modeling_hgrn2.py:230
↓ 1 callers
Class
HGRNBlock
fla/models/hgrn/modeling_hgrn.py:42
↓ 1 callers
Class
HGRNModel
fla/models/hgrn/modeling_hgrn.py:229
↓ 1 callers
Class
IntraCardCPBackend
Intra-card context parallel backend for chunk_gated_delta_rule_fwd_h.
fla/ops/common/backends/intracard.py:29
↓ 1 callers
Class
KDABlock
fla/models/kda/modeling_kda.py:42
↓ 1 callers
Class
KDAModel
fla/models/kda/modeling_kda.py:259
↓ 1 callers
Class
KDATileLangBackend
fla/ops/kda/backends/tilelang/__init__.py:17
next →
1–100 of 403, ranked by callers