Return ``True`` iff ``block.attn`` is a softmax-attention camctrl variant (``_SoftmaxUCPESinglePathLiteLA`` or registered aliases). D

Return the post-ReLU key scale used by frame-wise GDN.

Create a 2D spatial blend mask with linear ramps in overlap regions. Interior pixels get weight 1. Pixels in the overlap band of a non-edge

Move optimizer state to the specified device.

Normalize the poses such that the ref0 camera is the identity and the ref1 camera is unit distance to the ref0 camera.

Z stream: uses K (no RoPE). Cheaper than KV — no V load, no RoPE compute, no K_pair derivation.

Compute 1/RMS for one component of QKV over the full C = H*D channel dim. Args: qkv: (B, N, 3, H, D) idx: 0 for Q, 1 for K, 2 for

Project Q/K/V for the softmax main branch, apply norm and RoPE. Returns post-norm, post-RoPE, post-bf16 cast tensors without running SDPA, so

Run ``F.scaled_dot_product_attention(q, k, v)`` with FA-friendly head_dim padding. FlashAttention-2 only supports head_dim in {32, 64, 128, 256}.

"Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at t

Diffusion forward corruption process. Input: - clean_latent: the clean latent with shape [B*T, C, H, W] - noi

Compute the noised input xt = alpha_t * x + sigma_t * noise. Args: x: A `torch.Tensor` with shape `(batch_size, *shape)`

Diffusion forward corruption process. Input: - clean_latent: the clean latent with shape [B*T, C, H, W] - noi

Add all parameters of module to the params list. The parameters of the given module will be added to the list of param groups, with s

Add a transformation to the dataset.

Run all_gather on arbitrary picklable data (not necessarily tensors) Args: data: any picklable object Returns: list[data]

Analyze chunk pattern and return vectorization strategy. Detects special patterns that allow hybrid vectorization: - uniform: All chunks equa

q: [B, Lq, Nq, C1]. k: [B, Lk, Nk, C1]. v: [B, Lk, Nk, C2]. Nq must be divisible by Nk. q_lens

r"""require setting 'batch_per_epoch' before building optimizer & lr_scheduler

Get current cache mode.

Compute the entire variational lower-bound, measured in bits-per-dim, as well as other related quantities. :param model: the

Drop-in for fused_gdn_chunkwise.cam_scan_bidi_chunkwise. CUDA fast path when head_dim==128 & fp32 contiguous inputs (the deployed sana_wm con

Drop-in for fused_gdn_chunkwise.cam_scan_chunkwise (the STREAMING/cached causal cam scan). CUDA cam_phase_a + cam_phase_c (read q/k/v [B,H,D,N] di

Sum a forward camera scan and a separately-gated reverse scan. Chunk-causal camera attention needs the reverse branch to use boundary-masked

Chatbot function to generate responses based on the given prompt and optional image.

Check whether the current kv_cache is valid

Returns binned height and width.

Remove old checkpoints if the number exceeds max_checkpoints. Only the main process performs the actual deletion to avoid race conditions

Clear the current mask

Collect cached modules from a single transformer block

Compute the log-likelihood of a continuous Gaussian distribution. :param x: the targets :param means: the Gaussian mean Tensor. :para

Convert the diffusion network's noise prediction to x0 predidction. noise: the predicted noise with shape [B, C, H, W] xt: th

Convert the diffusion network's noise prediction to x0 predidction. noise: the predicted noise with shape [B, C, H, W] xt: th