Args: x: [B, 224, 224, 3] preprocessed image (channels-last). Returns: [B, 257, 1280] CLS + patch token feat

Forward pass for t2v and i2v. Args: x: Input latent [F, H, W, C_in] (channels-last) t: Timestep [1]

Compute binned relative position bias

Uses SAM to calculate the image embedding for an image, and then allow repeated, efficient mask prediction given prompts. Ar

Predicts masks given an image and prompt embeddings, using a transformer architecture. Args: transformer_dim (in

Using a SAM model, generates masks for the entire image. Generates a grid of point prompts over the image, then filters low q

Args: img_size (int): Input image size. patch_size (int): Patch size. in_chans (int): Number of input ima

Args: dim (int): Number of input channels. num_heads (int): Number of attention heads in each ViT block.

Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. qkv_bias (bool):

Args: kernel_size (Tuple): kernel size of the projection layer. stride (Tuple): stride of the projection layer.

A transformer decoder that attends to an input image using queries whose positional embedding is supplied. Args:

A transformer block with four layers: (1) self-attention of sparse inputs, (2) cross attention of sparse inputs to dense inputs, (3)

SAM predicts object masks from an image and input prompts. Args: vision_encoder (ImageEncoderViT): The backbone used to

Encodes prompts for input to SAM's mask decoder. Args: embed_dim (int): The prompts' embedding dimension ima