LaMP: Learning Robust Latent Motion Prior for Optimization-Based Human Motion Generation

Xiaozhong Lyu Korrawe Karunratanakul Kaifeng Zhao Siyu Tang

ETH Zürich

International Conference on 3D Vision (3DV), 2026

Paper Code BibTeX

LaMP provides a universal latent motion prior for motion blending, editing, collision avoidance, and refinement — Our proposed **Latent Motion Prior (LaMP)** framework provides a robust, universal latent motion prior suitable for both text-to-motion generation and optimization-based downstream tasks. **Motion Blending:** LaMP generates a smooth transition (yellow) between two distinct input motions (blue), such as standing and performing a cartwheel. **Motion Editing:** the model adapts a motion sequence to reach a specific target position (red sphere). **Collision Avoidance:** given an input motion that intersects with an obstacle (left), LaMP optimizes the trajectory to avoid the obstacle while preserving the original motion semantics (right). **Motion Refinement:** LaMP removes artifacts from a noisy input (left) to produce a physically plausible, refined motion (right).

Abstract

We present Latent Motion Prior (LaMP), a novel framework for learning a generalizable human motion prior that enables efficient optimization for a wide range of motion-related tasks, including text-to-motion generation, motion editing, motion blending, motion refinement, and environment-aware collision avoidance.

LaMP employs a body part-based encoder to learn a disentangled latent representation of human motion, together with a masked training strategy that encourages the model to capture the most informative structural and dynamic aspects of the motion. As a result, LaMP produces a robust and expressive latent space that serves as a latent motion prior across diverse downstream tasks.

We evaluate the learned representation on a wide range of optimization-based downstream tasks. Experimental results show that the current families of text-to-motion models are generally not suitable to serve as a motion prior, while LaMP consistently outperforms the state-of-the-art methods across all optimization tasks.

Method

LaMP architecture: masked motion autoencoder, latent diffusion model, and optimization-based downstream tasks — **Approach overview of LaMP.** Our key insight is that the quality of a diffusion-based motion prior heavily depends on the structure of its latent space. A part-based encoder preserves local kinematic detail while a spatial-temporal masking strategy distills the most informative features from redundant raw motion, yielding a compact yet expressive latent space.

Stage (a) — Masked Motion Autoencoder. We first train a masked autoencoder to learn a robust latent space for human motion. Each training sequence is encoded using six part-based encoders and one full-body encoder that combines them. During training, a portion of the motion sequence is randomly masked and the model reconstructs it from the remaining visible parts.
Stage (b) — Latent Diffusion Model. We freeze the pretrained encoder to map motion data into the latent space, then train a latent diffusion model following the standard text-to-motion setting.
Stage (c) — Optimization-Based Downstream Tasks. The resulting model serves as a general motion prior for downstream tasks — motion refinement, editing, blending, and collision avoidance — where the desired motion is found by optimizing the latent diffusion noise.

Results

Text-to-Motion Generation

On HumanML3D, LaMP achieves performance comparable to state-of-the-art diffusion models for text-to-motion. While VQ-VAE-based methods obtain the best raw generation scores, they cannot be optimized toward downstream objectives. In the tasks below we show that although some diffusion baselines achieve higher text-to-motion metrics, their latent distributions are far less robust for optimization — leading to failures on the downstream motion tasks where LaMP excels.

Motion Blending

Given a naïve concatenation of two distinct test motions, the goal is to generate a smooth transition between them. LaMP produces natural transitions from its motion prior, while EMDM fails to produce transition frames, MDM outputs unnatural poses, and MLD misses the upcoming motion.

The two input motions are shown in blue; the generated transition is shown in white/yellow. Each panel compares EMDM, MDM, MLD, and LaMP (ours) on the same input.

Two motion-blending examples. Only LaMP generates a coherent, natural transition between the two input motions while preserving their style.

Motion Refinement

Given a noisy input motion, refinement projects it back onto the motion prior to remove artifacts and recover a realistic sequence. LaMP produces smooth results, whereas EMDM still outputs noisy motion, MDM yields low-quality motion, and MLD loses pose details.

The noisy input motion is shown in blue; each method outputs the refined motion. Panels compare EMDM, MDM, MLD, and LaMP (ours).

Two motion-refinement examples. LaMP removes noise while staying faithful to the original input, producing physically plausible, temporally smooth motion.

Motion Editing

Given a test motion and a sampled target location, the task edits the motion so its pelvis reaches the target while preserving the original motion style. LaMP reaches the target while retaining the input style; MLD fails to reach the target, and MDM alters the motion semantics. EMDM is excluded here because it is not invertible.

The input test motion is shown in white and the sampled target location is the red dot. Panels compare MDM, MLD, and LaMP (ours).

Two motion-editing examples. LaMP reaches the sampled target position while preserving the semantics and style of the original motion.

SDF Collision Avoidance

Given a test motion surrounded by obstacles, the task optimizes the motion to avoid collisions while preserving its semantics. LaMP preserves the motion style and drives the collision loss to zero. MLD lacks fine-grained control, and MDM avoids the obstacle but produces floating artifacts.

The input test motion is shown in white and obstacles are the yellow spheres. The per-method collision loss is annotated above each panel (MLD, MDM, and LaMP, ours).

Two collision-avoidance examples. LaMP steers the body clear of the obstacles (collision loss 0.00) while keeping the original motion style intact.

Citation

@inproceedings{lyu2026lamp,
  title     = {LaMP: Learning Robust Latent Motion Prior for
               Optimization-Based Human Motion Generation},
  author    = {Lyu, Xiaozhong and Karunratanakul, Korrawe and
               Zhao, Kaifeng and Tang, Siyu},
  booktitle = {International Conference on 3D Vision (3DV)},
  year      = {2026}
}