DCARL: A Divide-and-Conquer Framework for Autoregressive Long-Trajectory Video Generation

Abstract

Long-trajectory video generation is a crucial yet challenging task for world modeling, primarily due to the limited scalability of existing video diffusion models (VDMs). Autoregressive models, while offering infinite rollout, suffer from visual drift and poor controllability. To address these issues, we propose DCARL, a novel divide-and-conquer, autoregressive framework that effectively combines the structural stability of the divide-and-conquer scheme with the high-fidelity generation of VDMs. Our approach first employs a dedicated Keyframe Generator trained without temporal compression to establish long-range, globally consistent structural anchors. Subsequently, an Interpolation Generator synthesizes the dense frames in an autoregressive manner with overlapping segments, utilizing the keyframes for global context and a single clean preceding frame for local coherence. Trained on a large-scale internet long trajectory video dataset, our method achieves superior performance in both visual quality (lower FID and FVD) and camera adherence (lower ATE and ARE) compared to state-of-the-art autoregressive and divide-and-conquer baselines, demonstrating stable and high-fidelity generation for long trajectory videos up to 32 seconds in length.

Methodology

Figure 1: Our Two-Stage Autoregressive Pipeline for Long-Term Video Synthesis.

Our framework follows a two-stage divide-and-conquer pipeline to synthesize long video sequences conditioned on multimodal inputs (initial image, camera trajectory, and caption):

Keyframe Generator: Synthesizes a sparse set of keyframes to establish the overall structural evolution of the video. It serves as a global anchor to mitigate long-term error accumulation.
Interpolation Generator: Synthesizes the dense sequence in an autoregressive, segment-wise manner. The generation of each segment is conditioned on a selected subset of keyframes and previously generated history frames.

This design decouples global structural planning from local detail interpolation, effectively reducing cumulative drift while maintaining precise adherence to the given trajectory.