Flow Matching (from an applied robot learning perspective)

November 16, 2025

I wrote this given how satisfying I found the concept of flow matching to be, theoretically and empirically. I will walk through its basic theoretical derivation, recent variants, before finally running them on a toy example.

First, it helps to understand what a generative model is in the context of robotics. It is easy to see what that means in other areas: like vision, where you try to “generate” images, or in language, where a model “generates” text. In robotics, we want to “generate” actions that, if sent to a robot, would allow it to interact with the world in a purposeful way.

Flow matching is seen today as the state-of-the-art method for action generation. Compared to ACT, it can capture multi-modal behaviors rather than collapsing to a single mode. Unlike VQ-BeT, it operates directly in continuous action space. And relative to its closest predecessor, ADP, it offers faster and more stable inference.

Lets take a look at what a typical flow-matching based closed-loop inference look like at a high level, and build backwards from there:

1. Robot captures observations $o_i$$o_i$ from its environment.

2. The observation $o_i$$o_i$ gets encoded into a compant latent vector through an encoder $z_i=E_{\varphi }(o_i)$$z_i = E_\phi(o_i)$.

3. A random action $a_{i,0}$$a_{i,0}$ is sampled from a predetermined stochastic distribution $p_0(a)$$p_0(a)$

4. The sampled action $a_{i,0}$$a_{i,0}$ is evolved using a velocity field $v_{\theta }(a_{i,t},t,z)$$v_\theta(a_{i,t}, t, z)$, integrating it from $t=0\to 1$$t=0\rightarrow1$

5. The final evolved action $a_{i,1}$$a_{i,1}$ is sent to the robot for execution (sent open-loop if it's a chunk of actions)

6. Go to step 1

If we think of our expert actions $a_{i,1}$$a_{i,1}$ as coming from an unknown target distribution $p_1(a)$$p_1(a)$, and the initial action samples $a_{i,0}$$a_{i,0}$ that we begin our evolving process from to be drawn from a known prior distribution $p_0(a)$$p_0(a)$, then we can define a time-dependent probability density path $p_t(a)$$p_t(a)$ that connects both distributions smoothly as follows:

This simply means that at $t=0$$t = 0$, the distribution is our predefined prior $p_0(a)$$p_{0}(a)$, and at $t=1$$t = 1$, its the target distribution $p_1(a)$$p_1(a)$ that we assume represent the expert actions. As $t$$t$ increases from $0$$0$ to $1$$1$, samples from the distribution $p_t(a)$$p_t(a)$ gradually evolve from random noise toward our expert-like actions.

Since we know what the distribution is at $t=0$$t = 0$ (we predefine it), we need a mechanism that can continuously push it along a path toward the expert distribution as time progresses. This can be done by defining a velocity field $v_t(a)$$v_t(a)$, which specifies for every point $a$$a$ in the action space and time $t$$t$ an instantaneous direction so that the overall distribution gradually evolves toward $p_1(a)$$p_1(a)$. This velocity field is sufficient to construct a time-dependent flow $\varphi :[0,1]\times {\mathbb{R}}^d\to {\mathbb{R}}^d$$\phi : [0,1] \times \mathbb{R}^d \rightarrow \mathbb{R}^d$, defined by the following ordinary differential equation (ODE):

By now, the motivation for learning the velocity field $v_t$$v_t$ should be clear. We can use it to construct the flow ${\varphi }_t$$\phi_t$, which tells us how to evolve an action sampled from our known distribution over time. Ideally, this learned field should match the true underlying velocity field $u_t(a)$$u_t(a)$ that moves samples along the probability path $p_t(a)$$p_t(a)$. This leads us to the Flow Matching (FM) objective:

In practice, optimizing ${\mathcal{L}}_{FM}(\theta )$$\mathcal{L}_{FM}(\theta)$ is not directly possible, since it requires access to the true velocity field $u_t(a)$$u_t(a)$, which can only be computed if we knew all the intermediate distributions $p_t(a)$$p_t(a)$. In reality, we only have access to expert action samples $a_{i,1}$$a_{i,1}$ from the final distribution $p_1(a)$$p_1(a)$, making the direct FM objective intractable.

Instead of trying to model the entire probability path $p_t(a)$$p_t(a)$ and its vector field $u_t(a)$$u_t(a)$, however, we can simplify the problem by defining a conditional path $p_t(a\mid a_1)$$p_t(a \mid a_1)$ for each expert action $a_1$$a_1$ that describes how a single sample should move smoothly from the prior $p_0(a)$$p_0(a)$ toward a small neighborhood around that specific expert action. This new objective is referred to as the Conditional Flow Matching (CFM) objective:

Here, $u_t(a\mid a_1)$$u_t(a \mid a_{1})$ is a conditional target vector field that defines how samples from $p_0(a)$$p_0(a)$ should flow toward a data point $a_1$$a_{1}$. While it might seem like a suboptimal relaxation of the original FM objective to do this on a "per sample" level, it happens to be mathematically equivalent to optimizing this CFM loss and therefore leads to the same learned velocity field! There exist an infite number of ways to construct these conditional paths between random samples and expert actions, but a simple choice is to just define their mean and std to change linearly through time, giving us this conditional flow:

With this choice of conditional paths, we get what is reffered to as the Optimal-Transport Conditional Flow Matching (CFM) objective become:

Now that we have everything defined, we are ready to see how the generative model can actually be trained using data tuples of $(o_i,a_{i,1})$$(o_i, a_{i,1})$.

1. Draw a pair of $(a_{i,1},o_i)$$(a_{i,1}, o_{i})$ from our demonestrations dataset.

2. Encode the observation $o_i$$o_{i}$ using the encoder network $z_i = E_\phi(o_i).

3. Sample a random action $a_{i,0}$$a_{i,0}$ from our prior distribution $p_0(a)$$p_0(a)$, and a random time $t\sim \mathcal{U}(0,1)$$t \sim \mathcal{U}(0, 1)$.

4. Compute the intermediate action by linearly interpolating between the noise sample and the expert action:

   $$\begin{array}{}\text{(4)}& a_{i,t}=(1-(1-{\sigma }_{min})t)a_{i,0}+t{a}_{i,1}\end{array}$$

5. Compute the target (ground-truth) velocity for this interpolated action using the conditional flow formulation:

   $$\begin{array}{}\text{(5)}& u_t(a_{i,t}\mid a_{i,1})=\frac{a_{i,1}-a_{i,t}}{1-t}=a_{i,1}-(1-{\sigma }_{min})a_{i,0}\end{array}$$

6. Pass the interpolated sample $a_{i,t}$$a_{i,t}$, and the encoded observation $z_i$$z_i$ through the velocity model:

7. Compute the training loss using the Conditional Flow Matching objective:

8. Backpropagate the loss and update the model parameters $\theta$$\theta$ with gradient descent.