(Left) Visualization of learned goal-conditioned values on antmaze-large-stitch. The red star denotes the goal position. The GCIVL agent with MLP-parameterized value exhibits incorrect generalization, while QRL with IQE does not.
(Middle) Ablation experiments on value function architecture. The inductive bias of IQE effectively mitigates overgeneralization.
(Right) Performance in maze navigation and robotic manipulation tasks. No single method shows consistently high performance.
While the standard MLP-parameterized value function suffers from value overgeneralization, value functions with structural inductive bias can mitigate overgeneralization.
Our experiments lead to two key conclusions.
(i) The QRL objective induces severe training instability in robotic manipulation tasks, highlighting the necessity of TD learning for achieving robust performance across diverse domains.
(ii) While GCIVL with IQE improves upon GCIVL with MLP by mitigating overgeneralization, the IQE itself can degrade performance in some tasks.
We propose Latent Alignment Network (LAN), a value-function architecture that aligns goal-conditioned value estimates with the geometry of a learned latent space. The core idea is to represent the goal-conditioned value as the negative Euclidean distance between a state embedding and a goal embedding.
Here, \(\varphi_S : \mathcal{S} \rightarrow \mathbb{R}^d\) and \(\varphi_G : \mathcal{G} \rightarrow \mathbb{R}^d\) denote the state and goal representation networks, respectively. Unlike metric-based representations that use a shared embedding function, LAN adopts asymmetric representations, where \(\varphi_S \neq \varphi_G\).
GCIVL with LAN outperforms alternative architectures, including quasimetric architectures (IQE and MRN), the metric-based parameterization (Hilbert), and MLP. Moreover, these performance gains are consistent across diverse tasks, addressing the limitations of existing value function architectures.
We train the LAN-parameterized value function using an expectile TD objective, following the IVL framework. This objective propagates sparse goal-reaching rewards through temporal-difference learning while preserving the distance-based structure imposed by LAN.
However, in long-horizon tasks, TD learning alone can produce sharp local fluctuations in the value function because sparse rewards must propagate over many steps. To stabilize value learning, we introduce a local continuity regularization term that penalizes excessive value changes between neighboring states.
The final objective combines the expectile TD loss with the local continuity regularizer:
This regularization encourages locally smooth value functions while preserving global Bellman consistency, leading to more stable value learning in sparse-reward goal-conditioned tasks.
The MLP variant denotes the HIQL baseline. On maze navigation tasks, the IQE, MRN, and Hilbert variants outperform the MLP baseline, and achieve performance comparable to LAN in antmaze-giant. These results suggest that architectural inductive biases can improve long-horizon value learning, supporting our Findings 1. However, in the scene dataset, LAN achieves an 88% success rate, whereas IQE, MRN, and Hilbert remain below 20%, and the MLP baseline stays below 40%. Overall, these results indicate that LAN outperforms existing quasimetric architectures within the same hierarchical framework.
| Algorithm | Medium | Large | Giant | Rel. Drop (%) ↓ |
|---|---|---|---|---|
| LAVL (ours) | 94 | 85 | 85 | 9.6% |
| OTA | 89 | 78 | 68 | 23.1% |
| CGCIVL | 84 | 66 | 35 | 58.3% |
| QRL | 58 | 45 | 22 | 61.7% |
| HIQL | 87 | 51 | 21 | 75.4% |
| GCIVL | 48 | 16 | 0 | 100.0% |
Average success rate (%) for different maze sizes in OGBench. Relative Drop (Rel. Drop) denotes the percentage decrease in success rate from medium to giant. These results demonstrate that LAVL maintains a robust success rate as the horizon increases, with only minimal performance degradation.
OTA, CGCIVL, and HIQL exhibit average relative drops of 18.6%, 19.1%, and 36.9%, respectively. In contrast, LAVL shows a much smaller drop of 1.1%, indicating improved robustness to trajectory fragmentation and stitching. These results highlight that LAVL is less sensitive to dataset-induced horizon truncation, which is critical for long-horizon goal reaching under offline data constraints.
Latent Dimension: LAVL exhibits stable performance across a wide range of latent dimensions, from 16 to 256. This robustness simplifies practical deployment, as careful tuning of the latent dimension is unnecessary.
Continuity Regularization: continuity regularization yields clear performance gains on challenging tasks, by stabilizing local fluctuation in the value function. Its effect is most pronounced on pointmaze-giant, where the success rate increases from 35% to 95%, and it also improves performance onantmaze-giant and scene.
We implement a variant of LAVL, termed LAVL-HV (Hierarchical Value), which uses separate value functions for the highlevel and low-level policies. Given that LAVL-HV differs from HIQL only by replacing the high-level value function, this result indicates that the value function of LAVL provides a more informative high-level learning signal for high-level policy learning. Moreover, LAVL consistently outperforms LAVL-HV, suggesting that a unitary LAVL value function also benefits low-level policy learning through tighter coupling with the subgoal representation.
@article{kang2026latent,
title={Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning},
author={Kang, Hyungkyu and Kim, Byeongchan and Oh, Min-hwan},
journal={arXiv preprint arXiv:2605.25740},
year={2026}
}