Please use this identifier to cite or link to this item: https://research.matf.bg.ac.rs/handle/123456789/470
Title: Neural Algorithmic Reasoners are Implicit Planners
Authors: Deac, Andreea
Velǐckovíc, Petar
Milinković, Ognjen 
Bacon, Pierre Luc
Tang, Jian
Nikolić, Mladen 
Affiliations: Informatics and Computer Science 
Informatics and Computer Science 
Issue Date: 1-Jan-2021
Rank: M33
Related Publication(s): Advances in Neural Information Processing Systems
Abstract: 
Implicit planning has emerged as an elegant technique for combining learned models of the world with end-to-end model-free reinforcement learning. We study the class of implicit planners inspired by value iteration, an algorithm that is guaranteed to yield perfect policies in fully-specified tabular environments. We find that prior approaches either assume that the environment is provided in such a tabular form-which is highly restrictive-or infer "local neighbourhoods" of states to run value iteration over-for which we discover an algorithmic bottleneck effect. This effect is caused by explicitly running the planning algorithm based on scalar predictions in every state, which can be harmful to data efficiency if such scalars are improperly predicted. We propose eXecuted Latent Value Iteration Networks (XLVINs), which alleviate the above limitations. Our method performs all planning computations in a high-dimensional latent space, breaking the algorithmic bottleneck. It maintains alignment with value iteration by carefully leveraging neural graph-algorithmic reasoning and contrastive self-supervised learning. Across eight low-data settings-including classical control, navigation and Atari-XLVINs provide significant improvements to data efficiency against value iteration-based implicit planners, as well as relevant model-free baselines. Lastly, we empirically verify that XLVINs can closely align with value iteration.
URI: https://research.matf.bg.ac.rs/handle/123456789/470
ISBN: 9781713845393
ISSN: 10495258
Appears in Collections:Research outputs

Show full item record

SCOPUSTM   
Citations

7
checked on Dec 24, 2024

Page view(s)

23
checked on Dec 24, 2024

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.