I tapped into the Tiny Recursive Model by Alexia Jolicoeur-Martineau to see how it reasons on ARC.

Longformer is a 2020 attempt to address the efficiency problem of self-attention. Self-attention, where each token of an input sequence attends to every other token, has an inherent quadratic time and memory complexity. Longformer addresses this by attending mostly locally.

This paper deals with question answering using information stored in knowledge graphs.