Paper notes: σ-GPTs, A New Approach to Autoregressive ModelsSeptember 14, 2024 Authors Arnaud Pannatier, Evann Courdier, François Fleuret Paper arxiv.org/abs/2404.09562 Code github.com/idiap/sigma-gpt Demo arnaudpannatier.ch/sigma-gpt/ Year 2024 read more
Paper notes: Longformer, The Long-Document TransformerNovember 23, 2023 Authors Iz Beltagy, Matthew E. Peters, Arman Cohan Paper arxiv.org/abs/2004.05150 Code github.com/allenai/longformer Year 2020 Longformer is a 2020 attempt to address the efficiency problem of self-attention. Self-attention, where each token of an input sequence attends to every other token, has an inherent quadratic time and memory complexity. Longformer addresses this by attending mostly locally. read more
Paper notes: Reasoning over Paths in Knowledge Bases using Reinforcement LearningNovember 07, 2023 Authors Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, Andrew McCallum Paper arxiv.org/abs/1711.05851 Code github.com/shehzaadzd/MINERVA/tree/master Year 2017 This paper deals with question answering using information stored in knowledge graphs. read more