Window Token Transformer: Can learnable window token help window-based transformer build better long-range interactions?

Mao, JW; Chang, YQ; Yin, XS

Yin, XS (通讯作者),Wenzhou Inst Hangzhou Dianzi Univ, Wenzhou 325038, Peoples R China.

NEUROCOMPUTING, 2023; 559 ():

Abstract

Compared with the vanilla transformer, the window -based transformer offers a better trade-off between accuracy and efficiency. Although the window -b......

Full Text Link