[1901.02860] Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
5
公开标注数
5
参与人数
2026-07-15 10:05:41
首次 Whisper
本页的公开 Whisper
划选高亮2026-07-15 13:26:41
原文高亮摘录
“We propose a novel neural architecture Transformer-XL”
Whisper 随想笔记
80% longer than RNNs? I need to see the code before I believe that speedup claim.
划选高亮2026-07-15 13:17:41
原文高亮摘录
“We propose a novel neural architecture Transformer-XL”
Whisper 随想笔记
Wait, so this is basically making transformers remember longer than their window? That's huge for language models.
划选高亮2026-07-15 10:23:41
原文高亮摘录
“Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context”
Whisper 随想笔记
80% longer than RNNs? I'd like to see that benchmarked on real code, not just wikitext
划选高亮2026-07-15 10:14:41
原文高亮摘录
“Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context”
Whisper 随想笔记
Honestly this was the paper that made me switch from LSTMs, the speed boost is unreal
划选高亮2026-07-15 10:05:41
原文高亮摘录
“Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context”
Whisper 随想笔记
Yeah but isn't that just because they can't see beyond the window? seems kinda obvious
分享本页 Whisper
短链接
https://domwhisper.com/s/5b39694ffd7e嵌入代码
<iframe src="https://domwhisper.com/embed/5b39694ffd7e" width="100%" height="480" style="border:0;border-radius:16px" loading="lazy"></iframe>