Evidence-ledger draft

Evidence-Ledger Synthesis of Transformer Position Encoding Evolution — Claim ledger

CSV-backed claim ledger tying paper claims to paper IDs and evidence status.

paper_idclaimclaim_statusevidence_statussource_depthsource_quotepage_or_sectiontaxonomy_fitaudit_status
2507.23083v1In this work, we propose CARoPE (ContextAware Rotary Positional Embedding), a novel generalization of RoPE that dynamically generates head-specific frequency patterns conditioned on token embeddings.supportedhas evidence rowfull-textFor example, at a se- quence length of 1024, CARoPE reduces perplex- ity by more than 60% compared to RoPE in the GPT-Tiny model (36.74 vs. 81.27).p1in-scope: taxonomy category matchpass; full-text verified; report=audit_report.md
2511.09146v2•We show that RoPE’slow-frequency alignmentinduces attention heads with long-range dependency capability, while extrapolative heads are intrinsically low-rank and benefit from preserved positional encoding.supportedhas evidence rowfull-textAmong many ap- proaches (Press et al., 2021; Chen et al., 2023b; Su et al., 2024; Peng et al., 2023; Wang et al., 2021), Rotary Position Embedding (RoPE) (Su et al., 2024) is widely used because it encodes rel- ative positions within dot-product attention and often extrapolates well to longer contexts.p1in-scope: taxonomy category matchpass; full-text verified; report=audit_report.md
2104.09864v5We introduce a novel method, namely Rotary Position Embedding(RoPE), to leverage the positional information into the learning process of PLMS.supportedhas evidence rowfull-textHowever, when increasing the maximum input text length to 1024, RoFormer outperforms WoBERT by an absolute improvement of 1.5%.p2in-scope: taxonomy category matchpass; full-text verified; report=audit_report.md
2604.09742v1While the rotation in RoPE can be efficiently implemented using matrix operations, the accompanying split and merge steps—implemented as vector operations—introduce non-negligible computational overhead.supportedhas evidence rowfull-textMetrics.We report bothspeedup timest 0/tand speedup percentage(t 0−t)/t 0, wheret 0denotes the baseline runtime andtdenotes the optimized runtime using RoME. 5.2.p2in-scope: taxonomy category matchpass; full-text verified; report=audit_report.md
2502.11664v4To overcome these issues, we propose Video Rotary Position Embedding (VRoPE), a novel positional encoding method tailored for Video-LLMs.supportedhas evidence rowfull-textSpecifically, VRoPE achieves an accuracy that is 32.19 points higher than RoPE and 14.22 points higher than RoPE-3D when the number of input frames increases to 1024-1216.p1in-scope: taxonomy category matchpass; full-text verified; report=audit_report.md