| 2507.23083v1 | In this work, we propose CARoPE (ContextAware Rotary Positional Embedding), a novel generalization of RoPE that dynamically generates head-specific frequency patterns conditioned on token embeddings. | supported | has evidence row | full-text | For example, at a se- quence length of 1024, CARoPE reduces perplex- ity by more than 60% compared to RoPE in the GPT-Tiny model (36.74 vs. 81.27). | p1 | in-scope: taxonomy category match | pass; full-text verified; report=audit_report.md |
| 2511.09146v2 | •We show that RoPE’slow-frequency alignmentinduces attention heads with long-range dependency capability, while extrapolative heads are intrinsically low-rank and benefit from preserved positional encoding. | supported | has evidence row | full-text | Among many ap- proaches (Press et al., 2021; Chen et al., 2023b; Su et al., 2024; Peng et al., 2023; Wang et al., 2021), Rotary Position Embedding (RoPE) (Su et al., 2024) is widely used because it encodes rel- ative positions within dot-product attention and often extrapolates well to longer contexts. | p1 | in-scope: taxonomy category match | pass; full-text verified; report=audit_report.md |
| 2104.09864v5 | We introduce a novel method, namely Rotary Position Embedding(RoPE), to leverage the positional information into the learning process of PLMS. | supported | has evidence row | full-text | However, when increasing the maximum input text length to 1024, RoFormer outperforms WoBERT by an absolute improvement of 1.5%. | p2 | in-scope: taxonomy category match | pass; full-text verified; report=audit_report.md |
| 2604.09742v1 | While the rotation in RoPE can be efficiently implemented using matrix operations, the accompanying split and merge steps—implemented as vector operations—introduce non-negligible computational overhead. | supported | has evidence row | full-text | Metrics.We report bothspeedup timest 0/tand speedup percentage(t 0−t)/t 0, wheret 0denotes the baseline runtime andtdenotes the optimized runtime using RoME. 5.2. | p2 | in-scope: taxonomy category match | pass; full-text verified; report=audit_report.md |
| 2502.11664v4 | To overcome these issues, we propose Video Rotary Position Embedding (VRoPE), a novel positional encoding method tailored for Video-LLMs. | supported | has evidence row | full-text | Specifically, VRoPE achieves an accuracy that is 32.19 points higher than RoPE and 14.22 points higher than RoPE-3D when the number of input frames increases to 1024-1216. | p1 | in-scope: taxonomy category match | pass; full-text verified; report=audit_report.md |