Publications

An activation-keyed delta-rule-based momentum update that adapts momentum’s forgetting rates to input direction frequencies, improving language-model pretraining efficiency without additional persistent optimizer state.

A dynamic rotary positional encoding scheme that adapts temporal positions to local acoustic context.

A control-theoretic framework linking Muon’s spectral equalization to lower cumulative training error.