Physics of Language Models: Architecture Design and the Magic of Canon Layers