Time Is All We Need