RL in Name Only? Analyzing the Structural Assumptions in RL Post-Training