Microsoft has accidentally revealed GPT-6.1 Sol uses the same base weights as GPT-6 Sol, but with only 2 inference passes instead of 3. : r/singularity
this is correct, i have worked on looped transformers and made a few <100m parameter models and basically from 3 loop you start to see diminishing returns What would make the most sense to me is if the above write-up were simply an error. What would make sense is if GPT-6.1 Sol were the same as GPT-6.0 Astra, but with fewer iterations of the loop (2 instead of 3). That automatically makes it cheaper and (assuming that 3 iters work well for Astra) slightly less capable. It also fits well with the statement that the "real" 6.1 got shelved (couldn't be brought up to standard until dev day, officially for safety reasons), but they simply re-released an already approved, capable model (6.0) as their 6.1 Sol, but dialled down the loop in order to make it cheaper. This makes sense. The posted tweet doesn't really. Unless "base model weights" just means that they did some extra reinforcement learning on top of 6.0 Sol (with 2 iterations), which ended up improving the model. > Unless 'base mod