What happens to model performance on the popular DeepSWE coding benchmark if you make small alterations to the questions, like changing a file name to lowercase? Models that once looked very similar to each other no longer look so similar. maxxbench.unlap.ai/research/why...
Private to this visit. Text and context are processed by AI.