Intro2 months back, I started the Prime Residency with Florian Brand as my peer or shall I say my verifier. My first task was to port the Agents’ Last Exam Linux CLI subset to verifiers. Then one day Florian asked me to look into the data and this quest slowly evolved into finding issues in the benchmark and the making of what we call ALE-Gold. It was more challenging than I expected. I was new to this kind of work, so there was some skill issue on my part. Along the way, I also hit benchmark defects, inference provider failures,...