Evaluating SotA LLM Models trying to solve a net-new LeetCode style puzzle
I am sure a lot of you would have seen this particular meme template. It has given rise to entire genre of tiktoks where girls are amazed at how much calculation guys do to pick which stall to go to in a crowded row of urinals.
I actually even made a whole game to place people on urinal stalls using various code-generating tools like bolt.new to have fun with this
Anyway, that led me to wonder, how would today’s (that is Jan 2025) state of the art LLM models fare on solving a LeetCode-style puzzle of optimally placing people in a row of urinal stalls which are already partially occupied.
There are 8 urinal stalls and some people are already using some of them. We have to place one new person in that stall. The current state of the urinal stalls is represented by a 8 digit number like 01100110 which means there are people occupying stalls 2, 3,6 and 7 After the new person is placed at a position, the final "comfort" score is calculated. We go from one end to another, and for every person (i.e. a '1') we do the following
For the following existing urinal stall configurations, give the answer that in which position should the new person be added, and what will be the total comfort score of that configurations. If there are more than one positions where the highest comfort score can be achieved, any of those positions is a correct answer. (stalls are counted 1-indexed, starting from left).
