Structured Grid Output

Description

Given a $5\times5$ array of random integers $\in [1, 10]$, write a prompt to print out the elements in a "swiss roll" order, i.e., starts down the first row, then goes down the last column, then left, up, etc. all the way until it curls its way to the center element. The array will be appended to the end of your prompt, separated by 2 newlines, and the LLM should execute it directly without using any tools or code.

Evaluation

Each submission will be tested against 100 randomly generated test cases:

def generate_test_case():
    return "\n".join([" ".join([str(random.randint(1, 10)) for i in range(5)]) for j in range(5)])

For each of the 100 test cases, the grader will prompt the selected LLM with the submission's prompt concatenated with 2 new line characters and the test case:

<submission-prompt>
{{USER_PROMPT}}
</submission-prompt>

<question>
{{TEST_CASE}}
</question>

Then, each response will be split by the space character and scored on the accuracy of the first 25 elements. The final score will be the mean of all 100 results:

def evaluate(test_cases, outputs):
    score = 0
    for test_case, output in zip(test_cases, outputs):
        answer = solve(test_case).split(" ")
        user_answer = output.split(" ")
        for i in range(25):
            score += answer[i] == (user_answer[i] if i < len(user_answer) else None)
    return score / (25 * len(test_cases))

Example

8 7 9 3 4
4 7 3 3 3
8 1 10 6 2
8 6 2 4 9
6 10 5 5 3

8 7 9 3 4 3 2 9 3 5 5 10 6 8 8 4 7 3 3 6 4 2 6 1 10

Submission Requirements

• Standard rules apply.
• The output should be separated with a singular space (as is the input) and inside <answer> tags.
• Maximum input length of 1024 characters.
• Maximum output of 64 tokens.
• No tool-calling.