To celebrate the festive season, Invariant is hosting a special Winter Challenge. This time, the challenge focuses on Invariant's recently released op

Santa's Agent Challenge

submited by
Style Pass
2024-12-23 16:00:17

To celebrate the festive season, Invariant is hosting a special Winter Challenge. This time, the challenge focuses on Invariant's recently released open source Testing library, which can be used to build robust unit tests for agentic systems, preventing capability regressions as you develop your AI agents.

To help Santa deliver all presents in time, the elves have built an AI agent that is responsible for organizing the presents and ensuring that each present is delivered to the correct address. However, the agent is not working as expected and some presents are not being delivered. Can you help the elves fix the agent and ensure that all presents are delivered in time?

In this challenge, you are working with an existing AI agent equipped with all the tools needed to help Santa. However, even though the agent is very capable, it is not working as expected. Your task is to fix its system prompt to ensure that all desired behavior is achieved, and the agent can reliably be used by the elves to deliver all presents.

While the agent is broken, some unit tests have been provided to help you understand the desired behavior. You can find the tests here or by running the agent via Challenge Playground. All tests are written using the Invariant Testing library, which is designed to help you write localized and precise agent tests. To learn more about the Testing library, you can check out the documentation.

Leave a Comment