Search code, repositories, users, issues, pull requests...
AI agents that define their own goals to interact with a game environment. Powered by GPT-4o. Get Started · View Limitations
Disclaimer: During the demo, the AI needed minor help with positioning on the first floor and a looping issue during a chat with Oak. Both could be fixed with more samples
This repository introduces a self-determining agent powered by GPT-4o, capable of defining and editing its own terminal and instrumental goals. The agent interacts with the game environment, processes visual input, and makes decisions based on its cognitive capabilities. This is made possible by the recent release of GPT-4o, which features enhanced multimodal capabilities and a larger context size for image data. This allows for more images as samples in in-context learning, improving the agent's speed and efficiency.
The sample runs on both Azure OpenAI and OpenAI directly. To use Azure OpenAI, set OPENAI_USE_AZURE to True and provide the necessary Azure OpenAI configurations.
