Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
A couple of sampled outputs from GPT-4: Merlin:1+2,Arthur:3,Lancelot:0,Merlin:3+4,Arthur:7,Lancelot:1 and Merlin:1+2,Arthur:3,Lancelot:0,Merlin:3+4,Arthur:7,Lancelot:1,Merlin:5+6,Arthur:11,Lancelot:2,Merlin:7+8,Arthur:15,Lancelot:3,Merlin:9+10,Arthur:19,Lancelot:4
The functionality is useful in the product/real life settings as basic sequence following and interplay of multiple actors is of importance to many real life scenarios.
🚨 Please make sure your PR follows these guidelines, failure to follow the guidelines below will result in the PR being closed automatically. Note that even if the criteria are met, that does not guarantee the PR will be merged nor GPT-4 access granted. 🚨
In order for a PR to be merged, it must fail on GPT-4. We are aware that right now, users do not have access, so you will not be able to tell if the eval fails or not. Please run your eval with GPT-3.5-Turbo, but keep in mind as we run the eval, if GPT-4 gets higher than 90% on the eval, we will likely reject since GPT-4 is already capable of completing the task.