LLMs struggle to explain themselves
The interactive demo below generates a random program and uses that to compute three number sequences. The LLM is given two of those as examples and asked to pick the third out of a lineup. Click expand to see the actual messages sent to/from the LLM. You can run things yourself if you click Settings and enter an Anthropic API key: check out the FAQ for more details!
So it’s probably a good idea not to connect this demo to a malevolent superintelligent LLM that knows zero-day exploits for your browser. Having said that:
To provide an Anthropic API key, go to api.headers.x-api-key and replace the {{TODO: ...}} placeholder. The settings are just YAML. The api section comes after the prompts so you’ll have to scroll a little.
You can change a lot through the settings: not just the prompts, but also the HTTP requests the demo makes (by default it’s configured to use Claude 3.5 Sonnet by Anthropic). For example, to remove the eval_js tool, you can set "tools": [] in the value of api.body.
You should be able to configure the demo to hit e.g. OpenAI or Mistral’s APIs just by changing settings. You’d have to change the JavaScript under api.parse function to handle their output. Just make sure that your function hews to the same format:
