Short writeup of recent challenges found in LLMs models (May 2023) – Random Notes – Some random post of my study research and other random stuff
Recently I have been spending more time reading than training LLMs (Large Language Models) as research publication in these domain started to surface due to hype started since release of ChatGPT around Nov 2022.
However, from myself and my peers personal encounters with employing AutoGPT to handle real world problems, the results more often than not dissapoints me. Of course don’t take me on face value that GPT-4 is bad, its still useful for other stuff but certainly not the job cutting excuse C-level came up to fire normies ~~ and not that company is burning out of cash~~.
Luo, Hongyin et al. “SAIL: Search-Augmented Instruction Learning.” recently shows search results based response have problem with selecting the relevant reference out from the search result. This study shows that event ChatGPT with search sometimes failed to reference the relevant answer and started to hallucinate. Even in Bard with retrieval, the result also contain some imaginary facts tweet.
Since the release of Schick, Timo et al. “Toolformer: Language Models Can Teach Themselves to Use Tools.”, there are many research and commercial attempt in “teaching” LLMs to use as many online tools as possible. For example, knowing when to use search engine is pretty useful to solve the knowledge cutoff problem.
