An early concern with OpenAI's ChatGPT was the knowledge cutoff issue -- which means the large language model (LLM) only has knowledge of the world un

SearchGPT explained: Details about OpenAI's search engine

submited by

Style Pass

2024-10-25 20:00:06

An early concern with OpenAI's ChatGPT was the knowledge cutoff issue -- which means the large language model (LLM) only has knowledge of the world until a certain date. For example, the GPT-4o model that powers ChatGPT as of this writing has a knowledge cutoff of October 2023. With the older GPT-4 model, that cutoff was September 2021.

In contrast, a traditional search engine such as Google has no such knowledge cutoff. With Google search, the internet is constantly crawled to add the freshest possible information to the index. The challenge of knowledge cutoff in an LLM is one that multiple vendors are looking to solve with search capabilities powered by generative AI (GenAI). OpenAI is one of those vendors.

GPT stands for generative pretrained transformer. The GPT family of LLMs including GPT-3, GPT-4 and GPT-4o are the foundational LLMs developed by OpenAI that power ChatGPT. With SearchGPT, OpenAI will bring some of the power of a traditional search engine and pair it with the capabilities of its GPT LLMs. There will be no knowledge cutoff with SearchGPT, as it mines real-time data from across the web.

SearchGPT is currently an OpenAI prototype, launched to a limited number of test users. There is also a waitlist for those who want to be part of the test group in the future. Test users include individuals as well as publishers. OpenAI is inviting publishers into the SearchGPT evaluation to ensure they are satisfied that their content is appropriately represented in search results.