Mojeek Search Summaries
If you take from the web, you should give back. Search engines like Google, Bing and Mojeek do that with hyperlinks; sending traffic back to the web-pages which they have crawled and indexed. The voluntary agreement underpinning them, expressed through website robots.txt files, has been based on the legal concept of fair usage.
Generative AI products started to break this principle, notably with chatbots based on Large Language Models (LLMs). The breakout success of ChatGPT started a commercial race, which has accelerated this process. It’s an issue that both its creator and publishers are extremely concerned about, as can be seen from the many lawsuits and data agreements being fought over.
We support the open web and have concerns about the current trend as we have discussed before. It is also why we proposed the NoML meta tag. Importantly at Mojeek we have always played fair, respecting robots.txt, and simply providing links back to the websites that allow us to crawl.
Still we do recognise that, despite their inherent tendency to hallucinate, LLMs can offer convenience and help with efficient research. They can be very useful for providing summaries during informational discovery and learning.
