A moderately substantial demo application in Dust - Actors for Java - in which we show how to build pipelines to find and process information using La

Search code, repositories, users, issues, pull requests...

submited by

Style Pass

2024-10-28 17:30:08

A moderately substantial demo application in Dust - Actors for Java - in which we show how to build pipelines to find and process information using Large Language Models.

We create a news reader which, when given a topic, finds valid RSS news feeds that will supply news articles about that topic. Classes of interesting information (entities) are also supplied by the user, and will be identified in the articles and logged.

The aim here is not so much to develop a fully-usable application (although surely one could easily be built on this foundation), but rather to show how easy it is to set up NLP pipelines in Dust where various stages of the pipeline have dialogs with ChatGPT.

dust-feeds - Built on the above provides Actors for processing and managing RSS feeds, crawling web sites and using SearXNG for web search.

dust-nlp - Actors to interact with OpenAI and Ollama LLMs (and a generic Actor for OpenAI-like apis which can be tailored to fit other providers).