GitHub - iantbutler01/ditty: Finetuning various models.
A very simple library for finetuning Huggingface Pretrained AutoModelForCausalLM such as GPTNeoX, leveraging Huggingface Accelerate, Transformers, Datasets and Peft
Ditty has support for LORA, 8bit, and fp32 cpu offloading right out of the box and assumes you are running with a single GPU or distributed over multiple GPUs by default.
Pipeline is responsible for running the entire show. Simply subclass Pipeline and implement the dataset method for your custom data, this must return a torch.utils.data.DataLoader
Trainer does what it's name implies, which is to train the model. You may never need to touch this if you're not interested in customizing the training loop.
Data wraps an HF Dataset and can configure length grouped sampling and random sampling, as well as handling collation, batching, seeds, removing unused columns and a few other things.
The primary way of using this class is through the prepare method which takes a list of operations to perform against the dataset. These are normal operations like map and filter.
