sahil280114/codealpaca

This is the repo for the Code Alpaca project, which aims to build and share an instruction-following LLaMA model for code generation. This repo is fully based on Stanford Alpaca ,and only changes the data used for training. Training approach is the same.

The Code Alpaca models are fine-tuned from a 7B and 13B LLaMA model on 20K instruction-following data generated by the techniques in the Self-Instruct [1] paper, with some modifications that we discuss in the next section. Evals are still a todo.

Current release contains the data generation procedure, dataset, and training code. Model weights aren't part of the release for now, to respect OpenAI TOS and LLaMA license.

[1]: Self-Instruct: Aligning Language Model with Self Generated Instructions. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi. https://arxiv.org/abs/2212.10560

data/code_alpaca_20k.json contains 20K instruction-following data used for fine-tuning the Code Alpaca model. This JSON file is a list of dictionaries, each dictionary contains the following fields:

Leave a Comment