Experiments with NLP and GPT-3
The shock and awe strategy of the VC funded AI companies seems to be working. Their goal is to make sure that everyone believes:
The shock and awe did work. To some extent. But once open source takes over, and we start chipping away one block at a time and solve the problems in a much more efficient way, we will see why they wasted so much of compute and data to build these systems.
Thanks to Meta open sourcing Llama2, we could see that building a GPT4 level open source model is possible. All it needed was compute. And since Llama2 weights were open sourced, we don’t need to train again.
1. Use cheap labor under horrible conditions to collect and clean data. ( https://time.com/6247678/openai-chatgpt-kenya-workers/)
We have already shown that we can collect data in a fun way during the Chandamama kathalu datathon with participation from students.
We have also collected speech data in a unique fun way last month which we will unveil at the end of the month. With almost zero effort while having fun we were able to collect 1.5 million voice samples. It's possible.
