My colleague Mercedes Bunz made me aware of this allegedly leaked document from a Google engineer, in which they make the case for open-sourcing their models. TLDR; The argument is that owning and cultivating the ecosystem for innovation is more valuable than keeping the models fenced off.
Caveats notwithstanding, say for instance that the diminishing value of training does not account for people’s salaries in publicly-funded institutions, it is still an interesting read, especially the timeline narrating all the developments. Good for teaching, but also to make sense of the various recent moves in the field.
Some of the models and mentioned: LLaMA ― Meta Alpaca― Stanford Alpaca LoRA― Stanford + Eric Wang. See paper here. A Chatbot interface for Alpaca Dolly 15k instructions dataset
databricks-dolly-15kis a corpus of more than 15,000 records generated by thousands of Databricks employees to enable large language models to exhibit the magical interactivity of ChatGPT.
GPT4all― open stack pipeline based on JPT-J and LLaMA Vicuna
an open-source Chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT
This las one was developed by a student-led university consortium called LMSYS Org!