[Linkpost] Alpaca 7B release | Budget ChatGPT for everybody?

post by Felix Wolf · 2023-03-17T13:08:49.536Z · EA · GW · None comments

This is a link post for https://www.youtube.com/watch?v=xslW5sQOkC8

The new YouTube Channel AI-Explained gives a summary about the new Alpaca 7B model. It is an instruction-following language model, you can find the blog post here.

I will now quote the video description, which has a short summary and lists its sources, and give my own two cents.

8 years of cost reduction in 5 weeks: how Stanford's Alpaca model changes everything, including the economics of OpenAI and GPT 4. The breakthrough, using self-instruct, has big implications for Apple's secret large language model, Baidu's ErnieBot, Amazon's attempts and even governmental efforts, like the newly announced BritGPT.
I will go through how Stanford put the model together, why it costs so little, and demonstrate in action versus Chatgpt and GPT 4. And what are the implications of short-circuiting human annotation like this? With analysis of a tweet by Eliezer Yudkowsky, I delve into the workings of the model and the questions it rises.
Web Demo: https://alpaca-ai0.ngrok.io/
Alpaca: https://crfm.stanford.edu/2023/03/13/... 
Ark Forecast: https://research.ark-invest.com/hubfs... 
Eliezer Tweet: https://twitter.com/ESYudkowsky/statu... https://twitter.com/ESYudkowsky/statu... 
Self-Instruct: https://arxiv.org/pdf/2212.10560.pdf 
InstructGPT: https://openai.com/research/instructi... 
OpenAI Terms: https://openai.com/policies/terms-of-use 
MMLU Test: https://arxiv.org/pdf/2009.03300.pdf 
Apple LLM: https://www.nytimes.com/2023/03/15/te... 
GPT 4 API: https://openai.com/pricing 
Llama Models: https://arxiv.org/pdf/2302.13971.pdf 
BritGPT: https://www.theguardian.com/technolog... 
Amazon: https://www.businessinsider.com/amazo... 
AlexaTM: https://arxiv.org/pdf/2208.01448.pdf 
Baidu Ernie: https://www.nytimes.com/2023/03/16/wo... 
PaLM API: https://developers.googleblog.com/202... 

Using a second AI to multiply the self-instruct training data and the resulting feedback loop at such an early stage could lead to a drastic improvement in cost efficiency. I have to reevaluate my AI timeline again.


What are your thoughts about Alpaca 7B and Stanford's CRFM publication of the new model? Are the presented terms and conditions enough to permit misuse?

Dall-E depiction of an alpaca sitting infront of an computer in comic style.
An alpaca sitting in front of a computer, depicted by DALL-E

None comments

Comments sorted by top scores.