Open Source AI Will Outcompete Google & OpenAI, Leaked Document Claims
"We aren’t positioned to win this arms race and neither is OpenAI. While we’ve been squabbling, a third faction has been quietly eating our lunch. I’m talking, of course, about open source."
- "At the beginning of March the open source community got their hands on their first really capable foundation model, as Meta’s LLaMA was leaked to the public."
- "It had no instruction or conversation tuning, and no RLHF. Nonetheless, the community immediately understood the significance of what they had been given."
- "A tremendous outpouring of innovation followed, with just days between major developments (see The Timeline for the full breakdown). Here we are, barely a month later, and there are variants with instruction tuning, quantization, quality improvements, human evals, multimodality, RLHF, etc. etc. many of which build on each other."
- "Most importantly, they have solved the scaling problem to the extent that anyone can tinker. Many of the new ideas are from ordinary people. The barrier to entry for training and experimentation has dropped from the total output of a major research organization to one person, an evening, and a beefy laptop."
- "In many ways, this shouldn’t be a surprise to anyone. Directly competing with open source is a losing proposition."
"Who would pay for a Google product with usage restrictions if there is a free, high quality alternative without them?"
- "And we should not expect to be able to catch up. The modern internet runs on open source for a reason. Open source has some significant advantages that we cannot replicate."
- "Paradoxically, the one clear winner in all of this is Meta."
- "Because the leaked model was theirs, they have effectively garnered an entire planet's worth of free labor. Since most open source innovation is happening on top of their architecture, there is nothing stopping them from directly incorporating it into their products."
- "Google should establish itself a leader in the open source community, taking the lead by cooperating with, rather than ignoring, the broader conversation."
"The text [excerpts above] is a very recent leaked document, which was shared by an anonymous individual on a public Discord server who has granted permission for its republication. It originates from a researcher within Google. The document is only the opinion of a Google employee, not the entire firm."
- Click here to read the full document.
