Of Leaks and Llamas: The Great Open/Closed Debate

On 4 May, a purported leaked document from Google appeared online. The document, titled “We have no moat, and neither does OpenAI”, seems to be an admission that the big companies working on AI are unable to keep competing with open-source researchers. This document, and admission, created quite a stir. To understand why, we need to take a step back...

Tale as Old as ...

The question of whether AI research should be open source has long been a hot topic of debate in the AI community.

On the one hand, proponents of open source argue that making AI research openly available to the public will encourage collaboration and innovation, ultimately leading to the development of better technologies. Open source allows for transparency and accountability. This is particularly important in areas such as healthcare, where the consequences of AI errors could be catastrophic. There are also concerns that closed AI research may lead to a concentration of power in the hands of a few large tech companies that have the resources to dominate AI development.

On the other hand, arguments against open-source AI research include concerns about the security and privacy of AI technology. After all, open-source code can be easily accessed and manipulated by malicious actors. There are also concerns about the intellectual property rights of researchers, who may not want their ideas and discoveries to be freely available to anyone.

Finally, there are those who feel that having too many people (or, perhaps, anyone at all) work on AI is irresponsible because of the potential dangers posed by rogue “superintelligent” AI which may end up wanting to harm humans.

The Case for Openness: Meta releases LLaMA

All of this is why, on 23 February of this year, a llama caused so much excitement and controversy online. This was the day when the AI research company Meta (formerly Facebook) released its language model LLaMA, not as a chatbot, but instead as an open-source package that could be requested by anyone in the AI community. This meant that the code was now publicly available to be viewed and modified. The company stated that democratizing access this way was meant to encourage research into AI. Still, access was restricted to serious researchers and had to be requested through a form.

Well...so much for that. A week later the code had been leaked on 4chan. Innovations emerged rapidly from this point forward. For example, just over a week later Artem Andreenko, CEO of SentientWave, managed to run the model on a Raspberry Pi.

The technology was out in the wild, and everyone wanted a piece of it. Within a short time, the global open-source community replicated sophisticated steps such as instruction tuning and RLHF (reinforcement learning from human feedback) – all at low cost.

Case Closed: OpenAI releases GPT-4

At about the same time, in the middle of March, OpenAI announced their next-generation AI language model, GPT-4. Many people expressed disappointment that this model was not open source as well. This was not the first time that OpenAI decided not to fully open source their work, although the company has (as their name suggests) historically advocated for open-source research and collaboration.

With the launch of GPT-4, OpenAI did not provide any information on the data used to train the system, its energy costs, or the specific hardware or methods used to create it. This secrecy was criticized by many in the AI community, including Elon Musk, for going against the company's founding ethos as a research organization and making it difficult for others to replicate their work.

After the release of GPT-4, OpenAI's chief scientist and co-founder, Ilya Sutskever, defended the company’s decision in an interview with The Verge, stating that the reasons for not sharing more information about GPT-4 were fear of competition and concerns over safety. Sutskever warned that these models are potent, that they're becoming more and more so, and that they can potentially cause a great deal of harm.

Some experts countered that open-source research is actually necessary for ensuring the safe and ethical use of AI. If the research itself has the potential to be harmful, the argument goes, that makes it even more crucial to publish its details openly. This way, other researchers can help develop safeguards against possible threats.

Ben Schmidt from Nomic AI told The Verge that the lack of visibility into the data used to train GPT-4 makes it difficult to determine where the system can be safely applied and to identify potential problems. He added that for individuals to make informed decisions about the system's limitations, they need to understand what the model does and what assumptions it makes.

William Falcon, CEO of Lightning AI, stated that he understood the decision from a business perspective but believed that it could set a "bad precedent" for the AI community. Falcon asked how the community should react if the model fails or causes problems, and how ethical researchers could propose solutions and identify flaws if they do not have access to the system's inner workings.

In reaction, Sutskever said he agreed that open-sourcing models can help develop safeguards. He added that for this reason OpenAI was not completely closed, but had provided access to its systems for certain academic and research institutions to study.

Jess Whittlestone, head of AI policy at The Centre for Long-Term Resilience, suggested to The Verge that it should not be left to individual companies to make decisions on AI practices. She suggested codifying practices and having independent third parties scrutinize the risks associated with certain models before releasing them to the world.

Google ‘Has no Moat’

It was in this context that, on 4 May, SemiAnalysis posted what they claimed was a leaked (and, apparently, verified) document from a researcher within Google. This was the document titled “We have no moat, and neither does OpenAI”, referred to at the beginning of this post.

The document claims that Google (and, for that matter, OpenAI) is not positioned to compete against open source and has, as a matter of fact, already lost the struggle for AI dominance. Open-source models are faster, more customizable, more private, and more capable pound-for-pound than Google's models. They are solving major problems and doing so in weeks instead of months. The document indicates that Google is concerned that the open source movement has the ability to scale their AI projects in a way closed source cannot.

The document notes that the open-source community quickly understood the significance of Meta’s leaked LLaMA model and were able to innovate and develop new ideas within days, solving the scaling problem to the extent that anyone can experiment with AI training. This ability to scale quickly is a significant concern for Google, as what took them months and years to build took open source only a matter of days.

According to the document, the success of open source projects like Stable Diffusion, which overtook the popularity of OpenAI's DALL-E in just a few weeks, is a warning sign for Generative AI like Bard and ChatGPT. Within just three weeks of its release, Stable Diffusion became much more popular than DALL-E, as shown by the Google Trends timeline.

Google is worried that similar events could occur with their own models, posing an existential threat to their business.

According to the document, Google’s engineers are worried about the fast and inexpensive process of creating and improving open-source models, which is well-suited to a collaborative approach. The memo notes that new techniques, such as LoRA, can fine-tune language models quickly and cheaply, creating models that are comparable to those made by Google and OpenAI.

The author reflects on Google's shortcomings, stating that iterative improvements made in the open-source world eventually dominate, making a full retrain expensive. The author concludes that the global collaborative nature of open source is more efficient and much faster at innovation, making direct competition with open source a losing proposition for Google.

The author of the document finds some comfort in the fact that open-source innovations are free and Google can also benefit from them. The author suggests that Google should try to dominate here in the same way that they dominate the open-source Chrome and Android platforms. The document refers to Meta's success with their open-source LLaMA language model and concludes that Google should establish itself as a leader in the open-source community. This may involve relinquishing some control over their models, the document says, but this is a compromise that is necessary for innovation. The author suggests that Google should take uncomfortable steps, such as publishing the model weights for small Universal Language Model variants.

So what can we learn from this leaked document?

For starters, it confirms the observation that open source is dominating the AI field and suggests that Google may change its strategy to join the open-source movement and dominate it in the same way they did with Chrome and Android. The document suggests that Google can take the lead by making the first move towards open source.

'Horrible Disaster Plan'

For so-called “AI Notkilleveryoneists” such as (perhaps the most well-known) Eliezer Yudkowsky, any suggestion of open-source AI is a literal death sentence. This group believes that wide collaboration on AI is not necessarily a good thing because it puts too much power in the hands of people who may not be qualified to use it responsibly.

“The openness thing is a horrible disaster plan,” Yudkowsky told Logan Bartlett in a recent podcast.

The debate continues.