comment by evelynciara ·
2021-06-15T20:59:05.257Z · EA(p) · GW(p)
Practical/economic reasons why companies might not want to build AGI systems
(Originally posted on the EA Corner Discord server.)
First, most companies that are using ML or data science are not using SOTA neural network models with a billion parameters, at least not directly; they're using simple models, because no competent data scientist would use a sophisticated model where a simpler one would do. Only a small number of tech companies have the resources or motivation to build large, sophisticated models (here I'm assuming, like OpenAI does, that model size correlates with "sophisticated-ness").
Second, increasing model size has diminishing returns with respect to model performance. Scaling laws usually relate model size to training loss via a power law, so every doubling of model size results in a smaller increase in training performance. And this is training performance, which is not the same as test set performance - increases in training performance above a certain threshold are considered not to matter for the model's ultimate performance. (This is why techniques like early stopping exist - you just stop training the model once its true performance stops increasing.)
(Counterpoint: Software systems typically have superstar economics - e.g. the best search engine is 100x more profitable than the second-best search engine. So there could be a non-linear relationship between model performance and profitability, such that increasing a model's performance from 97% to 98% makes a huge difference in profits whereas going from 96% to 97% does not.)
Third - and this reason only applies to AGI, not powerful narrow AIs - it's not clear to me how you would design an engineering process to ensure that an AGI system can perform multiple tasks very well and generalize to new tasks. Typically, when we design software, we create a test suite that evaluates its suitability for the tasks for which it's designed. Before releasing a new version of an AI system, we have to run the entire test suite on it and make sure it passes. It's obviously easier to design a test suite for an AI that is designed to do a few tasks well than for an AI that's supposed to be able to do any task. (On the flip side, this means that anyone seeking to design an AGI would have to design a way to test it to ensure that it's (1) actually an AGI and (2) performant.) (While generality isn't strictly necessary for AIs to be dangerous, I believe many of us would agree that AGIs are more dangerous as x-risks than narrow AIs.)
Fourth, setting out to create AGI would have a huge opportunity cost. Yes, technically, humans are probably not the absolute smartest, most capable beings that evolution could have built, but that doesn't mean that building a smarter AGI machine would be profitable. It seems to me that humans have a comparative advantage in planning etc. while "technology as a whole" will have a comparative advantage in e.g. doing machine vision at scale. So most firms ought to just hire a bunch of humans and design/purchase technological systems that complement humans' skill sets (this is a common idea about how future AI development will go, called "intelligence augmentation").