It was my impression OpenAI is concerned about AI safety, since it's backed by and founded by Elon Musk and Sam Altman who have expressed their concern for AI risks, and when interviewed on the topic of OpenAI, Musk and Altman made clear they think OpenAI's work, mission, and policies will bring the world closer to AGI while also ensuring it's safe. Multiple times now people have told me OpenAI isn't working on or about AI safety. I think what they mean is "AI safety" is something Nate points out in this article:
http://futureoflife.org/2016/01/02/safety-engineering-target-selection-and-alignment-theory/
Safety engineering, target selection, and and alignment research are 3 types of technical work/research, and more broadly, strategy research, moral-theory/machine-ethics, and collaboration-building are part of AI safety as well. So, it seems when people like Rob Bensinger or Victoria Krakovna tell me OpenAI doesn't seem like it will be doing AI safety in the near future, they won't be focusing on any of these areas. It seems to me OpenAI is, among other things, fostering alignment and capabilities research for AI in general without an explicit focus on the safety aspect. The 'open-source' component of OpenAI seems to be an effort towards creating a groundwork for strategy research or collaboration-building. Perhaps OpenAI is assuming safety engineering is part and parcel of capabilities research, and or perhaps that OpenAI can, with its influence in AI research in general, nudge capabilities and alignment research in the direction of safety concerns as well. My model of OpenAI's reasoning is that if all their research is the main force spurring capabilities research, it being open-source for everyone will level the playing field, not allowing any one company or other entity get ahead of the game without being examined by others, safety-concerned or not, and thus safety research can be injected into capabilities research in a broad way. Meanwhile, it seems folks like Scott Alexander, Jim Babcock and others have put forward that this approach is insufficient to precipitate AI safety research so it isn't outpaced by the relevant capabilities research, as it doesn't need to be a malicious entity, or one making philosophical failures, which makes AI dangerous, but technical failures in implementing an AGI which would also make it dangerous.
A month ago I made a big list of who is working on AI safety, as far as I could tell, and I included organizations like Google DeepMind and OpenAI because they've expressed a strong concern for AI safety and are now very big players in the field of AI more generally. Now, I'm understanding what people mean when they say OpenAI may not have anything to do with AI safety, because they're greatly mistaken about what AI safety really requires. So, I can exclude them from future versions of my list, or mention them but include caveats when I turn it into a proper discursive article. However, that still leaves the problem that some of the biggest players in AI in general think what they're doing will help with AI safety but it may actually make the future of AI more dangerous, and other big players in the field like Google might not worry about safety in their own AI research because they feel organizations like DeepMind and OpenAI have them covered. If this is the case, then it seems the mistaken understanding of the technical nature of AI safety needs to become as diffuse as increased awareness of it. This is a problem which needs solving.
My general sense from talking to Ilya Sutskever a few months ago was that the OpenAI people are interested in AI safety, but not planning to work on it in their first year or two, as they focus on gaining momentum with deep learning research. I expect they will figure out which AI safety directions they are interested in working on after their main research pipeline is more established.
ReplyDelete