Confusion on OpenAI's relationship to the field of AI safety

It was my impression OpenAI is concerned about AI safety, since it's backed by and founded by Elon Musk and Sam Altman who have expressed their concern for AI risks, and when interviewed on the topic of OpenAI, Musk and Altman made clear they think OpenAI's work, mission, and policies will bring the world closer to AGI while also ensuring it's safe. Multiple times now people have told me OpenAI isn't working on or about AI safety. I think what they mean is "AI safety" is something Nate points out in this article:

http://futureoflife.org/2016/01/02/safety-engineering-target-selection-and-alignment-theory/ Safety engineering, target selection, and and alignment research are 3 types of technical work/research, and more broadly, strategy research, moral-theory/machine-ethics, and collaboration-building are part of AI safety as well. So, it seems when people like Rob Bensinger or Victoria Krakovna tell me OpenAI doesn't seem like it will be doing AI safety in the near future, they won't be focusing on any of these areas. It seems to me OpenAI is, among other things, fostering alignment and capabilities research for AI in general without an explicit focus on the safety aspect. The 'open-source' component of OpenAI seems to be an effort towards creating a groundwork for strategy research or collaboration-building. Perhaps OpenAI is assuming safety engineering is part and parcel of capabilities research, and or perhaps that OpenAI can, with its influence in AI research in general, nudge capabilities and alignment research in the direction of safety concerns as well. My model of OpenAI's reasoning is that if all their research is the main force spurring capabilities research, it being open-source for everyone will level the playing field, not allowing any one company or other entity get ahead of the game without being examined by others, safety-concerned or not, and thus safety research can be injected into capabilities research in a broad way. Meanwhile, it seems folks like Scott Alexander, Jim Babcock and others have put forward that this approach is insufficient to precipitate AI safety research so it isn't outpaced by the relevant capabilities research, as it doesn't need to be a malicious entity, or one making philosophical failures, which makes AI dangerous, but technical failures in implementing an AGI which would also make it dangerous.

A month ago I made a big list of who is working on AI safety, as far as I could tell, and I included organizations like Google DeepMind and OpenAI because they've expressed a strong concern for AI safety and are now very big players in the field of AI more generally. Now, I'm understanding what people mean when they say OpenAI may not have anything to do with AI safety, because they're greatly mistaken about what AI safety really requires. So, I can exclude them from future versions of my list, or mention them but include caveats when I turn it into a proper discursive article. However, that still leaves the problem that some of the biggest players in AI in general think what they're doing will help with AI safety but it may actually make the future of AI more dangerous, and other big players in the field like Google might not worry about safety in their own AI research because they feel organizations like DeepMind and OpenAI have them covered. If this is the case, then it seems the mistaken understanding of the technical nature of AI safety needs to become as diffuse as increased awareness of it. This is a problem which needs solving.

The Death of the Republican Party Has Been A Long Time Coming

Everyone in the media is talking about how the Republican Party is collapsing. I think it's just become undeniable the Republican Party cannot be what it once was, what it wants to be, and what it was hoping it still might be. I don't think the collapse of the Republican Party started in 2016, or in 2014 or 2012 during the elections during which the Tea Party swept into success. I think the Republican Party started collapsing after the United States started gradually losing faith in their competence after the second term of George W. Bush. I think W. was the worst president in modern American history, at least since Nixon. As a Canadian, I have a different perspective, so maybe most Americans would disagree with me, but I think W. was even worse than Nixon. I'd venture W. might be the worst president in the post-War era, and I only say that because the pre-War era was a sufficiently different time, and also because I don't know the history well enough, to feel comfortable making predictions which go before that. I don't think it would be too hard for someone who knew their American history better to convince me W. was the worst president of the last 100 years, and one of the worst of all time.

Sure, there are plenty of Americans who feel every politician and Obama as well are wiping their butts with the Constitution, but there is still enough of a coherent core left of centre that the Democratic Party hasn't been imploding for the last 8 years, and isn't imploding right now.

Thomas Jefferson once wrote: "The tree of liberty must be refreshed from time to time with the blood of patriots and tyrants." As a modernist who has observed the quality of life in the developed world achieve wonders we're all lucky to have, I'm much opposed to bloody revolutions which would threaten all that is stable in democracies with elections that still work. However, I am glad Donald Trump is causing a crisis and a revolution in the Republican Party that is causing the spirit of the old guard to stare into the face of its doom. To cleanse the establishment so seems important enough sometimes I think it's worth the trade of putting Trump in the White House if the Republican Party dies. Barring authoritarian nightmare scenarios, Trump will only be the crazy President of the United States for four years until he's booted out in 2020, and then the Republican Party will be dead. As long as Trump can't succeed in violating human rights left, right and centre, his winning the Republican nomination for the Presidency might be good insofar as this will destroy the Republican Party and it will prevent a dangerous fundamentalist like W. from returning to the White House for a very long time as well.

On 'rationalist', 'effective altruist', and Labels-As-Personal-Identities

In a conversation about other things, Julia Galef of the Center For Applied Rationality mentioned as an aside she doesn't identify as an 'EA'. Of course, this is presumed by all to be shorthand for 'effective altruist'. I've been reading for a while these undercurrents that the identity of 'effective altruist', as a noun, as a personal label anyone can freely choose for themselves rather than a category they may or may not match, waters down what 'EA' really means, or could mean, and is becoming problematic. Of course, there are those who would say building this into some sort of ingroup-y, tribe-like identity has always been a problem, perhaps since its conception. It's a lot of the same problem many express with just about anyone identifying with the term 'rationalist', and that profession being accepted as long as that person can send the right signals, i.e., a surface-level understanding of the relevant memes, to the rest of the self-identified 'rationalist' community.

I know Rob Bensinger has for a long time expressed a preference for people referring to themselves as 'aspiring rationalists', or 'aspiring effective altruists'. I think this won't work, as that's so long a phrase as to strike most as unwieldy in casual parlance. At best, I think people will shrug, assume others will know the 'aspiring' is implied and implicitly tacked onto the use of the terms 'EA', 'effective altruists', and/or 'rationalists'. Of course, others *won't* or *don't* assume that, and then somewhere in each of our minds we assume we're 'bona fide' EAs, or rationalists, that we're the real deal, whatever that is supposed to mean. I think this has already happened. I don't perceive a way to enforce this norm of thinking of ourselves, not only as individuals, but as a collective, as constantly aspiring to these ideals as an asymptote to be approached but never achieved, us being the limited and epistemically humble humans we are, unless someone like Will MacAskill and/or Eliezer Yudkowsky wrote repeated implied injunctions against anyone referring to themselves as anything but 'aspiring', when relevant.

Ryan Carey once expressed a sentiment that 'effective altruist' is something, as the movement grows, will become a shorthand for those who are involved with the global community, the 'intellectual and social movement', that call itself "effective altruism". He predicted the term 'effective altruist' will come to refer to people who identify with it, without being especially problematic. This will happen in the same way 'Democrat' or 'Republicans' refers to Americans who identify with particular political parties, without anyone assuming someone affiliated with one party or the other being for democracy and against republics, or visa-versa. I rule this prediction is correct, and has already come to pass. I thus recommend people stop making as big a deal about how the term 'effective altruist' is used. I'm not as enmeshed with the rationality community, but for policy on what to think of and how to use the label and personal identity of 'rationalist', I defer to Scott Alexander's essay 'Why I Am Not Rene Descartes'.

Most people probably haven't noticed this, but I have stopped tending to use the term 'effective altruist'. Sometimes, in discourse when everyone else is using it, I feel forced to use the shorthand 'EA', or 'EAs'. It's just convenient and I don't want to break my or anyone else's flow. However, I mentally think of this as meaning a 'community member'. That is, an 'EA' is, to me, someone who has chosen to in some capacity be involved with the body of other people known as 'effective altruism'. The extent to which one is an 'EA' is the extent to which one involves themselves in this community. A 'hardcore EA' is someone who has perhaps made their involvement in effective altruism their primary community, as opposed to some other social category, subculture, social movement, etc. The abbreviation composed of two letters, 'EA', implies this, without implying one is necessarily someone is particularly effective, altruistic, or effectively altruistic. Some people who identify as EAs will not be particularly effective or as altruistic as they ought to be, and some who explicitly eschew the label will match in their actions the ideals of effective altruism better than most. In this sense I pragmatically accept 'EA'-as-community-member, while banishing from my personal lexicon thinking of some people as truly or real 'effective altruists', and others not being so. There are just individual humans who take actions, and ever bigger groups of them, be them subcultures, companies, or nations, who coordinate some of their actions or beliefs towards common goals. When there is greater coordination among greater number of peoples, it's to varying extents useful and efficient to think of them as a unified, cohesive, atomic agent.

I know this puts me in a position which may strike most as odd. I'm just putting my thoughts down for reference. However, I hope people will remove from the marker in their brain labelled 'EA' or 'effective altruist' that there's a strong correlation or implication that anyone who uses that term to refer to themselves as automatically way more effective, more altruistic, more rational, or otherwise better than anyone else they meet in life. There may be a weak correlation there, but to the extent you interact with individuals, EA, rationalist, or otherwise, get to know them first.

Evan's Dictionary

Sometimes I make up words. I think making up words is a valid strategy, especially for writers discussing relatively novel ideas, or novel perspectives on old ideas, if the majority of humans who would end up reading or hearing them would agree the meaning assigned to the word intuitively works. Thus, here are the words which, while I don't claim exclusive title to adding them to the English lexicon, seem rarely used before I thought of them myself. Additions to this dictionary are ongoing for the indefinite future. Suggested additions from friends of useful words they've made up and/or think I should also use, especially as the Evantionary see below) gains clout, are welcome and encouraged.

Dankularity: noun. A memetic dankularity is a joke and possibly a real prediction I've posited of some given subculture becoming primarily dominated by dank memes and other dankness-related phenomena as opposed to the domination of all other factors combined. 'Dankularity' as a word and idea is inspired by the idea of a technological singularity, as opposed to 'singularity' as its used in physics or other scientific disciplines.

Endarkenment: noun. The opposite of 'enlightenment'. Verb form: (to) endarken. Adj/Adv. form: endarken

Endodus: noun. The opposite of 'exodus'.

Evantionary: proper noun. A term I myself (Evan) and others may use to refer to 'Evan's Dictionary' (this here document). A portmanteau of 'Evan's' and 'Dictionary'.

What Should I Blog About?

I've been thinking about what topics I should blog about. I recently read The Sense of Style by Steven Pinker, and one if his pieces of writing advice was to know one's audience. There are plenty of things I personally want to write about. Of course, most of the things I want to write about most are longer research projects. I have plenty of time and urge to write shorter or lighter pieces in between more major essays such as "Major Types of Global Risks". So, I'm open to taking feedback on things I should write about in my spare time, as my readers would find them useful. My readers are, at the moment, mostly just a large group of my friends. Feel free to comment and provide feedback on this post here, or on any other site you encounter it. If there is something in particular you'd like to see me write about, let me know. Things in particular I was thinking of writing about:

1. A Guide to Making Memes

Yeah, this one is a completely serious suggestion. I mean, there isn't much serious about making Internet memes. But I've become somewhat notorious for making memes, and I'm surprised by this. I'm surprised because while others are impressed with my meme output, from the inside, it feels rather easy. I think becoming good at making memes is easier than lots of people think, and I could write some pointers for how to get started.

2. "How to Make Reddit Work For You"

While on the topic of wasting time on the Internet: Reddit. I've noticed over the last few years there are a lot of people who think of Reddit distastefully, because they've had bad experiences on there, or they've heard so many bad things about it. Some of these reasons are because there is a pernicious culture on Reddit, of flame wars propagated by a dank hive of neckbeards, and nary any subreddit, no matter how isolated can avoid it. Or something like that. I don't know why people really avoid Reddit, and I don't much care. However, it's a great platform that gets a bad rap for ideas associated with it.

I've optimized my use of Reddit. Whenever I visit Reddit, I only have good experiences. It's all about subreddit subscription management. Of course, plenty of users do this. I want to write a simple guide for how one can render Reddit not only benign instead of pernicious, not just boring instead of aggravating, but actually useful and interesting and exciting and sometimes amazing.  Essentially, I've figured out how to make Reddit into my virtual beautiful bubble, an enclave on the Internet which doesn't suck, and I want to show others how to do so for themselves.

3. Rationality and Effective Altruism Explainers

One thing I find quite enjoyable, and I am willing to spend my time doing, is to provide explainers on all sorts of topics in the rationality and effective altruism communities. Now, I'm not just talking about thought experiments, or heuristics and biases, one can look up on Google or Wikipedia to find out how they work. All subcultures, rationality and effective altruism included, gradually developed their own quirks. Sometimes there are weird quirks and cultural trends, idiosyncratic pieces of history, which can only be gleaned through procedural knowledge and a wide variety of sources. A confusion about these cannot always be solved by googling. Sometimes these questions can only be answered, or at least answered simply and clearly, based on experience. I've been in each of these communities for several years, so I think I usually have the experience to satisfactorily answer these questions. If I don't, I'll at least know someone who does, so I can forward the question along to them. Also, I have a decent memory, better than most, and a willingness to explain things in great detail. For example, look how long this blog post, just about other blog posts I might write, is. That's lots of details. I'm a thorough guy.

So, feel free to ask me questions about anything related to rationality or effective altruism, or to explain my weird eclectic opinions on any specific subject therein.

Major Types of Global Risks

Summary: In terms of which global risks we should currently be directing resources to research and mitigate, I've reached the same conclusion as the Future of Life Institute and the perspective laid out in 'Global Catastrophic Risks' (2008). That is, risks from emerging or existing technologies which could critically threaten the stability of human civilization are the ones to prioritize. In this I include most risks which include a anthropogenic factor of component, such as climate change. I describe what I currently perceive as the most plausible mechanism for catastrophic harm for each of these type of risks. I give treatment to other risks, and conclude systemic (e.g., macroeconomic, socio-political) instability and nanotechnology are both types of risks which themselves don't currently pose a global risks, but, for different reasons, each ought to remain on the radar of an integrated assessment of global catastrophic risks. 

Since I've started dedicating serious time and thought to thinking about global catastrophic risks (GCRs), I've assumed there are four types of risks humanity faces and should dedicate the bulk of its risk-mitigation efforts to. However, I realized just because this is my perspective doesn't mean it's everybody's perspective. So, I should check my assumptions. They are below. This is also an invitation to voice disagreement, or suggest other risks I should take more seriously.

Nuclear Weapons

I'm assuming the inclusion of nuclear war is so obvious it doesn't warrant further explanation. For the record, aside from nuclear war, I include situations in which one or more nuclear weapons are detonated. Here is an excerpt from Global Catastrophic Risks (2008) detailing nuclear risks which don't begin as war[1].

• Dispersal of radioactive material by conventional explosives ('dirty bomb')

•  Sabotage of nuclear facilities 

•  Acquisition of fissile material leading to the fabrication and detonation of a crude nuclear bomb ('improvised nuclear device') 

•  Acquisition and detonation of an intact nuclear weapon

•  The use of some means to trick a nuclear state into launching a nuclear strike.

(Anthropogenic) Environmental and Climate Change

 The tail risks of climate change could pose a global catastrophe. However, there seems other potential GCRs as a result of environmental change caused by human activity, but aren't also the result of increased atmospheric concentrations of CO2 and other greenhouse gases. Such risks possibly include peak phosphorus, soil erosion, widespread crop failure, scarcity of drinkable water, pollinator decline, and other threats to global food security not related to climate change. There are also potential ecological crises, such as a critical lack of biodiversity. Whether biodiversity or wild life are intrinsically valuable, and whether humanity ought to care about the welfare and/or continued existence of species other than itself are normative questions which are orthogonal to my current goals in thinking about GCRs. However, it's possible the mass extinction of other species will harm ecosystems in a way which proves catastrophic to humanity regardless of how much we care about things other than our own well-being. So, it's worth paying some attention to such environmental risks regardless.

When we talk about climate change, typically we're thinking about anthropogenic climate change, i.e., climate change influenced or induced by human action. However, there are a variety of other GCRs, such as nuclear fallout, asteroid strikes, supervolcanoes, and extreme radiation exposure, which would result in a sort of "naturally" extreme climate change. Additionally, these GCRs, alongside systemic risks and social upheaval, could disturb agriculture. Therefore, it seems prudent to ensure the world have a variety of contingency plans for long-term food and agricultural security, even if we don't rate anthropogenic climate change as a very pressing GCR.

Biosecurity Risks 

When I write "biosecurity", I mostly have in mind either natural or engineered epidemics and pandemics. If you didn't know, a pandemic is an epidemic of worldwide proportions. Anyway, while humanity in the past has endured many epidemics, with how globally interconnected civilization is in the modern era, there is more of a risk than ever before of epidemics spreading worldwide. Also, other changes in the twenty-first century seem like they greatly increase the risk of major epidemics, such as the rise of antibiotic resistance among infectious pathogens. However, there is a more dire threat: engineered pandemics. As biotechnology becomes both increasingly powerful and available over time, there will be more opportunity to edit pathogens so they spread more readily, cause higher mortality rates, or are less susceptible to medical intervention. This could be the result of germ warfare or bioterrorism. Note a distinct possibility that what an offending party intends as only a limited epidemic may unintentionally metastasize into a global pandemic. Scientists may also produce a potentially catastrophic pathogen which is then either released by accident, or stolen and released into the environment by terrorists.

Other potential biosecurity risks include the use of biotechnology or genetic modification that threatens global food security, or is somehow able to precipitate an ecological crisis. As far as I know, less thought has been put into these biosecurity risks, but the consensus assessment also seems to be they're less threatening than the risk of a natural or engineered pandemic.

In recent months, the world has become aware of the potential of 'gene drives'. At this point, I won't comment on gene drives at length. Suffice to say I consider them a game-changer for all considerations of biosecurity risk assessment and mitigation, and I intend to write at least one full post with my thoughts on them in the near future.

Artificial Intelligence Risks

2015 is the year awareness of safety and security risks from Artificial Intelligence (AI) went "mainstream". The basic idea is smarter-than-human AI, also referred to as Artificial General Intelligence (AGI), or machine/artificial superintelligence (MSI, ASI, or just "superintelligence") could be so unpredictable and powerful once it's complete humanity wouldn't be able to stop it. If a machine or computer program could not only outsmart humanity but think several orders of magnitude faster than humanity, it could quickly come to control civilizational resources in ways putting humanity at risk. The difference in intelligence between you and an AGI as feared isn't the same as difference between you and Albert Einstein. When concern over AI safety is touted, it's usually in the vein of a machine smarter than humanity to the degree you're smarter than an ant. Likewise, the fear is, then, AGI might be so alien and unlike humanity in its thinking that by default it might think of extinguishing humanity not as any sort of moral problem, but a nuisance at the same level of concern you give to an ant you don't even notice stepping on when you walk down the street. Technology which AGI might use to extinguish humanity is various types of robotics, or gaining control of other dangerous technologies mentioned, nuclear weapons or biotechnology.

While there are plenty of opinions on when AGI will arrive, and what if any threats to humanity it will pose, concern for certain sorts of AI risks is warranted even if you don't believe risks from machines generally smarter than humans are something to worry about in the present. "Narrow AI" is AI which excels in one specific, but not all domains. Thus, while narrow AI doesn't pose danger on its own, or does anything close to what humans would call thinking for itself, computer programs using various types of AI are tools which could either be weaponized, or which could accidentally cause a catastrophe, much like nuclear technology today. Artificial General Intelligence isn't necessary for the development of autonomous weapons, such as drones which rain missiles from the sky to selectively kill millions, or to justify the fear of potential AI arms races. Indeed, an AI arms race, much like the nuclear arms race during the Cold War, might be the very thing which ends up pushing AI to the point of general intelligence, which humanity might then lose control of. Thus, preventing an AI arms race could be doubly important. Other near-term (i.e., in the next couple decades) development in AI might pose risks even if they're not intended to be weaponized. For example, whether its through the rise of robotics putting the majority of human employment at risk, or losing control of the computers behind algorithmic stock trading, human action may no longer be the primary factor determining the course of the global economy humanity has created.

"Why does effective altruism buy bed nets instead of investing in gene drives [to eliminate mosquitoes forever]?"

Summary: a friend asked me this question on Facebook. I ended up writing a rather extensive response, and I believe this question will be asked again, so I've written up this answer to be shared as needed. The short answer is effective altruism is a social movement of thousands of middle-class individuals who aren't coordinated enough to invest tens of millions of dollars into a single project, and billionaire investors or philanthropists associated with the movement appear for various reasons not inclined to invest in independent research of this sort. Moreover, while gene drives show promise as technology to eliminate mosquitoes, figuring out the best way to fund research to ensure the best outcome is difficult. 

1. Despite appearances, effective altruism isn't yet a coordinated elite movement that can design top-down solutions to the all the world's problems we want to solve. We're bottom-up movement of viewers just like you.

2. Well, if a few thousand middle-class members of the effective altruism movement can't fund research into gene drives to eliminate mosquitoes, why not all these billionaires like Peter Thiel and Elon Musk we see in the headlines? Well, they're more into exclusively funding blue-sky projects and moonshots to save the world. They're the sorts of folks who would fund an EA-style project to use gene drives. Musk, Thiel and the other Silicon Valley billionaires are willing to fund innovation they control themselves, but not the innovation others can provide. It's a Catch-22. This is a problem the effective altruism community hasn't solved yet. Other examples of this trend include:

• Elon Musk and Peter Thiel collectively giving one billion dollars over the next several decades to OpenAI, while neither of them has ever given more than $500,000 in one year to the already-existing Machine Intelligence Research Institute, which may be the primary organization on Earth responsible for making AI safety a legitimate cause rather than sci-fi doomsday nonsense.
• Google establishing biotech company Calico to run the anti-aging revolution in-house, rather than giving more than a relative pittance of their own money to fund the research at the Strategies for Engineering Negligible Senescence (SENS), which laid much of the groundwork which makes a bold organization like Calico seem plausible in the first place.

If you know a source of or a solution to the problem of billionaire bias whereby they're confident they can save the world by themselves because they're already such supergeniuses, rather than somewhat lucky run-of-the-mill geniuses, and declining to sufficiently fund scientists who aren't billionaires only because they spent the last decade building the fields of research these billionaires have just started paying attention to, we'd be eternally grateful.

Once this problem is solved, not only will we get started on the gene drives, but we'll do the rest of it too.

3. In the meantime, the two billionaires who are more on board with effective altruism are the foundation Good Ventures. Again, the effective altruism movement:

• A. isn't at the scale yet where it can realistically undo bans on DDT[1], if that was a policy we were pursuing, but dang it, it's trying. Seriously, it's beginning to experiment with influencing federal policies in the United States in the last 18 months, but this takes a lot longer than running a RCT for patch solutions like bednets. EA isn’t jumping headlong into influencing any and all policies, because it wants to first find its feet in policy areas which it's confident it can stably influence and which are well within the Overton window, like domestic criminal justice reform.
• B.  Open Phil is looking into funding breakthroughs in scientific research, but it turns out evaluating how to do this strategically, rather than rationalizing coincidental externally-positive innovations and claiming it's a miracle of the peer-review system or whatever, is difficult. So, it takes time to have calibrated confidence in how to make things like gene drives or geoengineering or whatever go off without a hitch. If you think it's easy, go apply to be a researcher at the Open Philanthropy Project. They're hiring.
• C. In the mean time, members of organizations like Giving What We Can and the supporters of Givewell-recommended charities see value in continually donating a couple thousand dollars to AMF here or there because malaria kills hundreds of thousands every year right now, and it's unrealistic for them to think if they spend three years putting that money into their savings account they can build a nest egg which will allow them to bankroll a several-million dollar research program.

Gene drives started existing, like, what six months ago? Why isn't *anyone*, or *everyone* funding this yet? Why didn't the United Nations pass an accord to receive one trillion dollars in funding from the governments of the world to research gene drives the day after this op-ed came out? I would ask you to be more patient with the effective altruism movement. It’s not an omnipotent community.

[1] In a prior comment in the discussion, my interlocutor mentioned how if effective altruism wasn’t willing to invest in gene drives, it could at least try to do something which would more effectively eradicated mosquitoes and prevent the spread the diseases they bear. He stated undoing bans on DDT might be able to accomplish this better than the current strategy of purchasing LLINs on the cheap via AMF. It is this comment I’m referencing. I neither endorse nor condemn a policy of undoing bans on DDT, and I make no claim recommending the effective altruism movement or any associated organization begin doing so.