Vernor Vinge gave a talk in the Long Now Foundation seminar series last week (which is great, by the way, you should go if you can). Stewart Brand sent out an email summary but it isn’t on the web site yet.
As Brand says, “Vinge began by declaring that he still believes that a Singularity event in the next few decades is the most likely outcome — meaning that self-accelerating technologies will speed up to the point of so profound a transformation that the other side of it is unknowable. And this transformation will be driven by Artificial Intelligences (AIs) that, once they become self-educating and self-empowering, soar beyond human capacity with shocking suddeness.”
At Stewart’s request, Vinge’s talk was about knowable futures – which by definition mean that the singularity doesn’t happen. But the follow up questions and discussion after the talk were mostly about the singularity.
All of this has crystallized my view of the singularity. The path isn’t all that strange, but I now have a much better sense of the details, and see aspects that haven’t been covered in any essays or science fiction stories I know.
One point that came out at Vinge’s talk is important for context. Vinge (and I) aren’t imagining a singularity as an absolute point in time, and in that sense the term “singularity” is quite misleading. The sense of singularity arises because looking further up the curve from a given point, we see “so profound a transformation that the other side of it is unknowable.” However as we’re moving along the curve we won’t perceive a discontinuity; the future will remain comprehensible to some extent, though our horizon may get closer. However, we also won’t be the same people we are now. Among other things we’ll only be able to understand what’s going on because we are gradually integrated into hybrid human / machine systems. I’ll discuss how that will happen as we go along.
What’s missing from current discussions of the singularity?
Above I claim that the path I imagine has “some features that haven’t been a part of any essays or science fiction stories about approaching the singularity”. So I’ll elaborate on that first.
A major enabler of progress for systems (computer or human / computer hybrid) is their ability to quickly and accurately learn effective models of parts of their environment.
Examples of current model learning technology:
- model road conditions by tracking roads (Thrun);
- model safe driving by watching a driver (Thrun);
- model the immediate visual indicators of a “drivable road” by watching the road right in front of the vehicle (Thrun);
- model the syntactic and semantic mapping between two languages by analyzing parallel texts (many);
- model the motion style of dancers (Brand);
- model the painting style of various painters (Lyu)
- model users’ choices of topics, tags etc. for text (lots of people);
- etc. etc. etc.
A few observations about this list:
- Often model learning delivers performance comparable to humans. Thrun’s models performed at near human levels in adverse real-world conditions (130 miles of driving unpaved, unmarked roads with lots of obstacles). The best parallel text language learning is near mediocre human translation ability. Model learning performs far better for spam filtering than hand built filters. Statistical modeling of painting style can identify artists as well as human experts. Etc.
- Model learning is quickly getting easier. Thrun created three quite robust modeling capabilities in less than a year. A basic parallel text translation system can be created with off the shelf components. Fairly good text classification can be done with simple open source software. Etc.
- None of these models is part of a reflexive loop, except in the extremely weak sense that e.g. spam filtering contributes to the productivity of developers who work on spam filtering.
Generally, discussions of and stories about AI, the singularity etc. don’t emphasize this sort of modeling. For example, when have we seen a story in which robots could quickly and accurately imitate the body language, intonations, verbal style etc. of the people they are dealing with? But this is a basic human ability, and it probably is essential to human level interaction.
The larger story
The modeling I discuss above is just one of a set of trends that taken together will lead to the singularity (unless something stops us, like nuclear war, pandemic virus, or some as yet unexplained metaphysical impossibility). If all the following trends continue, I think we’ll get to the singularity within, as Vinge says, “a few decades”.
Increasing processing power, storage, and bandwidth per $
It seems like Moore’s law and its various relatives will continue to smile upon us for a while yet, so I don’t think we need to worry about this.
Also my intuition (for which I have very little evidence) is that we can do the vast majority of what we need with the processing power, storage and bandwidth we already have. Of course, moving up the curve will make things easier, and perhaps most importantly, economically accessible to many more people.
Huge amounts of real world data easily accessible
This is the sine qua non of effective model learning; any area where we don’t have a lot of raw data can’t be modeled effectively. Conversely, if we do have lots of data, even simple approaches are likely to produce fairly good models.
This is taking care of itself very nicely, with immense amounts of online text, click streams, video chat, surveillance cameras, etc. etc. As the model acquisition ability is built, the data will be available for it.
General purpose, high performance population-based computing
All model learning is population-based – that is, it works with distributions, not crisp values. Continued progress in modeling will lead to more and more general grasp of population-based computing. Conversely, general population-based computing will make writing new modeling tools much easier and faster. Also, population-based computing makes it immensely easier to scale up using massively parallel, distributed systems.
Right now we have piecemeal approaches to population-based computing, but we don’t have a family of general mechanisms, equivalent to the “instruction stream + RAM” model of computing that we know so well. I think a better conceptual synthesis is required here. The synthesis may or may not be a breakthrough in the sense of requiring big conceptual changes, and/or providing big new insights.
Tightly coupled hybrid human / machine systems
To make hybrid systems more powerful, we need high bandwidth interaction mechanisms, and UI support to maximize coupling. Multi-touch displays, for example, allow much higher bandwidth human input. Good speech understanding would also help. Etc. In the opposite direction, visual and auditory output needs to make good use of human ecological perception (e.g. our ability to notice potential predators or prey during a jungle walk without consciously looking for them, our ability to unconsciously read subtle shades of feeling on people’s faces, our ability to unconsciously draw on context to interpret what people say, etc.).
Lots of people are working on the underlying technology for this. I don’t know of any project that is explicitly working on “ecological” machine – human communication, but with better modeling of humans by machines, it will probably come about fairly naturally. I don’t see a need for any big synthesis or breakthrough to make this all work, just a lot of incremental improvement.
As an aside, I bet that the best ideas on full engagement of people’s ecological abilities for perception and action will come from gaming. The Wii is already a good example. Imagine a gaming system that could use all joint angles in a player’s arms and hands as input without requiring physical contact. This would certainly require modeling the player at several levels (physical, motor control, gesture interpretation). Such a UI will only be highly usable and natural if it supports rapid evolution of conventions between the machine and the human, largely without conscious human choice.
General, powerful model learning mechanisms
This is a huge topic and is getting lots of attention from lots of different disciplines: statistics, computer science, computational neuroscience, biological neuroscience, differential topology, etc.
Again here I doubt that we need any big synthesis or breakthrough (beyond the underlying general model of population computing above). However this is such a huge area that I have trouble guessing how it will evolve. There is a significant chance that along the way, we’ll get insights into model learning that will cause a lot of current complexity to collapse down into a much simpler and more powerful approach. This isn’t essential, but obviously it would accelerate progress.
Reflexive improvement of hybrid systems
This is where the big speedup comes. Existing hybrid systems, such as development teams using intensive computer support, are already reflexively improving themselves by writing better tools, integrating their social processes more deeply into their networks, etc. Once the machines in a hybrid system can model their users, this reflexive improvement should accelerate.
I don’t see any serious obstacles to user modeling in development systems, but I also don’t see any significant examples of it, which is somewhat puzzling. Right now, as far as I’m aware, there are no systems that apply user modeling to accelerating their own development.
Such systems are technically possible today; in fact, I can imagine feasible examples fairly easily. Here’s one: A machine could “watch” a developer identifying interesting events in trouble logs or performance logs, learn a model that can predict the person’s choices, and use the model to filter a much larger collection of log data. Even if the filter wasn’t all that good, it would probably help the human to see patterns they’d otherwise miss, or would take much longer to find. The human and the machine could continue to refine and extend the filter together. This kind of model learning isn’t terribly challenging any more.
We may be stuck in a conceptual blind spot here, as we were with hypertext prior to Berners-Lee. If so, the Christiansen disruptive innovation scenario will probably apply: A new “adaptive” development environment will be created by someone outside the mainstream. It will be much less functional than existing development environments, and will be denounced as a total pile of crap by existing development gurus. However it will be adopted rapidly because it fills a huge unmet need for much easier, more tightly coupled human / machine development.
At the risk of dangerous hubris, I think these ingredients are all we need to produce the singularity. Specifically, I believe that no conceptual breakthroughs are required beyond those listed above. (Of course unexpected breakthroughs might occur and accelerate things.) None of the ingredients seems particularly risky, although most of us (probably including me) would have been astounded by current model learning if it was demonstrated five years ago. (Bernie Widrow and a few other people knew we could do this even thirty years ago.)
How long will it be before things get really weird?
Systems will be hybrid human / machine up to the singularity, and maybe far beyond (allowing for some equivocation about what counts as “human”). I don’t expect the banner of progress to be carried by systems with no humans in them any time soon.
The rate of improvement of reflexive hybrid human / machine systems determines the rate of progress toward the singularity. These are systems that can monitor their own performance and needs, and adapt their own structure, using the same kinds of processes they use for anything else. This kind of hybrid system exists today – online software development communities are examples, since they can build their own tools, reconfigure their own processes, etc.
The power of these systems is a function of the depth of integration of the humans and machines, and the breadth of the tasks they can work on (flexibility). Since they can apply their power to improving themselves, increasing their power will generally accelerate their improvement.
We already learn models of our machines. Good UIs and programming environments are good because they help us build effective models. To really accelerate progress, our machines will have to model us, and use those models to improve the coupling between us and them. The depth of integration, and thus the power, and thus the rate of improvement of human / machine hybrids will depend on how accurately, deeply, and quickly machines can model humans.
The big unknowns in determing how fast we’ll move are
- how quickly model learning will evolve,
- how effectively we’ll apply model learning to enhancing reflexive hybrid human / machine development environments and
- how quickly we’ll transition to population-based computing.
Probably I can do a worst case / best case / expected case trend projection for these three unknowns, but this will take some further research and thought.
Vinge talked a bit about different trajectories approaching the singularity – in particular, he distinguished between “hard takeoffs” in which the necessary technology is developed in one place and then spreads explosively, and “soft takeoffs” in which the technology development is more open and a large fraction of the population participates in the takeoff.
My analysis has some strong implications for those scenarios, and introduces a new, intermediate one:
A hard takeoff
For example due to a large secret military project. This is possible, but unlikely. Such a takeoff would have to create human / machine hybrids far more powerful than any generally available, and at the same time, prevent the techniques involved from diffusing out into the world.
Over and over, we’ve seen that most closed research projects fail, because they lack the variety and external correctives that come from being embedded in a larger network of inquiry. Also, the very institutional mechanisms that can create a big project and keep it secret tend to prevent creative exploration of the marginal ideas that are probably needed for success.
Note: This point is about research projects, not engineering projects that operate within good existing theories. Thus e.g. the Manhattan project is not a counter-example.
A soft takeoff
This is very likely as long as progress depends on combining innovations across a wide range of groups, and I think this is likely to be essential throughout the process. As things speed up, the dynamics are increasingly sensitive to the ratio between the rate of local change and the rate of diffusion of innovations. Aside from institutional barriers, software innovations in a networked world are likely to diffuse extremely quickly, so the network is likely to approach takeoff as a whole.
As we get closer to the singularity, a third possibillity emerges. Powerful reflexive systems may enable local, disruptive innovation, for example by relatively small groups that keep their activities secret. These groups would not be able to “outrun” the network as a whole, so they would not have a sustainable lead, but they could temporarily be very disruptive.
In Rainbow’s End Vinge describes a group like this, who are the villains, as it happens. Another example would be a horribly lethal disease created in a future rogue high school biology lab.
These localized disruptive innovations could make the world a very uncomfortable place during the transition. However, a much larger population of networked hybrid systems with local computing power and high bandwidth connections will be able to move faster and farther than such a small group. Furthermore, as we’ve increasingly been seeing, in a highly networked world, it gets very hard to hide anything substantial, and our emergency response keeps getting faster.
How will normal humans experience the singularity?
Humans who are not part of a hybrid system (and probably there will be lots of them) will gradually lose their ability to understand what’s going on. It will be as though their horizon shrinks, or perhaps as though their conceptual blind spots expand to cover more of their world. They will find larger and larger parts of the world just incomprehensible. This could produce a lot of social stress. Maybe it already does – for example, this could be a contributing factor to the anxiety that gets exploited by “stop the world, I want to get off” movements.
We don’t experience our blind spot as a hole in our visual field because our brain fills it in. I think this is a good analogy for how we deal with conceptual blind spots – we fill them in with myths, and we can’t tell (unless we do careful analysis) where our understanding ends and the myths begin. So non-hybridized humans mostly won’t be aware that a lot of the world is disappearing into their blind spots, they will spontaneously generate myths to “wrap” the incomprehensible parts of their world. Some will defend the validity of those myths very aggressively, others will accept them as “just stories”. Again this is pretty consistent with the rise of retrograde movements like intelligent design and modern geocentrism.
This phenomenon is closely related to Clark’s law: “Any sufficiently advanced technology is indistinguishable from magic.” “Sufficiently advanced” just means “beyond my understanding”.
A phase transition
Putting this all together, I now think that we actually could end up with a fairly sharp discontinuity that plays the role of the singularity, but is spread out in time and space.
This could happen if the transition through the singularity is effectively a phase transition. A metaphor may help. If we put a pot of water on the stove and turn on the heat, for a while all the water heats up, but not uniformly – we get all sorts of inhomogeneity and interesting dynamics. At some point, local phase transitions occur – little bubbles of water vapor start forming and then collapsing. As the water continues to heat up, the bubbles become more persistent, until we’ve reached a rolling boil. After a while, all the water has turned into vapor, and there’s no more liquid in the pot.
We’re now at the point where bubbles of hybrid systems (such as “gelled” development teams) can form, but they aren’t all that stable or powerful yet, and so they aren’t dramatically different from their social environment. Their phase boundary isn’t very sharp.
As we go forward and these bubbles get easier to form, more powerful and more stable, the overall social environment will be increasingly roiled up by their activities. As the bubbles merge to form a large hybrid network, the contrast between people who are part of hybrid systems and normal people will become starker.
Unlike the pot that boils dry, I’d expect the two phases—normal people and hybrid systems—to come to an approximate equilibrium, in which parts of the population choose to stay normal indefinitely. The Amish today are a good example of how a group can make that choice. Note that members of both populations will cross the phase boundary, just as water molecules are constantly in flux across phase boundaries. Amish children are expected to go out and explore the larger culture, and decide whether to return. I presume that in some cases, members of the outside culture also decide to join the Amish, perhaps through marriage.
If this is correct, as the singularity approaches, the “veil” between us and the future will become more opaque for normal people, and at the same time will shift from a “time-like” to a “space-like” boundary. In other words, the singularity currently falls between our present and our future, but will increasingly fall between normal humans and hybrid systems living at the same time. Hybrid systems will be able to “see into” normal human communities – in fact they’ll be able to understand them far more accurately than we can now understand ourselves – but normal humans will find hybrid communities opaque. Of course polite hybrids will present a quasi-normal surface to normal humans except in times of great stress.
By analogy with other kinds of phase changes, the distance we can see into the future will shrink as we go through the transition, but once we start to move toward a new equilibrium, our horizons will expand again, and we (that is hybrid systems) may even be able to see much further ahead than we can are today. Even normal people may be able to see further ahead (within their bubbles), as long as the equilibrium is stable. The Amish can see further ahead in their own world than we can in ours, because they have decided that their way of life will change slowly.
I’d love to read stories set in this kind of future world. I think the view of a normal human, watching their child go off into the hybrid world, and wondering if they will return, would be a great topic. The few stories that touch on this kind of transition are very poignant: for example Tunç Blumenthal’s brief story in Vinge’s Across Realtime, and the story of Glenn Tropile’s merger into the Snowflake, and then severance from it, in Pohl and Kornbluth’s Wolfbane.