Turking! The idea and some implications

I recently read an edited collection of five stories, Metatropolis; the stories are set in a common world the authors developed together. This is a near future in which nation state authority has eroded and in which new social processes have grown up and have a big role in making things work. In some sense the theme of the book was naming and exploring those new processes.

One of those processes was turking. The term is based on Amazon’s Mechanical Turk. Google shows the term in use as far back as 2005 but I hadn’t really wrapped my mind around the implications; Metatropolis broadens the idea way beyond Amazon’s implementation or any of the other discussions I’ve read.

Turking: Getting bigger jobs done by semi-automatically splitting them up into large numbers of micro-jobs (often five minutes long or less), and then automatically aggregating and cross-checking the results. The turkers (people doing the micro-jobs) typically don’t have or need any long term or contractual relationship with the turking organizers. In many cases, possibly a majority, the turkers aren’t paid in cash, and often they aren’t paid at all, but do the tasks as volunteers or because they are intrinsically rewarding (as in games).

One key element that is distinctive to turking is some sort of entirely or largely automated process for checking the results — usually by giving multiple turkers the same task and comparing their results. Turkers who screw up too many tasks aren’t given more of those tasks. Contrast this with industrial employment where the “employer” filters job candidates, contracts with some to become “employees”, and then enforces their contracts. The relationship in turking is very different: the “employer” lets anybody become an “employee” and do some tasks, doesn’t (and can’t) control whether or how the “employee” does the work, but measures each “employee’s” results and decides whether and how to continue with the relationship.

This is an example of a very consistent pattern in the transition from industrial to networked relationships: a movement from gatekeeping and control to post hoc filtering. Another example is academic publishing. The (still dominant) industrial model of publishing works through gatekeeping — articles and books don’t get published until they are approved through peer review. The networked model works through post hoc processes: papers go up on web sites, get read, commented and reviewed, often are revised, and over time get positioned on a spectrum from valid/valuable to invalid/worthless. The networked model is inexorably taking over, because it is immensely faster, often fairer (getting a few bad anonymous reviews can’t kill a good paper), results in a wider range of better feedback to authors, etc.

It seems quite possible — even likely — that post hoc filtering for work will produce substantially better results than industrial style gatekeeping and control in most cases. In addition to having lower transaction costs, it could produce better quality, a better fit between worker and task, and less wasted effort. It also, of course, will change how much the results cost and how much people get paid — more on that below.

Amazon’s Mechanical Turk just involves information processing — web input, web output — and this is typical of most turking today. However there are examples which involve real world activities. In an extreme case turking could be used to carry out terrorist acts, maybe without even doing anything criminal — Bruce Sterling has some stories that explore this possibility. But there are lots of ordinary examples, like counting the empty parking spaces on a given block, or taking a package and shipping it.

Examples

  • Refugees in camps are turking for money. The tasks are typical turking tasks, but the structure seems to be some more standard employment relationship. If there were enough computers, I bet a high percentage of the camp residents would participate, after some short period in which everyone learned from each other how to do the work. Then the organizers would have to shift to turking methods because the overhead of managing hundreds of thousands of participants using contracting and control would be prohibitive.
  • A game called FoldIt is using turking to improve machine solutions to protein folding. Turns out humans greatly improve on the automatic results but need the machine to do the more routine work. The turkers have a wide range of skill and a variety of complementary strategies, so the project benefits from letting a many people try and then keeping the ones who succeed. (This is an example where the quality is probably higher than an industrial style academic model could generate.) The rewards are the intrinsic pleasure of playing the game, and also maybe higher game rankings.
  • There’s a startup named CrowdFlower that aims to make a business out of turkingrowdFlower has relationships with online games that include the turking in their game play. So the gamers get virtual rewards (status, loot). I can easily imagine that the right turking tasks would actually enhance game play. CrowdFlower are also doing more or less traditional social science studies of turking motivations etc. Of course the surveys that generate data for the research are also a form of turking.
  • Distributed proofreading. OCR’d texts are distributed to volunteers and the volunteers check and correct the OCR. (They get both the image and the text.) The front page goes out of its way to note that “there is no commitment expected on this site beyond the understanding that you do your best.” This is an early turking technology, and works in fairly large chunks, a page at a time. It may be replaced by a much finer grained technology that works a word at a time — see below.
  • Peer production (open source and open content). An important component of peer production is small increments of bug reporting, testing, code review, documentation editing, etc. Wikipedia also depends on a lot of small content updates, edits, typo fixes, etc. These processes have the same type of structure as turking, although they typically hasn’t been called turking. The main difference from the other examples is there’s no clear-cut infrastructure for checking the validity of changes. This is at least partly historical, these processes arose before the current form of turking was worked out. The incentive — beyond altruism and the itch to correct errors — is that one can get credit in the community and maybe even in the product.
  • I just recently came across another good example that deserves a longer explanation: ReCaptcha. It is cool because it takes two bad things, and converts them into two good things, using work people were doing anyway.

    The first bad thing is that OCR generates lots of errors, especially on poorly printed or scanned material — which is why the distributed proofreading process above is required. These can often be identified because the results are misspelled and/or the OCR algorithm reports low confidence. From the OCR failures, you can generate little images that OCR has trouble recognizing correctly.

    The second bad thing is that online services are often exploited by bad actors who use robots to post spam, abusively download data, etc. Often this is prevented by captchas, images that humans can convert into text, but that are hard for machines to recognize. Since OCR failures are known to be hard for machines to recognize correctly, they make good captchas.

    Recaptcha turns the user effort applied to solving captchas, which would otherwise be wasted, into turking to complete the OCR — essentially very fine grained distributed proofreading. Recaptcha figures out who’s giving the correct answers by having each user recognize both a known word and an unknown word, in addition to comparing answers by different users. Users are rewarded by getting the access they wanted.

    Note that if spammers turk out captcha reading (which they are doing, but which increases their costs significantly) then they are indirectly paying for useful work as well. Potentially Recaptcha could be generalized to any kind of simple pattern recognition that’s relatively easy for humans and hard for machines, which could generate a lot of value from human cognitive capacities.

    Some implications

    It seems that over time a huge variety and quantity of work could be turked. The turking model has the capacity to productively employ a lot of what Clay Shirky calls our “cognitive surplus”, and also whatever time surplus we have. Many unemployed people, refugee populations and I’m sure lots of other groups have a lot of surplus. As Shirky points out, even employed people have a discretionary surplus that they spend watching TV, reading magazines, playing computer games, etc. However right now there’s no way to bring this surplus to market.

    Switching from industrial relationships (heavyweight, gatekeeping and control) to networked relationships (lightweight, post hoc filtering) reduces per task transaction costs to a tiny fraction of their current level, and makes it feasible to bring much of this surplus to market.

    The flip side of that of course is that the more this surplus is available for production, the less anyone will get paid for the work it can do. Already in a lot of existing turking, the participants aren’t getting paid — and in many cases the organizers aren’t getting paid either. Also, more or less by definition, the surplus that would be applied to turking currently isn’t being used for any other paid activity, so potential workers aren’t giving up other pay to turk. Therefore, I expect the average payment for a turked task to approach zero, for both turkers and organizers. Usually there will still be rewards, but they will tend to be locally generated within the specific context of the tasks (online community, game, captcha, whatever). Often the entity that generates the rewards won’t won’t get any specific benefit from the turking — for example, in the case of ReCaptcha, the sites that use it don’t particularly benefit from whatever proofreading gets done.

    Mostly turking rewards won’t be measurable in classical monetary terms — in some cases rewards may involve “in game” currency but this doesn’t yet count in the larger economy. In classical monetary terms, the marginal cost of getting a job turked will probably approach the cost of building, maintaining and running the turking infrastructure — and that cost is exponentially declining and will continue to do so for decades.

    This trend suggests that we need to find some metric complementary to money to aggregate preferences and allocate large scale social effort. But I’m not going to pursue that question further here.

    Obviously it will be important to understand what types of work can be turked and what can’t. For example, could the construction of new houses be turked? That may seem like a stretch, but Habitat for Humanity and other volunteer groups do construct houses with a process very much like turking — and of course this has a long history in the US, with institutions like barn raising. Furthermore the use of day labor isn’t that different from turking. I’d guess that within ten years we’ll be turking much of the construction of quite complex buildings. It is interesting to try to imagine what this implies for construction employment.

    Realistically, at this point we just don’t know the limits of turking. My guess is that the range of things that can be done via turking will turn out to be extremely broad, but that it will take a lot of specific innovations to grow into that range. Also of course there will be institutional resistance to turking many activities.

    When a swarm of turkers washes over any given activity and devours most of it, there will typically be a bunch of nuggets left over that can’t be turked reliably. These will probably be things that require substantial specialized training and/or experience, relatively deep knowledge of the particular circumstances, and maybe certification and accountability. Right now those nuggets are embedded in turkable work and so it is hard or impossible to figure out their distribution, relative size, etc. For a while (maybe twenty years or so) we’ll keep being surprised — we’ll find some type of nuggets we think can’t be turked, and then someone will invent a way to make most of them turkable. Only if and when turking converges on a stable institution will we be able to state more analytically and confidently the characteristics that make a task un-turkable.

    Another issue is security / confidentiality. Right now, corporations are willing to use turking for lots of tasks, but I bet they wouldn’t turk tasks involving key market data, strategic planning, or other sensitive material. On the other hand, peer production projects are willing to turk almost anything, because they don’t have concerns about maintaining a competitive advantage by keeping secrets. (They do of course have to keep some customer data private if they collect it at all, but usually they just avoid recording personal details.) I’d guess that over time this will give entities that keep fewer secrets a competitive advantage. I think this is already the case for a lot of related reasons because broadly speaking “Trying to keep secrets imposes huge transaction costs.” Eventually keeping big secrets may come to be seen as an absurdly expensive and dubious proposition and the ability to keep big pointless secrets will become an assertion of wealth and power. (Every entity will need to keep a few small secrets, such as the root password to their servers. But we know how to safely give everyone read only access to almost everything, and still limit changes to those who have those small secrets.)

    There’s lots more to say, but that’s enough for now.

    The two faces of Avatar

    Avatar just keeps demanding a bit more analysis.

    To recap, I agree the story is an embarrassingly naive retread of the “white man goes native and saves the natives” plus gooey nature worship. But…

    I also believe the world Pandora, as shown to us in the movie, can’t be confined within that story. It keeps escaping and cutting across or contradicting the premises of the narrative, as discussed in many nerd posts including mine.

    So Avatar has two very different faces, and different personalities to go with them. And I think this goes back to the basic character of the social processes used to create Avatar. No, seriously, stay with me for a minute and I’ll explain.

    For our purposes we can say there are three modes of production in films and a lot of other activities: craft, industrial, and networked. Of course any real film is produced through a mix of these.

    A film made by a small team working on their own (with or without a presiding genius) is an example of craft production, just like similar teams producing ceramic tiles or houses.

    A film made in a “factory” environment along with many others (like The Wizard of Oz or Casablanca) is an example of industrial production.

    And a film made by multiple loosely coordinated groups with different expertise is an example of network production.

    Network production is now nearly universal in large films, but before Avatar I can’t think of any examples of network production driving the film content. Generally the network mostly fleshes out content dictated by a small team that is using craft production. (If you can think of good previous examples, please comment or email, I’d really like to know.)

    In Avatar, Cameron wanted a lot of depth in his world, and had the money and skills to pull together a network to produce it. Pandora was created by a huge collaboration between ecologists, biologists, linguists, artists, rendering experts and so forth. The collaboration also necessarily included software and hardware experts who built the computer networks, and project managers who shaped the social network, and these people were no doubt also very engaged with the ideas about Pandora and contributed to its character in significant ways. Cameron was of course involved, but the depth and complexity of the world (and the network) meant that most of the decisions had to be internal to the network.

    So Avatar inevitably has two faces. The plot arc, the characters and the dialog were crafted by Cameron. Much of the commercial success of the film no doubt is due to his judgements about what would work in that domain. But Pandora, and probably much of the human tech in the film was created by a social network that was focused on scientific (as well as artistic) verisimilitude, conceptual integrity across a wide range of disciplines and scales, and our best current skills for designing and managing big networks of people and machines. And a significant amount of the success of the film is due to the richness and coherence of the vision generated by the network.

    In some sense Cameron was responsible for both faces. In one case he was directly shaping the content. In the other, he was shaping and directing the social network that produced the content. But the two forms of production generate very different kinds of results, and those generate the divergent critical reactions that tend to focus either on the story or on the world.

    This analysis brings into focus a question on which I have no information, but which I think is important to our deeper understanding of Avatar and our thinking about the successors it will inevitably inspire. Who defined the parts of the world that bridge between the network and the story? For example, in Pandora, animals, Na’vi and trees can couple their nervous systems to each other. This coupling plays a role in the story, but it could have been avoided in some cases, and made less explicit and more “magical” in others. On the other hand this coupling mechanism is constitutive of key parts of Pandora such as the “world brain”, and it drastically affects our understanding of the nature of Pandora and its possible history.

    Did the network come up with this coupling as a way to make mind transfer — a part of the story that would otherwise have been magic — into science? Or was it somehow integral to Cameron’s vision of Pandora? Or — more likely — some combination of those.

    Bubbles of humanity in a post-human world

    Austin Henderson had some further points in his comment on Dancing toward the singularity that I wanted to discuss. He was replying to my remarks on a social phase-change toward the end of the post. I’ll quote the relevant bits of my post, substituting my later term “netminds” for the term I was using then, “hybrid systems”:

    If we put a pot of water on the stove and turn on the heat, for a while all the water heats up, but not uniformly–we get all sorts of inhomogeneity and interesting dynamics. At some point, local phase transitions occur–little bubbles of water vapor start forming and then collapsing. As the water continues to heat up, the bubbles become more persistent, until we’ve reached a rolling boil. After a while, all the water has turned into vapor, and there’s no more liquid in the pot.

    We’re now at the point where bubbles of netminds (such as “gelled” development teams) can form, but they aren’t all that stable or powerful yet, and so they aren’t dramatically different from their social environment. Their phase boundary isn’t very sharp.

    As we go forward and these bubbles get easier to form, more powerful and more stable, the overall social environment will be increasingly roiled up by their activities. As the bubbles merge to form a large network of netminds, the contrast between people who are part of netminds and normal people will become starker.

    Unlike the pot that boils dry, I’d expect the two phases–normal people and netminds–to come to an approximate equilibrium, in which parts of the population choose to stay normal indefinitely. The Amish today are a good example of how a group can make that choice. Note that members of both populations will cross the phase boundary, just as water molecules are constantly in flux across phase boundaries. Amish children are expected to go out and explore the larger culture, and decide whether to return. I presume that in some cases, members of the outside culture also decide to join the Amish, perhaps through marriage.

    After I wrote this I encountered happiness studies that show the Amish are much happier and dramatically less frequently depressed than mainstream US citizens. I think its very likely that the people who reject netminds and stick with GOFH (good old fashioned humanity) may similarly be much happier than people who become part of netminds (on the average).

    It isn’t too hard to imagine why this might be. The Amish very deliberately tailor their culture to work for them, selectively adopting modern innovations and tying them into their social practices in specific ways designed to maintain their quality of life. Similarly, GOFH will have the opportunity to tailor its culture and technical environment in the same way, perhaps with the assistance of friendly netminds that can see deeper implications than the members of GOFH.

    I’m inclined to believe that I too would be happier in a “tailored” culture. Nonetheless, I’m not planning to become Amish, and I probably will merge into a netmind if a good opportunity arises. I guess my own happiness just isn’t my primary value.

    [A]s the singularity approaches, the “veil” between us and the future will become more opaque for normal people, and at the same time will shift from a “time-like” to a “space-like” boundary. In other words, the singularity currently falls between our present and our future, but will increasingly fall between normal humans and netminds living at the same time. Netminds will be able to “see into” normal human communities–in fact they’ll be able to understand them far more accurately than we can now understand ourselves–but normal humans will find hybrid communities opaque. Of course polite netminds will present a quasi-normal surface to normal humans except in times of great stress.

    By analogy with other kinds of phase changes, the distance we can see into the future will shrink as we go through the transition, but once we start to move toward a new equilibrium, our horizons will expand again, and we (that is netminds) may even be able to see much further ahead than we can are today. Even normal people may be able to see further ahead (within their bubbles), as long as the equilibrium is stable. The Amish can see further ahead in their own world than we can in ours, because they have decided that their way of life will change slowly.

    Austin raises a number of issues with my description of this phase change. His first question is why we should regard the population of netminds as (more or less) homogeneous:

    All water boils the same way, so that when bubbles coalesce they are coherent. Will bubbles of [netmind] attempt to merge, maybe that will take more work than their hybrid excess capability provides, so they will expend all their advantage trying to coalesce so that they can make use of that advantage. Maybe it will be self-limiting: the “coherence factor” — you have to prevent it from riding off at high speed in all directions.

    Our current experience with networked systems indicates there’s a messy dynamic balance. Network effects generate a lot of force toward convergence or subsumption, since the bigger nexus tends to outperform the smaller one even if it is not technically as good. (Here I’m talking about nexi of interoperability, so they are conceptual or conventional, not physical — e.g. standards.)

    Certainly the complexity of any given standard can get overwhelming. Standards that try to include everything break down or just get too complex to implement. Thus there’s a tendency for standards to fission and modularize. This is a good evolutionary argument for why we see compositionality in any general purpose communication medium, such as human language.

    When a standard breaks into pieces, or when competing standards emerge, or when standards originally developed in different areas start interacting, if the pieces don’t work together, that causes a lot of distress and gets fixed one way or another. So the network effects still dominate, through making pieces interact gracefully. Multiple interacting standards ultimately get adjusted so that they are modular parts of a bigger system, if they all continue to be viable.

    As for riding off in all directions, I just came across an interesting map of science. In a discussion of the map, a commenter makes just the point I made in another blog post, that real scientific work is all connected, pseudo-science goes off into little encapsulated belief systems.

    I think that science stays connected because each piece progresses much faster when it trades across its boundaries. If a piece can’t or won’t connect for some reason it falls behind. The same phenomenon occurs in international trade and cultural exchange. So probably some netminds will encapsulate themselves, and others will ride off in some direction far enough so they can’t easily maintain communication with the mainstream. But those moves will tend to be self-limiting, as the relatively isolated netminds fall behind the mainstream and become too backward to have any power or influence.

    None of this actually implies that netminds will be homogeneous, any more than current scientific disciplines are homogeneous. They will have different internal languages, different norms, different cultures, they will think different things are funny or disturbing, etc. But they’ll all be able to communicate effectively and “trade” questions and ideas with each other.

    Austin’s next question is closely related to this first one:

    Why is there only one phase change? Why wouldn’t the first set of [netminds] be quickly passed by the next, etc. Just like the generation gap…? Maybe, as it appears to me in evolution in language (read McWharter, “The Word on the Street” for the facts), the speed of drift is just matched by our length of life, and the bridging capability of intervening generations; same thing in space, bridging capability across intervening African dialects in a string of tribes matches the ability to travel. Again, maybe mechanisms of drift will limit the capacity for change.

    Here I want to think of phase changes as occurring along a spectrum of different scales. For example, in liquid water, structured patterns of water molecules form around polar parts of protein molecules. These patterns have boundaries and change the chemical properties of the water inside them. So perhaps we should regard these patterns as “micro-phases”, much smaller and less robust than the “macro-phases” of solid, liquid and gas.

    Given this spectrum, I’m definitely talking about a “macro-phase” transition, one that is so massive that it is extremely rare in history. I’d compare the change we’re going through to the evolution of the genetic mechanisms that support multi-cellular differentiation, and to the evolution of general purpose language supporting culture that could accumulate across generations. The exponential increases in the power of digital systems will have as big an impact as these did. So, yes, there will be more phase changes, but even if they are coming exponentially closer the next one of this magnitude is still quite some time away:

    • Cambrian explosion, 500 Million Years ago
    • General language, 500 Thousand Years ago
    • Human / Digital hybrids (netminds), now
    • next phase change, 500 years from now?

    Change vs. coherence is a an interesting issue. We need to distinguish between drift (which is fairly continuous) and phase changes (which are quite discontinuous).

    We have a hard time understanding Medieval English, as much because of cultural drift as because of linguistic drift. The result of drift isn’t that we get multiple phases co-existing (with rare exceptions), but that we get opaque history. In our context this means that after a few decades, netminds will have a hard time understanding the records left by earlier netminds. This is already happening as our ability to read old digital media deteriorates, due to loss of physical and format compatibility.

    I imagine it would (almost) always be possible to go back and recover an understanding of historical records, if some netmind is motivated to put enough effort into the task — just as we can generally read old computer tapes, if we want to work hard enough. But it would be harder for them than for us, because of the sheer volume of data and computation that holds everything together at any given time. Our coherence is very very thin by comparison.

    For example the “thickness” of long term cultural transmission in western civilization can be measured in some sense by the manuscripts that survived from Rome and Greece and Israel at the invention of printing. I’m pretty sure that all of those manuscripts would fit on one (or at most a few) DVDs as high resolution images. To be sure these manuscripts are a much more distilled vehicle of cultural transmission than (say) the latest Tom Cruise DVD, but at some point the sheer magnitude of cultural production overwhelms this issue.

    Netminds will up the ante at an exponential rate, as we’re already seeing with digital production technology, blogging, etc. etc. Our increasing powers of communication pretty quickly exceed my ability to understand or imagine the consequences.

    Rationality is only an optimization

    I’m reading a lovely little book by H. Peyton Young, Individual Strategy and Social Structure, very dense and tasty. I checked out what he had done recently, and found “Individual Learning and Social Rationality” in which, as he says, “[w]e show how high-rationality solutions can emerge in low-rationality environments provided the evolutionary process has sufficient time to unfold.”

    This reminded me of work by Duncan Foley on (what might be called) low-rationality economics, beginning with “Maximum Entropy Exchange Equilibrium” and moving to a more general treatment in “Classical thermodynamics and economic general equilibrium theory“. Foley shows that the equilibria of neoclassical economics, typically derived assuming unbounded rationality, can in fact be approximated by repeated interactions between thoughtless agents with simple constraints. These results don’t even depend on agents changing due to experience.

    So from the careful, well grounded results by these two scholars, I’d like to take an alarmingly speculative leap: I conjecture that all rationality is an optimization, which lets us get much faster to the same place we’d end up after sufficiently extended thoughtless wandering of the right sort. This hardly makes rationality unimportant, but it does tie it to something less magical sounding.

    I like this way of thinking about rationality, because it suggests useful questions like “What thoughtless equilibrium does this rational rule summarize?” and “How much rationality do we need to get close to optimal results here?” In solving problems a little rationality is often enough, trying to add more just may just produce gratuitous formality and obscurity.

    At least in economics and philosophy, rationality is often treated as a high value, sometimes even an ultimate value. If it is indeed an optimization of the path to thoughtless equilibria, it is certainly useful but probably not worthy of such high praise. Value is more to be found through comparing the quality of the equilibria and understanding the conditions that produce them, than by getting to them faster.

    Leaving knowledge on the table

    Yesterday I had a very interesting conversation with an epidemiologist while I was buying a cup of coffee (it’s great to live in a university town).

    She confirmed a dark suspicion I’ve had for some time — large population studies do a terrible job of extracting knowledge from their data. They use basic statistical methods, constrained by the traditions of the discipline, and by peer review that has an extremely narrow and wasteful view of what count as valid statistical tools. She also said that even if they had the freedom to use other methods, they don’t know how to find people who understand better tools and can still talk their language.

    The sophisticated modeling methods that have been developed in fields like statistical learning aren’t being applied (as far as either of us know) to the very large, rich, expensive and extremely important datasets collected by these large population studies. As a result, we both suspect a lot of important knowledge remains locked up in the data.

    For example, her datasets include information about family relationships between subjects, so the right kind of analysis could potentially show how specific aspects of diet interact with different genotypes. But the tools they are using can’t do that.

    We’d all be a lot better off if some combinations of funding agencies and researchers could bridge this gap.

    Dancing toward the singularity

    Vernor Vinge gave a talk in the Long Now Foundation seminar series last week (which is great, by the way, you should go if you can). Stewart Brand sent out an email summary but it isn’t on the web site yet.

    As Brand says, “Vinge began by declaring that he still believes that a Singularity event in the next few decades is the most likely outcome — meaning that self-accelerating technologies will speed up to the point of so profound a transformation that the other side of it is unknowable. And this transformation will be driven by Artificial Intelligences (AIs) that, once they become self-educating and self-empowering, soar beyond human capacity with shocking suddeness.”

    At Stewart’s request, Vinge’s talk was about knowable futures – which by definition mean that the singularity doesn’t happen. But the follow up questions and discussion after the talk were mostly about the singularity.

    All of this has crystallized my view of the singularity. The path isn’t all that strange, but I now have a much better sense of the details, and see aspects that haven’t been covered in any essays or science fiction stories I know.
    More below the fold

    Ego, enforcement costs, and the unitary executive

    Preparing the ground

    This is the first of a few posts on this topic. Here I ask the motivating question, and review evidence on human (and animal) consciousness that will help to answer it.

    Jerry Fodor, with his usual brilliant phrasing of bad ideas, sums up the issue:

    If… there is a community of computers living in my head, there had also better be somebody who is in charge; and, by God, it had better be me.
    In Critical Condition

    More generally, human beings seem to feel that any time there is a population working together, somebody has to be in charge — and it had better be “one of us”.

    But this is at best a debatable proposition, and it sometimes has very bad consequences. Furthermore, as I will discuss in subsequent posts, it acts as a barrier to population thinking. Why do we hold to it so strongly, and often (like Fodor) treat it as an axiom in philosophical, social, or political reasoning? And more practically, how can we adopt a more sensible stance?

    I believe we can start to answer the first question by looking at the function of consciousness.

    Bernard Baars, in his Cognitive Theory of Consciousness proposes a functional account of consciousness: an animal needs to be able to make coherent responses to potential threats or opportunities, and more generally to act as a coherent whole. Uncoordinated behavior is likely to be ineffective or even damaging. Consciousness provides a common frame that multiple sensory-motor systems can use to integrate their responses; most importantly, it helpt the animal to generate appropriate coordinated behavior in novel situations.

    Libet’s experiments throw interesting additional light on the role of consciousness. Summarizing somewhat brutally, Libet found that our conscious awareness lags our perceptual stimulation, response, and even voluntary decisions by 200-500 milliseconds (1/5 to 1/2 second). Sometimes responses triggered by conscious awareness can intervene in an ongoing process and stop it, but in other cases the conscious awareness comes too late. We have probably all had the experience of seeing a glass tip over or fall off a table and being unable to get our body to move quickly enough to avert the mess, even though it seemed like there was enough time. The problem, of course, is that “we” (i.e. our conscious awareness) didn’t “get the news” until it was too late to do anything — and our sensory-motor subsystems weren’t ready to respond pre-consciously to that class of event because we weren’t in the right (conscious) “frame of mind” to prepare them.

    All of this is consistent with Baar’s hypothesis. If consciousness mainly provides a “frame of mind” that helps (fairly autonomous) subsystems coordinate their activities, then it can still work fine even if it tracks events with some delay. Routine activities can proceed with minimal conscious awareness. Drastic interruptions or significant violations of expectations will generate an orienting response that pushes conscious awareness to the forefront.

    This is a good story and may well be true — the empirical jury is still deliberating, though research continues to generate supporting evidence, making a favorable verdict increasingly likely.

    But this story doesn’t account for the rhetorical verve and intensity of Fodor’s comment, or more generally our passionate attachment to feelings of comprehensive self-awareness and self-control. Why do we typically, like Fodor, feel uncomfortable with the possibility of being a community of subsystems, often loosely coordinated, with our conscious self acting as an intermittent framing mechanism for their activities? Surely this is a better description of our ordinary experience than vivid, comprehensive internal perception of a luminous self?

    I’ll take up these questions in the next post of this series.

    Language and populations

    After posting my ignorant non-argument about philosophical theories of reference, I was happy to see a post on The Use Theory of Meaning at The Leiter Reports that seemed highly relevant, with abundant comments. Alas, on reading it I found that it was unhelpful in two ways. First, generally the arguments seemed to assume as background that the meaning of linguistic expressions was universal (much more on this below). Second, the discussion was obviously confused – by which I mean that participants disagreed about the meaning of the terms they used, how others paraphrased their positions, etc.

    If the first problem is correct, then the whole discussion is fairly pointless. Furthermore I think the first problem creates the conditions for the second, because an assumption of universal meanings for expressions is so far from any actual situation in language that an attempt to base theories on it is likely to lead to chaos.

    Here is a clear example of this assumption of “universal meaning”: William Lycan in Some objections to a simple ‘Use’ theory of meaning says “[A rule for the meaning of a name must be] a rule that every competent speaker of your local dialect actually obeys without exception, because it is supposed to constitute the public linguistic meaning of the name.” “A rule that every competent speaker… obeys” is universal in just the sense I mean.

    Now, this simply isn’t an accurate way to look at how people actually use language. I hope any readers can see this if they think about some examples of creating and understanding expressions, but I’m not going to argue for it now – maybe in another post. I can imagine all sorts of responses: Chomsky’s competence stance, claims that we have to talk that way to have a meaningful (!) or useful (!) theory, statements that it is some sort of harmless idealization (of a different sort from competence), etc. However given the messes in the philosophy of language now which are (in my opinion) largely due to this background assumption, and the concrete results in linguistics and machine learning that show we can get along just fine without it, I reject any such claim. Again, I’m not going to try to substantiate these bald claims right now – but I’m confident I can, and the Steels paper in the earlier post is a good example.

    As my earlier post says, what we actually have is a population. To take the story further, each member has dispositions (rules if you will) about how to use a term, how to compose terms to create more complex meanings, or decompose expressions to recover their meanings, etc. But the dispositions of each member of the population will in general be different in all sorts of ways from those of other members. There is no requirement that these dispositions be completely describable, any more than your disposition to shape your hand as you reach for a cup is completely describable – though they might be remarkably consistent in some ways. As a result, no matter how narrowly we define the circumstances, two members of the population will quite likely differ in some details of their use of expressions in those circumstances.

    Even with no total agreement in any particular, language works because (again as mentioned in the earlier post) people can resort to context and can create more context through interaction while trying to understand or make themselves understood. This resort prompts us to adjust our usage dispositions over time to bring them closer together, when we find such adjustment helpful and not too difficult. However it also implies the meaning of any given expression may depend in an unbounded way on its context.

    I’ll end this with comments on two related issues. First, even apparently consonant ideas, such as Wittgenstein’s “family resemblances”, typically embed the background “universal meaning” assumption. In Wittgenstein’s metaphor the word “game” refers to a particular family, held together only by those resemblances – but the family is treated as a universally accepted meaning for the term, albeit not conveniently delimited by necessary and sufficient conditions. My use of overlapping (and largely consonant) dispositions is not equivalent to this, as I hope is obvious, perhaps with a little thought. However of course overlapping dispositions can easily give rise to meanings that fit Wittgenstein’s “family resemblances”, and the relationship between two different speakers usage dispositions for a given term should perhaps be seen as a family resemblance.

    Second, such things as Gettier problems and difficulties with vagueness seem to me to arise quite directly from this assumption of universal meaning. Given the context dependence of meaning in my proposed (very fundamental) sense, it is not surprising that unusual contexts induce incoherence in our intuitions about meaning. The interpretation of our claims that we’ve seen a barn will depend on whether the listener knows there are lots of fake barns about (and knows that we know or don’t know). A population with varying dispositions about the boundaries of Everest will produce something very like supervaluation, and our actual use of language will take that into account. And so forth.

    What’s wrong with Stephen Turner’s Social Theory of Practices

    In The Social Theory of Practices: Tradition, Tacit Knowledge, and Presuppositions (1994), Stephen Turner mounts a sophisticated attack on the idea of “social practices” as some kind of supra-individual entities. (I will call these “coarse grained entities” below, to avoid the value laden implications of “higher level” or similar locutions.) This attack is part of a broad effort to support methodological individualism and attack any theory or evidence that contradicts it.

    This problem is important in its own right, but it gains additional significance in the context of population thinking. If “only the population is real” then should we regard claims about coarse grained entities as fictitious? Of course my answer is “no”, but this requires careful analysis. We want to develop a firm distinction that allows us to adopt a realistic ontology for these coarse grained entities, but reject any treatment of them as abstract entities that somehow exist independent of the population.

    Turner’s book is worth a response because it is a relatively clear and thoughtful examination of the argument against supra-individual entities. Analyzing Turner can help us figure out why he (and the methodological individualists in general) are wrong, but also can bring into clearer focus the nature, dynamics, and importance of coarse grained entities.

    More below the fold

    Population thinking and institutions– quick sketch

    Here are a few ideas about how population thinking applies to various institutions, to be elaborated in some future posting:

    Yochai Benkler’s peer-production arguments are based on population thinking. He explores the implications of recruiting project members from large populations, where the potential members have substantial variation in “fit” to the project.

    Eric Raymond’s reasoning in “The Cathedral and the Bazaar” is based on population thinking in ways similar to Benkler’s (not surprisingly). “With many eyes all bugs are shallow” only makes sense in the context of a population with the appropriate variation in debugging skills. The whole “bazaar” idea is basically a population-based approach, while “cathedrals” (in Raymond’s usage) very much attempt to approximate ideal types. (Cathedral construction was probably much more population-based in real cases.)

    Large open source projects are decisively population oriented. Consideration of these projects brings to the surface a central problem in applying population thinking to institutions. At any given time much of the structure and activity is defined by a small number of participants, and it is hard to regard them as a “population” – their individual characteristics and their relationships have a big influence on the project. Over a somewhat longer time scale, however, the populations (of users, contributors, partners, etc.) in which they are embedded have a major and often decisive influence on the evolution of the project. Often the core group is replaced over time (usually voluntarily) and the replacement process depends on the larger populations as much as the prior structure of the core group. (I’m talking here as though there was a clear boundary for the core group, but of course that is importantly incorrect. The core group merges into the population and individuals can migrate smoothly in and out, without necessarily crossing any clear-cut boundaries.)

    Large companies should probably be understood (and managed) largely in terms of population thinking. Legal frameworks impose a significant amount of typological structure on companies (corporations, tax rules, accounting rules, etc. etc.). However when a company of more than a few people succeeds it is because of population dynamics – the dynamics of the internal population structure, and the dynamics of the external populations of potential partner companies and/or populations of individual customers.

    The only workable view of science that I know is population based. (Often this is called “evolutionary epistemology”, but a lot of work still has to be done to figure out why science works so much better at generating knowledge than other institutions, even given the evolutionary framework.) Previous views, and most current views, attempt to find “rules” for scientific activity that guarantee (an approximation to) “truth” – obviously typological thinking, and more importantly clearly wrong; this “rule governed” approach can’t be made to work either retrospectively (trying to account for the real history of science) or prospectively (trying to give scientists recipes for how to do their job).

    Operations like Ebay depend critically on population thinking. Ebay is a place for populations to coordinate. (Not so much true of Amazon, though the book reviews etc are population oriented.)

    Duncan Foley has shown that population models of economic activity (using thermodynamics) can produce economic equilibria that model real economic data better than the standard neo-classical mechanisms (which are based on deterministic auction processes). Furthermore the equilibria produced by these population models are much more robust than neo-classical equilibria under random perturbation. Also the neo-classical process of convergence to equilibrium depends on absurd assumptions about the rationality of the participants and the information processing capacity of the market mechanisms; the assumptions required for a population approach are completely reasonable.

    Next Page »