Chicago Prediction Markets Summit

Blogging a conference - especially one as niche as Prediction Markets - is an uncertain outcome at best but the topic is an important enough one for Aurora to sponsor.

I'm speaking this afternoon from 2:30 to 3:00, so I'm really interested in how the other speakers address the topic.

I myself will admit to being the greenest of the group and really only here to demonstrate the application set among the CI world. But really, I'm here to learn, rather than teach. We'll see how it goes...

As background, this past February I heard of a Prediction Markets conference John Maloney from Colabria was putting on in New York City,  so because my wife Tina was having a baby a couple of weeks later, I sent Derek and Greg out to see what it was all about. Needless to say, their reconnaissance proved interesting enough that we jumped on board to sponsor this Chicago event.

Thankfully, the Gleacher Center at University of Chicago has a nice WiFi connection so blogging is possible.

University of Chicago
Gleacher Executive Center

---

The first speaker - Robin Hanson - is a professor and economist at George Mason University and a well-known pioneer in the PM field. Starting out very basic and ramping quickly to his experience with all sorts of markets where he has studied the costs and benefits of placing bets on future outcomes.

Robin was one of the guys who worked on DARPA's Policy Analysis Market that was killed by Congress back in 2003. Brendan Koener had a great Slate.com Explainer on the subject at the time: 

The Pentagon has scrapped its plans to operate the Policy Analysis Market, which would have allowed online traders to wager on the likelihood of future terrorist attacks. Aside from commodities like pork bellies, what sorts of futures can wannabe brokers buy and sell?

A whole galaxy, thanks to the proliferation of Internet-based prediction markets, also known as decision markets. These online bazaars allow punters to plunk down money, real or imagined, on the potential of films, ideas, or the U.S. military's success in snagging Saddam Hussein. It may sound like nothing more than glorified sports gambling, but many economists believe that such markets can suss out vital, hidden information about future eventsmuch in the same way that a soaring stock on Wall Street can indicate that good things are afoot for the company in question. That's why the Defense Advanced Research Projects Agency has been funding so much research on the topic, hoping that prediction markets can assist military planners.

The granddad of online prediction markets is the Iowa Electronic Markets, which was started in 1988 to forecast the fortunes of presidential candidates; the market now covers the Fed's interest rate decisions as well. IEM participants can use real money, with starting accounts capped at $500. The market is regulated by the Commodity Futures Trading Commission. (Those with more Teutonic tastes may prefer Wahl$treet, a market for futures on German politics.)

Sports fans can take advantage of their nightly SportsCenter viewings on the Athletic Stock Exchange. Not much money to be made here, as the entry fee is $10 and the athletes trade for just a few pennies. But sports aficionados may enjoy grabbing a share of Bobby Labonte (Ticker symbol: LBNT) or Hideki Matsui (MATS) and seeing where fortune takes them. There are also veritable athletic indexes, jock versions of the S&P 500, like the Quarterback Pool, which aggregates the fortunes of all National Football League signal callers.

If news rather than sports is your informational pastime of choice, check out the current-events section at TradeSports.com, an Ireland-based betting service that made a name for itself when its market more or less predicted the date of Saddam Hussein's ouster. A future on a Saddam arrest this month just went up three points, in light of the capture of one of his chief bodyguards.

Not all prediction markets require that you risk actual cash. By far the most famous is the Hollywood Stock Exchange, which permits trade in both "MovieStocks" (pegged to the box-office success of upcoming releases) and "StarBonds" (tied to the future fabulousness of silver screeners), all in fictional money. Today, for example, the HSX is bearish on the soon-to-be-released S.W.A.T., down $5.67 on a volume of nearly 7 million shares. Angelina Jolie, whose Tomb Raider sequel disappointed over the weekend, is also having a rough Tuesday; her StarBond is down a buck.

The Foresight Exchange Prediction Market allows traders to bet on the likelihood of a range of events, from the resignation of Donald Rumsfeld by October to a devastating earthquake in the western United States by 2010. (A celebrity version of the Foresight Exchange is Long Bets, where pundits are encouraged to lay down a few thousand bucks on such outré prophecies as whether there'll be a four-day work week in the year 2070.)

If you ever had trouble making sense of the blogosphere, Blogshares may help separate the wheat from the chaff. No money's exchanged on this marketthough there is a $500 contest taking place right nowbut it does give bloggers bragging rights as to the popularity of their daily thoughts among Web surfers. Curiously, Slate resident blog "Kausfiles" doesn't appear to be listed on the exchange. Explainer eagerly awaits the IPO.

Bonus Explainer: At first glance, the trading of weather derivatives on the Chicago Mercantile Exchange may seem like a prediction market of sortsafter all, weather forecasting seems ripe for such wagering. But these investors are using the market to hedge their bets, not to get rich off hunches about upcoming weather. For example, an energy company that craves a long, hot summer in a certain citythe better for folks to run their air conditioners incessantlywill manage risk by purchasing futures that predict a cooler than anticipated summer. That way, they won't get burned so badly if Mother Nature doesn't comply. Since these investors are interested parties whose betting strategies are designed solely to manage risk, the weather derivatives market isn't a classic predictions market. 

---

Justin Wolfers from Wharton at Penn took over and concentrated on the notion of PM as an information aggregation mechanism which is the nature of my interest as well.

After a quite intro and good deal of integral mathematics to show that PMs work and do effectively predict the future, Justin called in to question the usage of mean (average) opinions as being efficient - that is, better than some other mechanism for predicting the outcome in an efficient way.

Taking a mean of people's beliefs BEFORE or AFTER the prediction market is the key information input - markets are better than surveys if the surveyed understand where the market is pointing.

He ended with a comparison of what PMs are and are not: they're not new, magical or aggregators of preferences; but they are aggregators of information (and the beliefs that spring from them), while being very much non "Web 2.0" wiki-like or open source means of judging likelihood of outcomes.

Time for the break but I hope to get a copy of the decks from these guys - great stuff!

---

 

John Delaney, the CEO of Trade Exchange Network was next and had some great observations as well - the transcript of his New York talk is available too which has some great insights:

Our industry is about to experience a period of massive growth. Growth in the region of 500%+ year on year will be achieved I believe. Exactly when that will happen, I will not predict, but it will happen!

This growth probability is just one of the reasons why I am absolutely thrilled to be involved in this industry of ours. However against the reward that this growth can offer to us all the industry related risks are all the more concerning.

Over the last few weeks I have communicated with friends of our Exchange some of who are here and others such as Justin Wolfers, and Robin Hanson, (we all wish him a speedy recovery) on what they saw as some of the most significant risks to our industry.

While many potential risks exist we see the most significant being...

1. Insufficient Mainstream Public Validation
2. A Material PR/Legal Shock or Shocks
3. Illiquidity / Fragmentation
4. A slow progression to Prediction Markets being considered just a fad and general fatigue with prediction markets

Lets discuss these risks briefly

Lack of Mainstream Validation: We all believe in the predictive power of our markets but I believe the USE of the predictive information needs to be validated more strongly. Until this happens I believe our industry is at risk of not achieving all that it can.

The closure of the DARPA sponsored project which Robin Hanson was involved in would have been a validation, possibly a great validation. Had the project lived the US Department of Defence would had directly validated prediction markets. That would have been very supportive of our industry.

Anecdotally we spoke to Newsweek just after the closure of the PAM project and I recall the journalist in question could not understand why we were so disappointed as a result of the closure.

Other potential risks stem from the possibility of a material negative shock or shocks.

PR Shock: For example lets assume that our political markets that showed strong predictive power for the 2004 US political presidential did not do a good predictive job for the 2006 mid-terms. We would and should fully expect that it would be newsworthy how poor the markets predicted the outcome. Of course we do not predict that this will happen but it is possible and therefore a risk.

Similarly lets say a corporate entity announced that the much-hyped prediction markets had caused it to misallocate significant resources to a project and therefore the company would not meet earnings estimates.

Fragmentation and Illiquidity: Where public markets are concerned fragmentation and illiquidity is far from optimal. Fragmented and illiquid markets deliver a bad customer experience. They enter orders and their orders are not taken or see no resting orders. There is some correlation between liquidity and the level of involvement and the predictive power of a market. It was great to hear that Chris Hibbert was concerned with same fragmentation and illiquidity issues and suggested co-operation and standardization may help.

However against that concern relating to illiquidity and fragmentation is the concern about dominance. Our industry at its early stage of development is unlikely to be served well by a single dominant player. Lets say a single platform were to own the entire space innovation as just one example may be compromised.

Fatigue with Prediction Markets: The final thematic risk is a slow development of a belief that prediction markets are just a fad. The longer our industry is around without additional significant mainstream validation the greater this risk becomes.

Now lets move on from our 4 identified risk themes, lack of sufficient validation, a major shock or shocks, illiquidity / fragmentation and lack of interest in The Prediction Markets Fad to some things we can do as a crowd to help.

OK, so they are some of the risks, of course there are more now lets consider the responses that we can and should take.

Lack of Validation: What can we do to validate the markets and show the way to others. Well I think there is a lot we can and should do. And I think we should do it now without delay.

Negative Events: Lets assume there is a negative event it will happen. Sometimes people read a market that is 90 bid that it will absolutely happen. Statistically this market will not be predictive 10% of the time. Many people do not understand this.

So, lets anticipate the negative event or events and prepare for them now and not go scrambling to respond if there is a problem.

Fragmentation: Lets co-operate and co-ordinate where we can and minimize fragmentation. For example co-ordination relating the specification of contracts that are ubiquitous would be beneficial and enhance general understanding of the public.

OK, so they are just some of the generic and thematic responses we can and should consider. However I would like to make a firmer and specific request that we should consider and then execute on.

Lets develop on the great work of these meetings and go the next step by creating an industry association. Our association should be very transparent and for everyone interested in prediction markets and not just operators. It should represent the industry in the widest sense.

Our industry association can do all the traditional things associated with an industry association such as lobbying, representing and sponsoring. But it can also so many unique things.

First it can launch a world's first industry prediction market. Let the prediction market have its own prediction market. The Prediction Market Industry Prediction Exchange. And lets do it now. It was great to hear Thomas Malone (MIT) and Bo Cowgill (Google) suggest we co-operate.

Ladies and Gentlemen: there is Windom [in] Crowds but there is also Strength in Numbers.

Furthermore our prediction market association, and I talk about it as it is already in existence, can have a multi platform (TradeSports / News Futures, Iowa, etc) database of results and how various trading categories did. This publicly accessible information can protect against an aberration (markets that do an unexpectedly poor job of predicting) if it were to happen or possibly limit PR related damage if it did.

Finally our industry association can do the standard things that all such associations can do. For example sponsor research, act as a forum for the extended community and lots besides.

Lets make this happen!

OK lets try a little experiment.

www.predictionx.com is a URL we registered recently to run this little experiment. Our experiment lists 6 or 7 industry relevant contracts such as:

• Major Industry Growth in 2006
• Major Corporate Validation in 2006
• Major Investment Firm Validation in 2006
• and a number of others'

On the back of my business card you will fine a unique login and password. I ask you all to login and contribute your opinion to the collective wisdom. The results will be clear for all to see and should convince those who need convincing that risks exist. We can use this platform to list additional claims / contracts of course.

In conclusion, we are in a fantastic young industry with massive potential. However there are risks. Risks that we all face both individually and collectively. We address our individual risks internally but the collective risks should be addressed in a co-ordinated manner.

We at Trade Exchange Network are very keen to work with you all to address these risks and will be proactively reaching out to relevant parties in the near future.

So, what do you think? That 500% YoY growth is a pretty impressive idea but I don't know - making this less obscure (the Yahoo! and Google angle from NYC can change that) will be key. In fact, the final line of his presentation was "participation is key". The questions afterward captured this idea - dumb crowds to yield information that can be monetized or smart crowds (experts) who can have their opinions aggregated...? Both are cool ideas.

---

Next up, David Perry, president and founder of Consensus Point, a software firm providing hosted and behind-the-firewall PM capabilities, gave an overview of the background and basis for their business. (Sorry David, I couldn't Google a headshot of you for my little ad-hoc gallery here.)

David mentioned that General Electric was a client and they were using it to predict things like advertising uptake around the NBC Universal shows that would be coming out in the next season

David specifically cited a few applications that are right down the middle of where CI can use PM techniques:

• Forecasting: sales revenue, market share, consumer demand, project management, predicting trends, regulatory approval

• Product Development: pricing, market research, new technologies, alternatives to focus groups (cost effective and quick but also scale to unlimited number of questions because traders themselves select the questions they want to answer).

Scenario examples were for a medical device manufacturer that was unsure what to charge for a new appliance and related monthly service charges as well as a retailer trying to predict the sales volume of a new video game system by predicting unit sales of different systems (PS3 versus Xbox 360). Another was to build a "Washington Stock Exchange" to address polling shortcomings by more quickly and accurately collect political intelligence and then make the data available to consultants, candidates and the media.

His observations about the industry (nascent as it is) were also interesting demonstrating major growth in the past 18-24 months with projects still being largely by early adopters and senior management and that trend appears likely to continue in the future.

---

Next, Emile Servan-Schreiber from NewsFutures was up talking about some of the common questions they see from prospective clients:

Have you done this before? A slide of logos ranging from Yahoo! to Lilly, SAIC, Abbott to Siemens, Corning and Arcelor, plus public and non-profit groups such as the World Economic Forum. Examples of applications include sales and price forecasting, project completion/prioritization and allocation of resources, project and product evaluations (again as a potential replacement for focus groups) and environmental monitoring.

Prediction trading crowds vary from internal (mostly frontline employees) to external which could include either restricted (value chain partners, consumer panels and invited experts) or open (Yahoo! Tech Buzz, Davos Global Risks or the Corning LCD TV Futures markets) which are all branded, attract a certain demographic and therefore have an influence in terms of aggregating expert opinions... but all so far with "play money" rather than "real money" subsidized by the companies themselves.

Lilly's prediction market was more accurate than its lobbyists in predicting the Medicare drug benefit because of late night "arm twisting" at 4:00 AM and we got our drug benefit, such as it is. Interestingly, the likelihood never fell below 40 percent. The other cool measurement was that Lilly asked if the top 10 pharma salesforces would increase and an uneducated crowd predicted the outcome - yes, it increased - based on a $10,000 reward for the best of these uneducated traders. Cool stuff! But finally, Lilly used just 12 salespeople to predict the earnings forecast and successfully beat the official forecast of monthly drug sales! WOW! One month ahead was pretty good - but even two QUARTERS ahead it was at least FIFTY PERCENT BETTER than the official forecast. Dang!

In fact, the farther out you look, the better the PM will be than the official forecast - because the closer you get to the timeframe the more information the official forecasters will have and the better their forecast will be. Another example was with Siemens in mobile phones - again, the further out, the better the crowd predicted the forecast than the official.

One really important factor is getting rewarded for being right but being right before everybody else. They collected confidence ratings as well in these forecasts but didn't really weight the data - it was more as a psychological prop for the trader... but they could not compare certainty of one person to the certainty of another - only one's own certainty today versus certainty tomorrow.

Predicting the impact of advertising with just a few dozen employees from multidisciplinary roles evaluating, for example, a TV, newspaper or magazine ad - and then they might just cancel the campaign if it's not looking to have a big impact.

Rewards + Recognition + Relevance = Participation

---

Cass Sunstein, University of Chicago Law School Professor, was the special keynote speaker and spoke on the subjects included in his new book "Infotopia: How Many Minds Produce Knowledge".

Studying mock juries to aggregate opinion about trial damages outcome - if jurors think behaviors are bad before they deliberate they think they're worse afterward and the opposite is true as well. Dollar awards always go up after deliberation however. All these data were based on 500 six-person mock juries, the largest of its kind. When it comes to judges - when Democratic appointees sit and talk together they get very liberal and Republican appointees behaved precisely the opposite, getting very conservative when they deliberate together.

He described a phenomenon called "informational cascades" (information will be unhelpful) or "reputational cascades" (information will be punished) in which serial opinions require greater and greater amounts of information in order to counteract the inertia or momentum and cause dissenters to self-silence or equipoise (great word!) in order to avoid the reputational downside risk of disagreement.

Hayek's "marvel" of the pricing system is the analog here because it summarizes the opinions of crowd-based markets so that they could predict the outcome. The mechanism described by Suriweicki's Wisdom of Crowds was validated a couple of hundred years ago by Condorcet's "Jury Theorem" where the group will increase the likelihood that they'll "get it right" as the group's size gets larger.

The advantage of the prediction market method over regular surveys is that the incentive for rewards create incentive to add their intelligence and understanding to the outcome. Simple really. The more expert the bettors the more accurate the prediction because of the risk/reward ratio.

The CIA would not have failed in predicting whether Weapons of Mass Destruction would be found in Iraq but informational and reputational cascades could have hampered that according to the Jury Theorem. Pretty cool. Group polarization and hidden profiles were the problem here as well.

Final comments on questions around: direct democracy - Athenian democracy versus Rousseau democracy as alternative approachs but the market provides a correction to these two; how does deliberation work or not work has to be seen through the lens of prediction markets where these factors of group polarization, hidden profiles and cascades don't apply to PMs.

Final smark-aleck comment - he asked what the weight of the space shuttle was and whether people could reliably guess that - I actually knew that answer - 4.5 million pounds gross weight at launch - so I'm going to open my session this afternoon with that.

---

LUNCH BREAK

---

Chris Hibbert, Principal Investigator at CommerceNet, talked about the open source app he's been working on for the past several years called Zocalo.

Chris believes that the reason PMs aren't as widely used is because the software isn't widely available. Google and Yahoo! can go out and just build their own apps but for most companies they just can't afford it. His goal is to bring the price down to the point where the ideas can propogate.

He's building a Web site where people can use his app to try PMs out at a low cost and then extend the open source app on their own as developers.  Chris cites several examples available online to document some of the actual applications:

Company	Subject	Organizer	References
HP	sales level	Charles Plott	Time, Plott & Chen
Eli Lilly	Drug efficacy	Eli Lilly	Time
Microsoft	developer acceptance of new releases	Todd Proebsting	Time
Intel	assignment of chip production to plants	Tom Malone	Time
British Petroleum	Pollution Credit trading	internal	Tom Malone: The Future of Work
Siemens	software development scheduling	Gerhard Ortner	Ortner
Google	software schedules, product deployment	Bo Cowgill, et. al	Google Blog
Corning, Arcelor	unknown	Emile Servan-Schreiber	MarginalRevolution Blog
Rite-Solutions	product ideas, suggestion box	internal	Home Page (look under Strokes of Genius:Innovation Engine), NYTimes
unnamed german mobile phone company (it's Siemens)	usage levels of new phone models	Spann & Skiera	Internet-Based Virtual Stock Markets for Business Forecasting

---

I was just on - stunned silence was my reward - let's hope that doesn't mean they hated it. In chatting with people at the break it seemed to come across okay but we'll see. My slides are attached.

---

Michael Gorham, Director of the IIT Center for Financial Markets at the Stuart Grad School of Business at the Illinois Institute of Technology was on next, talking about Regulating Event Markets in terms of whether gambling commissions such as the CFTC (the Commodity Futures Trading Commission) would be able to have oversight over prediction markets like they do over lotteries and other gambling venues.

Microsoft prevented this regulation by avoiding "consideration" in giving the stake of the funds to bettors and letting them keep the winnings but took back the stake, while Intrade simply operates offshore.

---

We wrapped things up with a panel discussion featuring most of the people who spoke throughout the day.

Manipulation was the subject of play-money markets like ProTrade and the Foresight Exchange from Consensus Point (because the money is free so a fictional account generated at will can have its liquidity sacrificed to some other account's benefit).

The willingness to be a market-maker - that is, put up a bunch of money to get people to bet for or against an outcome - more liquidity (cash on the line or stake up for bet) is better in terms of getting better results - the "thicker and deeper" the market the more accurate outcome you'll see.

Product features and the ability to predict what features will succeed in a real marketplace at a particular price (conjoint analysis) is a big opportunity - poll 100 people and have another 100 people bet on the outcome of that poll to get at a better, "deeper" prediction of the outcome of what those product features should be.

Teaching business students how to setup and read the outcomes of prediction markets to make aggregated decisions is a necessary set of skills to teach but could take a generation to make that important.

An observation was made about how there are very few women involved in this market but the same was true about the Internet in general when women slowly but inexorably took over.

Real-Money vs. Play-Money - which markets are more effective. Real money is easier to recruit people to the market but not more effective if there's a termination date and then gain reputation cred or a prize at the end. Another factor is to add a lottery intermediary - so the best performers are rewarded with more lottery "tickets" or chances to win the lottery for the prize. This seems to give people incentive to behave more logically. It's not gambling if there's no element of chance. NewsFutures holds a monthly auction so people can use their play-money to win a prize (and outbid other players) in order to liquidate their winnings or decide to reinvest their portfolio at some point - the same is true on Tradesports from Intrade though too.

Attachment	Size
prediction_markets_summit_chicago_20060607.ppt	2.77 MB
notesfromthursdaywitharik.txt	1.63 KB