Sunday, March 20, 2011

Impact of a decision-support system on user behavior

An interesting aspect of OR and business analytics practice is the chance to observe the impact of a decision support system (DSS) on paying customers who deploy it in real-life situations to improve business decisions. A good DSS tends to make users more innovative - they feel more secure that they have 'science' and automation working for them, and try to maximize benefits and progress toward becoming super-users.

A robust DSS should anticipate and react practically to a wide range of user inputs. In particular, if the DSS is based on a historical data-driven analytical model, inputs that are within the historical range would produce outputs roughly based on 'interpolations' - more reliable. On the other hand, an input not seen in prior history is (again, roughly speaking) equivalent to operating a heavy machine outside the functional limits, and may produce extrapolations that may make analytical sense, but little business sense. The latter input scenario makes the DSS look 'silly' because the business-savvy user would have provided a far better manual answer for such new inputs, because unlike the DSS that is just a computer program, they 'know'.

A bad DSS is one that is not trusted by users. There's more to it that good analytics. They may turn off the analytical portions and just use the automating features. Usually, it is equally if not more important to get the non-analytical portions right to ensure that customers are always getting some value for money. Then there's my pet 'timely failure theory' gleaned the hard way. If "infeasible" is an utterly unavoidable option in a DSS output for what ever reason, then make sure that such failures are immediate. Nothing irritates twitter-era humans more than having to wait 20 minutes to find out that nothing is going to happen.

In the ongoing cricket world-cup, the new umpire decision review system (UDRS) that predicts 'outs' using the Hawk-eye ball-tracking technology is turning out to be an example of a good DSS. The machine-free accuracy rate of umpires was 92%, which jumps to 98% with DSS-assisted decisions. If the DSS determines that the predicted result is marginal, the original decision of the umpire stands.

In the past, if the users (umpires) had a slight doubt, they would err on the side of caution. In cricket, that would amount to a 'not-out' ruling in favor of the batsman. That behavior appears to have changed a bit. Now, the decisions are slightly more 'aggressive' i.e., in favor of what the DSS decision is likely to be if a TV-referral were to be made. If the DSS is a more accurate predictor (which it appears to be), then this is likely to be a positive change in user-behavior, on average. A big chunk of the 6% error reduction may well have come from a reduction in the number of 'false not-outs'. Given that cricket today is skewed in the favor of batsmen, this is a welcome development.

On the other hand, cricket administrators have to work hard on defining a simple and effective operating range for the DSS. Otherwise, detractors (batsmen!) will use this as an excuse to get the DSS banished. Similarly, the time taken by the incumbent process to process and return a response to an input review is far too long . A batsman who is going to ruled 'out' by a damn machine after playing an idiotic shot doesn't want the global television audience to see replay after embarrassing replay of his moment of madness.

Sunday, March 13, 2011

Analytics and Cricket - VI: Improving the Cricket World-cup Schedule

The second-most watched single-sport tournament on the planet (after the soccer world cup) is underway in the subcontinent. The ICC cricket world cup is held once every 4 years and the latest edition is being jointly hosted by India, Sri Lanka, and Bangladesh. The original list also included Pakistan, but a militant attack on the Sri-Lankan national cricket team's bus there two years ago led to safety concerns. Fueled by the Indian economy, cricket is now a multi-billion dollar sport and the creation of a professional city-franchise based sports league in India three years ago is the latest example of this phenomenon. For some reason, cricket has always had an intimate connection with Operations Research (search for cricket in the archive). Today's post presents an overview of the potential benefits of using OR-enhanced scheduling in cricket.

The 2011 edition stretches over six weeks, and considering the fact that the number of teams who qualified for the finals is only 14, the duration of the tournament appears to be far too long, and has come in for considerable criticism. For example, the FIFA soccer world cup with more than twice the number of teams in the fray was completed in one month. There have been quite a few instances where the idling time for teams between any two games has been close to one week. Given this (and to minimize the number of boring mismatches), a recommendation for the 2015 world cup has been to reduce the number of teams from 14 to 10.

The 2011 WC format
The 2011 tournament schedule can be found here. We provide a quick overview of the format and 'constraints' here. The teams have been divided into two groups of 7. Each team in a group plays every other team in its own group. The top 4 teams from each group after the league phase enter the knock-out stage that includes 4 quarter-finals, 2 semis, and a grand finale. Each game lasts up to 8 hours, and is typically played in a day-night format, starting around 2pm and ending by 10pm. The majority of the revenue comes from TV ads. The media rights for this edition were sold for $2 Billion to ESPN. League matches involving host nations (and India in particular) garner huge ratings. Marquee match-ups are typically scheduled during weekends. Given the magnitude of revenue at stake, one can guess that even a 1% improvement to the schedule in terms of increased viewership and reduced player fatigue (thereby resulting in better match quality for fans) can add a lot of value.

A sample of hard and soft constraints
A minimum gap of two days between successive matches is a must to ensure adequate time for rest and travel. Minimizing total travel cost and idle time appears to be desirable. Host nations play their games on home turf to the extent possible. Day games (9am start) are also possible, if the morning fog/mist/dew factors are not overwhelming. On the other hand, locations with consistent dew problems at night are better suited for day matches to ensure a more fair contest. Reserve days are required for games that are washed-out due to inclement weather during the knock-out stage of the tournament, and any feasible schedule must take that into account. Of course, the OR-driven Duckworth-Lewis rules for rain-interrupted games are used to maximize the chances of getting a fair contest under the circumstances.

The Big Picture
The global cricket season is itself pretty packed and the overall schedule has come in for much criticism due to player burn out as well as overselling the game. As a cricket fan, it is pretty obvious that the status-quo is so dismal that better scheduling can maximize long-term revenue while also minimizing burn-out. Overselling is a major issue, not only because it kills the golden goose (long-term fan involvement in the game), but the number of inconsequential games being played has lead to match-fixing and spot-fixing (similar to point-shaving in basketball). The money involved in illegal betting is mind-blowing, and many suspect that it already is or could become another source of income for terrorist groups operating in the Af-Pak area. One does not expect OR to help resolve all these issues, but it can certainly be applied to some of the key ones it is well suited for, and the cascading positive effects can make a difference to the overall situation.

Potential OR Approaches
The Traveling Tournament Problem (TTP) popularized by Dr. Mike Trick at CMU appears to be a good starting point for improving the schedule for future cricket tournaments, including the world cup and the IPL, and ultimately, the global cricket season itself. Exact or heuristic approaches that combine constraint programming with MIP models appear to be well-suited to such problems given the complex and 'idiosyncratic nature' of some the scheduling constraints that tend to be imposed in cricket, as well the value added by even small scheduling improvements.

Looking Forward: The 2015 World Cup and Beyond
The next edition with be the "Anzac" world cup, hosted jointly by Australia and New Zealand. The distances between some cities that are likely to host some of the matches can be enormous (e.g. Perth and Sydney are more than 4000 miles apart) or relatively tiny (e.g. intra-NZ games, or Australian east-coast games (Melbourne-Adelaide is less than 800 miles), and the value that can be obtained by adopting optimized schedules can be significant.

I hope this post presents a reasonable high-level overview of cricket tournament schedules and motivates interested OR'ers to further investigate this problem. As a cricket tragic and OR professional, I would be happy to contribute toward any such effort.

(To be submitted as an entry to the March-2011 Informs Blog challenge: OR and Sports)

Thursday, March 3, 2011

Low Probability High Consequence Sporting Wins

LPHC events are always interesting since the implications (conditional expected cost), given the occurrence of the event, tend to have cascading side-effects. In risk-management analytics (e.g. optimal Hazmat routing), limiting the conditional expected risk turns out to be important. On the other hand, in the realm of sport, LPHC events are often quite desirable. Who doesn't love to hear and re-hear those stories about back-to-the-wall fight-backs and come-from-nowhere wins? However, among such magic moments, only a small subset have long-term and wide-ranging implications. Often, we have to wait for years or decades to see how the story unravels and how far the ripple effects go.

Yesterday's massive sporting upset of England by Ireland in the cricket world cup - yet another (truly) sensational match in my hometown within a week - has the potential to fall under this category for a variety of reasons. First some highlights of Kevin O'Brien's amazing counter-attack.

USA beat England in the 1950 soccer world cup - an incredible upset, but one that ultimately did not make much of a difference to the sport (arguably). On the other hand, Joe Namath's 'guaranteed' win in Super-bowl III could be termed a LPHC event since it seems to have played a big part in strengthening the NFL brand by creating a gripping storyline for the future. Or more recently, the 'Invictus' rugby story of South African Springboks that united a nation on the verge of being torn apart by the after-effects of the inhuman apartheid - yes, it is a true story. Sports, like politics is local and everybody has their favorite LPHC picks. For brevity, the top two on my short-list of LPHC sporting upsets are:

1. The 1980 miracle on ice. The uplifting impact of this story on people who haven't even stepped on ice, and the inspiring convergence of the sequence of events, imho, transcends sport, nationality, and time.

2. Was the 1980s the last and greatest decade of pure sporting action around the world, across all sport? India's cricket win over the invincible Caribbean world champions in the 1983 world cup final. If I recall, the odds of team-India winning the world-cup was something like 50, 000 to 1. A eleven-person nobody-group of not particularly athletic cricketers, speaking different languages, practicing many different religions, and coming from different backgrounds, yet each proudly representing a poverty-ridden, but democratic nation of 800 million hopefuls. Competing against the supremely powerful, atrociously talented, all-conquering team that was the West Indies. Watch the trailer to the wonderful documentary film tribute to this Windies team ("Fire in Babylon").

Notice the almost subdued celebrations in those days, and compare with the over-the-top ones by Indian cricketers today!

Incredible though that day was, nobody predicted the after-effects.Events on that London day started a three-decade long perfect cricketing storm that is now a multi-billion dollar professional sports-and-media franchise with a billion-person advertising market, and growing.

Ireland as a nation is financially reeling. Many people are moving out of Ireland, bringing back memories of the potato famine days in the 19th century. Cricketing-wise, they may not even be allowed to compete in the next world cup in 2015. As if all this wasn't enough, they still have to overcome (?) the proverbial luck of the Irish. Can they do the impossible?