Wednesday, September 21, 2011

Anatomy of a scam

Once, even twice is a coincidence. But three is a pattern worthy of a second look.

Exhibit 1: The financial monoliths that rode the cash wave in an ocean of American tax payer money have pretty much gotten away scot-free. Their criminal greed has been marketed as a simple combination of market upredictability, non-robust math models, and corporate irresponsibility.

Exhibit 2: The response from the Pakistani government after the double-tap delivered to Osama Bin Laden a few hundred yards away from their West Point, was to admit 'gross incompetence' rather than confess to any willing participation in hiding a notorious fugitive. No punishment and continued pouring of billions of dollars down the drain. They happily take out a full page Ad in the Wall Street Journal on the 11th of this month to celebrate.

Exhibit 3: The government of India is for all practical purposes run by the Darth Vader-like Italian-born Sonia Maino, who has propped up an equally complicit 80+ year old mute puppet as the prime minister to take the heat. The current regime that has ruled India for 50 of its 60-odd post-independence years is neck deep in a series of corruption scandals and midnight arrests of peaceful anti-corruption activists. The most blatant of these is the so-called '2G scam', where billions of dollars (1.86 trillion ₹) worth of public money in form of lucrative bandwidth was all but given away to friends and family. Only the most junior ministers (belonging to the coalition-party :) are in jail. Their defense is rather innovative but inevitably based on this same theme - 'negligence' and 'uncertainty', rather than admitting to any criminal wrong-doing or fraud.

Case 3 is an interesting example. This tab has already touched upon it twice before and arguments outlined turned out to be in line with what the Harvard-affiliated anti-corruption lawyer Dr. Subramanyam Swami used in his own article to describe the reasons for the mess.

The defense in all three examples of colossal fraud essentially argue that they merely maintained the 'status quo', claiming ignorance of the true value of doing the (obviously) right thing. Their second line of attack is to plead down the severity of the charge all the way to a misdemeanor. In exhibit 3, this is being done along the following flimsy lines "since the true value of the resource can only be determined if an auction had actually taken place, the figures quoted are cooked up by vested interests". Two factors go against such an argument:

1. This figure was calculated by a government agency (!) - the Comptroller and Auditor General (CAG).

2. Unless the CAG has performed a rigorous math-based analysis and run simulations to determine the maximal revenue obtainable from selling a scarce resource in a gigantic market like India, the quoted figure that was based on an average or a reasonable auction scenario is more likely to be a lower bound on the true cost of the swindle.

How dependable are 'opportunity cost', and more generally such "if" based decision models? A paper that I co-authored as a student of civil engineering many years ago happens to be based on this idea and was used in highway resource planning in Virginia. Often times, business value of an analytics idea can only be viably demonstrated by calculating 'what would have happened to a set of past outcomes had this OR method been used instead". On the other hand, if a researcher were to build up a ladder that consists of several degrees of conditional dependence to arrive at a final value, then such chain-of-events driven claims have to closely scrutinized to ensure that we simply do not end up with 'noise'.

Sunday, September 18, 2011

Book Review: Five Point Someone

5.someone is not a book about Operations Research. It is the title of a wildly successful first book by Chetan Bhagat in 2004. He is now India's most successful English paperback novelist. While 5.s is primarily an entertaining waiting-room read, it is also an indictment of India's higher education system that has so successfully strangled creativity. Nowhere is this 'success' more visible than in India's elite Institutes of Technology (IIT) that have incredibly low acceptance rates. While the IITs have indeed become a great global brand name, they have also failed to meet India's genuine domestic engineering and technological R&D needs. Having scraped through this very system a couple of decades ago (only recently have the resultant PTSD symptoms gone away!), the urge to relive the past was non-existent until recently when the hilarious movie version came out and commented on here. Also found some O.R connections in the plot, which is the main reason for this post.

The story is about three brilliant high-school students "who never came second in their class ever" until they make it into the IIT's mechanical engineering program, where their Grade-Point Average (GPA) is in the 5.0/10.0 region that relegates them to the back-benches. The story is about the combined desperate attempts of these 'three idiots' to push that magic number up into a respectable region before they graduate, while at the same time, allocating a respectable amount of time for all-night weed-Vodka sessions with Pink-Floyd on the rooftop of their dormitory building. The resulting series of comic failures within this GPA mouse race (since "rats are smart") includes a near-suicidal jump from the 9th floor of a campus building. Furthermore, one of the three 'idiots' is dating the daughter of their arch nemesis, the Mech HoD (see 'translations' at the end of this post) who happens to teach Indem (i.e. Operations Research and Management Science).

The story took an interesting turn for me when i discovered that the villain of the piece is an OR guy. His first scene in the book has him lecturing about scarce resource scheduling and management in a garment-tailoring factory setting. The resultant mathematical analysis leads to 'optimization equations' which at least one of the idiots finds extremely interesting. Great. However, the smartest of the three idiots differs and criticizes this abstract reduction of human effort to equations saying that 'these are people we are talking about, not robots'. Nevertheless, this doesn't discourage them from optimally pooling their resources and partitioning their classroom attendance to reduce their effort by 66% while also maintaining their GPA. They also estimate the probability-weighted cost of traffic cop fines if they were in fact caught triple-riding on the motor scooter owned by one of them to be far less than the accrued benefits.

Response from Critics
The style of writing is informal and filled with campus-slang Indian English, which may affect a few sensibilities and result in some wincing for those who read 'seriously good' contemporary Indian authors like Aatish Taseer. The author (then a rank amateur and an engineering techie) typed the whole thing up himself using MS Word. The book is available at Amazon, where the critical ratings, mostly based on all too-serious analysis of the language style and delivery, are at odds with the unprecedented popularity of the substance of the book within India itself. One reason is that most Indian students are required to be reasonably fluent in 2-4 Indian languages (apart from English, C, C++, Java, ...) in addition to being exposed to dangerously high radiation levels of math and science. Consequently, they never get a chance or feel the need to master any one of these languages until it is too late (this tab is a classic example). The book can be enjoyable if you can cut through the almost blog-like writing style as well as some awkwardly written situations, and let your inner undergrad-rebel escape. The incidents described in the book are something that engineering students across India, both in and out of the IIT system, can readily identify with. The U.S education system with its growing and almost exclusive emphasis on standardized testing and credentialing (i.e the "degree"), along with a reduction in the overall number of quality tech jobs, may well be on its way there too.

Mech = Mechanical Engineering
HoD = Head of Department
Indem = Industrial Engineering and Management
Mugger = One who excels in the art of rote-learning
Ragging = Hazing
Arbit = Arbitrary
Parantha = An unleavened Indian flat bread.

Update 9/19/2011: usual typos and more OR ideas in the book.
Update 1/19/2013: Just watched Paper Chase (1973). Let's just say that the plot and some of the scenes described in Chetan Bhagat's book resemble those in the movie, with a low probability of this being a coincidence. (Interestingly, CB was displeased with the low-key credit given to his book in the Bollywood version.)

Saturday, September 10, 2011

In Memoriam: 10 years after O.R's Apollo-13 mission

Two years ago, this tab recognized the amazing but well-hidden contribution of OR practitioners to the post 9-11 recovery plans at US airlines. Does your company have the right people to handle such an emergency? It's almost ten years to the day since OR's finest hour, and after journeying through a couple of different industries since then, my personal conclusion is that it was not a coincidence that those US airlines that best overcame this nightmare also had on their payroll seasoned OR experts.
So it's worth revisiting this topic in a Q&A form to examine the impact that OR can have during such situations. It is not intended to be an 'OR hagiography' - the stock of 'traditional' OR has in fact shrunk drastically since 2001, while some different new avenues have opened up.

Q. Was OR really critical to the post 9-11 schedule recovery effort?
Absolutely. The key ingredient to finding a solution to this 'mother of all one-time decision problems' was the presence of a significant number of experienced personnel in the airline with strong OR skills. why? Large airlines that operate thousands of flights on a daily basis use expensive aircraft that burn precious jet fuel, and are crewed by tens of thousands of trained professionals both on and off the ground. All this requires a well-synchronized and manageable 'program of activities', i.e., schedule of tasks that airplanes and crews follow. Every tiny portion of schedule involves complex combinatorial and constrained decision making to make the best use of these scarce and expensive resources. Nobody in an airline understands and applies the fundamental science behind this (yes, there is one) better than OR folks.

Q. Other than the tragic occasion itself, what was so unique and challenging about this problem in particular?
The scope: mammoth - hundreds of planes and thousands of crews stranded across North America in all kinds of airports.

The stakes: huge - the quicker an airline could get back on its feet to resume normal operations, the less money it bled (the slide into eventual bankruptcy began around the same time period).

The technical challenge: very high, but this was just a part of the problem. Research to design a brand new decision feasibility and optimization model that would be actually operational, and manage a gazillion possibilities that satisfy several new constraints, work well within a brand new implementation that gets around tons of IT and engineering issues within a couple of days - then to be used for a few days before being discarded for ever. This was the perfect R&D storm.

The task: a two-stage problem that would first bring our planes and crews back to their domiciles and then get them back to a regular operational mode through the end of the month.

Q. I'm intrigued. What the heck is "OR" and what kind of OR skills were needed to weather this perfect storm?
I forgot. O.R stands for Operations Research. OR folks tend to be reclusive but do make some noise about duality once in a while. As a rule, we generally believe in hiding our best achievements as well as the real name for our field. Dissident, mediocre OR types believe that we ever so often give away our flagship award ('Edelman') to bungling senior non-OR execs (who then hire OR grads to fix the problems they created), and we all know how well they have run this planet the past decade. Sour grapes aside, OR folks deal with the science and engineering of better. For example, we try our best not to tell you how to run your business, but we can and will tell you how you could run it more efficiently and effectively. Visit for details.

To the second part of your question: Airline-OR research experience combined with solid implementation skills was the need of the hour. During the high-flying days, airline R&D intellectual property in certain business areas was 2-3 years ahead of the best output from any university. Consequently, the OR crisis team was able to quickly take apart existing business-critical decision systems (that OR guys built and managed in those days) and extract big chunks off the million line monolith of mess-C code for reuse within this one-time model.

Q. I'm now an OR expert, so i'm jumping gears. Computationally, what optimization technique was crucial?
Column generation (and emacs) turned out to be the most important weapon in the arsenal 10 years ago. We were already routinely solving mammoth monthly planning instances and steadily pushing the envelope through intense research into improved algorithms as well as million$ cutting edge parallel hardware that airlines could afford then, so multiplying the problem size by another trillion did not really matter in the end. Now there is flexibility for you.

Q. I never realized that OR, in the right hands, and for the right reasons, would be so powerful and practical. Should every company maintain an OR team?
Any medium to large sized company, and not just airlines, would be well advised to maintain and hold on to an experienced and skilled OR team having some implementation experience. OR is ultimately about practical problem solving rather than an consultancy exercise that leaves critical (and insightful) details as an exercise to the reader. Routine planning problems may well be handled using OR-commodity software packages, but the day that even a mild Apollo-13 like decision problem threatens to disrupt your company's fundamental operations and plans, nobody can help put Humpty-Dumpty together again (and in an optimal and efficient manner) like an OR team can; seen this happen just ten years ago.

Thanks, OR guy. I heard the analytics was the real flavor of this decade and ...
Sorry to cut you short, we are analytics too, for a long time now. We change our names like winamp skins.


9/13/11: fixed typos