The cricket 20-20 world cup is going on in the West Indies, and a potentially great match between the West Indies and England was cut-short today by rain. Upon resumption after the rain stopped, the chasing team's target was reduced using the D/L rules from 192 @ 9.6 runs per over to just 60 in the 6 overs that was possible, with all ten wickets available. What a farce! As mentioned in an earlier post (as well as in ORMS today by the creators - Operations Research (O.R.) guys whose names appear in the title of this post), The D/L model is used to forecast the runs target for the second team. This is a fantastic analytical model that works splendidly for the 50-over format. Why? Because this was adopted about 20 years after 50-over cricket was popularized, and D/L had plenty of varied data to calibrate their model and estimate goodness of fit. On the other hand, everybody assumed that the same model would work like a charm for the 20-over format, since D/L works with the % of overs remaining, etc, so its just a case of using a different multiplier, right? Wrong.
Nobody in the International Cricket Committee (ICC) bothered to even do a cursory analysis of how this model would perform in T20 games - a typical O.R. case study of blind trust in a black-box solution that works fine in normal conditions but fails when the problem slightly changes. And thus we recognize a weak spot in this model. It is going to take time to gather the data needed for better calibration, but what do we do until then?
Like any parameter estimation problems in statistics, O.R and econometrics, this one also requires a significant number of strongly good quality, non-collinear historical observations to work really well. The three years so far has been insufficient. Is 7 more years of international T20 cricket sufficient? 17 more years? While T20 is also cricket (at least when Sachin or Mahela bat), the dynamics is quite different from the 50-over format. Teams are 'all-out' or close to all-out far less frequently compared to the 50-over game, and every cricket fan knows that a wicket in a 50-over game is disproportionately more valuable compared to a wicket in a 20-over game. Does 2 wickets in a T20 game equal in value, the loss of one wicket in a 50-over game? As we being to think about this, we realize that the risk-reward-resource model used by D/L could be quite different, what with just 120 balls per innings. Or it could be the same in principle and its just a simple recalibration. On the other hand, An T-20 over is 5% of an inning, compared to 2% for the longer format. Does this huge reduction cause some boundary condition effects that need to accounted for? Is there a possibility that that we never find the amount of good quality data in my lifetime to make this same model work reliably for T20 games? I think it is time to look inside the model and confirm first if the fundamental assumptions and modeling constructs continue to hold in a T20 situation. Clearly with the 3 years of data we have had so far, it appears to be off the mark. In fact, even in the 50-over game, the model is known to have some bias the favors the team batting second, which however, is not severe enough to warrant replacement. However, we should be looking at fundamental modeling extensions if we find intrinsic problems with the D/L model applied to T20.
Cricket is perhaps the most unpredictable of all sports and is called the game of 'glorious uncertainties'. I just saw an international team lose 5 wickets in a single over and yet end up winning the match comfortably. It's also embraced a modern and sophisticated O.R. solution to weather-interrupted matches, but please, lets get our modeling straight. This kind of uncertainty is great for the O.R person in me, but not at all enjoyable as a cricket fan, and unlike business, cricket is far too serious to left totally to O.R types like me.
Monday, May 3, 2010
Thursday, April 29, 2010
Can OR provide strategy for the World Chess Championship Players?
Among the very many great sporting events that remain hidden away from the island of the USA is the ongoing battle for the World Championship in Chess. The classy defending champion, Vishy Anand is a sentimental favorite, given that he's based in Chennai, India (Madras) where I studied. The challenger is the Bulgarian Veselin Topalov, who before the title bout started, had a slight head-head advantage over Anand. To add to this, The volcano in Iceland meant that Vishy had to endure a several-day road trip across Europe to get to the venue in time after the chess authorities only granted him an one-day extension instead of a three-day break he asked for. Vishy promptly lost the first game, but won two of the next three to open a slender one-point lead. For those who remember the Fisher-Korchnoi-Karpov-Kasparov days in the cold war era, 21st century chess still remains an incredible mental sport where supreme ego, psychological gamesmanship, and sharp analytical intellects clash to create some amazing drama.
A great blog to cover chess is maintained by Susan Polgar (one of the famous trio of Polgar sisters from Hungary) now residing in Texas. An incredible talent herself, she won an under-11 girls chess competition in her country undefeated at the age of 4, and is arguably the world's greatest female player. She asks the question - How should Topalov plan his strategy for the remaining games?
The first to reach 6.5 points in this 12-game series wins, and with Anand at 2.5 currently, a risky approach may cost Topalov many games, whereas a placid approach may enable Anand to force some quick draws (0.5 points each). I wonder what statistics, OR, and game theory has to say with regards to the optimal policy to adopt for either player?
If you are far behind in points, then it may pay to throw caution into the winds, since there is little to lose, while in the current situation, the risk and reward is still somewhat balanced.
Does a player with a lead of one point or more simply play to force quick and safe draws?
Interesting questions. Some answers would be nice.
correction (April 30) - Susan Polgar may not even be the best chess player in her family, let alone the world :-) That credit probably goes to her sister Judith Polgar.
A great blog to cover chess is maintained by Susan Polgar (one of the famous trio of Polgar sisters from Hungary) now residing in Texas. An incredible talent herself, she won an under-11 girls chess competition in her country undefeated at the age of 4, and is arguably the world's greatest female player. She asks the question - How should Topalov plan his strategy for the remaining games?
The first to reach 6.5 points in this 12-game series wins, and with Anand at 2.5 currently, a risky approach may cost Topalov many games, whereas a placid approach may enable Anand to force some quick draws (0.5 points each). I wonder what statistics, OR, and game theory has to say with regards to the optimal policy to adopt for either player?
If you are far behind in points, then it may pay to throw caution into the winds, since there is little to lose, while in the current situation, the risk and reward is still somewhat balanced.
Does a player with a lead of one point or more simply play to force quick and safe draws?
Interesting questions. Some answers would be nice.
correction (April 30) - Susan Polgar may not even be the best chess player in her family, let alone the world :-) That credit probably goes to her sister Judith Polgar.
Wednesday, April 21, 2010
The Informs Practice Conference and the OR think tank misses a trick or two
How can Informs make the Practice conference even better? An advantage of being an unofficial reporter is that I can avoid self-congratulatory blog posts and actually criticize without any sugar coating - in the hope that we get out of our comfort zone and make this an even better event next year.
Clearly some things were out of their control. All the OR folks in the world would not have been able to predict the impact of a volcano in Iceland on the travel plans of overseas visitors to the conference. Also, Dr. Micheal Trick, whose pioneering web page on O.R was the main source of information as well as inspiration for graduate students like me in the 1990s, and motivated me to join this exciting field, was missing, and one can't fault Informs for this. I was really looking forward to shaking his hands and thanking him for his service. 'Marketing in Online Social Spaces,' by Kevin Geraghty, Vice President, Research & Analytics, of 360i was a really good one (somehow I forgot to cover this in my daily conference tab). Kevin was providing an example of marketing campaigns using social networking data. He found out (using completely public domain tools!) that in the OR blog world, to the surprise of many, a certain Lieutenant in the Navy had more 'online friends' than Dr. Trick, so if one were to promote some hypothetical OR product, then he should be chosen as a first reviewer, assuming that those friends were OR types rather than 'sailors'. He also obtained other funny personal trivia from public domain, that I'll just leave out.
A second peeve I had was the highly limited lunch and dinner options for vegetarians (Two boiled asparagus roots and a turdy-looking cuboid of tofu does not an Edelman banquet make!). This should not be difficult to fix. If this doesn't change, I frankly don't see much point in shelling out two grand and semi-starve most of the conference. Thankfully, due to the purely individual initiative of the obviously superb Hilton staff, i was not totally inconvenienced. Kudos to those guys. They got their 'hospitality management OR' right.
Third, attendees should be able to obtain access to the video archive of the talks. Static slides don't cut it anymore. Unlike academic conferences, the value of practice-oriented conferences often lies in what is said in between slides.
On the positive side, the posters were a big hit. One can engage the presenters informally and in 5-10 minutes get a high-level idea of what their innovation is about. And the good thing is that you can visit them in your own time and network too. For example, I found out that the Sandia Labs in beautiful New Mexico, has this really cool Python-based modeling language (PYOMO?) that they used for stochastic programming. Can't wait to try it out. At the MPL booth, I found out that that they are making software available for free on a Windows environment. In tandem with COIN-OR (which they package MPL with, i think), you have a solid modeling and optimization package, free!
My best talks in no particular order - Sanjay Saigal (Intechne) on uncertainty , Jeffrey Cramm (Univ of Cincinnatti) on practical OR, Kevin Geraghty (360i) on social networking, John Osborne (Kroger) on OR innovation against all corporate odds, and any Edelman presentation.
overall grade: 7/10
I'll end on a warning note. The bottom line goal is that if people are thinking analytics, then O.R should not be far off from their thoughts. Well so far, O.R has been losing this battle on many fronts. Clearly, we do not want to lose our existing membership in any OR-friendly industries (most representative of the ones who showed up). However, we should be doing much more to attract members from the non-traditional, emerging industries (very few of those). At the end of the day, OR is an applied field, and while the analytics turf can be defended in journals, textbooks, and conferences, it can only be won in hard-fought battles by in-the-trenches OR foot soldiers, who need be to well-equipped and trained to build innovative, scalable, practical products and solutions for real-world problems in the 21st century - that is increasingly going to be marked by many terabytes of noisy data. When we start with "min z = c.x + y: ax <= b", O.R academic programs should first be teaching how and where to get the "a, b, c" in this and what is really means, rather than taking a short-cut straight to 'x, y, z' in the abstract world, like we have been doing the last few decades. If you have other questions, ping me and I'll tab it here ...
Clearly some things were out of their control. All the OR folks in the world would not have been able to predict the impact of a volcano in Iceland on the travel plans of overseas visitors to the conference. Also, Dr. Micheal Trick, whose pioneering web page on O.R was the main source of information as well as inspiration for graduate students like me in the 1990s, and motivated me to join this exciting field, was missing, and one can't fault Informs for this. I was really looking forward to shaking his hands and thanking him for his service. 'Marketing in Online Social Spaces,' by Kevin Geraghty, Vice President, Research & Analytics, of 360i was a really good one (somehow I forgot to cover this in my daily conference tab). Kevin was providing an example of marketing campaigns using social networking data. He found out (using completely public domain tools!) that in the OR blog world, to the surprise of many, a certain Lieutenant in the Navy had more 'online friends' than Dr. Trick, so if one were to promote some hypothetical OR product, then he should be chosen as a first reviewer, assuming that those friends were OR types rather than 'sailors'. He also obtained other funny personal trivia from public domain, that I'll just leave out.
A second peeve I had was the highly limited lunch and dinner options for vegetarians (Two boiled asparagus roots and a turdy-looking cuboid of tofu does not an Edelman banquet make!). This should not be difficult to fix. If this doesn't change, I frankly don't see much point in shelling out two grand and semi-starve most of the conference. Thankfully, due to the purely individual initiative of the obviously superb Hilton staff, i was not totally inconvenienced. Kudos to those guys. They got their 'hospitality management OR' right.
Third, attendees should be able to obtain access to the video archive of the talks. Static slides don't cut it anymore. Unlike academic conferences, the value of practice-oriented conferences often lies in what is said in between slides.
On the positive side, the posters were a big hit. One can engage the presenters informally and in 5-10 minutes get a high-level idea of what their innovation is about. And the good thing is that you can visit them in your own time and network too. For example, I found out that the Sandia Labs in beautiful New Mexico, has this really cool Python-based modeling language (PYOMO?) that they used for stochastic programming. Can't wait to try it out. At the MPL booth, I found out that that they are making software available for free on a Windows environment. In tandem with COIN-OR (which they package MPL with, i think), you have a solid modeling and optimization package, free!
My best talks in no particular order - Sanjay Saigal (Intechne) on uncertainty , Jeffrey Cramm (Univ of Cincinnatti) on practical OR, Kevin Geraghty (360i) on social networking, John Osborne (Kroger) on OR innovation against all corporate odds, and any Edelman presentation.
overall grade: 7/10
I'll end on a warning note. The bottom line goal is that if people are thinking analytics, then O.R should not be far off from their thoughts. Well so far, O.R has been losing this battle on many fronts. Clearly, we do not want to lose our existing membership in any OR-friendly industries (most representative of the ones who showed up). However, we should be doing much more to attract members from the non-traditional, emerging industries (very few of those). At the end of the day, OR is an applied field, and while the analytics turf can be defended in journals, textbooks, and conferences, it can only be won in hard-fought battles by in-the-trenches OR foot soldiers, who need be to well-equipped and trained to build innovative, scalable, practical products and solutions for real-world problems in the 21st century - that is increasingly going to be marked by many terabytes of noisy data. When we start with "min z = c.x + y: ax <= b", O.R academic programs should first be teaching how and where to get the "a, b, c" in this and what is really means, rather than taking a short-cut straight to 'x, y, z' in the abstract world, like we have been doing the last few decades. If you have other questions, ping me and I'll tab it here ...
Tuesday, April 20, 2010
Informs Practice Conference 2010 - Day 3
The day started off with an encore presentation by the edelman winner - Indeval. Interesting talk. The theoretical stuff was not particularly interesting, but the fact that they got some OR stuff to work in real time in a mission-critical system, and involving billions of dollars is really cool. Next, i managed to attend a couple of optimization-focused talks on Approximate Dynamic Programming at Schneider Trucking, followed by 'the practice of the alternative' by Dr. Jeffrey Camm from the U of Cincinatti. He is from the Brown-Rosenthal school of practical OR, which i heartily subscribe to as well, and it was probably the most informative talk of the meet for me.
The rest of the day was devoted to the energy industry. We had another plenary by Richard O'Neill, Chief Economic Advisor to the Fed Energy Regulatory Commission. This guy went into some depth on electrical circuits and mixed integer programs. Quite unexpected, but it was great for us optimization practitioners. Then I got to listen to more presentations on energy-related topics including analytics for the smart-grid.
All in all, it was an enjoyable conference, even if one can attend only 10-12 of the 80 presentations on offer. Great location, excellent hotel service. Good job, Informs!
The rest of the day was devoted to the energy industry. We had another plenary by Richard O'Neill, Chief Economic Advisor to the Fed Energy Regulatory Commission. This guy went into some depth on electrical circuits and mixed integer programs. Quite unexpected, but it was great for us optimization practitioners. Then I got to listen to more presentations on energy-related topics including analytics for the smart-grid.
All in all, it was an enjoyable conference, even if one can attend only 10-12 of the 80 presentations on offer. Great location, excellent hotel service. Good job, Informs!
Monday, April 19, 2010
Informs Practice Conference 2010 - Day 2
The day started off with a plenary by a senior guy in Walt Disney. Equally interestingly, he worked at PeopleExpress decades ago, now part of Airline Revenue Management folklore. the key takeaway was that smart OR ultimately improves the odds in your favor by one or two percentage points, and that is a really big deal. Following that, there was an incredible variety of interesting topics to choose from, many of which were scheduled at the same time. So I tried to avoid MBAs, vendors, as well as academic types and listen to the in-the-trenches practice guys. The first one was the head of R&D in Kroger, a group that's 2 years old in an 126-year old company. This talk focused on how to cut thru the (126 years of ) red tape to get genuinely valuable work done. Very interesting. Quotes included "you should be willing to bet your job that your project idea works ..." and a need for passion. Every body's hand in the audience went up when he asked how many people in the audience liked their jobs. Not surprising. Practical OR is fun.
The next interesting talk was by Dr. Sanjay Saigal on probability management. He is a non-conformist and funny, and he put on a real show, and i really wished this talk had continued for another 15-20 mins. Great topic.
All in all, I missed several great talks. If anything, the practice conference has an abundance of riches in terms of the high-quality content presented. I'm distraught that I may have to skip a talk by the uber-brilliant Dr. Ellis Johnson tomorrow to catch another one at the same time that is equally exciting and pertinent to my current line of work.
One of the the 'birds of a feather' discussion in the evening focused on the role of O.R in analytics. I've already talked about the identity crisis facing OR'ers in a prior post, and INFORMS, as well as OR academic programs should act soon to fix this gap. The master of ceremonies for the Edelman awards later mentioned (or paraphrased) that OR is the most important invisible profession in the world today.
Finally, i sat in on an Edelman finalist presentation by the New Brunswick department of Transportation, Canada, since they were my sentimental pick - NB is just three hours further east of my place in Eastern Maine. In the end, the bankers won it. Interestingly, almost every single entry featured a company partnering with a university or a OR software vendor.
Today, I managed to spot two OR all-time greats, Dr. Cynthia Barnhart, and Peter Kolesar. Too bad I did not get a chance to interact with them, given that they were involved as an Edelman judge, and finalist, respectively.
The next interesting talk was by Dr. Sanjay Saigal on probability management. He is a non-conformist and funny, and he put on a real show, and i really wished this talk had continued for another 15-20 mins. Great topic.
All in all, I missed several great talks. If anything, the practice conference has an abundance of riches in terms of the high-quality content presented. I'm distraught that I may have to skip a talk by the uber-brilliant Dr. Ellis Johnson tomorrow to catch another one at the same time that is equally exciting and pertinent to my current line of work.
One of the the 'birds of a feather' discussion in the evening focused on the role of O.R in analytics. I've already talked about the identity crisis facing OR'ers in a prior post, and INFORMS, as well as OR academic programs should act soon to fix this gap. The master of ceremonies for the Edelman awards later mentioned (or paraphrased) that OR is the most important invisible profession in the world today.
Finally, i sat in on an Edelman finalist presentation by the New Brunswick department of Transportation, Canada, since they were my sentimental pick - NB is just three hours further east of my place in Eastern Maine. In the end, the bankers won it. Interestingly, almost every single entry featured a company partnering with a university or a OR software vendor.
Today, I managed to spot two OR all-time greats, Dr. Cynthia Barnhart, and Peter Kolesar. Too bad I did not get a chance to interact with them, given that they were involved as an Edelman judge, and finalist, respectively.
Sunday, April 18, 2010
Informs Practice Conference 2010 - Day 1
Getting from North Eastern Maine to Orlando involved going thru Detriot. For some reason, this US carrier seems to 'dynamically' assign gates at DTW to arriving aircraft, so we "arrived" 30 minutes ahead of schedule, but arrived 30 minutes later. This is not the first time it's happened. Anyway, the weather is Orlando is great compared to Maine which was in the low 40s when I left ...
The workshops on Day 1 were quite useful. Forio Business solutions had some nice system dynamics tools for building snazzy looking web-simulations. I managed to get through one Markdown Optimization example, simulating different price elasticities. The next workshop was enjoyable as well as informative. Getting to to see the legendary Dr. Bixby in person was cool. Gurobi 3.0 now has a parallel barrier solver in place, and I verified that this one is deterministic. Their dev team is sure keeping a fast pace of major releases and their benchmark results continue to impress and I resolved to learn Python. Finally, the third workshop was with OPTMODEL, SAS's versatile modeling and optimization language / procedure. They displayed some nice decomposition approaches to a Kidney exchange and ATM optimization problems, all deployed within OPTMODEL. I felt that the Kidney exchange model (KEM) could have benefited from some specialized TSP subtour constraints, but then again, some nice work on display by the young OR experts from this company.
It was nice to catch up with old airline colleagues, and INFORMS had some vegetarian food, thankfully. Finally, it was nice to meet Dr. Ravi Ahuja, another O.R. giant, in person. These were the stand-out moments for day 1 - a rare chance of interacting with the stalwarts of our discipline in person.
The workshops on Day 1 were quite useful. Forio Business solutions had some nice system dynamics tools for building snazzy looking web-simulations. I managed to get through one Markdown Optimization example, simulating different price elasticities. The next workshop was enjoyable as well as informative. Getting to to see the legendary Dr. Bixby in person was cool. Gurobi 3.0 now has a parallel barrier solver in place, and I verified that this one is deterministic. Their dev team is sure keeping a fast pace of major releases and their benchmark results continue to impress and I resolved to learn Python. Finally, the third workshop was with OPTMODEL, SAS's versatile modeling and optimization language / procedure. They displayed some nice decomposition approaches to a Kidney exchange and ATM optimization problems, all deployed within OPTMODEL. I felt that the Kidney exchange model (KEM) could have benefited from some specialized TSP subtour constraints, but then again, some nice work on display by the young OR experts from this company.
It was nice to catch up with old airline colleagues, and INFORMS had some vegetarian food, thankfully. Finally, it was nice to meet Dr. Ravi Ahuja, another O.R. giant, in person. These were the stand-out moments for day 1 - a rare chance of interacting with the stalwarts of our discipline in person.
Monday, April 12, 2010
Doogie, Darwin, Dowry, and the TSP
A couple of teenagers from the U.S. visited the beautiful IIT campus in Madras (Chennai), India in 1989-90. They were not there to attend the popular collegiate cultural festival 'Mardi Gras' as it was known back in those days, but to present a research paper on AIDS. They happened to be brothers, Balamurali Ambati, and Jayakrishna Ambati, who completed medical school at a fairly young age. Per Wikipedia, BA graduated from the Mount Sinai school of medicine at the age of 13, and become a qualified doctor at 17 in 1995.
Today, the Ambani brothers hog the media space in India as they seek to become richer, but for a brief while in the 1990s, the elder Ambati brother got entangled in a 'dowry harassment' scandal. Dowry harassment reports was big news in India, with the per-capita dowry-deaths in line with the number of 'murder for insurance' cases in the US, or the wife-beating cases in Switzerland. Anyway, reports indicate that the case fell apart after the bride's father was recorded on tape trying to extort a few hundred big ones in blackmail money. Unfortunately for the elder brother, it looks like like he had to cool his heels in India until this case was wholly resolved, losing a good two years in the "youngest achiever" race, which has since become an idiotic, even deadly craze in Southern India. This is in contrast with the more comical approach in Northern India and Pakistan, where many kids are 2-5 years older than their official age. If you were that skinny, baby-faced runt in a middle school in Bangalore, he would be that guy with the stubble in the last bench, and the captain of your school's football (soccer) and (field-) hockey teams.
Pardon the digression. Around the time the Ambati brothers visited the IITM campus (A former student reminisces here) to talk about AIDS, they were also the primary authors of this published paper on the traveling salesman problem. The title is exciting, but a tad misleading, in that it hints at a polynomial time algorithm for the NP-Hard TSP. It resembles a randomized heuristic approach based on the theory of natural selection, and appears to possess good computational properties, and has been cited more than once in followup research in this area. On the other hand, I don't think even Doogie did any OR work, real or fictional.
Today, the Ambani brothers hog the media space in India as they seek to become richer, but for a brief while in the 1990s, the elder Ambati brother got entangled in a 'dowry harassment' scandal. Dowry harassment reports was big news in India, with the per-capita dowry-deaths in line with the number of 'murder for insurance' cases in the US, or the wife-beating cases in Switzerland. Anyway, reports indicate that the case fell apart after the bride's father was recorded on tape trying to extort a few hundred big ones in blackmail money. Unfortunately for the elder brother, it looks like like he had to cool his heels in India until this case was wholly resolved, losing a good two years in the "youngest achiever" race, which has since become an idiotic, even deadly craze in Southern India. This is in contrast with the more comical approach in Northern India and Pakistan, where many kids are 2-5 years older than their official age. If you were that skinny, baby-faced runt in a middle school in Bangalore, he would be that guy with the stubble in the last bench, and the captain of your school's football (soccer) and (field-) hockey teams.
Pardon the digression. Around the time the Ambati brothers visited the IITM campus (A former student reminisces here) to talk about AIDS, they were also the primary authors of this published paper on the traveling salesman problem. The title is exciting, but a tad misleading, in that it hints at a polynomial time algorithm for the NP-Hard TSP. It resembles a randomized heuristic approach based on the theory of natural selection, and appears to possess good computational properties, and has been cited more than once in followup research in this area. On the other hand, I don't think even Doogie did any OR work, real or fictional.
Sunday, April 11, 2010
All Set for the INFORMS Practice Conference
Wonders never cease. One advantage of working for a solvent company is that it provides a rare chance of attending a major conference within the U.S. The INFORMS practice conference seemed like a good choice. Besides, the annual INFORMS conference is a few months away, and one never knows how the travel budget is gonna change. A greedy approach works better here... It's been eons since the previous conference - not surprising if you spent dog-years in the mostly-bankrupt airline industry.
Given the short notice, I'm presenting absolutely nothing, get four days off from work to listen to cool OR guys talk, and the plan is just to learn as much as possible and be an on-site reporter. Please email me at shivaram (dot) subramanian (at) gmail.com, if you are interested in talking OR during the meet. I will be posting daily tabs of the conference here, so watch this space. If you would have liked to be at the conference but could not make it, please email me any topics you would like me to cover here, and I will do my best. As always, any tips on the optimal way to cover conferences is welcome.
For those practitioners interested in the costs involved, here's the lowdown. Airfare is about 400$. 3-4 day hotel stay is about 700-900$ (now I know how it feels to be on the receiving side of Pricing optimization). I'm staying an additional day (Sunday) to take advantage of the technology workshops and network. Registration fees for non-members is about 900$. The total cost, including daily expenses is in the ballpark of $2500. It remains to be seen if the feedback and new ideas that one can get out of this outweighs these costs. Last year's Edelman work was fantastic, and hopefully this year will be just as good.
Given the short notice, I'm presenting absolutely nothing, get four days off from work to listen to cool OR guys talk, and the plan is just to learn as much as possible and be an on-site reporter. Please email me at shivaram (dot) subramanian (at) gmail.com, if you are interested in talking OR during the meet. I will be posting daily tabs of the conference here, so watch this space. If you would have liked to be at the conference but could not make it, please email me any topics you would like me to cover here, and I will do my best. As always, any tips on the optimal way to cover conferences is welcome.
For those practitioners interested in the costs involved, here's the lowdown. Airfare is about 400$. 3-4 day hotel stay is about 700-900$ (now I know how it feels to be on the receiving side of Pricing optimization). I'm staying an additional day (Sunday) to take advantage of the technology workshops and network. Registration fees for non-members is about 900$. The total cost, including daily expenses is in the ballpark of $2500. It remains to be seen if the feedback and new ideas that one can get out of this outweighs these costs. Last year's Edelman work was fantastic, and hopefully this year will be just as good.
Saturday, March 27, 2010
Analytics and Cricket - II : The IPL effect
This is second in the series of articles on O.R. and cricket. Click here for the first part, done a while ago.
The Indian Premier League (IPL) is close to becoming the number one Indian global brand - not just the number one sports brand. It has overtaken past colonial stereotypes (such as snake charmers, elephants, and Maharajahs), current pop stereotypes (IT outsourcing brands like Infosys, Wipro, et al, knowledge-brands like the IIT graduate, etc). The two newest franchise teams unveiled in this fledgling three-year old league were purchased for $333M, costing more than a couple of current NHL teams. Sports has become big business, even as the cricket fan in me rebels against this. Several owners have 'Bollywood' connections. Not surprising, given that these movie types make so many expensive flops year after year, the risk level for a cricket venture is surely much lower.
This IPL season is on YouTube now after a pioneering deal with Google, and this experiment serves as a nice dress rehearsal for the search engine company toward more such live streaming ventures in the future. In terms of audience size, it's easily a factor of ten-twenty bigger than that for NCAA basketball. India has a lot of cricket-crazy people. I've provided the YouTube link for my favorite match of the tournament so far: Bangalore v Mumbai. This is the shortest form of cricket played where each innings lasts twenty overs and the entire game is completed in three hours.
We will cover two new analytical induced innovations observed in this season's IPL.
First, the number of run-outs (analogous to a baseball strike-out where a player doesn't make it to a base in time) seems to have increased dramatically. Why? It looks like team statisticians have noticed that a traditionally weak area of teams is fielding and the probability of a direct hit on the stumps is low. This reduces the risk of getting run-out and the reward for stealing an additional run against statistically poor fielding teams may be well worth the risk. Teams that do not improve their fielding will probably see this hit-probability decrease. Teams will take more chances against you and more members in your team will have the opportunity to show-case their non-athletic, keystone kops-like fielding prowess leading to a deterioration in stats. Conversely, good fielding teams can improve their hit-probability stats and reap the reward in terms of effecting more run-outs. Teams of both kinds can be seen. The ones adopting better fielding standards are at the top of the points table.
A second analytic innovation is the form of a special T-20 (twenty-over cricket) bat and is now the most famous mongoose in India (that's the brand name for this bat). It has a handle as long as the blade itself, with the total length of the bat itself being constant. Statistics show that in this form of the game, oftentimes, half a bat is often better than a full-one, if optimally designed! Don't believe it? See this YouTube clip of Matt "the bat" Hayden, the first player in the IPL to use this bat. He is certainly not going to be the last.
So why is the mongoose effective? In the most serious form of cricket (test cricket), a full bat is a must. It's a longer game (over 5 days) and the chances of getting out is much, much higher over time and you want a bat as large as a barn door to prevent the ball from disturbing your stumps. From the T20 perspective, the ball travels the longest when it hits the sweet spot of the bat (roughly three-fourth of the way down a bat), and combined with the fact that getting out in T20 is not such a big deal, you end up with the mongoose, which is essentially just a long handle and a reinforced lower half, like a pendulum. It's made of wood just like the traditional bat, just as long, and roughly the same weight. For a given period of time at the crease, you are more likely to get out using the mongoose, but the expected number of runs (specifically in the form of hitting sixers) you could score before that happens can be much higher, thus making it an attractive trade-off in certain T20 match situations.
The Indian Premier League (IPL) is close to becoming the number one Indian global brand - not just the number one sports brand. It has overtaken past colonial stereotypes (such as snake charmers, elephants, and Maharajahs), current pop stereotypes (IT outsourcing brands like Infosys, Wipro, et al, knowledge-brands like the IIT graduate, etc). The two newest franchise teams unveiled in this fledgling three-year old league were purchased for $333M, costing more than a couple of current NHL teams. Sports has become big business, even as the cricket fan in me rebels against this. Several owners have 'Bollywood' connections. Not surprising, given that these movie types make so many expensive flops year after year, the risk level for a cricket venture is surely much lower.
This IPL season is on YouTube now after a pioneering deal with Google, and this experiment serves as a nice dress rehearsal for the search engine company toward more such live streaming ventures in the future. In terms of audience size, it's easily a factor of ten-twenty bigger than that for NCAA basketball. India has a lot of cricket-crazy people. I've provided the YouTube link for my favorite match of the tournament so far: Bangalore v Mumbai. This is the shortest form of cricket played where each innings lasts twenty overs and the entire game is completed in three hours.
We will cover two new analytical induced innovations observed in this season's IPL.
First, the number of run-outs (analogous to a baseball strike-out where a player doesn't make it to a base in time) seems to have increased dramatically. Why? It looks like team statisticians have noticed that a traditionally weak area of teams is fielding and the probability of a direct hit on the stumps is low. This reduces the risk of getting run-out and the reward for stealing an additional run against statistically poor fielding teams may be well worth the risk. Teams that do not improve their fielding will probably see this hit-probability decrease. Teams will take more chances against you and more members in your team will have the opportunity to show-case their non-athletic, keystone kops-like fielding prowess leading to a deterioration in stats. Conversely, good fielding teams can improve their hit-probability stats and reap the reward in terms of effecting more run-outs. Teams of both kinds can be seen. The ones adopting better fielding standards are at the top of the points table.
A second analytic innovation is the form of a special T-20 (twenty-over cricket) bat and is now the most famous mongoose in India (that's the brand name for this bat). It has a handle as long as the blade itself, with the total length of the bat itself being constant. Statistics show that in this form of the game, oftentimes, half a bat is often better than a full-one, if optimally designed! Don't believe it? See this YouTube clip of Matt "the bat" Hayden, the first player in the IPL to use this bat. He is certainly not going to be the last.
So why is the mongoose effective? In the most serious form of cricket (test cricket), a full bat is a must. It's a longer game (over 5 days) and the chances of getting out is much, much higher over time and you want a bat as large as a barn door to prevent the ball from disturbing your stumps. From the T20 perspective, the ball travels the longest when it hits the sweet spot of the bat (roughly three-fourth of the way down a bat), and combined with the fact that getting out in T20 is not such a big deal, you end up with the mongoose, which is essentially just a long handle and a reinforced lower half, like a pendulum. It's made of wood just like the traditional bat, just as long, and roughly the same weight. For a given period of time at the crease, you are more likely to get out using the mongoose, but the expected number of runs (specifically in the form of hitting sixers) you could score before that happens can be much higher, thus making it an attractive trade-off in certain T20 match situations.
Tuesday, March 16, 2010
The Dangerfield Syndrome - To Patent, Publish or Present?
Before we start, as always, this tab entry is purely a product of personal opinion, watching much comedy central, and is solidly based on the wild conspiracy theory (if this isn't dual noise, what is?) It is completely unrelated to any real-world company or university, including the ones I worked for in the past or present. On the other hand, it is well and truly dedicated to every Rodney Dangerfield in O.R. practice (you belong to this club only if you already knew that).
In academia, a professor would be happy to get their work published in a leading peer-reviewed O.R journal and often, this acceptance defines the degree of success or failure of a research project. Almost all new ideas in a university make it to some journal or conference, but very few practical innovations in the industry gain visibility. They are either patented, or remain hidden as a company's intellectual property. At the end of the day, the Edelmans, like the Oscars, are as much (and probably more) a tribute to an organization's upper management for being O.R. friendly, as it is to the guys in the trenches who actually pull off the O.R. innovations.
So how does the common O.R. guy in the company gain peer-recognition? (suggestions most welcome here :-)
After prototyping is done, what does the practitioner do? Sit around tooling, while waiting for analytical support calls ? There are a few choices (or few choices if you have a bad boss), depending on when your next project starts. You can try to patent the most practical approach. After all, in practice, the proof is in the pudding. Or you can publish a non-proprietary version of your findings. A cynic may say that what this really means is that you released an 'unpractical' version, but this not the case - at least not always. Patenting is common place in the IT world, but a relatively less popular option in the O.R. world. A patent in your resume can make you look more 'result-oriented'. A future employer may be worried if he/she observes way too many publications during your past job ("so did u do any real work this millennium?").
On the other hand, if you plan to move on to academia after a while, publishing is not such a bad idea. And it's not bad for a future industry job as well, since it serves as a solid, peer-reviewed reference for your scientific skills. It's also a good marketing and recruiting tool, since your company gains recognition in the scientific community as a place that promotes cutting edge R&D. A good manager would recognize these benefits. A common fear that is unfounded is that publishing = giving away your intellectual property. As long as you keep your engineering ideas out of scope, my own experience is that it can be counter-productive for a competitor to try and directly reproduce a product from a journal paper that appears in print 2-4 years after the idea was implemented in a product (or not). Add to that a couple of years that it takes to go from idea to finished product, and you get the picture. Then you realize that you would have been better off building something for today's customer on your own, rather than relying on recycled ideas.
Patenting versus publishing is a personal choice. Acads are shocked to see frivolous patents. But for every seemingly hideous patent, there exists a new publication for yet another factor-of-2 approximation algorithm for an NP-Hard problem that can be readily 'solved' in milliseconds for real data instances. The answer to the original question is not clear cut. And what about presenting at conferences?
Nowadays, companies count the number of patent applications filed by their R&D group. If such a metric is employed, perhaps patenting should be the first option. This scenario is more likely if you work in a non-traditional O.R. industry, and in such a situation, presenting at a conference can provide visibility and is a nice trade-off. Conference presentations takes up much less time than publishing, and you can get your ideas out there quickly. On the other hand, conference proceedings seem to be valued less in our field. Perhaps practitioners can present more of their work at a dedicated conference. These should be a conference of practitioners, and organized by, and for the benefit of practitioners. That would be wicked. Why the Gettysburg clause you ask? When profs meet, its an conference, but when practitioners and engineers meet, its labeled a workshop. As we all know, when the rocket stays up, it's hailed a scientific success, and if not, its an engineering failure. There's still no respect for 'R&D'.
In academia, a professor would be happy to get their work published in a leading peer-reviewed O.R journal and often, this acceptance defines the degree of success or failure of a research project. Almost all new ideas in a university make it to some journal or conference, but very few practical innovations in the industry gain visibility. They are either patented, or remain hidden as a company's intellectual property. At the end of the day, the Edelmans, like the Oscars, are as much (and probably more) a tribute to an organization's upper management for being O.R. friendly, as it is to the guys in the trenches who actually pull off the O.R. innovations.
So how does the common O.R. guy in the company gain peer-recognition? (suggestions most welcome here :-)
After prototyping is done, what does the practitioner do? Sit around tooling, while waiting for analytical support calls ? There are a few choices (or few choices if you have a bad boss), depending on when your next project starts. You can try to patent the most practical approach. After all, in practice, the proof is in the pudding. Or you can publish a non-proprietary version of your findings. A cynic may say that what this really means is that you released an 'unpractical' version, but this not the case - at least not always. Patenting is common place in the IT world, but a relatively less popular option in the O.R. world. A patent in your resume can make you look more 'result-oriented'. A future employer may be worried if he/she observes way too many publications during your past job ("so did u do any real work this millennium?").
On the other hand, if you plan to move on to academia after a while, publishing is not such a bad idea. And it's not bad for a future industry job as well, since it serves as a solid, peer-reviewed reference for your scientific skills. It's also a good marketing and recruiting tool, since your company gains recognition in the scientific community as a place that promotes cutting edge R&D. A good manager would recognize these benefits. A common fear that is unfounded is that publishing = giving away your intellectual property. As long as you keep your engineering ideas out of scope, my own experience is that it can be counter-productive for a competitor to try and directly reproduce a product from a journal paper that appears in print 2-4 years after the idea was implemented in a product (or not). Add to that a couple of years that it takes to go from idea to finished product, and you get the picture. Then you realize that you would have been better off building something for today's customer on your own, rather than relying on recycled ideas.
Patenting versus publishing is a personal choice. Acads are shocked to see frivolous patents. But for every seemingly hideous patent, there exists a new publication for yet another factor-of-2 approximation algorithm for an NP-Hard problem that can be readily 'solved' in milliseconds for real data instances. The answer to the original question is not clear cut. And what about presenting at conferences?
Nowadays, companies count the number of patent applications filed by their R&D group. If such a metric is employed, perhaps patenting should be the first option. This scenario is more likely if you work in a non-traditional O.R. industry, and in such a situation, presenting at a conference can provide visibility and is a nice trade-off. Conference presentations takes up much less time than publishing, and you can get your ideas out there quickly. On the other hand, conference proceedings seem to be valued less in our field. Perhaps practitioners can present more of their work at a dedicated conference. These should be a conference of practitioners, and organized by, and for the benefit of practitioners. That would be wicked. Why the Gettysburg clause you ask? When profs meet, its an conference, but when practitioners and engineers meet, its labeled a workshop. As we all know, when the rocket stays up, it's hailed a scientific success, and if not, its an engineering failure. There's still no respect for 'R&D'.
Tuesday, February 23, 2010
O.R. Ball - Defense and Offense
Often times, the difference between the theory and practice of O.R. is the same as that between a carefully choreographed brain surgery at Boston General versus brain repair performed at the MASH 4077. In the former case, the idea is come back with a generic breakthrough that can be safely re-used for all surgeries. In the latter case, the aim is to get through the Korean war (on TV albeit) with the least number of casualties among the huge number of wounded that randomly show up.
The textbook approach prides itself on being data-agnostic and coming up with new "small polynomial" techniques that exploit special structure in a problem class. Data agnosticism is something we practitioners can ill-afford since the proof is in the pudding. If we build the best recipe that is optimized for no more than 6-7 guests, and we never expect to see more than ten guests, ever, then it is pointless to fuss over a complex recipe-generator that optimally serves a thousand guests, but can't serve five as quickly and as well. And if there are thousands of such five-guest parties to be served, the former approach wins hands down. Of course, now if we were optimally designing a dam or two, things would be a little different since strict service levels come into play. So it is case dependent, and bringing your skill and art into play to deal with such differences and putting your money where your model is, happens to be one of the reasons why O.R. practice is never dull.
Textbooks recommend that we exploit problem structure. Industrial optimization exploits problem structure to an extent, but can and should exploit the structure in data. The Simplex method that is making such a robust comeback via GUROBI is worst-case exponential and rarely does a bad job because good implementations thrive on real-world LP data and the numerical properties of computers. In the real world, even strongly NP-hard optimization problems are quite manageable. Do not let textbooks scare you! After all, there were no computers around when architects in South India optimally designed the Brihadeeswara temple a thousand years ago - the design required that the tall temple's shadow be constrained to within its (convex) perimeter any time of day. They achieved an elegant and 'feasible' design that stands 216 feet high while using incredibly heavy granite stones to do this. And yes, the location of the symbolic 'idol' of the deity coincides with the centroid of the overall structure. Now this has got to be one great place for spiritual reflection or thinking O.R!
It's a strange dichotomy in the global O.R. community. The universities are focused on 'defense' and abhor seeing any "2^n" kind of numbers anywhere - a systematic O.R. approach that generates lower bounds to protect your answers, but is also capable of forcing turnovers, i.e., can be quickly converted into good quality solutions with a little bit of imagination. The book on the Traveling Salesman Problem by the stalwarts of our discipline is quite fascinating in this context. The O.R. business community goes with whatever brings home the Dosa, exponential or otherwise, and there is healthy scorn for the 'defense' part. Approximation / local optimum / randomized methods can be thought of as being part of the 'offense'. It gets the glory and can be used to quickly initiate a product, and that perhaps is a reason why many a young practitioner is hooked to it. However, it is generally a good idea to consider using both approaches. Over the product life cycle, it is usually the best defense-offense combo that wins the game, all other things being equal.
The textbook approach prides itself on being data-agnostic and coming up with new "small polynomial" techniques that exploit special structure in a problem class. Data agnosticism is something we practitioners can ill-afford since the proof is in the pudding. If we build the best recipe that is optimized for no more than 6-7 guests, and we never expect to see more than ten guests, ever, then it is pointless to fuss over a complex recipe-generator that optimally serves a thousand guests, but can't serve five as quickly and as well. And if there are thousands of such five-guest parties to be served, the former approach wins hands down. Of course, now if we were optimally designing a dam or two, things would be a little different since strict service levels come into play. So it is case dependent, and bringing your skill and art into play to deal with such differences and putting your money where your model is, happens to be one of the reasons why O.R. practice is never dull.
Textbooks recommend that we exploit problem structure. Industrial optimization exploits problem structure to an extent, but can and should exploit the structure in data. The Simplex method that is making such a robust comeback via GUROBI is worst-case exponential and rarely does a bad job because good implementations thrive on real-world LP data and the numerical properties of computers. In the real world, even strongly NP-hard optimization problems are quite manageable. Do not let textbooks scare you! After all, there were no computers around when architects in South India optimally designed the Brihadeeswara temple a thousand years ago - the design required that the tall temple's shadow be constrained to within its (convex) perimeter any time of day. They achieved an elegant and 'feasible' design that stands 216 feet high while using incredibly heavy granite stones to do this. And yes, the location of the symbolic 'idol' of the deity coincides with the centroid of the overall structure. Now this has got to be one great place for spiritual reflection or thinking O.R!
It's a strange dichotomy in the global O.R. community. The universities are focused on 'defense' and abhor seeing any "2^n" kind of numbers anywhere - a systematic O.R. approach that generates lower bounds to protect your answers, but is also capable of forcing turnovers, i.e., can be quickly converted into good quality solutions with a little bit of imagination. The book on the Traveling Salesman Problem by the stalwarts of our discipline is quite fascinating in this context. The O.R. business community goes with whatever brings home the Dosa, exponential or otherwise, and there is healthy scorn for the 'defense' part. Approximation / local optimum / randomized methods can be thought of as being part of the 'offense'. It gets the glory and can be used to quickly initiate a product, and that perhaps is a reason why many a young practitioner is hooked to it. However, it is generally a good idea to consider using both approaches. Over the product life cycle, it is usually the best defense-offense combo that wins the game, all other things being equal.
Tuesday, February 9, 2010
Blog Review: The Age of Analytics
Web-tooling on the Monday after Superbowl for analytic content, I found this great new blog 'The Analytic Age' for O.R. practitioners and enthusiasts alike.
About the author - Dr. V. Ramakrishnan, has a strong background in O.R. and has a distinguished track record as an analytics entrepreneur, among other things. He currently teaches at the MIT Sloan School, and was VP-Chief Scientist at Profit Logic (and later at Oracle Retail), a company which designed and built the hugely successful Markdown Optimization Product, and was acquired by Oracle a few years ago.
The first commentary on analytics in the blog provides valuable real-world insight into the practice of this craft. It is encouraging to learn that businesses are looking for believable and optimized decision analytic tools. The kind of 'A' that is not just about how cool the technology inside is, or how good it looks on the outside, or leaves you stranded with an impressive array of numbers and summary reports; rather, the kind of 'A' that provides reliable prescriptive answers based on sound business sense. These business requirements directly play to the strength of O.R.
There's magic, and then there is practical magic. The latter is more likely to bring the audiences back.
About the author - Dr. V. Ramakrishnan, has a strong background in O.R. and has a distinguished track record as an analytics entrepreneur, among other things. He currently teaches at the MIT Sloan School, and was VP-Chief Scientist at Profit Logic (and later at Oracle Retail), a company which designed and built the hugely successful Markdown Optimization Product, and was acquired by Oracle a few years ago.
The first commentary on analytics in the blog provides valuable real-world insight into the practice of this craft. It is encouraging to learn that businesses are looking for believable and optimized decision analytic tools. The kind of 'A' that is not just about how cool the technology inside is, or how good it looks on the outside, or leaves you stranded with an impressive array of numbers and summary reports; rather, the kind of 'A' that provides reliable prescriptive answers based on sound business sense. These business requirements directly play to the strength of O.R.
There's magic, and then there is practical magic. The latter is more likely to bring the audiences back.
Wednesday, January 20, 2010
The iPod and the O.R. Designer
The web is full of business articles that extol the benefits of a great, overall user experience that has come to define the iPod product. These articles roughly state that we knowingly accepted a discrete approximation of continuous music and enjoyed the experience because it was delivered in a really convenient manner. Business commentators even go so far as to say that features don't matter any more. Not surprisingly, there are huge lessons to be learned for those us who make a living by building apps with 'O.R. inside' for business end-users.
If your OR program in school was part of the Industrial and Systems Engineering (ISE) department, then we would recognize that these are not novel ideas. We have seen this before in our ISE labs and it resembles 'Human Factors' engineering. It was always exciting to know what the HFE guys were up to - they were generally designing all this ergonomic stuff - keyboards, wheelchairs, etc. We O.R. grads were optimizing big-deal industrial problems with sophisticated math and did not pay enough attention to such "low-tech and qualitative" HFE ideas. Designing for an optimal business-user experience can be thought of as an area that combines OR with HFE. First a look at a partial checklist of design questions that we can relate to.
Does your O.R. app:
- require the user to have an O.R. PhD to operate?
- feel paranoid about the sophistication of the O.R. technology inside?
- obsess about run time with little regard for quality?
- have bewildering layers of menus filled with dials for 'costs', 'penalties', etc.?
- change answers wildly with small changes in input?
- come back with a blank stare if it encounters infeasibility somewhere inside?
...
You get the idea. An O.R. practitioner not only has to worry about the stuff that is inside - traditionally, we've been good analysts and trained for that, but it's about time we become equally good designers - few, if any, O.R. graduate programs teach that. Perhaps we should be teaming up with our next door HFE neighbors on this. As we continue to explore this theme, we will notice interesting connections between what's inside and outside the app.
If your OR program in school was part of the Industrial and Systems Engineering (ISE) department, then we would recognize that these are not novel ideas. We have seen this before in our ISE labs and it resembles 'Human Factors' engineering. It was always exciting to know what the HFE guys were up to - they were generally designing all this ergonomic stuff - keyboards, wheelchairs, etc. We O.R. grads were optimizing big-deal industrial problems with sophisticated math and did not pay enough attention to such "low-tech and qualitative" HFE ideas. Designing for an optimal business-user experience can be thought of as an area that combines OR with HFE. First a look at a partial checklist of design questions that we can relate to.
Does your O.R. app:
- require the user to have an O.R. PhD to operate?
- feel paranoid about the sophistication of the O.R. technology inside?
- obsess about run time with little regard for quality?
- have bewildering layers of menus filled with dials for 'costs', 'penalties', etc.?
- change answers wildly with small changes in input?
- come back with a blank stare if it encounters infeasibility somewhere inside?
...
You get the idea. An O.R. practitioner not only has to worry about the stuff that is inside - traditionally, we've been good analysts and trained for that, but it's about time we become equally good designers - few, if any, O.R. graduate programs teach that. Perhaps we should be teaming up with our next door HFE neighbors on this. As we continue to explore this theme, we will notice interesting connections between what's inside and outside the app.
Subscribe to:
Posts (Atom)