The exit polls for the recently concluded Indian elections threw up a spectrum of results. Several Cable-TV networks ran their own polls, most of their numbers falling within a seemingly reasonable range, barring a public research group called 'Todays Chanakya', whose numbers were literally off the charts, predicting a massive win for Narendra Modi. People began to take averages of these polls to come up with an 'expected result', and many of these 'poll of polls' excluded TC's result as an outlier, discarding it as unbelievable.
I spent quite a bit of time looking at the meager information provided in the todayschanakya.com (TC) website before the results were announced. Buzz-words aside, what caught my attention was the meticulous attention they paid toward obtaining a representative data sample in every single constituency. Their prior track record in predicting elections in India was simply stunning. In a recent state election too, their prediction was an outlier, and turned out to be accurate. This data sampling step is important, especially given the incredibly diverse nature of India's population. Translating projected vote-shares into actual seats won in India's 'first past the post' system is an incredibly daunting problem. If your sample is even slightly messed up, then your seat predictions can be way off, regardless of the sophistication of the predictive analytics you employ. Human judgment and domain expertise is critical.
As this useful blog points out, it's not about 'sampling error', but sampling bias. And once we see this, it is not difficult to see why the English TV networks of India, virtually every single one a willing and well-compensated participant in the witch hunt of Narendra Modi since 2002, miserably fail in their predictions, time and again. Their reporting has rarely been fact-driven, and is usually ratings-driven. Few, if any on their payroll, are trained in the rigorous scientific method. Reporters appear to be hired based on ideology, west-accented English-speaking ability, and political connections rather than merit or technical proficiency. So, when by force of habit, you look for a sample that you like, then you will only get the predictions you want viewers to see in your TV shows, which has little to do with reality. The media witch hunt against Modi, like their exit polls, as is now known, was never fact-driven from day one. It was doomed from the start. After this election, few will take their "predictions" seriously again unless they reform.
TC's predictions were quite accurate. Modi indeed won in a landslide as they predicted, with the incumbent Nehru dynasty (aka "UPA" coalition) whose corruption almost surely qualifies as a crime against humanity, getting deservedly annihilated. On election day, at around 1-2:00 AM EST, while following the election trends, UPA was leading in about a hundred of the 543 seats up for grabs, way higher the predicted range of 61-79 seats that TC predicted they would get. However, as the day progressed, it was amazing to see UPA's leads petering out one by one, as if an invisible rope was magically pulling it back into the predicted range. Statistical destiny. Only two people appeared to be convinced about the result before May 16. TC, who adopted a scientific approach to gathering and analyzing data, and Narendra Modi, who created the history in the first place. Both of them dared to be different and put their reputations on the line, and were worthy winners.
This election result and Modi becoming the Prime Minister of India has taught many of us a scientific lesson. Data science is about being guided by facts, not emotion, or prejudiced opinion, or preferred outcome. Carefully constructed fact-driven methods are less likely to fail. Gujarat's development, both rural and urban, spearheaded by Modi for 12 years, is real, and cannot be falsified. It happened, and it is there to be seen regardless of what the New York Times tells you. I blogged in 2012 that the heavy-lifting done in Gujarat may pay rich dividends in the future. The people there lived that development and they knew, and the thousands of migrants returned from Gujarat to other states to speak about their experience there. TC's data sample accurately reflected this reality. The media-heads sitting in Delhi, London, and New York were high on ideology-meth, low on fact. Few visited the state of Gujarat to make a factual assessment. Some of the open-minded critics who did, ended up becoming Modi's strongest supporters. Not surprisingly, his fact-driven campaign won him every single parliamentary seat there. The amazing number of Indians cutting across religious, class, language, age, gender, and geographical 'barriers', who voted for Modi, too cannot be brushed aside. Facts cannot be ignored until time-travel becomes practical.
And here's another prediction, an easy one. Modi will probably become India's best, and most unifying leader since Mahatma Gandhi, if he isn't already that. If, as the Nehru dynasty says, "power is poison", India has surely found their Shiva.
Showing posts with label modeling corruption in OR models. Show all posts
Showing posts with label modeling corruption in OR models. Show all posts
Wednesday, May 21, 2014
Monday, December 30, 2013
Indian Intellectuals and the Fighter-Pilot Syndrome
Update: title changed
Legendary formula car racer Michael Schumacher suffered a serious injury in a skiing fall. As millions around the world pray for his safe recovery, a troubling question was triggered by this sad news:
"How likely is it for a skiing enthusiast, who is known to have made a successful career in the superfast and dangerous world of Formula car racing, to meet with a skiing accident?"
Does this conditional probability increase or decrease? I am not aware that Schumi claimed he was a skiing expert or thought of himself as one. This is just a sample of one and could just be a tragic coincidence. The question remains open and the focus of this post is on a related topic.
Here's a wikipedia blurb on a US Air Force officer John Stapp:
"During his work at Holloman Air Force Base, Stapp became interested in the implications of his work for car safety. At the time, cars were generally not fitted with seatbelts, but Stapp had shown that a properly restrained human could survive far greater impacts than an unrestrained one. Many traffic-accident deaths were therefore avoidable but for the lack of seatbelts. Stapp became a strong advocate and publicist for this cause, frequently steering interviews onto the subject, organizing conferences, and staging demonstrations (including the first known use of automobile crash test dummies). At one point, the military objected to funding work they believed was outside their purview, but they were persuaded when Stapp gave them statistics showing that more Air Force pilots were killed in traffic accidents than in plane crashes. The culmination of his efforts came in 1966 when Stapp witnessed Lyndon B. Johnson sign the law making manufacture of cars with seatbelts (lapbelts at that time) compulsory..."
Controlling fast jets did not give those pilots additional skills that made them equally safe at driving cars at some speed. Is it possible that this 'fighter pilot effect' gave them a false sense of security while driving the much slower motor cars? Similarly, safely driving ultra-fast cars shouldn't automatically make one an equally safe hi-speed skiing expert (update: initial reports indicate Schumacher was not going very fast). However, public belief in this 'fighter pilot syndrome' appears to exist at some level, and this is especially true in India. For example, if you win a Nobel Prize or for that matter, any prize in the west, then regardless of your field of expertise and your near-total ignorance about what makes India tick, you are given special powers that turn you into an expert on every topic under the sun (especially Indian culture and politics), overnight. Unlike Marxist economist Amartya Sen or India's egoistic movie stars, who don't need a second invitation, there are others who prefer not to make a fool of themselves in public. However, the Indian media does not spare them the embarrassment by demanding their "fighter pilot" advice on unrelated topics. This 'intellectual celebrity' feedback is then used to try and influence public opinion. A good example is the recent NDTV-25 debate panel on "secularism in India" compered by 2G-scam tainted journalist Barkha Dutt that included exactly one genuine expert, Arun Shourie, who knew what he was talking about, and bunch of other "experts".
All-weather experts and their Indian media co-pilots must be asked to wear their seat-belts and slow down before they take the Indian public for a ride.
Happy New Year. Drive Safe. Get well soon, Schumi.
Legendary formula car racer Michael Schumacher suffered a serious injury in a skiing fall. As millions around the world pray for his safe recovery, a troubling question was triggered by this sad news:
"How likely is it for a skiing enthusiast, who is known to have made a successful career in the superfast and dangerous world of Formula car racing, to meet with a skiing accident?"
Does this conditional probability increase or decrease? I am not aware that Schumi claimed he was a skiing expert or thought of himself as one. This is just a sample of one and could just be a tragic coincidence. The question remains open and the focus of this post is on a related topic.
Here's a wikipedia blurb on a US Air Force officer John Stapp:
"During his work at Holloman Air Force Base, Stapp became interested in the implications of his work for car safety. At the time, cars were generally not fitted with seatbelts, but Stapp had shown that a properly restrained human could survive far greater impacts than an unrestrained one. Many traffic-accident deaths were therefore avoidable but for the lack of seatbelts. Stapp became a strong advocate and publicist for this cause, frequently steering interviews onto the subject, organizing conferences, and staging demonstrations (including the first known use of automobile crash test dummies). At one point, the military objected to funding work they believed was outside their purview, but they were persuaded when Stapp gave them statistics showing that more Air Force pilots were killed in traffic accidents than in plane crashes. The culmination of his efforts came in 1966 when Stapp witnessed Lyndon B. Johnson sign the law making manufacture of cars with seatbelts (lapbelts at that time) compulsory..."
Controlling fast jets did not give those pilots additional skills that made them equally safe at driving cars at some speed. Is it possible that this 'fighter pilot effect' gave them a false sense of security while driving the much slower motor cars? Similarly, safely driving ultra-fast cars shouldn't automatically make one an equally safe hi-speed skiing expert (update: initial reports indicate Schumacher was not going very fast). However, public belief in this 'fighter pilot syndrome' appears to exist at some level, and this is especially true in India. For example, if you win a Nobel Prize or for that matter, any prize in the west, then regardless of your field of expertise and your near-total ignorance about what makes India tick, you are given special powers that turn you into an expert on every topic under the sun (especially Indian culture and politics), overnight. Unlike Marxist economist Amartya Sen or India's egoistic movie stars, who don't need a second invitation, there are others who prefer not to make a fool of themselves in public. However, the Indian media does not spare them the embarrassment by demanding their "fighter pilot" advice on unrelated topics. This 'intellectual celebrity' feedback is then used to try and influence public opinion. A good example is the recent NDTV-25 debate panel on "secularism in India" compered by 2G-scam tainted journalist Barkha Dutt that included exactly one genuine expert, Arun Shourie, who knew what he was talking about, and bunch of other "experts".
All-weather experts and their Indian media co-pilots must be asked to wear their seat-belts and slow down before they take the Indian public for a ride.
Happy New Year. Drive Safe. Get well soon, Schumi.
Sunday, September 1, 2013
Why the UPA will get re-elected in 2014: in 100 words or less
Stats indicate UPA to be the most corrupt government in Indian history, so it won't get reelected in the 2014 elections. Wrong. Here's proof.
1. Scam cash, embezzled dough, and black money is a sunk cost. Irrecoverable past.
2. INC has turned India's voting mass into MYLOPs.
By definition, MYLOPs are Markovian; their vision of the future is independent of past events, given instant gratification. It follows from (1) and (2) that whoever supplies the biggest freebie to the voting mass now, wins. UPA just designed the free lunch, transcending the NFL theorem.
No contest.
Notation
INC = Indian national congress, the dynastic party (ideology: Nehruvian socialism), which has wrecked India almost continuously since political independence (1947).
UPA = United Progressive Alliance = INC + clones, won the last two elections, seeks a hat-trick (= three-peat, in American).
MYLOPs = Myopic, Local Optimum seeking Pessimists. First introduced here.
Free lunch = FSB = A grossly underfunded food security bill, which provides a theoretical guarantee of free food for all poor people, but in reality is a food bank for UPA's vote bank.
Friday, December 21, 2012
The Scamster versus the Statesman (or why the Indian National Congress fancies its chances in 2014 and 2019)
A Scam Response Model
The loss due to government-blessed scams and failed populist schemes in India run in the order of Billions and Trillions of Indian Rupees in a country where the impoverished still form a sizable percentage and earning under or around the government designated poverty level of ₹20+ a day (50₹ ~ 1$). To many Indians, such twelve-zero numbers seem fantastic. Thus a person caught stealing thousands in a small town may receive a sound thrashing because people more clearly recognize the utility of amounts that actually show up in their historical experience. However, as the size of the theft increases beyond imaginable sums, the magnitude of the public response does not appear to proportionally increase. The law of diminishing returns seems to kick in, resulting in a concave response function. We postulate that:
Change in Public Response (ΔR) ∝ Change in # of Zeros (ΔZ) in Scam amount S , i.e
R = k log S, where k is a constant to be estimated using historical data.
The assumption of a logarithmic response model turns to be quite useful in the Indian context. For example, if political party A pulls off a 12-zero scam, and party B a 6-zero scam (using a log-10 base):
response(A) = 12k, and response(B) = 6k.
The response is only a factor of 2 more for the scam that was a million times bigger. This makes it easier for party-A to equate itself to party-B in the eyes of the public. It can have its cake and perhaps eat it too. Most Indians would not find it hard to map A & B to their real-life representatives.
Statesman versus Scamster - Round 1
A subjugated, under-informed electorate may exhibit a strong concave response that is characterized by relatively high sensitivity to small thefts at one end, and a disregard for mega-scams at the other end. We postulate that such a population's mental decision making model tends to be (1) myopic, (2) local optimality seeking, and (3) pessimistic (MYLOP). It accepts its terrible local living conditions provided the situation doesn't get perceptibly worse, and votes in polls based on this mental model. The immediate cause-and-effect tied to a local ₹100 theft represents a clear and present risk to current survival and is likely to elicit a swift and violent response, just like a populist cash-equivalent freebie elicits an equally enthusiastic response. However, a gigantic national scam that all but ensures that its future generations will see no improvement is shrugged off. MYLOP behavior is akin to a frog that allows itself to boiled alive because it does not register the gradual increase in water temperature until it is too late.
On the flip side, a government that is largely scam-free and claims to work on long-term growth, but is perceived to have marginally worsened the status quo can get booted out of power in the next election. Asking such a population to endure temporary 'hard choices' in order to be rewarded with medium-to-long term improvement ("no free lunch", "it will get worse before it gets better") implies a non-convex decision model - a hard sell since it clashes with the MYLOP attitude of the population. Scamsters + freebies trumps the Statesman here. Let's dig a little deeper into scams.
How to Hide a Scam?
Answer: A new scam that is an order of magnitude bigger. This can be explained via a mathematical model based on an interesting technical report by John Cook, M. D. Anderson Cancer Center (and author of the Endeavor blog):
The soft maximum of two variables is the function
g(x; y) = log(exp(x) + exp(y))
Given scams S1 and S2, the cumulative response from the public per our postulated model is log(S1 + S2). For example, the Commonwealth Games of India (CWG) plus a few other scams put together was a 9-zero scam, and this was followed by the 2G bandwidth sell-off, which was a thousand times bigger, i.e., a 12-zero scam. The cumulative response is:
log(10^12 + 10^9) = log(10.001 ^ 12) = 12.0005211273k,
a value that is barely more than the response to just the 2G scam itself (12k). In other words, the cumulative response to scams is approximately its soft maximum. The 2G scam simply eclipses the CWG scam in the mind of the public and the media. Even if there were 10 CWG-level scams, they'd be cumulatively eclipsed by a new 2G-level scam for all practical purposes and drop off the radar.
Scamster versus Scamster
Suppose a generally honest political party-B gets greedy seeing the muted public response and its forgiveness of prior scams. If it imitates party-A and pulls off a CWG-level scam, the public's anger would not be a 1000 times smaller. It would only be (12-9)/12 = 25% smaller. Having a clean prior record is not very helpful. Furthermore, if Party-B campaigned on a plank of 'transparent party that is different', it will be perceived as being not very different from party-A (75% alike) despite being 99.9% less corrupt compared to party-A, measured in terms of money siphoned off.
A MYLOP population that is forced to choose between two scamsters (even if their scam sizes are different) will pick the one that offers better freebies. If both provide equal freebies, the MYLOPs are likely to alternate between the two choices, every electoral cycle (e.g state of Tamil Nadu or Uttar Pradesh).
Statesman v/s Scamster - Round 2
Getting out a terrible locally-optimal situation may require a statesman to bring out and maintain a series of organic (sustainable and permanent) and highly visible improvements to first earn the trust of its MYLOP people over a sufficiently long period before unveiling any risky long-term development models. This means a lot of very hard infrastructure work: providing reliable 24x7 electricity, good roads, clean water, primary schools for girls, security for women, etc., to fundamentally alter the decision model of the population from MYLOP to a more optimistic and forward-looking model where people begin to see the possibilities within their grasp. An electorate that is moving along such a positive gradient is more likely to vote to continue along this path of incremental improvement without settling for "dont rock the old boat" and "here and now" populist freebies provided by the scamster. So the statesman can win (e.g. State of Gujarat), but it takes a lot of heavy lifting to be done upfront.
The two seemingly contradictory outcomes from the two state elections that were announced yesterday (Gujarat and Himachal Pradesh), as well as many other Indian elections in the recent past can be plausibly explained using these simple models.
Update 1: 01/21/2013
Interesting quote from 'Steve Jobs' top 5 mistakes" that directly relates to the above idea of hiding scams using bigger scams:
"... The lesson that I take from these defunct products is that people will soon forget that you were wrong on a lot of smaller bets, so long as you nail big bets in a major way (in Jobs's case, the iPod, iPhone, iPad, etc) .."
The loss due to government-blessed scams and failed populist schemes in India run in the order of Billions and Trillions of Indian Rupees in a country where the impoverished still form a sizable percentage and earning under or around the government designated poverty level of ₹20+ a day (50₹ ~ 1$). To many Indians, such twelve-zero numbers seem fantastic. Thus a person caught stealing thousands in a small town may receive a sound thrashing because people more clearly recognize the utility of amounts that actually show up in their historical experience. However, as the size of the theft increases beyond imaginable sums, the magnitude of the public response does not appear to proportionally increase. The law of diminishing returns seems to kick in, resulting in a concave response function. We postulate that:
Change in Public Response (ΔR) ∝ Change in # of Zeros (ΔZ) in Scam amount S , i.e
R = k log S, where k is a constant to be estimated using historical data.
The assumption of a logarithmic response model turns to be quite useful in the Indian context. For example, if political party A pulls off a 12-zero scam, and party B a 6-zero scam (using a log-10 base):
response(A) = 12k, and response(B) = 6k.
The response is only a factor of 2 more for the scam that was a million times bigger. This makes it easier for party-A to equate itself to party-B in the eyes of the public. It can have its cake and perhaps eat it too. Most Indians would not find it hard to map A & B to their real-life representatives.
Statesman versus Scamster - Round 1
A subjugated, under-informed electorate may exhibit a strong concave response that is characterized by relatively high sensitivity to small thefts at one end, and a disregard for mega-scams at the other end. We postulate that such a population's mental decision making model tends to be (1) myopic, (2) local optimality seeking, and (3) pessimistic (MYLOP). It accepts its terrible local living conditions provided the situation doesn't get perceptibly worse, and votes in polls based on this mental model. The immediate cause-and-effect tied to a local ₹100 theft represents a clear and present risk to current survival and is likely to elicit a swift and violent response, just like a populist cash-equivalent freebie elicits an equally enthusiastic response. However, a gigantic national scam that all but ensures that its future generations will see no improvement is shrugged off. MYLOP behavior is akin to a frog that allows itself to boiled alive because it does not register the gradual increase in water temperature until it is too late.
On the flip side, a government that is largely scam-free and claims to work on long-term growth, but is perceived to have marginally worsened the status quo can get booted out of power in the next election. Asking such a population to endure temporary 'hard choices' in order to be rewarded with medium-to-long term improvement ("no free lunch", "it will get worse before it gets better") implies a non-convex decision model - a hard sell since it clashes with the MYLOP attitude of the population. Scamsters + freebies trumps the Statesman here. Let's dig a little deeper into scams.
How to Hide a Scam?
Answer: A new scam that is an order of magnitude bigger. This can be explained via a mathematical model based on an interesting technical report by John Cook, M. D. Anderson Cancer Center (and author of the Endeavor blog):
The soft maximum of two variables is the function
g(x; y) = log(exp(x) + exp(y))
Given scams S1 and S2, the cumulative response from the public per our postulated model is log(S1 + S2). For example, the Commonwealth Games of India (CWG) plus a few other scams put together was a 9-zero scam, and this was followed by the 2G bandwidth sell-off, which was a thousand times bigger, i.e., a 12-zero scam. The cumulative response is:
log(10^12 + 10^9) = log(10.001 ^ 12) = 12.0005211273k,
a value that is barely more than the response to just the 2G scam itself (12k). In other words, the cumulative response to scams is approximately its soft maximum. The 2G scam simply eclipses the CWG scam in the mind of the public and the media. Even if there were 10 CWG-level scams, they'd be cumulatively eclipsed by a new 2G-level scam for all practical purposes and drop off the radar.
Scamster versus Scamster
Suppose a generally honest political party-B gets greedy seeing the muted public response and its forgiveness of prior scams. If it imitates party-A and pulls off a CWG-level scam, the public's anger would not be a 1000 times smaller. It would only be (12-9)/12 = 25% smaller. Having a clean prior record is not very helpful. Furthermore, if Party-B campaigned on a plank of 'transparent party that is different', it will be perceived as being not very different from party-A (75% alike) despite being 99.9% less corrupt compared to party-A, measured in terms of money siphoned off.
A MYLOP population that is forced to choose between two scamsters (even if their scam sizes are different) will pick the one that offers better freebies. If both provide equal freebies, the MYLOPs are likely to alternate between the two choices, every electoral cycle (e.g state of Tamil Nadu or Uttar Pradesh).
Statesman v/s Scamster - Round 2
Getting out a terrible locally-optimal situation may require a statesman to bring out and maintain a series of organic (sustainable and permanent) and highly visible improvements to first earn the trust of its MYLOP people over a sufficiently long period before unveiling any risky long-term development models. This means a lot of very hard infrastructure work: providing reliable 24x7 electricity, good roads, clean water, primary schools for girls, security for women, etc., to fundamentally alter the decision model of the population from MYLOP to a more optimistic and forward-looking model where people begin to see the possibilities within their grasp. An electorate that is moving along such a positive gradient is more likely to vote to continue along this path of incremental improvement without settling for "dont rock the old boat" and "here and now" populist freebies provided by the scamster. So the statesman can win (e.g. State of Gujarat), but it takes a lot of heavy lifting to be done upfront.
The two seemingly contradictory outcomes from the two state elections that were announced yesterday (Gujarat and Himachal Pradesh), as well as many other Indian elections in the recent past can be plausibly explained using these simple models.
Update 1: 01/21/2013
Interesting quote from 'Steve Jobs' top 5 mistakes" that directly relates to the above idea of hiding scams using bigger scams:
"... The lesson that I take from these defunct products is that people will soon forget that you were wrong on a lot of smaller bets, so long as you nail big bets in a major way (in Jobs's case, the iPod, iPhone, iPad, etc) .."
Tuesday, October 9, 2012
Predicting the Future Size of the Nehru Dynasty
A glance through Indian newspapers will tell you about corruption in the highest places - specifically within India's so-called first family of Gandhis. Those not familiar with Indian politics would be surprised to find 'Gandhi' and 'corruption' in the same sentence. The puzzle is quickly resolved once you discover that this Gandhi family thankfully does not have the Mahatma in their family tree. This is the family of Nehrus, and at some point, a surviving daughter married a relatively unknown chap bearing that hallowed last name, engineering the most profitable branding coup the world has ever seen.
It is easy to write reams about how this family has institutionalized poverty and corruption in India over the last 60 years but it suffices for the purposes of this post to note that starting a few years prior to India's political independence from the British in 1947, members of the Nehru dynasty have directly or indirectly controlled (and destroyed) the futures of several hundred million Indians. Sadly, their level of incompetence has increased every generation, and as their numbers slowly grow, it becomes important for Indians to know this: how many dynasty members will a person have to get through to reclaim power in New Delhi in the future? Take a quick look at these time-series data in 20-year chunks:
Date Number Dynasty members
1924-1944: 0 No Nehru calling the shots
1944-1964: 1 Jawaharlal Nehru
1964-1984: 1 Indira Nehru Gandhi
1984-2004: 2 Rajiv Gandhi & Sonia Gandhi
2004-2014: 3 SoniaG, RahulG, & PriyankaG
----------------------------------------------------------------
2014-2024: 3 SoniaG, RahulG, & PriyankaG
2024-2044: 5 SG, RG, PG and PG's two children
The numbers below the dashed line are future predictions based on the current family count, and assuming that Priyanka's two kids today (Rahul is unmarried with no pending paternity cases) will be baptized into the family tradition of absolute power in their 20s-30s, like every generation before them.
Of course, it should not be surprising that the counts shown above are exactly the first six numbers of the Fibonacci sequence. There is another interesting case of poetic injustice in the nomenclature that is hidden here. Calling them Fibonacci numbers perpetuates an injustice to the mathematicians in India who discovered the series a long time before Fibonacci and unlike the Gandhi-Nehru mix up, this fact was well-known outside India too.
It is easy to write reams about how this family has institutionalized poverty and corruption in India over the last 60 years but it suffices for the purposes of this post to note that starting a few years prior to India's political independence from the British in 1947, members of the Nehru dynasty have directly or indirectly controlled (and destroyed) the futures of several hundred million Indians. Sadly, their level of incompetence has increased every generation, and as their numbers slowly grow, it becomes important for Indians to know this: how many dynasty members will a person have to get through to reclaim power in New Delhi in the future? Take a quick look at these time-series data in 20-year chunks:
Date Number Dynasty members
1924-1944: 0 No Nehru calling the shots
1944-1964: 1 Jawaharlal Nehru
1964-1984: 1 Indira Nehru Gandhi
1984-2004: 2 Rajiv Gandhi & Sonia Gandhi
2004-2014: 3 SoniaG, RahulG, & PriyankaG
----------------------------------------------------------------
2014-2024: 3 SoniaG, RahulG, & PriyankaG
2024-2044: 5 SG, RG, PG and PG's two children
The numbers below the dashed line are future predictions based on the current family count, and assuming that Priyanka's two kids today (Rahul is unmarried with no pending paternity cases) will be baptized into the family tradition of absolute power in their 20s-30s, like every generation before them.
Of course, it should not be surprising that the counts shown above are exactly the first six numbers of the Fibonacci sequence. There is another interesting case of poetic injustice in the nomenclature that is hidden here. Calling them Fibonacci numbers perpetuates an injustice to the mathematicians in India who discovered the series a long time before Fibonacci and unlike the Gandhi-Nehru mix up, this fact was well-known outside India too.
Thursday, September 20, 2012
FDI in Indian Retail: Good or Bad for India? In 100 Words or Less
Is building a hospital in an Indian town good? sure
If congress workers construct it? no.
Building a road generally a good idea? yes.
If the congress party supervises? nope.
Whether 'FDI* in retail' is bad or good strategy for India is moot. If its execution is mangled, it will end up being a horrific loss. A 60+ year trail of evidence supports this indictment of the Congress party.
"Amateurs Talk about Strategy, Dilettantes Talk about Tactics, and Professionals Talk about Logistics". The original quote is usually attributed to General Omar. Bradley, the "People's General".
(FDI = Foreign Direct Investment. 'FDI in Retail' may well be the proverbial straw the breaks the ruling Congress party's back.)
If congress workers construct it? no.
Building a road generally a good idea? yes.
If the congress party supervises? nope.
Whether 'FDI* in retail' is bad or good strategy for India is moot. If its execution is mangled, it will end up being a horrific loss. A 60+ year trail of evidence supports this indictment of the Congress party.
"Amateurs Talk about Strategy, Dilettantes Talk about Tactics, and Professionals Talk about Logistics". The original quote is usually attributed to General Omar. Bradley, the "People's General".
(FDI = Foreign Direct Investment. 'FDI in Retail' may well be the proverbial straw the breaks the ruling Congress party's back.)
Monday, March 26, 2012
Gender-Shaping III: Is Amartya Sen's Missing Women Count Exaggerated?
This is the third post in this series on gender-shaping. The previous installment can be found here. Thanks to a twitter link, I came across a 2010 journal paper: "Missing Women: Age and Disease," Siwan Anderson (University of British Columbia) and Debraj Ray (New York University) published in Review of Economic Studies Vol.77.
This paper has among other things, investigated Amartya Sen's '100 Million Missing Women of India' claim that is attributed to systemic discrimination. Anderson and Ray have estimated the number of 'missing women' in India, China and Sub-Sahara Africa by age and cause-of-death (not done before) while also moving away from the simplistic aggregate sex ratios that were used as baselines in prior works. The authors make the following useful observations: Defining missing women by differences in aggregate sex ratios can be misleading, or uninformative (or both). It is misleading because different countries have different fertility and death rates, and (in particular) different age distributions. They will have different disease compositions.
They may also have different sex ratios at birth for genetic or environmental reasons that have nothing to do with missing females.
The procedure is also uninformative: we cannot tell at what ages the missing women are clustered, or what diseases are responsible. Thus, we cannot begin to ask about the various
channels: discrimination, biology, social norms, and so on. Answering these questions is of profound importance. By unpacking missing women by age and disease, our paper takes a limited and preliminary step in this direction."
From an OR perspective, we extensively rely on similar customer segmentation models (in revenue management for e.g), and this additional age- and causal-factor based segmentation appears to be quite important and yields two main results as well as a comparative result that may be interesting to an U.S audience:
1. A large fraction of the missing women in India are not infants (less than 20%) but adults, and is attributable to other factors like disease and injury, apart from any systemic discrimination. Consequently, any claim of exclusively female infanticide driven 'missing women' in India is rejected. On the other hand, this paper finds that 44% of China's missing women are in the prenatal age-group. Here is a snapshot of sex-ratio by age, taken from the Anderson & Ray paper:
2.The authors make an interesting comparative comparison with the U.S: "we observe some similarities between age-specific percentages of missing women in the historical United States (ca. 1900) and India or sub-Saharan Africa today".
3. The Sen count (100 million missing women) appears to have been calculated with respect to a specific counterfactual: The overall sex ratio for N. America, U.S and Japan. An alternative calculation by Coale (1991) comes up with a more conservative estimate of 60 million. Anderson and Ray perform similar calculations but at the segment level (i.e. by age-disease) and generate missing number estimates using more carefully chosen counterfactuals as the baseline and find approximately 20 million missing women in India (aggregated across all age groups), while the corresponding figure for China is 58 million. Furthermore, 'injury' is not an insignificant culprit in India across all age groups, a potentially worrying trend that its government must look into. (The paper alludes to the old bogey of 'dowry deaths' as a probable cause which may not turn out to be the case. A similar detailed analysis is required).
The findings of this paper also weakens a statement in a previous post on this topic that a skewed overall male:female ratio in a region is a 'scary indicator' of female infanticide being practiced there. My statement ignored the age distribution as well as the 'cause of death' dimension. Bad O.R, but I have Amartya Sen for company.
Thursday, December 1, 2011
OR models for discouraging criminal intent
Dr. Subramaniam Swamy, the brilliant lawyer who exposed the 2G scam in India is trying to raise awareness about this issue amongst the Indian public via a series of talks around the country. In one of his talks, he white-boards a simple but useful high-level model (shown below) to determine the level of financial penalty that should be imposed to deter future scams. This is especially important in the case of crooked politicians who tend to be thick-skinned and are willing to ride out their years in jail in order to enjoy their ill-gotten stash after 'serving out their term'. Conditioning on the chances of getting caught (p),
E[reward] = (1-p)R - p.D
where R = reward from the scam, and D = penalty to be paid if you are caught (primary decision variable). Swamy's point was that, in the Indian context, even though 'p' is relatively small in value for various reasons, all is not lost if 'D' can be made sufficiently large (proportional to 'R' times the odds of evasion) to ensure that the LHS value is unattractive enough.
Let's apply this model to the roads of India, where traffic violations are the norm. A given O-D (origin-destination) path is formed by several links, where link (i) is associated with a probability p(i), reward R(i), and penalty D(i). Crooks try to find the path of least resistance, e.g, smallest cumulative 'p' or max cumulative E[reward]. The police can counter this by a solving p-median type problem (this is a different 'p') where they can optimally position their scarce resource (traffic cops) to maximize an aggregate measure of expected penalty based on link traffic volumes. Given budget restrictions, these scarce resources have become even more scarce. However, it appears that in many other parts of the world, police departments have been smart enough to create a multiplier effect via an 'illusion of ubiquity' achieved via a 'ghost archetype' that periodically and randomly enforces the law in prominent locations in a particularly visible and pitiless manner. Recognizing the 'human element', in this case, the nonlinear 'customer response' to certain deployment scenarios can lead to a solution that increases the perceived probability p(i) for certain key links in the transportation network (beyond its actual statistical rate) without increasing actual capacity. Similarly, the presence of crooked 'bribe friendly' cops at busy intersections can lead to a nonlinear drop in public trust and perceived p(i), regardless of an increase in overall capacity.
To summarize, OR based decision support models can be effective in making a dent in corruption via analytically driven decisions that help maximize the perceived penalty for criminal/corrupt acts that in turn, leads to an actual incremental reduction in the relevant crime/scam rate, despite a scarcity of resources.
E[reward] = (1-p)R - p.D
where R = reward from the scam, and D = penalty to be paid if you are caught (primary decision variable). Swamy's point was that, in the Indian context, even though 'p' is relatively small in value for various reasons, all is not lost if 'D' can be made sufficiently large (proportional to 'R' times the odds of evasion) to ensure that the LHS value is unattractive enough.
Let's apply this model to the roads of India, where traffic violations are the norm. A given O-D (origin-destination) path is formed by several links, where link (i) is associated with a probability p(i), reward R(i), and penalty D(i). Crooks try to find the path of least resistance, e.g, smallest cumulative 'p' or max cumulative E[reward]. The police can counter this by a solving p-median type problem (this is a different 'p') where they can optimally position their scarce resource (traffic cops) to maximize an aggregate measure of expected penalty based on link traffic volumes. Given budget restrictions, these scarce resources have become even more scarce. However, it appears that in many other parts of the world, police departments have been smart enough to create a multiplier effect via an 'illusion of ubiquity' achieved via a 'ghost archetype' that periodically and randomly enforces the law in prominent locations in a particularly visible and pitiless manner. Recognizing the 'human element', in this case, the nonlinear 'customer response' to certain deployment scenarios can lead to a solution that increases the perceived probability p(i) for certain key links in the transportation network (beyond its actual statistical rate) without increasing actual capacity. Similarly, the presence of crooked 'bribe friendly' cops at busy intersections can lead to a nonlinear drop in public trust and perceived p(i), regardless of an increase in overall capacity.
To summarize, OR based decision support models can be effective in making a dent in corruption via analytically driven decisions that help maximize the perceived penalty for criminal/corrupt acts that in turn, leads to an actual incremental reduction in the relevant crime/scam rate, despite a scarcity of resources.
Wednesday, September 21, 2011
Anatomy of a scam
Once, even twice is a coincidence. But three is a pattern worthy of a second look.
Exhibit 1: The financial monoliths that rode the cash wave in an ocean of American tax payer money have pretty much gotten away scot-free. Their criminal greed has been marketed as a simple combination of market upredictability, non-robust math models, and corporate irresponsibility.
Exhibit 2: The response from the Pakistani government after the double-tap delivered to Osama Bin Laden a few hundred yards away from their West Point, was to admit 'gross incompetence' rather than confess to any willing participation in hiding a notorious fugitive. No punishment and continued pouring of billions of dollars down the drain. They happily take out a full page Ad in the Wall Street Journal on the 11th of this month to celebrate.
Exhibit 3: The government of India is for all practical purposes run by the Darth Vader-like Italian-born Sonia Maino, who has propped up an equally complicit 80+ year old mute puppet as the prime minister to take the heat. The current regime that has ruled India for 50 of its 60-odd post-independence years is neck deep in a series of corruption scandals and midnight arrests of peaceful anti-corruption activists. The most blatant of these is the so-called '2G scam', where billions of dollars (1.86 trillion ₹) worth of public money in form of lucrative bandwidth was all but given away to friends and family. Only the most junior ministers (belonging to the coalition-party :) are in jail. Their defense is rather innovative but inevitably based on this same theme - 'negligence' and 'uncertainty', rather than admitting to any criminal wrong-doing or fraud.
Case 3 is an interesting example. This tab has already touched upon it twice before and arguments outlined turned out to be in line with what the Harvard-affiliated anti-corruption lawyer Dr. Subramanyam Swami used in his own article to describe the reasons for the mess.
The defense in all three examples of colossal fraud essentially argue that they merely maintained the 'status quo', claiming ignorance of the true value of doing the (obviously) right thing. Their second line of attack is to plead down the severity of the charge all the way to a misdemeanor. In exhibit 3, this is being done along the following flimsy lines "since the true value of the resource can only be determined if an auction had actually taken place, the figures quoted are cooked up by vested interests". Two factors go against such an argument:
1. This figure was calculated by a government agency (!) - the Comptroller and Auditor General (CAG).
2. Unless the CAG has performed a rigorous math-based analysis and run simulations to determine the maximal revenue obtainable from selling a scarce resource in a gigantic market like India, the quoted figure that was based on an average or a reasonable auction scenario is more likely to be a lower bound on the true cost of the swindle.
How dependable are 'opportunity cost', and more generally such "if" based decision models? A paper that I co-authored as a student of civil engineering many years ago happens to be based on this idea and was used in highway resource planning in Virginia. Often times, business value of an analytics idea can only be viably demonstrated by calculating 'what would have happened to a set of past outcomes had this OR method been used instead". On the other hand, if a researcher were to build up a ladder that consists of several degrees of conditional dependence to arrive at a final value, then such chain-of-events driven claims have to closely scrutinized to ensure that we simply do not end up with 'noise'.
Exhibit 1: The financial monoliths that rode the cash wave in an ocean of American tax payer money have pretty much gotten away scot-free. Their criminal greed has been marketed as a simple combination of market upredictability, non-robust math models, and corporate irresponsibility.
Exhibit 2: The response from the Pakistani government after the double-tap delivered to Osama Bin Laden a few hundred yards away from their West Point, was to admit 'gross incompetence' rather than confess to any willing participation in hiding a notorious fugitive. No punishment and continued pouring of billions of dollars down the drain. They happily take out a full page Ad in the Wall Street Journal on the 11th of this month to celebrate.
Exhibit 3: The government of India is for all practical purposes run by the Darth Vader-like Italian-born Sonia Maino, who has propped up an equally complicit 80+ year old mute puppet as the prime minister to take the heat. The current regime that has ruled India for 50 of its 60-odd post-independence years is neck deep in a series of corruption scandals and midnight arrests of peaceful anti-corruption activists. The most blatant of these is the so-called '2G scam', where billions of dollars (1.86 trillion ₹) worth of public money in form of lucrative bandwidth was all but given away to friends and family. Only the most junior ministers (belonging to the coalition-party :) are in jail. Their defense is rather innovative but inevitably based on this same theme - 'negligence' and 'uncertainty', rather than admitting to any criminal wrong-doing or fraud.
Case 3 is an interesting example. This tab has already touched upon it twice before and arguments outlined turned out to be in line with what the Harvard-affiliated anti-corruption lawyer Dr. Subramanyam Swami used in his own article to describe the reasons for the mess.
The defense in all three examples of colossal fraud essentially argue that they merely maintained the 'status quo', claiming ignorance of the true value of doing the (obviously) right thing. Their second line of attack is to plead down the severity of the charge all the way to a misdemeanor. In exhibit 3, this is being done along the following flimsy lines "since the true value of the resource can only be determined if an auction had actually taken place, the figures quoted are cooked up by vested interests". Two factors go against such an argument:
1. This figure was calculated by a government agency (!) - the Comptroller and Auditor General (CAG).
2. Unless the CAG has performed a rigorous math-based analysis and run simulations to determine the maximal revenue obtainable from selling a scarce resource in a gigantic market like India, the quoted figure that was based on an average or a reasonable auction scenario is more likely to be a lower bound on the true cost of the swindle.
How dependable are 'opportunity cost', and more generally such "if" based decision models? A paper that I co-authored as a student of civil engineering many years ago happens to be based on this idea and was used in highway resource planning in Virginia. Often times, business value of an analytics idea can only be viably demonstrated by calculating 'what would have happened to a set of past outcomes had this OR method been used instead". On the other hand, if a researcher were to build up a ladder that consists of several degrees of conditional dependence to arrive at a final value, then such chain-of-events driven claims have to closely scrutinized to ensure that we simply do not end up with 'noise'.
Monday, November 22, 2010
Measuring the impact of corruption via OR models - the 2G scam in India
The recent 2G spectrum scam in India has taken corruption to epic levels. Large-scale theft is now being expressed as a percentage of India's GDP for convenience of notation. The amount of taxpayer money siphoned off due to the nefarious actions of certain senior ministers that resulted in an inefficient (non system-optimal) resource allocation wrt the 2G spectrum is estimated at 1760000000000 Rupees (1 US$ ~ 45 Indian Rupees), or 1.76 Trillion Rupees. This seems to be a conservative estimate.
If we compare the value of the corrupt allocation with that of the true system-optimal allocation, I wonder if that loss estimate would be even higher?
This large rupee number is something one usually throws out wildly, except that in this case, it is shockingly close to fact. Furthermore, well-known award-winning cable-news journalists (marketed as fair and balanced) have been implicated by an angry public and audio-tapes have surfaced that seem to allegedly point to their dual role as information-sharing lobbyists, working as mediators between coalition partners of the government to ensure a cover-up, as well as scripting and stage-managing TV shows and news articles to alter public opinion. This has been dubbed 'barkhagate' on the Internet - yet another a cliched 'gate' scandal, but this scandal makes Nixon look like an Eagle boy-scout. Twitter-istan is abuzz with #barkhagate.
Obama, during his recent visit to India, referred to the Indian Prime Minister as his 'Guru', partly due to the PM being an economics professor in a past life. Should he now be called the GGuru? He once was an admired man for pioneering India's economic reforms in the 1990s. Sadly, along with that has come scam after scam, and many in India get the feeling that the actual powerful core within the ruling coalition have this 80+ year old ex-professor set up as a fall guy for their series of epic embezzlements (2G is just the latest).
The fair bandwidth resource allocation problem is a very, very interesting OR challenge. Several cool mathematical models, including combinatorial auction, along with clever Benders decomposition based solution approaches have been invented to solve the resultant discrete optimization formulation (e.g., winner determination problem)
So how does corruption impact such OR models? It is an important as well as an interesting question that deserves more formal attention. If corruption is modeled explicitly within a model, then efficiency, cost-minimization, and revenue maximization are no longer the real objectives. Shadow prices and reduced costs will be misleading. Objective function cost coefficients are inflated or discounted based on the intent of the scam. A machine's throughput may be far less that what shows up on paper due to its unknown, substandard quality. The data will be really messy. Ethics-driven regulations and their corresponding constraints will be missing. By definition, optimization algorithms seek out extreme values and push the envelope. Unethically used, such methods will help maximize corruption.
Dubious organizations may simply place the blame on OR models and the analytics, rather than on the crooked ones who misuse it. Like journalism, whose reputation largely lays in tatters, corruption in analytics will have a devastatingly negative impact on the public perception of mathematicians and OR folks who have won respect as truth-seekers. Once lost, such hard-earned goodwill is almost impossible to regain. As OR people, we have a responsibility, both natural and inherited, to maintain high ethical standards and actively seek the truth (or in OR practice, 'the best obtainable version of the truth' as Carl Bernstein would say). After all, the entire theory of optimization and duality is ultimately based on the notion of fairness and rationality. The insidious noise that undermines fair duality has to be recognized early enough, and must be filtered out.
A question will be posted on OR-exchange to initiate a discussion on this important topic.
If we compare the value of the corrupt allocation with that of the true system-optimal allocation, I wonder if that loss estimate would be even higher?
This large rupee number is something one usually throws out wildly, except that in this case, it is shockingly close to fact. Furthermore, well-known award-winning cable-news journalists (marketed as fair and balanced) have been implicated by an angry public and audio-tapes have surfaced that seem to allegedly point to their dual role as information-sharing lobbyists, working as mediators between coalition partners of the government to ensure a cover-up, as well as scripting and stage-managing TV shows and news articles to alter public opinion. This has been dubbed 'barkhagate' on the Internet - yet another a cliched 'gate' scandal, but this scandal makes Nixon look like an Eagle boy-scout. Twitter-istan is abuzz with #barkhagate.
Obama, during his recent visit to India, referred to the Indian Prime Minister as his 'Guru', partly due to the PM being an economics professor in a past life. Should he now be called the GGuru? He once was an admired man for pioneering India's economic reforms in the 1990s. Sadly, along with that has come scam after scam, and many in India get the feeling that the actual powerful core within the ruling coalition have this 80+ year old ex-professor set up as a fall guy for their series of epic embezzlements (2G is just the latest).
The fair bandwidth resource allocation problem is a very, very interesting OR challenge. Several cool mathematical models, including combinatorial auction, along with clever Benders decomposition based solution approaches have been invented to solve the resultant discrete optimization formulation (e.g., winner determination problem)
So how does corruption impact such OR models? It is an important as well as an interesting question that deserves more formal attention. If corruption is modeled explicitly within a model, then efficiency, cost-minimization, and revenue maximization are no longer the real objectives. Shadow prices and reduced costs will be misleading. Objective function cost coefficients are inflated or discounted based on the intent of the scam. A machine's throughput may be far less that what shows up on paper due to its unknown, substandard quality. The data will be really messy. Ethics-driven regulations and their corresponding constraints will be missing. By definition, optimization algorithms seek out extreme values and push the envelope. Unethically used, such methods will help maximize corruption.
Dubious organizations may simply place the blame on OR models and the analytics, rather than on the crooked ones who misuse it. Like journalism, whose reputation largely lays in tatters, corruption in analytics will have a devastatingly negative impact on the public perception of mathematicians and OR folks who have won respect as truth-seekers. Once lost, such hard-earned goodwill is almost impossible to regain. As OR people, we have a responsibility, both natural and inherited, to maintain high ethical standards and actively seek the truth (or in OR practice, 'the best obtainable version of the truth' as Carl Bernstein would say). After all, the entire theory of optimization and duality is ultimately based on the notion of fairness and rationality. The insidious noise that undermines fair duality has to be recognized early enough, and must be filtered out.
A question will be posted on OR-exchange to initiate a discussion on this important topic.
Subscribe to:
Posts (Atom)