The One-Dollar Haircut

Every year, I travel more than 9K miles to my native place in Tamil Nadu, India, to get my $1 haircut.

(pic source: http://jhingu.com)

The local barber is an expert and does it the old-fashioned way. A comb to sift, and super-quick scissors acting as efficient cutting planes to produce a nice, convex-hull hairdo. No fancy machinery, seating, lighting, or A/C in his environment-friendly shop, and not a single nick, or hair out of place when he's done. This keeps his customers happy and operating cost low. It's a dollar regardless of crop density. Perhaps the argument is that for the sparse-headed, the cost of searching goes up even as the actual cutting time reduces. If the shop is full, spill-over customers can sit in makeshift chairs outside the shop, and sip chai from the adjoining 'tea-kadai'.

The Indian barber shop is also an instance of the 'elastic' capacity model that is key to understanding how the Indian economy has chugged along. Here is what Prof. R. Vaidyanathan, Professor at the Indian Institute of Management, Bangalore, in his superb book 'India, Uninc' has to say:

Data shows that it this 'unincorporated' economy built from the ground-up by entrepreneurs that is responsible for much of India's GDP and employment, and not the stock market that grabs headlines. Despite the recent crash of the global markets, the Indian elephant is likely to remain solid, like it did after 2008.

The traditional Indian way of doing business may be initially bewildering to the external observer, which may have led to this 'Uninc' being categorized by Nehruvian India's west-educated officials as "unorganized". Actually, it is anything but, and appears to be based on balancing constraint-enforcing 'order' and constraint-relaxing 'chaos'. This traditional Indian approach, which I personally view as dharmic optimization, yields astonishingly high and sustainable levels of quality and efficiency when done right, but can also be disastrous when either order or chaos is excessive.

End Note: Apparently, the global wig supply-chain sources much of its hair from India.

How many songs does your favorite internet radio station hold?

Can we even measure the number of songs 'stocked' by our favorite internet radio station (IRS) ? Douglas Hubbard who wrote "How to Measure Anything" would say yes. At least, we can get a reasonable and useful estimate. In fact, it may be tough to identify what an 'exact' answer is here since new songs may be continually added, and less popular ones deleted over time.

This problem appears to resemble another one: How many fish are there in a particular lake? As DH notes, just because we may never see all the fish the lake or hear every song in the IRS, it does not automatically mean that we cannot estimate its size. As part of my self-education in Indian logic, I understand this as follows: when we cannot directly perceive and verify (pratyaksha pramaana), we can try to infer this unobserved truth (~anumaana). Sometimes, we can use comparison (~upamaana) to solve this problem. Here are examples of historically significant events in India where estimating unobserved quantities turned out to be important:

1. Estimating the size of native volunteers ready to take on the occupying British Raj as part of 'Operation Red Lotus' in 1857 (covert sampling and proportions)

2. Estimating the number of refugees inside the Red Fort (without entering) after the 1947 partition of India (comparison of proportions)

3. Estimating the size of jute crop in 1930s Bengal (random sampling)

4. The number of civilians killed in the Bengali holocaust of 1943 (random sampling). The blog carries excerpts, but you should read Madhusree Mukerjee's book for an in-depth analysis.

Using the overlapping sample approach and the fish example in DH's book: catch some fish (say 1000), tag them, and let them swim happily around the lake. Again catch fish (say, 1000) and count the proportion of tagged fish (say 5%), indicating that we tagged roughly 5% of all the fish, giving us an estimate of about 20k. We can also calculate, say, a 90% confidence interval for the chosen sample size. Which brings me to the reason for this blog. I noticed that too many songs on the Ilayaraaja radio station offered by Google-music repeated during my office commutes last week. No problem re-listening to Ilayaraaja musical scores like this one (based on Shamukhapriya ragam is my semi-educated guess).



20 songs were mentally tagged on day-1 and released back into the radio station, and 20% of those tagged songs were streamed back to me on day-2. This experiment was repeated a couple of times, yielding similar results, indicating that Google-music's Ilayaraaja station roughly currently stocks about a hundred songs, and I will soon be hearing only repeats.

Mumbai Dabbawalas: Operational Excellence Based on Trust Chains and dharma

Most business case studies, research papers, and news articles on the Mumbai dabbawalas (MD) focus on, and rightly marvel at the fact the MDs were awarded a remarkable 'six sigma' western certification based on the less than one-in-million statistical error rate in their daily delivery of home-cooked meals to many thousands of Mumbai workers for over a hundred years now. As background, here is one of the earlier articles, written ten years ago, on this astonishing team. Please read this article first, since the remainder of the discussion will assume familiarity with the MD supply chain. The MDs have built a zero carbon-footprint, health-enhancing, almost perfectly-reliable, organically sustainable, hyper-personalized supply chain. Profitably achieving even one of these objectives would be considered a victory for modern day supply chains. Rather that again focus on what they achieved and how (and both questions are of great interest to the Operations Research and business analytics community), the more interesting questions for me as I began learning about the dabbawalas were:
how can they sustain this? and why do they?

Business case studies and news articles skip these questions. Luckily, I found at least two sources of information that I could tap into to try and answer these questions to my own satisfaction. We'll start with the 'how', and then the deeper, related question of 'why'.

How do they sustain it?
We turn to Prof. P. Kanagasabapathi's gem of a book "Indian Models of Economy, Business and Management (3rd Edition)", and see what he has to say about MDs. I must add this: I have come to learn, over the last few years that the Indian economy is like a complex number. It has a real part and an imaginary part. The latter is covered by CNBC, Bombay Stock analyses, and Ivy-league university economic theories faithfully imported and regurgitated in Indian colleges, and gobbled up by students and economists.  If we want to understand how the real part works, read this book, and Dr. R. Vaidyanathan's equally brilliant 'India, Uninc'. Both are data-driven books, and offer us the reality based on deep insights into how the Indian society and mind actually function. They are available in paper and Kindle editions, and ridiculously under-priced.

Let us see what Prof. PKS has to say (emphasis in bold / square bracket is mine):

"...the dabbawala business in Mumbai is an innovative business designed and executed by a group of uneducated and undereducated people from the ordinary classes to suit the local conditions. The business plan is so designed to make it perfectly functional, while at the same time keeping it cost effective.

"....see how the supply chain management works. The dabba-wallas are semi-literate people, with rural backgrounds from Pune district. They belong to the warrior [clan] of Malua, which fought for [the great Indian king Shivaji] in the earlier days. In the 1890s, one Mahado, a migrant from their area started the supply of lunch boxes. Today under the banner of Nutan Mumbai Tiffin-box Suppliers Association, more than 4500 of them are involved in supplying nearly 2,00,000 boxes every working day. Braving Mumbai weather conditions and difficulties involved in multiple transfer points, these `ordinary' people make only one mistake for eight million deliveries "I.It is no wonder that the Forbes Global Magazine gave them a Six Sigma efficiency rating on par with multinational companies such as Motorola and General Electric. In fact, the efficiency of dabbawallas is much better than the Six Sigma levels fixed by the experts for world class performances. As a result they are now admired as the model for the global corporations...While mega corporations use the most modern technologies and employ highly qualified technical and management experts to reach higher levels, how could these ordinary people from village backgrounds with only their common sense and limited resources better the benchmarks fixed for the most efficient companies in the world? Is it not due to the effective teamwork and most efficient planning? Are these people not practising the most effective methods to serve their customers? In the case of modern corporate sector, they search for different methods to sell their products. In some cases, the products may not be essential to the customers. But still they try to look for different ways to attract customers and sell their products. In the case of dabbawallas they have devised a way to serve the customers who would like to eat home-made food, and in the process they have made it so well that they have become world-class service providers.

We start to get some answers above on the 'how'. Now we turn to find answers for how they're able to sustain this:
".. community relationships provide certain benefits and cost advantages in business. One is trust, which is very important for business. Communities generate high levels of trust due to their close-knit relationships. Second is the lower transaction costs compared to the rates determined by the markets. As a result, efficiency is increased and costs are reduced..."

Trust. It can dramatically reduce costs and boost efficiency. Every multinational company's employee brochure strongly emphasizes the importance of 'trust' and 'teamwork'. However, it is not easy build a high degree of cross-organizational trust in today's western workplace. why? Dr. Kanagasabapathi quotes Francis Fukuyama:
"The ability to associate depends, in turn, on the degree to which communities share norms and values and are able to subordinate individual interests to those of larger groups. Out of such shared values comes trust, and trust, as we will see, has a large and measurable economic value." Trust results in `social capital.'"

Trust can arise out of that bi-directional relationship called mutual respect. This blog I wrote a few years ago views mutual respect as a necessary and sufficient condition for building trust in a workplace.  If one party dilutes its commitment, and merely tolerates rather than respect, this trust is broken. The modern workplace is contractual. Order is enforced, and chaos is contained by a variety of standard background checks, zero and non-zero tolerances, and legal penalties. One of India's most important thought leaders, S. Gurumurthy has pointed out that 'Capitalism requires employers, Communism banks on employees. Traditional Indian economies are entrepreneurial and require neither'. Rather, they rely on trust chains built upon mutual respect, which produce social capital. To learn more about social capital, Dr. P. Kanagasabapathi turns to Aiyer:
"From time immemorial, groups of people have created strong communities, based on commonly observed rules and mutual self-help. These social links discourage deviant behaviour through ostracism and other social penalties, create a climate of trust in which agreements are honoured and grievances redressed, and facilitate collective action against threats from outsiders and risks from natural disasters. This is social capital..."

Thus, the MDs have been able to sustain their dizzying levels of operational reliability and quality levels by achieving a high degree of trust within their organization and the mutual respect between self and customer.

Why do they?
The 'why' takes us even deeper. For this, I'm indebted to the head of the Indian embassy in Vancouver, Canada. Listen to his brief talk about the MDs in this You Tube video (watch from 9:33)




The Dabbawalas are doing their dharma, their cosmic duty. This idea of cosmic/sanctity implies an unmanifest motive to their duty. The effect that dharma has on the dabbawala output is real, measurable, and has made him go far beyond the so-called 6-sigma level of quality. As the consular head points out in his talk, when the Dabbawalas handle, deliver, and serve food, they also perform a sacred service, a seva.  As mentioned in a previous blog here, an Indian business model seeks not unconstrained profit maximization, and nor is it non-profit, for neither are sustainable, but it shubh laabh, or auspicious profitability. There is a harmonious balance between the worldly and the unmanifest. The Indian idea of seva is different from the western conception of service. As Gurumurthy once pointed out (via the wonderful anecdote of Swami Vivekananda and the philanthropist Rockefeller), the Indian giver thanks the receiver, not the other way around. While the manifest, material benefit does indeed go to the receiver, the higher, unmanifest benefit from this good Karma accrues to the giver. Who, then is the giver, and who is the receiver? The entrepreneurial Dabbawala will continue to serve, innovate, and sustain operational excellence provided there is: demand for his service, trust at every point of interaction in the supply chain, and he chooses to do his dharma.

The Contextual Optimization of Sanskrit

I'd written a blog for the INFORMS conference earlier this month based on my practice perspective, which emphasized the importance of contextual optimization rather than despairing over the 'not infallible' theoretical worst-case nature of certain mathematical problems. This is something well-internalized by those in the large-scale and non-convex optimization community, where 'NP-Hardness' is often the start, rather than the end point of R&D.

The wonderful 'Philip McCord Morse Lecture' at the recently concluded INFORMS conference in San Francisco by Prof. Dimitris Bertsimas of MIT touched upon this point, and the 'conclusions' slide in the talk explained this idea really well. To paraphrase, 'tractability of a problem class is tied to the context - whether the instances you actually encounter can be solved well enough'. I mentioned the Sulba Sutras in that blog - a well known body of work that epitomizes the Indian approach to mathematics as Ganitha - the science of computation. The genius, Srinivasa Ramanujan, was a relatively recent and famous example of a mathematician hailing from this tradition. The Indian approach is often algorithmic and more about rule generation than infallible theorem proving. Not that Indians shied away from proof ('Pramaana'. For example, see 'Yuktibhasa'). As I understand it, this sequential process of discovery and refinement does not lose sleep over theoretical fallibility, and consists of:
a) in-depth empirical observation of, and a deep meditation on facts,
b) insightful rule generation,
c) iterative, data-driven refinement of rules.
This quintessential Indian approach is applied not just to math, but to practically every field of human activity, including economics, commerce, art, medicine, law, ethics, and the diverse dharmic religions of India, including Hinduism and Buddhism. Panini's Sanskrit is a great example of this approach.

Panini, the famous Sanskrit grammarian (along with Patanjali) is perhaps the most influential human that much of the world does not know much about. His fundamental contributions to linguistics more than 2000 years ago continues to transform the world in many ways even today. Noted Indian commentator, Rajeev Srinivasan, has recently penned a wonderful article on Panini and Sanskrit. You can learn more about Panini's works by reading Dr. Subhash Kak's (OK State Univ) research papers (samples are here and here). This blog was in part, triggered by this article, and talks about Sanskrit and its contextual optimizations.

Abstract: Sanskrit recognizes the importance of context. Two examples that show how Sanskrit is optimized depending on the context, in two entirely opposite directions, is shown below.

Optimization-1. The grammar is designed to be entirely context-free as Rajeev Srinivasan's article explains, and anticipated the 'grammar' of today's high-level computing language by more than 2000 years: precise with zero room for ambiguity of nominal meaning. To the best of my knowledge, punctuation marks are not required, and order of the words can be switched without breaking down, although there may be personal preferences for some orders over the others, and the sentence remains unambiguously correct. An optimization goal here therefore is to generate a minimum (necessary and sufficient) number of rules that result in an maximally error-free production and management of a maximal number possible variations of Sanskrit text. In this case, Panini appears to have achieved the ultimate goal of generating a minimal set of rules that will produce error-free text, forever. There are other well-known optimizations hidden in the structure and order of the Sanskrit alphabet - more on that later.

Optimization-2. The final interpretation of many keywords in Sanskrit ARE contextual. Which means there are multiple, related interpretations for some words that have a nominal/generic meaning, but you have to optimize the final interpretation at run-time by examining the context of usage, to recover the most suitable specific choice. If the first optimization helped eliminate fallibility, this second optimization in a sense re-introduces a limited fallibility and a degree of uncertainty and freedom by design! This feature has encouraged me to reflect (recall Ganesha and Veda Vyasa), develop a situational awareness while reading, pay attention to the vibrations of the words, and grasp the context of keywords employed, rather than mechanically parse words and process sentences in isolation. A thoughtful Sanskrit reader who recognizes this subtle optimization comes away with a deeper understanding.  For example, Rajiv Malhotra, in his book 'Being Different' (now in the top-10 list of Amazon's books on metaphysics) gives us the example of 'Lingam'. This can mean 'sign', 'spot', 'token', 'emblem', 'badge', etc, depending on the context. Apparently, there are at least 16 alternatives usages for 'Lingam' of which one best suits a given context is picked, and not simply selected at random. And of course, the thousand contextual names ('Sahasranamam') for Vishnu is well known in India. Some well-known western and Indian 'indologists' have ended up producing erroneous, and often tragic translations of Sanskrit text either because they failed to recognize this second optimization, or because they misused this scope for optimization to choose a silly interpretation, leading to comic or tragic conclusions.

Again, this contextual optimization approach by the ancient Indians is not just restricted to Sanskrit, but is employed gainfully in many areas, including classical arts, management, healthcare, ethics, etc., and of course dharmic religion. This contextual dharmic optimization has perhaps helped India in getting the best out of its diverse society, as well as keep its Sanskriti refreshed and refined over time. For example, the contextual ethics of dharma (ref: Rajiv Malhotra's book) has a universal pole as well as a contextual pole that allows the decision maker faced with a dilemma, to not blindly follow some hard-wired ideological copybook, but contemplate and wisely optimize his/her 'run-time' choice based on the context, such that himsa is minimized (dharma is maximally satisfied). Some posts in this space has tried to explore the applications of this idea'.

An earlier blog discussed a related example of seemingly opposite goals for contextual optimizations. When it came to mathematical algorithms, data, and linguistic rules in Sanskrit, a goal was to be brief and dense, minimize redundancy, and maximize data compression, so that for example, an entire Sine-value table or generating the first N decimals of Pi can be both encoded and decompressed elegantly using terse verse. Panini's 'Iko Yan Aci' in the Siva Sutras is a famous example of a super-terse linguistic rule. On the other hand, when it comes to preserving long-term recall and accuracy of transmission of Sanskrit word meanings as well as the precise vibrations of mantras (e.g. Vedic chants) that are critically linked to the 'embodied knowing' tradition of India, the aim appears to be one of re-introducing controlled data redundancy to maximize recall-ability, and error-reduction. This optimization enabled Sanskrit mantras to be accurately transmitted orally over thousands of years.

To summarize, contextual optimization is a powerful and universal dharmic approach that has been employed wisely by our Rishis, Acharyas, Gurus, and thinkers over centuries to help us communicate better, be more productive, healthier, creative, empathetic, scientific, ethical, and interact harmoniously with mutual respect.

update 11/22/2014: 'optimize the final interpretation at parse-time / read-time' is the intent for optimization-2, rather than the computer-science notion of 'interpretation at run time'.

The Best Decisions are Optimally Delayed

The lessons learned from the last few years of practice have convinced me that analytics and OR (OR = Operations Research), or at least MyOR is mainly about learning the art and science of engineering an optimally delayed response. Good analytics produces an optimally delayed response.

But why introduce a delay in the first place? Isn't faster always better? 'Science of better' does not always mean 'science of faster'. From age-old proverbs we find that in between 'haste makes waste or knee-jerk reaction' and being 'too clever by half' lies 'look before you leap'. If we view Einstein from an OR perspective: "Make things as simple as possible, but not simpler", the reason seems clearer. We must make situation-aware and contextually-optimal decisions as fast as possible, or as slow as necessary, but not faster, or slower, i.e., there exists a nonzero optimal delay for every response decision. A middle path in between a quick-and-shallow suboptimal answer, or a slow-and-unwieldy 'over-optimized' recipe. Of course, one must work hard during this delay to maximize the impact of the response, and put Parkinson's law to good use, as suggested below:

#orms methods often *delay* product response to find superior answers. Tricky to pick optimal (delay, improvement) pair :)
— (@dualnoise) August 1, 2014

See this old post on 'optimally delaying an apology' to maximize benefit to the recipient, or recall the best players of every sport being able to delay their response by those few milliseconds to produce moments of magic, or Gen. Eisenhower delaying the call to launch D-Day. In the same way, a good OR/analytics practitioner will instinctively seek an optimal delay. For an example of this idea within an industrial setting, read this excellent article by IBM Distinguished Engineer J. F. Puget on taxicab dispatching that he shared in response to the above tweet. Implication: If your analytics system is responding faster than necessary, then slow it down a bit to identify smarter decision options. The 'slower' version of this statement is more obvious and is a widely used elevator pitch to sell high-performance analytics and optimization tools.

The history of the 'optimal delay' is many thousand years old, going back to the writing of the world's longest dharmic poem, the Mahabharata, which also includes within it, the Bhagavad Gita, one of the many sacred texts of Hinduism.

(pic link: http://www.indianetzone.com)

The story about how this epic came to be written is as follows:
Krishna Dvaipana (the Veda Vyasa) wanted a fast but error-free encoding of the epic that could be told and re-told to generations thereafter. The only feasible candidate for this task was the elephant-faced and much beloved Ganesha, the Indian god of wisdom and knowledge, and remover of obstacles. The clever Ganesha agreed to be the amanuensis for this massive project on the condition that he must never be delayed by the narrator, and must be able to complete the entire epic in ceaseless flow. Veda Vyasa accepted but had his own counter-condition: Ganesha should first grasp the meaning of what he was writing down. This resulted in a brilliant equilibrium.

Veda Vyasa composed the Mahabharata in beautiful, often dense verse that Ganesha had to decipher and comprehend even as he was writing it down without lifting the piece of his tusk that he had broken off to inscribe, from the palm leaves. If Ganesha was too slow, it would potentially give Vyasa the opportunity to increase the density and frequency of incoming verses that may overload even his divine cognitive rate. If he went too fast, he would risk violating Vyasa's constraint. Similarly, if Vyasa was too slow, he would violate Ganesha's constraint. If he went too fast, his verse would lose density and risk becoming error-prone, and of course, then Ganesha would not have to think much and perhaps write it down even faster. Imagine if you will, a Poisson arrival of verses from Vyasa divinely balanced by the exponentially distributed comprehension times of Ganesha. Writer and composer optimally delayed each other to produce the greatest integral epic filled with wisdom ever known; written ceaselessly in spell-binding Sanskrit verse, without error, and flowing ceaselessly to this day without pause.

I can think of no better way to celebrate Ganesha Chathurthi than to recall and apply this lesson in everyday life.

Building the Unsolvable Maze

I came across a tweet via Simon Singh, famous writer of books based on math-topics. I've read a couple of them: 'Fermat's last theorem' and 'The code book'. His tweet points to a picture of an amazing maze hand-drawn over 30 years ago in Japan. Although it is supposed to be 'unsolvable', some comments there claim that it could be solved very quickly if it was made publicly available. Among the very first papers I read after coming to the U.S to study traffic engineering (to understand the reasons for India's chaotic, maze like traffic) was about Moore's algorithm entitled "shortest path through a maze". Mathematically, the shortest path problem formulation has a couple of properties of small interest in the context of this discussion. It has no duality gap, and is totally unimodular: It is sufficient to solve the continuous 'relaxation' to recover an integral optimal solution to the primal or dual formulation. Wikipedia has a page on maze-solving algorithms.  Interesting as the optimization problem of finding an 'optimal route' to 'escape' this maze is, a more interesting question to me personally was: why would someone hand-build such an intricate maze over years; and then why not claim any credit for it? I have tried to interpret this based on my understanding of the Indian way.

(source link and main article at: http://imgur.com/gallery/4kyvVVb)

The intense concentration required for such a task is surely daunting: to at once elevate one's consciousness while also dissolving one's aham (ego) that hinders the mind from systematically growing a complex maze whose paths increase rapidly over time as more forks and merges are constructed. Paths that stop even as they begin, paths that ultimately lead nowhere, paths where you travel for a while, only to discover that you are back where you were before... and then after a lot of calm, refined, and introspective searching (not suffering), finding a path that leads one to satya (ultimate reality/truth) that transcends the maya of the maze that held us in its thrall. A path that dissolves the noisy duality between the world within the maze and without, uniting them harmoniously into a unified whole, even as the space enclosed within the maze maintains a provisional identity within this overall unity. And then perhaps a realization that there could be a pluralism of such (alternative optimal) transcendental paths to satya. A harmonious unity within multiplicity that celebrates its diversity, rather than a synthesized unity derived by optimizing the goal of orderly sameness. The latter produces an efficient monoculture, but one that invariably regards pluralism as a seed of chaos. The former, integral unity best represents the nature of the underlying philosophical unity of India that has continually preserved and refined its dharma civilization over several thousand years. This forms the dharmic basis for any reasonable 'idea of India'. I look forward to reading Rajiv Malhotra's new book on this subject.

Journeys that traverse such a path have led to amazing discoveries that enlightened the world, and will continue to do so. Perhaps it produced this captivating art that simultaneously appeals to the casual observer, the artist, the seeker, and the analyst alike; yet each of us seeing only a partial facet of its underlying truth. A work of art to which its 'creator' deliberately did not append a signature to, and claim ownership of, perhaps unwilling to disturb it's harmony. That is the way of the Yogi.

Dedicated to Rajiv Malhotra on the occasion of India's 65th republic day.Thank you.

The King and the Vampire - 3: The Flaw of Optimizing-On-Average

This is the third episode of the 'King Vikram and the Vetaal' series.

(link: juneesh.files.wordpress.com)

Dark was the night and weird the atmosphere. It rained from time to time. Eerie laughter of ghosts rose above the moaning of jackals. Flashes of lightning revealed fearful faces. But King Vikram did not swerve. He climbed the ancient tree once again and brought the corpse down. With the corpse lying astride on his shoulder, he began crossing the desolate cremation ground. "O wise King, it seems to me that your ministers are sometimes too quick in taking policy decisions based on an average scenario, ignoring the distribution. But it is better for you to know that such decisions invariably results in complaints. Let me cite an instance. Pay your attention to my narration. That might bring you some relief as you trudge along," said the Betaal, which possessed the corpse. 

 

(pic source link: pryas.wordpress.com)

The retailing vampire went on:
Once there lived a retailer who always optimized his scarce resource constrained planning problems based on an average planning week. It was a quick and easy heuristic, and he claimed that it worked just fine. One day, an OR practitioner challenged this assumption, quoting points from the well-known 'flaw of averages' book, and said that with a bit more effort, and without increasing the problem size much, one can generate true optimal merchandising decisions by including all scenarios, using CPLEX. This would also be relatively more robust in reality compared to optimizing to the average. The retailer replied that while this issue was of theoretical importance, it did not matter much in practice, unless he saw some hard evidence. The practitioner decided to make an empirical point and proceeded to evaluate a number of historical instances by evaluating the recommendations generated by these two approaches over all the scenarios. In 75% of the instances, the practitioner's method yielded relatively better metrics, while in 25% of the cases, the retailer's average method did better. The retailer took one look at the results and said that the experimental setup was erroneous, inconclusive, and remained unconvinced.

So tell me King Vikram, Why did the retailer conclude the test setup was faulty? Was he correct in this assessment? Which method is better in the retailer's context? Answer me if you can. Should you keep mum though you may know the answers, your head would roll off your shoulders!"

King Vikram was silent, then closed his eyes, as if going into a Yogic trance in a moment of intense meditation. He reopened his eyes soon enough along with a smile, and then spoke: "Vetaal, unlike the last time, this one is pretty easy, so let's take the first question first.

1. The retailer felt the experimental setup was flawed because, if the practitioner's method truly yielded optimal solutions as claimed, then it should have done just as well or better in 100% of the instances.

2. However, the retailer was wrong because his old method optimized against constraints based on an average scenario. There is no guarantee that his decisions will be feasible to the original problem, over all scenarios. Therefore, in the instances where the old method did better, the recommendations had to be infeasible, since we cannot, of course, find a feasible solution better than an optimal one.

3. So we now know that the old method generated infeasible solutions at least 25% of the time. If we ignore these cases where we know it was surely infeasible, and look at the remaining 75% of the cases where it may have been feasible, it did worse 100% of the time due to a combination of suboptimality and overly-constrained instances.

In every case, the old method was either infeasible or suboptimal. The average-based heuristic must be discarded.

No sooner had King Vikram concluded his answer than the vampire, along with the corpse, gave him the slip.

(pic source link: omshivam.files.wordpress.com)

Indian Intellectuals and the Fighter-Pilot Syndrome

Update: title changed

Legendary formula car racer Michael Schumacher suffered a serious injury in a skiing fall. As millions around the world pray for his safe recovery,  a troubling question was triggered by this sad news:

"How likely is it for a skiing enthusiast, who is known to have made a successful career in the superfast and dangerous world of Formula car racing, to meet with a skiing accident?"

Does this conditional probability increase or decrease? I am not aware that Schumi claimed he was a skiing expert or thought of himself as one. This is just a sample of one and could just be a tragic coincidence. The question remains open and the focus of this post is on a related topic.

Here's a wikipedia blurb on a US Air Force officer John Stapp:
"During his work at Holloman Air Force Base, Stapp became interested in the implications of his work for car safety. At the time, cars were generally not fitted with seatbelts, but Stapp had shown that a properly restrained human could survive far greater impacts than an unrestrained one. Many traffic-accident deaths were therefore avoidable but for the lack of seatbelts. Stapp became a strong advocate and publicist for this cause, frequently steering interviews onto the subject, organizing conferences, and staging demonstrations (including the first known use of automobile crash test dummies). At one point, the military objected to funding work they believed was outside their purview, but they were persuaded when Stapp gave them statistics showing that more Air Force pilots were killed in traffic accidents than in plane crashes. The culmination of his efforts came in 1966 when Stapp witnessed Lyndon B. Johnson sign the law making manufacture of cars with seatbelts (lapbelts at that time) compulsory..."




Controlling fast jets did not give those pilots additional skills that made them equally safe at driving cars at some speed. Is it possible that this 'fighter pilot effect' gave them a false sense of security while driving the much slower motor cars?  Similarly, safely driving ultra-fast cars shouldn't automatically make one an equally safe hi-speed skiing expert (update: initial reports indicate Schumacher was not going very fast). However, public belief in this 'fighter pilot syndrome' appears to exist at some level, and this is especially true in India. For example, if you win a Nobel Prize or for that matter, any prize in the west, then regardless of your field of expertise and your near-total ignorance about what makes India tick, you are given special powers that turn you into an expert on every topic under the sun (especially Indian culture and politics), overnight. Unlike Marxist economist Amartya Sen or India's egoistic movie stars, who don't need a second invitation, there are others who prefer not to make a fool of themselves in public. However, the Indian media does not spare them the embarrassment by demanding their "fighter pilot" advice on unrelated topics. This 'intellectual celebrity' feedback is then used to try and influence public opinion. A good example is the recent NDTV-25 debate panel on "secularism in India" compered by 2G-scam tainted journalist Barkha Dutt that included exactly one genuine expert, Arun Shourie, who knew what he was talking about, and bunch of other "experts".

All-weather experts and their Indian media co-pilots must be asked to wear their seat-belts and slow down before they take the Indian public for a ride.

Happy New Year. Drive Safe. Get well soon, Schumi.

Optimizing Kindle Book Rentals

When to buy at Amazon
Amazon just raised their 'free shipping' threshold to 35$, a few weeks before the holiday shopping season. This simple 'entropic optimization' approach, which utilizes Amazon's wish list to time-prioritize purchases remains valid, but requires an increased level of procrastination. What also caught my attention is Amazon's Kindle rental models. Beyond the initial sunk costs, virtual products are high margin, with negligible holding cost, besides an infinite, instantaneously replenish-able inventory. They are also scratch/damage proof. The only long-term downside to providing a renting option appears to be faulty pricing. If we price too low, we may turn many potential buyers into renters, and a high price may discourage potential renters. Let's look at a couple of (real) Kindle rentals for which I laboriously pulled data while watching Sachin Tendulkar's 199th cricket test match.

Kindle Book 1 (Undergrad Math textbook)
The minimum rental period is 60 days (50$), and the maximum (apparently) is around 360 days (140$), with the marginal price held approximately constant. We pay 30 cents for every extra rental day beyond the minimum period.

Kindle Book 1: Price Versus Rental Days

The cost (snapshot at the point of observation) of purchasing a permanent copy was 200$. If we plot the percentage price discount versus the rental period expressed as percentage of a year, we can see that the discount varies between 25% and 70% of the full cost. Approximately linear model employed for this book. Here, we can rent the book for an entire year without paying the full price.

Price Discount Percentage versus Rental Percentage-of-Year

Kindle book 2 (Advanced forecast-modeling textbook)

The second example is a bit more interesting. The content is technically far more sophisticated compared to Book 1, but the target market is different, and the number of (paper) pages is far lower, and so is the price. In both instances, the cheapest rental can be purchased at less than half the full price. There are roughly three different marginal prices employed within a rental period that varies between a minimum of 30 days (~$15) and a maximum of 365 days (~$35, also the full Kindle price). The corresponding breakpoints occur (roughly) near the 90-day, and 180-day rentals, respectively. If we restrict our attention to this rental time period, the price is concave, with the marginal prices decreasing as the rental period increases. It is preferable to simply buy the book rather than rent it for close to a year.

Kindle Book 2: Piece-wise linear rental pricing

A percentage based plot is show below, along with an empirical power-law pricing model (using Open Office) that looks like a near-perfect fit for this particular book rental. A log(x) model also works well in this instance.

Optimization Models

Seller: How would optimization scientists go about determining these marginal rental prices? Suggestions welcome. Perhaps ideas from analytical rental models for other products (cars, houses, equipment ...) can be used as a starting point to figure out this "information rental" model. Perhaps the pricing model can be initialized using historical rental data gathered for similar books. This being an online retail sales model, we can dynamically and frequently update these models or their parameters to maximize performance metrics. 

Buyer: From a user-perspective, if we can assign a value for owning a permanent copy, and have an informal mental model of the temporal utility of a rental as T(x), then (for example), we could solve some variation of this single-decision problem in 'x':

Maximize Value V = T(x)/f(x) (⇒ Maximize log T - log f, optionally)

l ≤ x ≤ u
to determine an optimal 'rent versus buy versus walk-away' decision based on our willingness-to-pay. Assuming a 1:1 mapping between 'f' and 'x', so we could transform any price range limit into an equivalent (l, u) bound on 'x'.
A simple way to solve this problem is to enumerate the values of V for all rental days using a spreadsheet.

Renting digital textbooks over a semester
Like thousands of Indian immigrants in the U.S, I came on an assistantship, carrying a couple of hundred bucks in my pocket that represented a big chunk of my parent's savings. I actually felt rich when I discovered that the take-home monthly income from my research assistant-ship after tuition fees turned out to be more than what my engineer dad was earning after decades of dedicated service in Nehruvian India. Until I saw the prescribed textbook prices, that is. Buying overpriced books was simply out of the question when the few copies in the library were already taken. There was no Amazon then, and it would've been amazing to have a rental option like this, especially when continued funding is dependent on maintaining good grades.

For example, if we only cared about a textbook for portions of a semester (our total planning horizon), and our total price budget is 'pmax', then we can informally solve a multiple-period version of the above optimization model to come up with a best waiting strategy and rent for one or more time periods ("quiz time") which maximizes our total T(x) and also keeps us within our "knapsack" like price budget for the semester. This policy of "rent as needed" may work well with book rentals having a constant marginal price. On the other hand it may be worthwhile renting fewer times for an optimally longer duration if the price is concave in the length of the rental, as it is in our second instance. 

Gandhi and Operations Research

October 2nd is the birthday of Mahatma Gandhi, a major spiritual force behind the Indian freedom movement of the 20th century. Gandhi-ji also was a fundamental and direct inspiration for Martin Luther King Jr.'s civil rights movement of the 1960s, and Nelson Mandela's struggle against apartheid. In this post, we attempt to examine his idea of ahimsa from an optimization perspective.

Update (Oct 5): This Huffington Post article provides amazing insight into Gandhi's ideas.

What is ahimsa?
Indian textbooks mention that Gandhiji brought the colonial empire in India to its knees by using ahimsa and sathyagraha (both were 'spelling bee' words a couple of years ago). These words have no equivalent in English, and are often used to imply "passive resistance", "pacifism", or "non-violence". A mathematical optimization model provides a more useful translation.

The popular Sanskrit verses on himsa and ahimsa given below was popularized by Gandhiji:
ahimsa paramo dharmaha,
dharma himsa tathaiva cha
[Oct 2018 update: the second verse has been attributed to Swami Chinmayananda)

My translation:
Non-harming is the greatest virtue;
So too is righteous harm.

The second line suggests that allowing cruelty to go unchallenged is equivalent to willingly permitting harm, and therefore, must be resisted. The verses are a combination of the ideal (global optimality = zero harm), and a context-dependent violation of that ideal (soft rule = minimize harm). 'Local harming' is permissible in rare circumstances when it results in an overall reduction in global harm. The gangrenous foot has to be amputated to save the body, or a terrorist who attacks innocents in a mall or a school has to be taken down by security forces. In a recent talk, Narendra Modi tells us a story of how Gandhi would request his assistant at Sabarmati Ashram to pour back half a cup of water back into the river, because all he wanted was half a cup. Minimal harm!

(updated October 2)
Optimization Model of Ahimsa
From an optimization modeling perspective, these ahimsa verses represents an objective function of minimizing harm. In normal circumstances, the optimal value should be zero, but in all circumstances, it should be minimal. When some non-zero harm is inevitable, the goal is to limit the total harm to a minimum, i.e., the employed level of harm is optimal if and only if it is necessary and sufficient to restore dharma. The 'necessary' condition implies minimalism of the counteracting harm, while the 'sufficient' condition implies the safe neutralization of the source of the harm. It's a tough balancing act for humans even though nature itself effortlessly adheres to Newton's third law. A pure hard-constraint version of ahimsa would discourage self-defense and even celebrate cowardice, while a pure soft constraint version could open the doors to unnecessary use of force, and justifying cost versus benefit approaches. (The legal system dictum of "let a hundred guilty go unpunished, but a single innocent should not be wrongly convicted" is an interesting case study in this regard.) Hence, applying any one of these two lines is an incomplete specification and can lead to unpredictable results.

We argued a while ago that these verses are an improved 'fail-safe' choice for the 'zeroth law' of robotics. In the real world, when we build decision models to aid decision making, we can optimize decision variables to pick a pareto-optimal solution that also results in the least disruption to the system ("don't fix what isn't broken"). For example, if we are scheduling workers to maximize efficiency or minimizing cost, then an optimal solution that also minimally disrupts (and preferably, enhances) their quality-of-work-life is more likely to be sustainable over the long run.

Gandhiji's Swaraj
Many feel that Gandhiji was partial to the first line, and quotes attributed to him support this claim. On the other hand, Gandhi's 'Hind Swaraj' and his lesser known quotes on preferring violent self-defense to cowardly capitulation suggests that he was aware of both verses. His book 'Hind Swaraj' (Indian self-rule) implies that his primary objective was not merely an overthrow of colonizers, but to achieve the strategic and deeper goal of ending the cultural genocide of India (restoring its Sanskriti and dharma). Applying the ahimsa verses would yield a path to Swaraj that results in minimal incremental harm to India's Sanskriti and dharma. Such a path may not necessarily also be optimal in terms of being the shortest-time path, or the least painful, or one that maximizes regained territory.

Sine-Generator in the Aryabhatiya, 1500 years ago

Came across this post by Rajiv Malhotra on facebook, and am posting a screenshot.  More research and findings continue to trickle in about the discoveries and creations of ancient Indian mathematicians, and this sloka (verse) is not an isolated example. The focus of this brief post is on the design of the sloka from the data compression optimization aspect.

Brevity and conciseness was important to ancient Indian scientists and linguists for various reasons. Their success in optimally designing such slokas to be memorized by maximizing the density of information carried per unit syllable, while also (and this is important) ensuring that the sloka is logically sequenced and constructed, and relatively easy to memorize, is utterly remarkable. In particular, the affection for the minimal among those Sanskrit grammarians is legendary. There is an oft-repeated saying in ancient India that the joy experienced by a grammarian who manages to achieve a half-a-syllable reduction is only equaled by their joy upon the birth of their child. The choice of the objective function drives the design approach. In contrast with the previous example, we have the Vedic chants in Sanskrit, the world's oldest, unbroken surviving oral tradition, where redundancy was deliberately added to minimize error in oral transmission (an achievement that may have contributed toward India having the world's oldest, unbroken civilization). In this case, what was memorized and repeated was much, much longer than the original verses in order to optimize information and audio fidelity.


(Mysore is a city in Karnataka, India, whose state language is Kannada)

Update (Sept 5, 2013)
The English transcript of the sloka can be found here in the Nature Journal (thanks to Srikrishna ‏for sharing this link).

Update 1: December 15, 2013
A superb post at http://hindufocus.wordpress.com on a compressed Sanskrit verse for remembering the value of Pi to 32 decimal places.
"..... Here is an actual sutra of spiritual content, as well as secular mathematical significance.

gopi bhagya madhuvrata
srngiso dadhi sandhiga
khala jivita khatava
gala hala rasandara

..... The translation is as follows:

O Lord anointed with the yogurt of the milkmaids’ worship (Krishna), O savior of the fallen, O master of Shiva, please protect me."

Why the UPA will get re-elected in 2014: in 100 words or less

Stats indicate UPA to be the most corrupt government in Indian history, so it won't get reelected in the 2014 elections. Wrong. Here's proof.

1. Scam cash, embezzled dough, and black money is a sunk cost. Irrecoverable past. 

2. INC has turned India's voting mass into MYLOPs.

By definition, MYLOPs are Markovian; their vision of the future is independent of past events, given instant gratification. It follows from (1) and (2) that whoever supplies the biggest freebie to the voting mass now, wins. UPA just designed the free lunch, transcending the NFL theorem. 

No contest.

Notation

INC = Indian national congress, the dynastic party (ideology: Nehruvian socialism), which has wrecked India almost continuously since political independence (1947).

UPA = United Progressive Alliance = INC + clones, won the last two elections, seeks a hat-trick (= three-peat, in American).

MYLOPs = Myopic, Local Optimum seeking Pessimists. First introduced here. 

Free lunch = FSB = A grossly underfunded food security bill, which provides a theoretical guarantee of free food for all poor people, but in reality is a food bank for UPA's vote bank.

Predicting Coconuts and Baby Chickens

The next few blogs cover observations from a recent visit to India and South-east Asia.

Earlier this month, I used to wake up at dawn every day to visit the farmer's market in my ancestral town in Tamil Nadu, India to purchase fresh vegetables for salad, curry, and sambhar.

The vegetables here are sold directly by farmers, and are simply delicious. The vegetables I purchase in US groceries may look better, but in terms of variety, flavor, and cost, are no match for what is on offer here in this village market.

I was introduced to a wise and elderly person selling coconuts in his stall.

My family buys coconuts exclusively from him. Coconuts, like the cow, have a special place in Hinduism. The Coconut is like the Kalpavriksha, the giving tree, while the cow is associated with the Kamadhenu, the wish fulfilling cow. The reason, as is usual with Hinduism, is both scientific and compassionate. The sanctity of a particular place, object, person, or living being is usually tied to net benefit of the entity with respect to the cosmos. The Indian cow and the coconut tree are incredibly giving in a variety of ways throughout their lifetime, demanding little in return. Worthy of emulation and praise.

Coconuts are often used in a Pooja (as a prayer offering) at temple. Often, Indians break a coconut prior to beginning an important task, or undertaking an tough journey, or launching a new project. When used for such a purpose, it is desirable that the coconut not be tender or rotten or dry, but perfectly fresh. Breaking a bad coconut at a temple is deemed to bring ill-luck (from a scientific perspective, this act aims to turn a complacent person/group into the more alert and focused type, pre-empting any mishap with positive action. Ill-luck is most certainly avoidable). However, there is no obvious, guaranteed way to tell from an external observation whether the inside of a coconut is spoilt or not. For example, roughly 50% of the coconuts I select at the local A&P grocery store turned out to be spoilt (The degree of spoil is actually a continuous variable, but for now, we simply treat it as a binary indicator). Thus, careful attention is paid to the selection of temple coconuts.

The white turbaned coconut vendor in the picture is an expert at picking out good coconuts. He has an excellent track record. I asked him about the various factors he takes into account prior to making his selection decision. Aside from a visual inspection, he conducts a quick 'sound test' to pick a coconut. If you let him know that this coconut is destined for a temple, he picks one that informally maximizes the probability of goodness. Also, spherical coconuts are preferred to the ellipsoidal ones, since they tend to shatter quite spectacularly like grenades, emphatically warding off ill-luck :) An empirical analytical model for predicting the probability of a good coconut can be constructed using a logit (logistical) model, for example:

log(odds of a good coconut) = A0 + A1 * thickness of the shell + A2 * latent quantity of coconut water inside + A3 * shape_factor +  A4 * sound_factor ... = U

By measuring the values of the explanatory variables in the RHS and recording the outcome after breaking the coconut, we can calibrate the coefficient vector (A) using historical data (maximum likelihood estimate, for example), giving us the following probability model that predicts the goodness of a coconut:

prob (good coconut) = exp(U) / (1 + exp(U).

When we pick coconuts, our mental decision model informally estimates the RHS for a given set of coconuts and picks the one that appears to maximize the odds. During the last few years, our vendor has picked just one bad coconut. It seems he was quite distraught, and replaced the coconut free of cost.

A related, traditional prediction problem that requires specialized skills and experience, is determining the gender of a baby chicken. Multiple techniques have been employed to make this prediction. Unlike coconuts, experts here can  make a conclusive determination (probability ~ 1.0). As the book "Moonwalking with Einstein" notes: "A good chicken sexer can identify the gender of approximately 1,000 chickens an hour and much of this has to do with their expert memory of chicken private parts".

Exhaustive Search for #orms books in South India

Blog#2 from India.

A bookstore I've regularly visited over the last couple of decades is the 150+ year old Higginbothams on Mount Road, Madras (Chennai).

(pic source link: wikipedia)

My first ORMS textbook: 'Linear Programming and Network Flows' by Bazaraa, Jarvis, and Sherali was purchased here. In recent years, finding the latest ORMS books here has become tedious. The Chennai Metro rail construction has adversely impacted the parking area in front. Yesterday's search spanned the entire second floor, and ORMS/Business Analytics books were found in the following sections: Management (Management Science books), Operational Management (OM, Supply Chain), Statistics (a slew of Intro2ORMS books, probability models, queuing theory), Linear Algebra (Linear Programming), Computer Science (computational complexity, graph theory, combinatorics), ... Perhaps this situation is a reflection of the field of ORMS itself. It's been a multidisciplinary area since the beginning.

Sapna book house in Bangalore has a good collection of computer science oriented material.

(pic source link: images.touristlink.com)

SBH once boasted of a huge collection of technical books, and it's still a pretty decent place to look for books relating to business analytics, data mining, and machine learning.

Gangarams is another well-known book store in Bangalore.

(pic source link: dnaindia.com)

I've found very few ORMS books here in the past, and I plan to skip it this time around.

Off late, flipkart.com, the Amazon-like online bookstore offers a massive assortment of books, including ORMS titles. It ships only to Indian addresses currently, accepts credit cards, and offers free home delivery within 2-7 business days. Despite these benefits, online shopping denies one the simple joy of a leisurely enumeration search in the afternoon, wading thru stacks of old, dusty tomes in search of that one hidden gem. In a totally random location at Higginbothams was a copy of 'Optimization for Machine Learning'.

Estimating the Number of Refugees inside the Red Fort, 1947

An interesting historical use of statistical thinking during a humanitarian crisis. The situation is as follows:

Britain decided to up and leave India one fine day, but not before partitioning the land into the Islamic state of Pakistan, and Gandhi's pluralistic India. Communal riots broke out in 1946-47 during the biggest cross-border exodus of populations in mankind at that point in time. In Delhi, the capital city, an unknown number of (the majority of the) Muslims who chose to remain in India arrived to seek shelter in the iconic red fort and also within the smaller area of Humayun's tomb. The rest of the situation is described by C. R. Rao, professor emeritus at Penn State, and ranked among the greatest living statisticians, in his 1989 lecture (emphasis and comments in parantheses added by me):"

"...The government employed contractors to feed them. The contractors used to submit bills to the government of the purchases they made of different commodities [that were in short supply] like rice, pulses, salt etc. , to feed the refugees. A Secretary to the government of India suspected that the contractors were over quoting the commodities they purchased and he thought of asking a statistician to go inside the Redfort and count the number of refugees. Perhaps there were no statisticians in Delhi at that time, he sent a telegram to the Indian Statistical Institute in Calcutta requesting for a statistician to be sent to Delhi immediately. Two statisticians who had experience in conducting sample surveys went to Delhi by air and on arrival they were taken to the Redfort. 

When they wanted to go inside the Redfort to see how they can count, the guards did not allow them to go inside as they were not members of the same community as the refugees. The problem then was to estimate the number of refugees without going inside the fort or having any  knowledge of the concentration of refugees inside the Fort. The experts had conducted household surveys in Calcutta and had some idea of per capita consumption of different commodities. Here statistical thinking came to their rescue. If R, P, and S are the amounts quoted by the contractors of rice, pulses and salt, and r, p, and s are the per capita consumptions of these commodities as estimated from household surveys, they argued that R/r, P/p and S/s are estimates of the same number of persons. Based on the figures quoted by the contractors, the three estimates based on rice, pulses and salt were, 30,253, 21,122, and 10,891 respectively. They chose the smallest number based on salt. Rice was the most expensive commodity at that time and apparently the contractors were over quoting the amount of rice purchased to make money. There was another protected area, the Humayun Tomb where [a smaller number] some have taken refuge. A volunteer belonging to the same community as the refugees entered the tomb and counted the number of refugees; The statisticians used the same formula there and found that the salt estimate agreed with the number counted. The government secretary was pleased and accepted the salt estimate..."

(pic source: wikimedia)

C. R. Rao was one of the early members of the renowned Indian Statistical Institute (ISI) that was founded by the legendary P. C. Mahalanobis, an institute the produces fine Operations Research grads to this day. He's briefly mentioned in an earlier post, and an upcoming post in this space will again be related to his work.

Historical Postscript
1. A story that involves covert sampling to predict the number of pro-India soldiers to provision during the 1857 war of independence in India is briefly covered in 'Operation Red Lotus'.

(pic source: http://whc.unesco.org)
2. The tomb, which was visited by President Barack Obama on his India visit, houses the mutilated remains of a so much more relevant person - Dara Sikoh, perhaps the Mughal who most deserved the suffix "great". The Red Fort (and the Taj Mahal) was built by Indian workers on the orders of Dara's father.

(pic source: wikipedia)

Dara was the most learned and compassionate in a dynasty characterized by excesses that causes resentment to this day. He respected, even celebrated the pluralism of the dharmic thought system that has defined India since times immemorial. In unbelievably stark contrast stands his brother, the fanatical Aurangazeb, who unfortunately for India, prevailed in a tense struggle for the throne and had Dara killed most gruesomely on charges of blasphemy, before proceeding to slaughter about 4.6 Million Hindus (NY Times, 2011), statistically ranking him at #23 in the all-time list of mass killers (WW2 is at #1).

June 6: edited typos.

Jugaad Innovation: Stuck in a Local Optimum

Updated July 5, 2013:
CNN link on Jugaad Innovation.

The book cover blurb sounded exciting: Do more with less. An alternative to risky expenditure-driven, resource-hungry growth using a "bottom up approach to frugal and flexible innovation". Endorsement from a salesforce.com CEO. No doubt, Jugaad is a useful concept - it's a Hindi word that implies a improvised and clever work-around, but there is such a thing as stretching an idea too far. Jugaad arises from the Indian way of doing "more with less" - possibly a public response to artificial scarcity induced by genocidal British colonization since the late 18th century when a resource-rich India's share of world GDP plummeted from 25% to a negligible quantity in a short time to turn it into from a knowledge and manufacturing economy into an agrarian, impoverished nation. A scarcity-driven economy that has been nurtured by Nehruvian socialist politics over the last 60+ years. However, the word itself ties in nicely with ideas in Operations Research that deal with the optimal allocation and utilization of scarce resources, and hence this post.

The book has some heart-warming and splendid examples of Indian innovation, which are nice to read. The remarkably decentralized and entrepreneurial nature of the workforce, and their seeming comfort in operating effectively within what appears as "chaos" to the external observer is one of the salient features of the native Indian economy. A discussion of the Indian practice of the "missed call" (the closest that mankind has come to 'half a bit' of information) adds humor. The authors could have lent depth to the book by exploring the deep-rooted cultural origins of the decentralization, and entrepreneurial spirit that drives the Indian way of doing things. They do make some useful comparisons between the profligacy and rigidity of some Western CEOs with the adaptive nature of the "jugaad entrepreneur". However, celebrating every success from Bharti airtel to PepsiCo's direct seeding to Mitticool (now that is really cool) as a win for "Jugaad" dilutes the message. When the book hyped the crappy 'Aakash' tablet, the populist, Indian government's tax-payer subsidized tablet for the unwashed masses (a cheaper and cynically sub-optimal alternative to constructing and maintaining primary schools), it was time to cry halt. Celebrating artificial scarce resource allocation, however efficiently done, feels like a cop-out.

Operations Research practice is not just about optimizing within constraints and declaring victory. Value can be unlocked by using OR to expose the more expensive bottlenecks in the system. Constraints that turn out to be artifacts must be eliminated whenever possible. I'm all for (also) 'thinking small', but not at the expense of losing the context.

The Scamster versus the Statesman (or why the Indian National Congress fancies its chances in 2014 and 2019)

A Scam Response Model
The loss due to government-blessed scams and failed populist schemes in India run in the order of Billions and Trillions of Indian Rupees in a country where the impoverished still form a sizable percentage and earning under or around the government designated poverty level of ₹20+ a day (50₹ ~ 1$). To many Indians, such twelve-zero numbers seem fantastic. Thus a person caught stealing thousands in a small town may receive a sound thrashing because people more clearly recognize the utility of amounts that actually show up in their historical experience. However, as the size of the theft increases beyond imaginable sums, the magnitude of the public response does not appear to proportionally increase. The law of diminishing returns seems to kick in, resulting in a concave response function. We postulate that:
Change in Public Response (ΔR) ∝ Change in # of Zeros (ΔZ) in Scam amount S , i.e


R = k log S, where k is a constant to be estimated using historical data.

The assumption of a logarithmic response model turns to be quite useful in the Indian context. For example, if political party A pulls off a 12-zero scam, and party B a 6-zero scam (using a log-10 base):
response(A) = 12k, and response(B) = 6k.

The response is only a factor of 2 more for the scam that was a million times bigger. This makes it easier for party-A to equate itself to party-B in the eyes of the public. It can have its cake and perhaps eat it too. Most Indians would not find it hard to map A & B to their real-life representatives.

Statesman versus Scamster - Round 1
A subjugated, under-informed electorate may exhibit a strong concave response that is characterized by relatively high sensitivity to small thefts at one end, and a disregard for mega-scams at the other end.  We postulate that such a population's mental decision making model tends to be (1) myopic, (2) local optimality seeking, and (3) pessimistic (MYLOP). It accepts its terrible local living conditions provided the situation doesn't get perceptibly worse, and votes in polls based on this mental model. The immediate cause-and-effect tied to a local ₹100 theft represents a clear and present risk to current survival and is likely to elicit a swift and violent response, just like a populist cash-equivalent freebie elicits an equally enthusiastic response. However, a gigantic national scam that all but ensures that its future generations will see no improvement is shrugged off. MYLOP behavior is akin to a frog that allows itself to boiled alive because it does not register the gradual increase in water temperature until it is too late.

On the flip side, a government that is largely scam-free and claims to work on long-term growth, but is perceived to have marginally worsened the status quo can get booted out of power in the next election. Asking such a population to endure temporary 'hard choices' in order to be rewarded with medium-to-long term improvement ("no free lunch", "it will get worse before it gets better") implies a non-convex decision model - a hard sell since it clashes with the MYLOP attitude of the population. Scamsters + freebies trumps the Statesman here. Let's dig a little deeper into scams.

How to Hide a Scam?
Answer: A new scam that is an order of magnitude bigger. This can be explained via a mathematical model based on an interesting technical report by John Cook, M. D. Anderson Cancer Center (and author of the Endeavor blog):

The soft maximum of two variables is the function
g(x; y) = log(exp(x) + exp(y))

Given scams S1 and S2, the cumulative response from the public per our postulated model is log(S1 + S2). For example, the Commonwealth Games of India (CWG) plus a few other scams put together was a 9-zero scam, and this was followed by the 2G bandwidth sell-off, which was a thousand times bigger, i.e., a 12-zero scam. The cumulative response is:
log(10^12 + 10^9) = log(10.001 ^ 12) = 12.0005211273k,
a value that is barely more than the response to just the 2G scam itself (12k). In other words, the cumulative response to scams is approximately its soft maximum. The 2G scam simply eclipses the CWG scam in the mind of the public and the media. Even if there were 10 CWG-level scams, they'd be cumulatively eclipsed by a new 2G-level scam for all practical purposes and drop off the radar.

Scamster versus Scamster
Suppose a generally honest political party-B gets greedy seeing the muted public response and its forgiveness of prior scams. If it imitates party-A and pulls off a CWG-level scam, the public's anger would not be a 1000 times smaller. It would only be (12-9)/12 = 25% smaller. Having a clean prior record is not very helpful. Furthermore, if Party-B campaigned on a plank of 'transparent party that is different', it will be perceived as being not very different from party-A (75% alike) despite being 99.9% less corrupt compared to party-A, measured in terms of money siphoned off.

A MYLOP population that is forced to choose between two scamsters (even if their scam sizes are different) will pick the one that offers better freebies. If both provide equal freebies, the MYLOPs are likely to alternate between the two choices, every electoral cycle (e.g state of Tamil Nadu or Uttar Pradesh).


 
Statesman v/s Scamster - Round 2
Getting out a terrible locally-optimal situation may require a statesman to bring out and maintain a series of organic (sustainable and permanent) and highly visible improvements to first earn the trust of its MYLOP people over a sufficiently long period before unveiling any risky long-term development models. This means a lot of very hard infrastructure work: providing reliable 24x7 electricity, good roads, clean water, primary schools for girls, security for women, etc., to fundamentally alter the decision model of the population from MYLOP to a more optimistic and forward-looking model where people begin to see the possibilities within their grasp. An electorate that is moving along such a positive gradient is more likely to vote to continue along this path of incremental improvement without settling for "dont rock the old boat" and "here and now" populist freebies provided by the scamster.  So the statesman can win (e.g. State of Gujarat), but it takes a lot of heavy lifting to be done upfront.

The two seemingly contradictory outcomes from the two state elections that were announced yesterday (Gujarat and Himachal Pradesh), as well as many other Indian elections in the recent past can be plausibly explained using these simple models.

Update 1: 01/21/2013
Interesting quote from 'Steve Jobs' top 5 mistakes" that directly relates to the above idea of hiding scams using bigger scams:
"... The lesson that I take from these defunct products is that people will soon forget that you were wrong on a lot of smaller bets, so long as you nail big bets in a major way (in Jobs's case, the iPod, iPhone, iPad, etc) .."

Operation Red Lotus and Jules Verne

Around the World in 80 Days
One of the few positive side effects of Hurricane Sandy knocking out power for nearly a week was the rediscovery of a childhood joy of reading history books and stories of valiant Indian queens and kings by the candlelight. Globe-trotting dreams of kids in 1970s socialist India who couldn't afford to set foot on an airplane were kindled by reading and re-reading stories like Jules Verne's fascinating adventure travelogue 'Around the World in 80 Days', a book written a hundred years earlier in 1873.

Indian Rebel
Just a few years before Jules Verne wrote some of his most popular books, the English government was engaged in a bloody, all-out war for economic power in the Indian subcontinent. The rebellion against the colonizers in 19th century India was not just a local mutiny of Sepoys (hired soldiers), and not just an uprising of the peasants; native rulers (Rajas), dashing queens (Ranis), naturalized Indian kings of the dying Mughal dynasty, military leaders, intellectuals, priests, bankers,  traders, farmers, ... entire populations participated in the struggle to free India socially, economically, and culturally in what came to be recorded as the first war of Indian Independence fought in 1857. This is one of the many excellent findings recorded in the historical book "Operation Red Lotus" written by Parag Tope, a descendent of Tatya Tope, chief coordinator of the Indian war machine during that time period.

Jules Verne's Captain Nemo of 1870 appears to be a character straight out of this war. A hi-tech sea-faring native prince seeking vengeance against the English, Nemo is not far from reality. Native rulers set up navies that relied on the construction of relatively sophisticated ships to safeguard overseas trade interests that were being overrun by the piracy of the English East India Company (EEIC). This is hardly surprising given that the world's first dockyard was built in India and the maritime prowess of native kings on both sides of the Indian coast is legendary. However, the key battles of the 1857 war were fought inland in Northern India.

Operation Red Lotus

A wealth of information from recently recovered original correspondence to Tatya Tope during this time period appears to have encouraged the writing of this book. This excellent review summarizes the findings. While there is ample evidence to suggest that the strategy and tactics employed by the Indian resistance in 1857 was indicative of a comprehensive and long-term approach, claims of an sustainable war effort cannot be taken seriously (to paraphrase Gen. Omar Bradley) unless it can be shown that there was a feasible logistics solution in place. Operation Red Lotus (ORL) answers this question in the affirmative.

pic source

The Lotus Code: Operations Research and Logistics?
ORL is mainly about how Tatya Tope and other leaders on the Indian side designed a simple and neat supply chain to support the war effort and maintain an army of newly unemployed native soldiers, bereft of any provisioning infrastructure. Tatya's first step, like we do in the corporate world today, was to forecast enterprise-level demand - but he had to do this without tipping off the Red Shirts. Toward this, he exploited the contempt the colonizers had for 'backward' Indian rituals and traditions. Tatya Tope secretively sampled demands at the platoon level using a red-lotus based numerical code. The English were puzzled to see a spate of red lotuses being passed around the garrisons in many cities they controlled and initially attributed it to yet another weird 'blood red' native custom. The number of petals in a red lotus is approximately the same as the number of soldiers in an EEIC platoon.

 (source colorbox.com)

A native soldier who volunteered would pluck a petal and pass the flower on. After all soldiers made their choice, what was left of the flower was sent back to Tatya. By counting the reeds and/or the remaining petals in a returned lotus from a particular geographical location, noting the number of lotus-sampled platoons, and rolling up these estimates to a regimental level, Tatya was able to predict the approximate size of the native army by location. Having this fundamental information at hand, the planners calculated the provisioning required to sustain the war effort in its various stages, and also finalize the line of attack to capture and hold Delhi, the capital, as well as other strategically important cities. The next challenge was to geographically allocate supplies to satisfy this location-specific demand forecast given that the emerging native army did not have the resources or the time to rapidly create and manage their own own supply line. The only viable solution was to outsource this task, which was accomplished via the Chapati (an unleavened Indian bread) code.

The Chapati Code
After the flood of lotuses, the English were bemused to see Chapatis being exchanged by the chiefs of some villages. Alarm bells went off and they managed to intercept Chapatis embedded with lotus seeds.

 (source food fun freak blog)

The English concluded that this was not another strange custom and that the exchange of lotuses and Chapatis was some kind of a secret handshake. Red lotuses weren't found in the hands of villagers, and seeded Chapatis were not found in garrisons. What historians did not fully realize was that the lotus and the Chapati distribution were the code; not merely Boolean flags, but a vector representation of numerical information relating to troop strength and battle plans. We now know that the Indian war planners sent Chapatis to the various villages along a directional chain graph on a Euclidean plane whose source was a major garrison, and the sink was an important military objective.

[ORL book extract]

The men of the villages that were ready to support the war effort would requisition and stock food grains, and the women of the village would be required to prepare food in quantities proportional to the number of Chapatis they received. The village chief would pass on a similar number of Chapatis to the next village in the designated chain. There is little doubt now that this was a coordinated and planned war to achieve self-rule.

Breaking Down The Supply Chain
When the time was ripe, the sepoys mutinied on the pretext of greased cartridges (immortalized by the legend of Mangal Pandey),  catching the EEIC and the Victorian government in England off guard, and the initial phase of the war went against the colonizers. However, once they recognized the logistical method employed by the natives, the reprisals were eerily Nazi-like: swift, decisive and genocidal, and in keeping with a pattern that was repeated a few times before India's political independence in 1947.  Villages on these chain graphs that supported the war effort were interdicted. Emergency laws were enacted that essentially gave English officers the license to kill civilians based on suspicion. Large populations, including women and children were wiped out, successfully disrupting the Indian supply chain.

 (source: wikipedia)

(source columbia.edu)

Once the logistical backbone was broken, it was a matter of time before the colonials won the war. However, it turns out that the war also had some cascading international effects.

Global impact of the 1857 Indian War of Independence
The London government withdrew huge sums of money from their banks and investments, including prominent financial institutions in the U.S, to raise sufficient capital to fund and ship out tens of thousands of fresh troops to India to win the war, while publicly spinning this fiasco as a localized case of mutiny. This behavior did not escape the attention of some Americans who questioned the official version.
.

  [ORL book extract, pages 86-87]

ORL notes another pertinent question that George Train asked: "England says they are short of funds. Where are the hundreds of millions of silver that have been shipped there [India], disturbing the currency of the world?

The withdrawal of such huge sums of money by the English from US banks exacerbated the panic of 1857, and the resultant liquidity problem severely impacted the Northern states in the US besides triggering the world's first ever global economic crisis. Furthermore, the war hit the EEIC-pirated textile goods exported from India to the west, which reacted by sourcing more cotton from the southern US states that was being produced at even cheaper rates using African slave labor, thereby boosting the Confederate economy and morale. Thus, the Anglo-Indian war of 1857 appears to also have had some impact on the subsequent American civil war of 1861-65.


Jules Verne's Phineas Fogg is most likely based on George Train, a remarkable real-life American adventurer-entrepreneur who went around the world a few times, including a round trip in 67 days.

Legacy
It is less known that the Indian resistance achieved a few tangible victories for India in the long term. For brevity, we will have to leave that discussion for another day. India lost the 1857 war but those who resisted genocide became legend; none more so than the heroic young queen, Rani Lakshmi Bai. This coming Monday, November 19 is her birthday. Reading the immortal line in the Hindi poem about her in the candlelight as a kid, and then again a few days ago about her incredible exploits in ORL even as hurricane winds howled outside, never fails to bring a lump to the throat.

खूब लड़ी मर्दानी वह तो झाँसी वाली रानी थी।।

"She who gallantly fought a man's battle, was our Queen of Jhansi".