Predicting Coconuts and Baby Chickens

The next few blogs cover observations from a recent visit to India and South-east Asia.

Earlier this month, I used to wake up at dawn every day to visit the farmer's market in my ancestral town in Tamil Nadu, India to purchase fresh vegetables for salad, curry, and sambhar.

The vegetables here are sold directly by farmers, and are simply delicious. The vegetables I purchase in US groceries may look better, but in terms of variety, flavor, and cost, are no match for what is on offer here in this village market.

I was introduced to a wise and elderly person selling coconuts in his stall.

My family buys coconuts exclusively from him. Coconuts, like the cow, have a special place in Hinduism. The Coconut is like the Kalpavriksha, the giving tree, while the cow is associated with the Kamadhenu, the wish fulfilling cow. The reason, as is usual with Hinduism, is both scientific and compassionate. The sanctity of a particular place, object, person, or living being is usually tied to net benefit of the entity with respect to the cosmos. The Indian cow and the coconut tree are incredibly giving in a variety of ways throughout their lifetime, demanding little in return. Worthy of emulation and praise.

Coconuts are often used in a Pooja (as a prayer offering) at temple. Often, Indians break a coconut prior to beginning an important task, or undertaking an tough journey, or launching a new project. When used for such a purpose, it is desirable that the coconut not be tender or rotten or dry, but perfectly fresh. Breaking a bad coconut at a temple is deemed to bring ill-luck (from a scientific perspective, this act aims to turn a complacent person/group into the more alert and focused type, pre-empting any mishap with positive action. Ill-luck is most certainly avoidable). However, there is no obvious, guaranteed way to tell from an external observation whether the inside of a coconut is spoilt or not. For example, roughly 50% of the coconuts I select at the local A&P grocery store turned out to be spoilt (The degree of spoil is actually a continuous variable, but for now, we simply treat it as a binary indicator). Thus, careful attention is paid to the selection of temple coconuts.

The white turbaned coconut vendor in the picture is an expert at picking out good coconuts. He has an excellent track record. I asked him about the various factors he takes into account prior to making his selection decision. Aside from a visual inspection, he conducts a quick 'sound test' to pick a coconut. If you let him know that this coconut is destined for a temple, he picks one that informally maximizes the probability of goodness. Also, spherical coconuts are preferred to the ellipsoidal ones, since they tend to shatter quite spectacularly like grenades, emphatically warding off ill-luck :) An empirical analytical model for predicting the probability of a good coconut can be constructed using a logit (logistical) model, for example:

log(odds of a good coconut) = A0 + A1 * thickness of the shell + A2 * latent quantity of coconut water inside + A3 * shape_factor +  A4 * sound_factor ... = U

By measuring the values of the explanatory variables in the RHS and recording the outcome after breaking the coconut, we can calibrate the coefficient vector (A) using historical data (maximum likelihood estimate, for example), giving us the following probability model that predicts the goodness of a coconut:

prob (good coconut) = exp(U) / (1 + exp(U).

When we pick coconuts, our mental decision model informally estimates the RHS for a given set of coconuts and picks the one that appears to maximize the odds. During the last few years, our vendor has picked just one bad coconut. It seems he was quite distraught, and replaced the coconut free of cost.

A related, traditional prediction problem that requires specialized skills and experience, is determining the gender of a baby chicken. Multiple techniques have been employed to make this prediction. Unlike coconuts, experts here can  make a conclusive determination (probability ~ 1.0). As the book "Moonwalking with Einstein" notes: "A good chicken sexer can identify the gender of approximately 1,000 chickens an hour and much of this has to do with their expert memory of chicken private parts".

Oracle Crosstalk 2010: Yes, OR can help retail

Analytics is a hot topic for retailers. Several presentations during this conference returned to this theme and is a huge positive sign for OR in this industry. However, how to apply science smartly to the tons of sales data being collected (and retailers collect this like holy dust nowadays) and to automate robust decision making is another question altogether - 99% of the retailers do not have the right skill sets to pull this off. It is not easy building this system in-house from the ground up, and companies like Oracle Retail can do this better and faster, allowing retailers to focus on their core business areas. Few if any, were even aware of 'OR' (is OR by any name still OR ?). On the other hand, the general consensus was that revenue and supply chain optimization is a must to compete in this tough economy (honey, i shrunk the margins). A panel of investment experts with representatives from major financial institutions, both government and private, felt that in addition to top-line growth, being able to optimize the use of scarce resources to achieve operational excellence is one of the keys for survival in this new economy.

The level of hype surrounding the virtual region that lies at the intersection of social networking and mobile technology is staggering with numbers going off the charts (literally). How can OR grab a share of the science pie here? For example, I liked the way Wet Seal has been innovating and surely this area is going to see new and innovative applications being developed soon. Experts from Google and other tech-heavy companies weighed in on this as well and all signs point to an interesting retail shopping experience in the near future. Get your scan-ready smart-phone out and interactively and collaboratively shop in a real store with your virtual friends. The line between brick-and-mortar stores and the virtual shopping world is rapidly blurring.

On a side note, it was great visiting Chicago after a few years. The single most beautiful landmark there is the dazzling Swaminarayan Mandir (loosely translates to 'temple') in Bartlett. Hand-sculpted from Italian Marble and Turkish limestone by thousands of Indian craftsman, this Mandir does not rely on machine-tools and does not contain a single metal part or fastener in keeping with the sacred Indian tradition of Mandir architecture. Interlocking stones are used throughout, with individual stones weighing between a few grams to 5.2 tons. Pretty amazing. It is open to the public and is a must-see for visitors. The gardens and fountains surrounding the temple are well-maintained and pleasing to the eye. Inside the complex, smartly designed fiber-optic lighting and hidden underground heating add an invisible high-tech layer to this construction, and combines with the silence that must be observed within the ethereal sanctum-sanctorum to create a deeply personal and spiritual experience. The food served in the community center in the basement is excellent, and is strictly vegetarian, of course.