Saturday, April 28, 2012

House Hunting Inefficiently

If you are looking to be more time-efficient in hunting houses, see a prior post here

Halfway through the house hunt, the process began to get mechanical. There was not much diversity in house designs, especially the newer ones, although there was this really nice and shiny new house adjacent to a cemetery, which threw me off a bit. Internet forum opinions ranged from "Loved it. Quietest neighbors I ever had" to "Hey, its Halloween every night here!".  The older ones seemed to have more character and personality. However, they came with their own maintenance list. I then got quite interested in the principles of the amazing ancient Indian science of construction and planning 'Vaastu Shastra' that actually turned out to be a pretty useful practical guide (better than my realtor sometimes). And so the search continued ..

A common reason for many houses in the U.S (perhaps not as common in India) being put up for sale is that the children grow up and move away, and their parents want to downsize. Such houses are inevitably full of memory trails frozen within photo frames: childhood sketches, family reunions, high-school trophies, often ending with snaps of a daughter's wedding. Looking at those pictures changed the objective function. What was 'just' a house to tick off the list, was for many years a home where a family was raised from the cradle, parents aged gracefully, and kids grew up with security. That is no easy thing to pull off in today's world, and perhaps there would be very few things more satisfying that emulating what some parents in those families did. It was quite humbling. Each such 'ordinary' house had an extraordinary American tale to tell, some happy, some not so. And no matter how much money one pours into home improvements, that unique signature of how a family lived in that home does not really go away; after all, a family breathes life into a house. 'Must-have' product attributes no longer seemed that important. House-hunting stopped, and the search for a home began. This approach may be inefficient, but it certainly feels more rewarding and less tedious.

Saturday, April 21, 2012

Housing Industry Analytics: Short-Sale versus Foreclosure

There's been an explosion of 'short sales' of housing properties by banks in the US market. In the east coast markets, there's been a 60-100% spike in short sales, why? If the banks wait too long and then foreclose, the properties loses too much value and the final returns may be lower. On the other hand, a short sale is immediate, and long term risk is eliminated. As the article in the NECN link above says:

"In a "short sale," banks agree to let someone who owes more on their mortgage than their home is worth to sell it to a new buyer, with the bank typically writing off tens of thousands of dollars in the process. But what the bank gains is avoiding the cost, protracted process and uncertainty around taking the home or condo by foreclosure and then trying to resell it as a bank-owned property."

A first look indicates that this is a decision optimization problem under uncertainty that is somewhat similar to (but not the same as) that faced by fashion retailers who are trying to clear their end-of-season perishable inventory. Do they markdown apparel right now or should they wait for some more time? If they wait, then over time, the 'fashion statement' value deteriorates and the retailer may have to more aggressively markdown to attract customers and clear inventory. On the other hand, if they markdown right now, that may turn out to be a hasty and expensive decision, with a certain probability. So really there are two decisions to be made: when to markdown and by how much?

A bank may own a majority stake in several properties whose values are depreciating over time in an over-capacitated market (property owners are unlikely to have the cash to maintain or make improvements), which they need to get off their books without losing much. Using stochastic optimization methods available in the field of Operations Research ("the science of better"), they may be able to do a much better job of profitably managing their inventory (it won't be surprising if they are already doing this). Stochastic optimization methods are specifically designed to work with probabilities of scenarios, as opposed to a deterministic approach that assumes everything is perfectly known in advance, although the latter often turns out to be a reasonable and quick first approximation. An alternative that may be especially appealing to pessimistic banks looking to avoid worst-case meltdowns is 'Robust Optimization' that can operate without formal probability distributions and can among other things, help minimize a bank's maximum regret. Another advantage of OR methods is that they are typically not capital intensive, and the ROI on successful projects can be remarkably high. In short, OR can be very, very useful here.

Tuesday, April 17, 2012

Analytics and Cricket - VIII: DRS & Bayes Theorem

In the last post on cricket, we mentioned that the false positive (F+) issue with the Decision Review System (DRS) employed in international cricket could be a deal-killer (see red zone in picture below).

In this post, we work out an illustrative numerical example using a well-known conditional probability model based on reasonable data derived from interviews of ICC personnel to show that the current F+ rate disproportionally reduces the efficacy of the DRS, causing it to operate only marginally more effectively that the human-only (umpire) method, and thus may not be worth the cost of maintenance unless the F+ rate is reduced to a more acceptable level.

For brevity, let's focus on bowler reviews in this example. A bowler will ask for a machine review of an umpire's original decision of not out, hoping to turn that into an 'out'. Umpires in the elite panel are themselves around 90% effective in making the right decision (so on average, only 10% of the subsequent DRS referrals should change the outcome if they work perfectly), so it is really that 10% gap that is the problem.

Today's cricket DRS system is claimed to be around 95% accurate in giving a batsman out, if in fact, the batsman is really out. Suppose the DRS also yields F+ results for just 1% of the bowler reviews, i.e. it gives a batsman 'out' when he is really 'not out' just like the umpire originally said. If 10% of the batsmen subject to bowler reviews are actually out (as obtained in the previous paragraph), what is the probability that a batsman is actually out given that the DRS overturns the umpire's decision to say he is out?

Answer: Let OUT be the event that the batsman reviewed is actually out (its complementary event is NOTOUT), and RED the event that DRS gave him out. The desired probability P(OUT|RED) is obtained using the Bayes formula by:

P(OUT|RED) = P(OUTRED)/P(RED)
Expanding out the terms, we can write this as
= [P(RED|OUT) x P(OUT)] /
[P(RED|OUT) x P(OUT) + P(RED|NOTOUT) x P(NOTOUT)]

= [0.95 * 0.1] / [0.95*0.1 + 0.01 * 0.9]
= 0.095/0.104 = 91%

Observations
1. Even a 1% F+ rate brings down the true efficacy of DRS, and it is not 95% as the ICC claims. The second term is a combination of F+ rate and human accuracy. Thus
P(OUT|UMPIRE SAYS OUT) = 90%
If the bowler asks for DRS review:
P(OUT|DRS SAYS OUT) = 91%
Not much of an improvement

2. The better the umpires get at their job, the worse the existing DRS will statistically perform. For example, if the umpires improve their upon their accuracy by just one percentage point, i.e. to 91%, the conditional accuracy of DRS changes to:
= [0.95 * 0.09] / [0.95*0.09 + 0.01 * 0.91]
= 90%


Thus, the tables are turned now and the DRS makes things worse for batsmen here and we may be better off not using DRS at all even if it is provided free of cost!

This second result may seem puzzling. Why does this happen? If the umpires get better, the frequency of true NOTOUT is 1% higher, and with the F+ rate held constant at 1%, there will be an increase in the total number of false positives over a period of time, in addition to a small decrease in count of true positives, thereby reducing the accuracy rate of the DRS.

You can plug in a variety of numbers to see what the corresponding results are. You can also perform a similar analysis for batsman reviews.

Recommendations:
1. Significantly cut down on the F+ rate and not just focus purely on increasing true positive rate

2. Improve the quality of original human decisions. This will reduce the dependence on DRS, encourage improvements in the DRS to keep pace, and obviously improve player attitude toward umpires.

3. If a brilliant cricketing instinct filled person like Mahendra Singh Dhoni talks about 'adulteration of human and machine', do think twice about it, he's got a useful math model behind this statement!

Reference: Introduction to Probability Models by Sheldon M. Ross. This example is a variation of an example from this book. Hope I did not mangle it.

Monday, April 16, 2012

Anatomy of an Online Debate

The Huffington Post ran a curious online debate a few days ago: Is Yoga a Hindu Practice? Let me state right of the bat that I thought the debate was utterly idiotic given that this question was like asking people to vote if baseball was quintessentially American, if the great pyramids were Egyptian, or if the great wall was built by the Chinese. After all, how the heck does a person debate against a fact? By twisting it into a silly insinuation about ownership. Nevertheless, let's look at the debate results measured by market-share for, against, and neutral to the topic, tracked before and after reading the debate. 


If this picture is unclear:
For, against, neutral (baseline) = (65%, 26%, 9%).
For, against, neutral (after reading debate) = (66%, 28%, 6%)
So the net result is that the 3% of the undecided split 1% toward 'for' and 2% toward 'against' after reading the debate.

Warning: If you are a predictive analytics connoisseur or swear by rigorous statistical methods, the rest of the post will be cringe inducing, so read on at your own risk.

Given that we have absolutely no data to use other than this pie chart, I tried to 'quick fit' a plausible Multinomial Logit choice model to these results with the aim of personally understanding how useful this debate really was. Toward this, I defined a utility function u(t) = exp (a0(t) + a1(t)), where:
a0 = baseline contribution for (t = aye, nay, neutral)
a1 = debate contribution
market share (t) = u(t) / (u1 + u2 + u3)

Using the 'before' pie chart, I obtained the following values:


where CONST = a0, and a1 = 0 at this point. Again, note that these are just plausible values and not statistically calibrated likelihood maximizing coefficients based on historical individual observations. Next, merrily using the  'after' pie chart to update the utility function taking the debate into account, I obtained the following values:



where DEBATE = a1 that is used to update the utility function. Note that the results don't really change dramatically.

Based on this plausible MNL model, we observe a positive value for a1 for both the 'for' and 'against' since their market-shares increase after the debate, and a negative value for 'neutral' since the debate forces a good chunk of the few fence sitters to switch. To measure the usefulness of the debate to each group, I looked at the ratio of "what was additionally useful" versus "prior understanding", i.e. the ratio utility_before/utility_after given in the last column. The results indicate that the debate itself was pretty close to useless to the overwhelming majority of the voters, i.e. for the 'aye' people (like me) and on the whole, the debate reinforced what they already knew, yielding a tiny usefulness change of about 0.4%. On the other hand, the debate was relatively more useful to the 'nay' people and it incrementally 'hardened their position' by about 6%. More than a third of the miniscule fence sitters actually took a stance and this debate appealed most to them.

Given that the voter response ('elasticity') for a particular choice-group to an event (debate) in an MNL model also depends on the incremental gain possible from their existing market-share, the degree of movement in market-shares for the three groups are not surprising, although the specific direction of the resultant net shift in market shares does indicate that the debate may have had something to do with it. Of course, one can game such online "changing minds" debates by entirely ignoring the debate and starting with a non-favorite position as the baseline and then simply selecting your most favorite position in the end.

On a side note, almost all the 'Yoga' practiced in the U.S and the west is really Yogasana, whose primary function is to help prep your mind and body for actual Yoga, which in turn has nothing to do with whether you can twist yourself into an exclusive USPO patent-protected double-pretzel or not, and everything to do with open-source inner-sciences that aim to rid a mind of ego, exclusivity, and dogma and reach higher levels of consciousness.