The Contextual Optimization of Sanskrit

I'd written a blog for the INFORMS conference earlier this month based on my practice perspective, which emphasized the importance of contextual optimization rather than despairing over the 'not infallible' theoretical worst-case nature of certain mathematical problems. This is something well-internalized by those in the large-scale and non-convex optimization community, where 'NP-Hardness' is often the start, rather than the end point of R&D.

The wonderful 'Philip McCord Morse Lecture' at the recently concluded INFORMS conference in San Francisco by Prof. Dimitris Bertsimas of MIT touched upon this point, and the 'conclusions' slide in the talk explained this idea really well. To paraphrase, 'tractability of a problem class is tied to the context - whether the instances you actually encounter can be solved well enough'. I mentioned the Sulba Sutras in that blog - a well known body of work that epitomizes the Indian approach to mathematics as Ganitha - the science of computation. The genius, Srinivasa Ramanujan, was a relatively recent and famous example of a mathematician hailing from this tradition. The Indian approach is often algorithmic and more about rule generation than infallible theorem proving. Not that Indians shied away from proof ('Pramaana'. For example, see 'Yuktibhasa'). As I understand it, this sequential process of discovery and refinement does not lose sleep over theoretical fallibility, and consists of:
a) in-depth empirical observation of, and a deep meditation on facts,
b) insightful rule generation,
c) iterative, data-driven refinement of rules.
This quintessential Indian approach is applied not just to math, but to practically every field of human activity, including economics, commerce, art, medicine, law, ethics, and the diverse dharmic religions of India, including Hinduism and Buddhism. Panini's Sanskrit is a great example of this approach.

Panini, the famous Sanskrit grammarian (along with Patanjali) is perhaps the most influential human that much of the world does not know much about. His fundamental contributions to linguistics more than 2000 years ago continues to transform the world in many ways even today. Noted Indian commentator, Rajeev Srinivasan, has recently penned a wonderful article on Panini and Sanskrit. You can learn more about Panini's works by reading Dr. Subhash Kak's (OK State Univ) research papers (samples are here and here). This blog was in part, triggered by this article, and talks about Sanskrit and its contextual optimizations.

Abstract: Sanskrit recognizes the importance of context. Two examples that show how Sanskrit is optimized depending on the context, in two entirely opposite directions, is shown below.

Optimization-1. The grammar is designed to be entirely context-free as Rajeev Srinivasan's article explains, and anticipated the 'grammar' of today's high-level computing language by more than 2000 years: precise with zero room for ambiguity of nominal meaning. To the best of my knowledge, punctuation marks are not required, and order of the words can be switched without breaking down, although there may be personal preferences for some orders over the others, and the sentence remains unambiguously correct. An optimization goal here therefore is to generate a minimum (necessary and sufficient) number of rules that result in an maximally error-free production and management of a maximal number possible variations of Sanskrit text. In this case, Panini appears to have achieved the ultimate goal of generating a minimal set of rules that will produce error-free text, forever. There are other well-known optimizations hidden in the structure and order of the Sanskrit alphabet - more on that later.

Optimization-2. The final interpretation of many keywords in Sanskrit ARE contextual. Which means there are multiple, related interpretations for some words that have a nominal/generic meaning, but you have to optimize the final interpretation at run-time by examining the context of usage, to recover the most suitable specific choice. If the first optimization helped eliminate fallibility, this second optimization in a sense re-introduces a limited fallibility and a degree of uncertainty and freedom by design! This feature has encouraged me to reflect (recall Ganesha and Veda Vyasa), develop a situational awareness while reading, pay attention to the vibrations of the words, and grasp the context of keywords employed, rather than mechanically parse words and process sentences in isolation. A thoughtful Sanskrit reader who recognizes this subtle optimization comes away with a deeper understanding.  For example, Rajiv Malhotra, in his book 'Being Different' (now in the top-10 list of Amazon's books on metaphysics) gives us the example of 'Lingam'. This can mean 'sign', 'spot', 'token', 'emblem', 'badge', etc, depending on the context. Apparently, there are at least 16 alternatives usages for 'Lingam' of which one best suits a given context is picked, and not simply selected at random. And of course, the thousand contextual names ('Sahasranamam') for Vishnu is well known in India. Some well-known western and Indian 'indologists' have ended up producing erroneous, and often tragic translations of Sanskrit text either because they failed to recognize this second optimization, or because they misused this scope for optimization to choose a silly interpretation, leading to comic or tragic conclusions.

Again, this contextual optimization approach by the ancient Indians is not just restricted to Sanskrit, but is employed gainfully in many areas, including classical arts, management, healthcare, ethics, etc., and of course dharmic religion. This contextual dharmic optimization has perhaps helped India in getting the best out of its diverse society, as well as keep its Sanskriti refreshed and refined over time. For example, the contextual ethics of dharma (ref: Rajiv Malhotra's book) has a universal pole as well as a contextual pole that allows the decision maker faced with a dilemma, to not blindly follow some hard-wired ideological copybook, but contemplate and wisely optimize his/her 'run-time' choice based on the context, such that himsa is minimized (dharma is maximally satisfied). Some posts in this space has tried to explore the applications of this idea'.

An earlier blog discussed a related example of seemingly opposite goals for contextual optimizations. When it came to mathematical algorithms, data, and linguistic rules in Sanskrit, a goal was to be brief and dense, minimize redundancy, and maximize data compression, so that for example, an entire Sine-value table or generating the first N decimals of Pi can be both encoded and decompressed elegantly using terse verse. Panini's 'Iko Yan Aci' in the Siva Sutras is a famous example of a super-terse linguistic rule. On the other hand, when it comes to preserving long-term recall and accuracy of transmission of Sanskrit word meanings as well as the precise vibrations of mantras (e.g. Vedic chants) that are critically linked to the 'embodied knowing' tradition of India, the aim appears to be one of re-introducing controlled data redundancy to maximize recall-ability, and error-reduction. This optimization enabled Sanskrit mantras to be accurately transmitted orally over thousands of years.

To summarize, contextual optimization is a powerful and universal dharmic approach that has been employed wisely by our Rishis, Acharyas, Gurus, and thinkers over centuries to help us communicate better, be more productive, healthier, creative, empathetic, scientific, ethical, and interact harmoniously with mutual respect.

update 11/22/2014: 'optimize the final interpretation at parse-time / read-time' is the intent for optimization-2, rather than the computer-science notion of 'interpretation at run time'.

Lassoing the Exponential

An abbreviated version was blogged for the INFORMS 2014 annual conference as 'Not Particularly Hard'.

-------------------------------------

While working on a new retail optimization problem a few weeks earlier, a colleague was a bit disappointed that it turned out to be NP-Hard. Does that make the work unpublishable? I don't know know, but unsolvable? No. The celebrated Traveling Salesman Problem (TSP) is known to be a difficult problem, yet Operations Researchers continue to solve incredibly large TSP instances to proven near-global optimality, and we routinely manage small TSP instances every time we lace our shoes. Why did I bring up laces? In a moment ...

Hundreds of problems that are known to be difficult are 'solved' routinely in industrial applications. In this practical context it matters relatively less what the theoretical worst-case result is, as long as the real-life instances that show up can be managed well enough, and invariably, the answer to this latter question is a resounding YES. The worst-case exponential but elegant 'Simplex method' continues to be a core algorithm in modern-day optimization software packages. 

This issue of contextual optimization is not a new one. For some ancient people who first came across 'irrational' numbers, it was apparently a moment of uneasiness: how to 'exactly' measure quantities that were seemingly beyond rational thought. For some others, it was not much of an issue. Indeed, there is an entire body of Ganitha (the science of calculations, or mathematics) work in Sanskrit, the 'Sulba Sutras', almost 3000 years old, where irrational numbers show up without much ado. 'Sulba' means rope or lace or cord. If we want to calculate the circumference of a circle of radius r, we can simply use (2πr) along with an approximation for 'π' that is optimally accurate, i.e., good enough in the context of our application. If we we did not have a good enough value for π, we could literally get around the problem: simply draw a circle of radius r, and line up a Sulba along its circumference to get our answer. For really large circles, we can use a scaled model instead of ordering many miles of Sulba. Not particularly hard. Encountering a really difficult optimization problem can be a positive thing, depending on how we respond to it.  Often, there are alternative approaches to business problems that at first glance, appear to have  insufficient data: this tempts us to throw in the towel and send the problem back to the customer and say "infeasible" or "unbounded". Instead, we can use a Sulba and Lasso our decision variables. This could well be an ORMS motto:

"When the going gets NP-Hard, Operations Researchers get going" :)

86400X speedup?

I once read a research paper that stated that their customized nonlinear solver reduced computational time for a particular problem class from days to seconds, i.e., something like an 86400X speedup. 

(pic source link: espn.go.com)

Digging a little deeper, it seems the authors did not notice prior work that solved similar sized instances of a more difficult discrete nonlinear case, using an analogous CPLEX-based approach, in a few seconds to a few hours in the worst case. Even a conservative 'from several minutes to a few seconds' mean-improvement is impressive (~100X faster). After deleting complicating side-constraints and relaxing integrality restrictions, the resulting continuous relaxation can indeed be solved really quickly compared to the original problem.

Amartya Sen recently confessed to pulling numbers out of thin air to grab people's attention, lending credence to the claims of his detractors. I hope O(claims) does not turn into a total marketing game in the future.

Synchronized Scheduling Case Study - Part 2

The case study was introduced in the previous post. Read the introduction here.

.... Despite their suboptimal approach, the schedulers of the LEG carrier did some nice work with their synchronization approach.

2SAT versus 3SAT
There is a jump in decision complexity when moving from a two-body conflict resolution problem to three-body conflict-resolution. Intuitively, we can observe this in linear programming (LP) problems. LPs limited to two variables per constraint are easier to solve. The dual (two constraints per variable) is a generalized network flow problem that can be solved using purely combinatorial methods. Intuitively, one can adopt an alternating heuristic that iteratively fixes one variable and optimizes the resulting univariate problem. In computational complexity theory, SAT-2 is easier to manage compared to SAT-3.

Rather than trying to simultaneously juggle flight attendant (FA), pilot, and aircraft maintenance schedules, the LEG schedulers adopted an simplification. They combined the two human elements into a single 'crew' scheduling entity. This can be achieved via "co-pairing", i.e., by building common pairings (partial schedules) for FAs and pilots employing X = {X_PILOT ∪ X_FA}, i.e., every pairing would be feasible to both pilot and FA contracts and FAA regulations. Column generation allows you incredible flexibility in accommodating such a change. This flexibility is invaluable when hit by a perfect storm. Now, they could work with a two-variable per constraint structure so that their telephonic "alternating heuristic" negotiations could converge in a timely manner. Unfortunately, such a beneficial reduction in complexity was accompanied by inefficiency. The approach erred inordinately on the conservative side. The resources were now too tightly integrated, resulting in a cost spike due to under-utilization of expensive resources. LEG's R&D team was invited to resolve this problem.

Integrated Crew and Aircraft Routing (ICAR)
The OR team quite naturally proposed an integrated scheduling approach that would reduce costs, improve reliability of operations, and also enable tactical synchronization of flight connections. However, the number of possible connections to be evaluated is O(n²) for a n-flight schedule. Methods in the literature are typically data-agnostic; sophisticated alternatives to the straightforward approach designed to perform an acceptable job even in the worst case. Luckily, LEG's OR team had to solve only LEG's problem and not that of every future airline and space shuttle.

The Beauty and the Brute Force
Given the problem size spread over a five-day period, there are a few million candidates to synchronize in the theoretical worst case. However, our focus is only on same and overnight connections, which reduces the candidate list to a few hundred thousand. When we recognize that we only have to focus on the set T of connections of the same sub-fleet type at the same airport (duh), and exploiting a few other contextual restrictions, this final number is data-driven. It turned out that there were no more than tens of thousands of synchronization possibilities to deal with, 99% less than the theoretical bound. This finding allowed the possibility of cheerfully discarding (as we found out a couple of years later when some of the first journal papers on ICAR came out) all the neat out-flanking theorems and approximations presented in the literature.  The method chosen may score low on aesthetics but was effective and flexible in managing the full blown ICAR instances in context. There is a childlike joy in discovering that enumeration solves your practical business problem - akin to finding out that this emperor has very little clothes. This basic counting step gave the team a clear goal.

Problem Statement (at last)
Given a flight schedule, jointly construct feasible crew pairings and aircraft strings such that their scheduling intersection (elements within the connection set T) are optimally synchronized.

And, yes, there were a few TSP (Traveling Salesman Problem) instances to be managed within ICAR. To be continued....

Synchronized Scheduling Case Study - Part 1

Synopsis 
When a low-cost airline (LCA) entered a short-haul aviation market many, many years ago, a legacy airline carrier (LEG) decided to compete with LCA on its own turf by matching the LCA's fare, frequency and super-quick aircraft turn-around times on ground ('turns' for short). This required a high degree of coordination between LEG's in-flight crews (pilots and flight-attendants), ground-crew, airplane maintenance planners, and the airline schedulers (including their scheduling optimization system). In particular, the utilization of the pilots and the flight-attendants had to be maximized. The challenge however was that all these different resources were part of separate organizational structures and were governed by different contractual rules, and the LEG idea eventually turned out to be cost-prohibitive and unreliable due to poor synchronization of resources.  On the other hand, the Operations Research (O. R) team built a specialized scheduling system for LEG (far ahead of the literature available at that time) that would've achieved the needful. Unfortunately, the silo-ed organizational structure hindered adoption.  Result:  Unlike the legendary revenue optimization contest where the legacy carrier triumphed, the tables were turned here, and the LCA flourished. The low-cost LEG idea was resuscitated many years later in a different context and market, and did a better job using the lessons learned from this early setback. In the next few posts, we will compare the two approaches used to synchronize these resource scheduling tasks and summarize the lessons learned.

Airline Scheduling Basics
Given an aircraft schedule, the resource planning is done sequentially in order of their profitability (or cost) to the airline. Thus:
LEG flight schedule → pilot schedule→ flight attendant schedule → aircraft maintenance routing

The cost-critical task here is the construction of "pairings" (or "strings" for aircraft), which denote partial (unassigned) solutions that represents partial tours of a resource. For LEG pilots (set of two persons), and FAs (set of three persons), this represents a tour in a time-space network that starts from a crew domicile (a small subset of airports in the LAC schedule), traverses a bunch of different airports over 1-5 duty days with intervening periods of rest known as layovers, before returning to the home domicile, while satisfying a myriad of contractual and FAA rules for work hours, rest time, etc. In short, the cost and constraints that determine each pairing is an extremely complicated nonlinear, non-smooth, non-convex, nonplussing mixture of on-duty + rest + reliability + crew work life balance + other path-dependent factors. 

Good quality pairings can be created using a dynamic column-generation procedure. A column here represents a pairing, and during an optimization run, thousands of pairings are generated and selected from among the many trillions of possibilities using a "branch and price" scheme. Eventually a subset of the generated columns are selected such that every flight in the time-space network is covered by exactly one pairing, and such that the total cost is minimized (where the actual 'cost' being optimized is a combination of a hundred-odd objective function components).

Note: An airline crew-pairing solver's set partitioning problem is arguably the easiest optimization model in the world to represent on paper:
{SPP: Minimize cx : Ax = 1, x binary, x ∈ X}

In contrast, the real-life computer code used to manage this simple model can easily run in the order of many hundred thousand lines (largely due to the complicated and detailed nature of X), with expensive parallel-processing machines, and an embarrassment of CPLEX licenses to match. Why? A small percentage point reduction in crew pairing cost (cx) can easily recover all these infrastructure costs and then some. Since crews get paid the max of planned and actual costs, these numbers are actual bottom-line savings. Furthermore, such scheduling tools can be mission-critical systems that run 365 days a year, requiring O.R product support staff to be on call like primary care doctors. 

Solution Approach
Downstream systems, and not of immediate interest to this post, combine these pairings into actual monthly schedules with pilot, first officer, and flight attendants assigned by solving yet another set-partitioning problem. Thus cost-effectively scheduling even a single resource requires a sophisticated solution approach. In those days, academic literature often contained interesting approaches that looked good on paper but proved to non-implementable because of the highly context-driven and data-driven nature of this problem that requires developers to pay careful attention to problem size, exploit special structures in SPP and X, and keep in mind certain carrier-specific objectives. On the other hand, interested readers can refer to this journal paper - the first one I read on the job. It is also the #1 cited INFORMS paper based on a recent ranking. Today, this portion of airline planning is typically outsourced to specialized airline crew-scheduling software vendors.

Pairings and Strings
A set of pilot pairings tell you, for any given LEG flight: the in-bound flight that the pilots arrived in, and the out-bound flight on which they will depart. Of course, if a LEG flight schedule is built with lots of tight turns (say 30 minutes or less) at an airport, then it makes sense from the point of view of operational reliability and cost, for the crews to stay with, and depart on the same airplane, assuming of course, that the aircraft routers will assign the same aircraft (tail number) to these flights. This is a resource synchronization rule: Keep crews and aircraft together to the extent possible while also keeping costs low and satisfying all contractual duty, rest, and maintenance rules.

The FA scheduling system can take pilot pairings as inputs to try and keep the FAs with the pilots. Unfortunately, since these crews have different operating hours and safety rules give the varying nature of their job descriptions, very often, pilots and FAs have to break up and depart on different flights. This is an operational nightmare when you are trying to work with tight turns to compete with LCA, and actual costs shoot up if you 'over-synchronize'. To make matters worse, the strings (partial aircraft maintenance routing solutions) also have to be tightly woven with crew pairings. It's a planners nightmare.  Believe it or not, LEG folks actually built much of these schedules separately ("legacy approach"), with each scheduler anticipating what the other person will do, often coordinating via phone-calls in this multi-person game. Result: an expensive and sometimes operationally brittle schedule that increased operational costs and ate up some customer goodwill, which eventually resulted in the plug being pulled. In the next part, we'll see how LEG's OR team analyzed this problem.

to be continued .....

Ekthetikophobia: The Fear of the Exponential

More specifically, the fear of having to solve an NP-Hard optimization problem to save your job. 

Do you or anyone else in your organization exhibit symptoms of Ekthetikophobia?

Sample Responses by Ekthetikophobes
1. Resign: Capitulation is by far the most common response, especially if the person does not have an ORMS background and is oblivious to the 'science of better'. Throw hands in the air, lose faith in humanity, and slurp spoonfuls of the nearest meta-heuristic algorithmic prescription provided by bees, ants, bacteria, squeaky wheels, mutant turtles, ... any nature cure that uses pseudo random numbers. This response is best captured by the Hindi proverb "Naach Na Jaane, Aaangan Teda",i.e., a bad dancer blames the uneven floor.

2. Controlled Panic. This is pretty typical of a generation of OR practitioners weaned on CPLEX. Like MDs trying to decode a deadly new strain of flu, the overriding urge is to throw money at the problem by ordering the latest versions of the baddest bunch of industrial strength optimization solvers in the market with matching supercomputing accessories to run massive benchmarks, and generally scare the pants off their IT managers who work with depression-era budgets.

3. Code. This is relatively more common to programming gurus and follows exactly one rule: If you throw sufficiently well-written code at any problem, it must work. This is so crazy, these guys are on to something here.

4. Publish. The median response from E.phobic research folks is to put this exhibit on a pedestal for everybody to gaze at. It's akin to a biologist discovering a new specie or an astronomer sighting a new planet that shouldn't have been there. Half of these people (surely OR types) will go on to demonstrate the monstrosity of this new problem based on diabolical worst case instances that even a God who plays dice would not inflict on mankind. The other, more elegant half (theoretical CS E.phobes) propose 'factor of 2' approximations that cover all remaining difficult instances missed by the Muggles, yet just as useful practically, before declaring victory. Then over the next two decades: keep shaving of that factor; rinse & repeat; it's a veritable cottage industry.

Exaggerations aside, ranked at the top of the most systematic, resourceful, innovative, and practically useful responses toward 'new' NP-Hard optimization problems must be the approach adopted by Dantzig, Fulkerson, and Johnson (1954) to tackle a 49-city Traveling Salesman Problem using 1950s computing technology and fearless minds that dared lasso the exponential.

July 22: minor fix: added missing text.