ML, Natural Language Processing, Panini and a 2500 year old puzzle

News:

You must have seen the recent news:?

Rishi Rajpopat, pursuing PhD in St. John's College in University of Cambridge, has solved a 2,500 year old problem posed by Panini's ancient text, Astadhyayi.

https://www.bbc.com/news/articles/cg3gw9v7jnvo

There are many news articles on this. Unfortunately none of them really explain what the puzzle or conflict was, why Panini's book on grammar is one of mankind's greatest achievements and why it can have profound impact on Man-Machine interaction..?

Panini and his book Ashtadhyayi:

Panini wrote this book on Grammar around 4'th Century BCE. This book is like a Computer Algorithm on Language Processing. It is rules to codify Natural Language (Sanskrit).

This book is a set of 4,000 rules of Grammar and Language.?

1. These rules teach how to form new words (from existing vocabulary) and new sentences in Sanskrit that are grammatically correct.?

2. Written in 8 chapters, this book is written in Sutra's; like poetic couplets..?

3. These rules are written and are to be used in sequence. That is rules written later build and use the rules written earlier in the sequence.?

Conflict and Puzzle

After Panini, Katyayana and others wrote commentaries and built on top of Panini's work.?There was one puzzle or conflict. What to do in cases where more than one rules apply??Panini himself addressed it in the same book. One of the Metarule he wrote was?

"Vipratisedhe Param Karyam"?

Katyayana (150-200 years after Panini) interpreted this as:?

"When there is a conflict between two rules of equal strength, the rule that comes later in the serial order wins." - Taking 'Later' as the meaning of Param.?

Example of Conflict:

Lets take the word: “Mantraih” which means From the Mantras. This word is derived from combining two root words, Mantra (chant) and Bhi (from). There are rules to combining two words.?

Now there are actually two rules from Panini’s text, that are application in this case.?

Rule 7.2.103 (from Chapter 7):?The rule says when combining 2 words: when the first word ends with ‘a’, change it to ‘e’ when the second word starts with ‘bh’. So, applying this, resulting word should be ‘Mantrebhi’.?

and

Rule 7.1.9 (from chapter 7):?The rule says when the preceding?word ends with an ‘a’ replace ‘bhi’ with 'ih’. So, applying this rule, resulting word should be ‘Mantraih’.

Now, we all know ‘Mantraih’ is the word used in Sanskrit to convey ‘From the Mantras’.?

This is a conflict as Rule 7.2.103 comes after Rule 7.1.9. So according to the meta rule, 7.2.103 should apply. But ‘Mantrebhi’ is not the word used.?

Breakthrough

Rishi went back to the original Panini text and relooked at the Meta rule:

"Vipratisedhe Param Karyam"?

He found that ‘Param’ has 2 prominent meanings: ‘Later’, and ‘Right’ (on the right side).?

Because the book is written rules that are to be applied sequentially, Param was interpreted as ‘Later’ as in ‘Later’ in the sequence. That interpretation has led to many conflicts and exceptions.

Now Rishi while looking at many exceptions and examples, realized that if ‘Param’ is interpreted as ‘Right side’, the meta rule’s meaning changes.?

"Vipratisedhe Param Karyam"?can also mean when there are conflicting rules, use the rule applicable to the ‘right side’ meaning rule that is applicable to the word on the right.

Implications

These codified grammar rules written in sequence to construct (and understand) words and sentences were ideal for coding Natural Language processing. This codification is not possible today. Panini's text was a possibility for using of Sanskrit language in Machine-Human interaction through natural language. Even the rules and the way they were constructed can teach us how to codify other languages (English or a new language for Machine-Human interaction).

This was problematic because these rules when applied created many exceptions. And there can't be exceptions when we are coding rules for Machine Learning as conflict resolution becomes tricky. This breakthrough doesn't solve all the exceptions but solves majority of them.