Recap over Rule Based Systems

Due to user interface based configurable products we usually neglect fundamentals so thought to add recap over these concepts.

Rule based systems are also called as Expert systems because it harnesses the knowledge from Experts which is in form of Rules.

So, what are rules?

Rules are nothing but patterns in your data and associating those patterns with Actions will transform that into Rule. ?It has two sides left hand side which consist of patterns (also called as Antecedents) and right-hand side representing Action or several Actions (also called as Consequences). So, antecedents followed by consequences represents knowledge in modular form.

Job of Rule Engine/ Inference Engine-

Rule based systems accepts input data, processes that data with help of defined rules and suggests the possible set of actions or final decision.

Job of rule engine or inference engine is to pick up qualified rules by looking into data and execute those rules to generate actions followed by final decision. Its a very simple process. SME knowledge or domain peoples are just required to define rules and inference engine take care for rest of part.

Modern rule-based system products / product companies are providing such platform for users to define control flow via User Interface where users are defining workflows, rules to conclude into final decision. If you look into history its actually similar to Imperative programming.

How do you design control flow?

There are 2 ways, we can design control flows – Goal Directed or Backward Reasoning and Data Driven or Forward Reasoning.

Goal Directed is something which looks for sub goals to achieve goal. Lowest level or outcome of following this journey will turn into action.

Example, if you want to go on nice vacation so you will first look into accommodation reservation, travel arrangements and based on availabilities you will start booking which defines the actions.

Data Driven is like algorithm which will read the input data and will decide which action need to be performed. In sense, what to do next will be decided by available data.

Example, KYC processes which try to match input data with details provided by different organization/ government organizations for the personals. ??

This is all about reasoning but programmatically how it will be achieved so the techniques are called as chaining.

There are two types of chaining- Backward chaining and Forward chaining. Fundamentally, backward chaining goes well with Goal Directed Reasoning and forward chaining goes well with Data Driven Reasoning. But its not hard and fast, you can use any based-on circumstances.

Backward Chaining starts with goal and then look backward to gather facts which will help to achieve goal. It uses up-down strategy going from top to bottom. It is used in game theories, evidence assistant systems, rule-based systems and AI. This method is known to find proof using depth-first search strategy.

Forward Chaining algorithm starts from known facts with help of input data, triggers all rules whose properties are satisfied and delivers conclusion to the known facts. This process repeats until the problem is solved. It follows a down-up strategy, going from bottom to top. It is used in CLIPS, business rule systems and manufacturing rule systems. This method uses breadth-first search as it has traverse through all the facts first.

So far so good...

As said earlier, most of organizations are using standard products to derive their rule-based outcomes. For this, they ask their SMEs to define workflow using user interface of these products. But,

Are you following these chaining concepts before designing? May be yes or no.

Do we know, the workflow what you have designed is performance centric? May be no (getting these stats only after going in production… or rarely in during system testing) or may be yes.

There is concept called as Entropy (very well known if you are aware of ML). So basically, it will provide measure of disorder or uncertainty in system. I haven’t seen this in any product which offers such upfront stats for workflow designed by user. But in summary, it will help to measure impurity of node in tree diagram.

Some Technological Flavors of Rule Based Systems-

• Java Based Business Rule Based Management Engine – User can add or configure rules in configuration file and can implement logic of rules in code. Example, JBPM.
• Spark Integration with Drool - Spark provides a faster application of Drools rules to the data than traditional single-node applications.
• ETL Based Rule Engine- Abinitio is ETL tool, graph-based data processing platform so Rules can be configurable in graph or even we can pull from in Database to arrive at decision.
• XML Based Rule Engines – Workflow or nodes defined in tree diagram can be transformed in XML and ingestion of data with execution of XML nodes will arrive at outcome.
• Database Driven Rule Based System – All the rules, rule execution will be managed by data entries recorded in databases and even user interface.
• JSON based Rule System- All the rule configuration and workflow execution managed in terms of JSON files which are very light weight. These JSON files can be transformed via translators or API’s to execute defined workflow.
• ML & AI based - Advance algorithms or techniques enabling to extract pattern from data and reduce even work of configuration. So such implementations are just need to be observed to maintain ethics.

Many more…

This is all just at high level and believe me decision science is very interesting topic once you enter into it.

Connect with me separately in case this attract your interest.