Introduction.
Decision automation helps enterprises make better, faster, error-free, and people-independent decisions thereby allowing exponential growth while ensuring compliance, minimizing risks, and reducing cost of operations. Decision automation uses artificial intelligence (AI), decision keys (data/metrics/indicators etc.), and business rules to help organizations automate the decision-making process. Decision automation is the foundation that enables adoption of AI and transformation of business processes and products. It acts as the most important lever to accelerate the digital journey.
A decision automation solution consists of a set of software (or technology) components and set of intelligence components. It requires information input from various internal and external systems (information sources) to apply the intelligence. And it makes the final decision available as real-time APIs (Application Programing Interface) or as updates to one or more databases.
The software or technology components enable the decision automation solution to obtain the information input (from the information sources), process that information, apply the intelligence, and provide a decision output.
The intelligence components comprise of machine learning models and decision rules (example – “if-then-else” logic) which are implemented using the software or technology components. The intelligence components are created from historical data using one or more machine learning algorithms (example – neural network, ensemble methods like gradient boosting & random forest, SVM, Regression etc.). They may also be derived from the implicit subject matter expertise of business experts and codified as business rules. Figuire-1 below depicts the components and output of a decision automation solution.
Figuire-1: Components of a decision automation solution.
Technology components.
Information bridges (pipes or connectors).
Information bridges allow the decision automation solution to connect to multiple internal and external information sources. Common internal information sources include Loan Origination Systems, Loan Management Systems, Insurance Administration Systems, Point of Sales Systems, Call Centre Systems, ERPs, etc. Some of the common external information sources include Credit Bureaus, Data providers – LexisNexis, Acxiom, Neilsen, D&B, IQVIA, External APIs for KYC verification, Government APIs, Account aggregator APIs etc.
The information bridges enable the decision automation solution to obtain the required information that forms the basis for decision automation. Very often information across multiple systems is required to automate a particular decision. Below are a few examples.
- Underwriting an application for home loan
- Application information – income, type of employment, etc.
- Information from bank statement – debit, credit, balance, salary credited in bank, etc.
- Property information – property value, loan to value ratio, type of property, location etc.
- Credit bureau information – delinquency, outstanding, sanction amount and utilization, inquiries, number and types of trades, credit activity etc.
- Making an offer at a point-of-sale terminal
- Past transaction amount and number.
- Points earned and burned.
- Customer profile – time since registration, type of customer, segment etc.
- Available offers.
The information obtained using the bridges, must be processed, and are used to create features that are used for applying machine learning models (example – create a risk score, customer lifetime value score etc.) and for calculating decision keys (example – Fixed Obligation to Income Ratio, Change in Leverage, Effective Interest Rate, Average ticket size per transaction etc.). In most cases, the decision keys and the prediction from models are used within a business rule to arrive at the optimal decision for the customer or transaction.
A best-of-breed decision automation solution must provide an intuitive and easy user interface to configure the information bridges. Such configuration must include easy connectivity to multiple databases and data stores, easy integration of APIs and scheduling batch and real-time data exchange.
Data processor.
Once the raw information is made available using the information bridges, it is often required to process that information before it is used to create features or decision keys. Some of the common processing include:
- Aggregation – Example: aggregating total value across number of purchases.
- Filtering – Example: filter out all banking transactions which are lower than a specific value or carried out before a certain cut-off date.
- Join – Example: Combine transaction information and points redemption information to create a customer-level information vector.
- Window operations – Example: Calculate change in ticket size between a transaction and its immediate predecessor.
- And many more.
The data processing can be performed using a programming interface, wherein users can run scripts using SLQ, Python, R, Java etc. To enable individuals without programming expertise to use the solution, an intuitive drag-and-drop interface for performing the data processing may also be very useful.
Feature builder.
Feature libraries are the heart of machine learning models. Many automated decisions require predictions form a machine learning model to be integrated in a business rule or used in isolation. To apply a machine learning model on the processed information, one must create features which form the input for machine learning models. Most analytically mature organizations use pre-built feature libraries for each key information source. Such feature libraries must be integrated within the decision automation solution. In addition, the solution must have the ability to add specific features for a given machine learning model.
Decision automation solutions must enable users to integrate existing feature generation programs and to create new ones. The solution should provide a programming interface and/or a drag-and-drop interface to enable creation of new features. It should be noted that usually data scientists or data engineers are responsible for creating new features; hence, a programming interface for feature creation may be more optimal in many cases.
To know about automated feature libraries from sources like Credit Bureau, Bank Statements, Income Tax Statement, Retail Loyalty Data, Telecom Transaction Data etc, please write to us @ contact@leobrix.com
Decision key builder.
Decision keys are metrics which are used by business users to develop decision rules or business rules. In most cases, business users use the prediction of a machine learning model and a set of decision keys to create the business rules. Below are a few examples.
Decision keys and model predictions are used for automating a credit decision.
- Decision keys
- Fixed Obligation to Income Ratio, Credit Bureau Utilization, Secured to Unsecured Leverage, Loan to Value Ratio, Bank Balance to EMI Ratio, Income to EMI ratio.
- Predictions (internal)
- Internal risk score
- Predictions (external)
- Bureau risk score.
Decision keys and model predictions to identify type of collection call to be a made to an early delinquent customer.
- Decision keys
- Outstanding Amount, Number of times Delinquent in the Past, Number of Past Promise to Pay, Number of Past Kept Promises, Number of Past Broken Promises, Bureau Leverage, On-us to Off-us Balance in the Bureau.
- Predictions (internal)
- Bucket Roll-Over Score, Self-Cure Score.
- Predictions (external)
- Bureau risk score.
Decision keys for making a real-time offer at point-of-sale.
- Decision keys
- Average Ticket Size, Average Interpurchase Time, Basket Diversity Index, Time Since Lats Transaction, Current Offers Available.
- Predictions / output of machine learning models (internal)
- Customer Lifetime Value Score, Customer Segment.
Examples of combining the above keys and predictions for creating business rules are discussed in the subsequent sections. To know more about Industry Specific Decision Keys, please write to us @ contact@leobrix.com
Models register and machine learning implementor.
Machine learning models are an integral part of most decision automation flows. The output of a model may be used as is to automate a decision or it may be used in conjunction with other decision keys (within a business rule). Hence, a decision automation solution must provide the utility to implement a machine learning model along with the pipeline required to obtain the required data and create the features for applying the model. This utility is sometimes referred to as Machine Learning Operations (ML Ops).
The model implementation component should also be capable of implementing model explain-ability methods like LIME or Shapley Values. This enables solution to identify the reason code or the key features that are driving the prediction for a specific case.
In addition to the ability to implement a model, it is often considered a best practice to provide a model registry. The model registry allows the registration of a model along with the model meta-data (information about the model – the features used, description of the features, development time, reference development data sets, development docs etc.). This feature allows users to know the details about the model and update future versions of the model.
Business rules engine.
Business rules are created using decision keys and predictions from machine learning models. The rules incorporate the implicit subject matter expertise of the business experts / process owners. The rules are created as decision trees; wherein each terminal node of the decision tree represents a recommended decision. To know about complete rule sets for key decision automation solutions, please write to us @ contact@leobrix.com. Below are a few examples of decision rules.
Rules used for automating a credit decision.
The rules are applied on a new application. Below is a simplified business rule for this use case.
Rules used to identify types of collection call.
The rules are applied on any account which has a DPD of 7 days or more. Below is a rule set for one of the sample nodes:
Decision – Harsh call
- Outstanding balance greater than threshold.
- Of-us balance more than threshold.
- One or more broken promises in the past.
- At least once more than 60 DPD in the past.
- Belongs to risky geo code.
Rules used to identify point-of-sale offer.
The rules are applied when a customer makes a purchase, and a match is made with the loyalty database using a mobile number or any other identifier.
Decision – get 10% off on next purchase.
- Time since last purchase is greater than average interpurchase time.
- Belongs to top tier as per transaction frequency and value.
- Average ticket size (calculated over the last 3 months) is greater than threshold.
The above are sample decision rules; real-decision rules are more exhaustive and involve a wide set of decision nodes.
Model governance.
Machine learning models form an integral part of most decision automation flows. Hence, it is very critical that one monitors those models and ensures that they are performing as per expectation. A detailed discussion on model governance is beyond the scope of this paper; interested readers can write to us @ contact@leobrix.com. However, a brief set of metrics that are used for tracking the performance of models are outlined below:
- Stability and data drift: Population Stability Index (PSI) at the overall level and for each feature.
- Model accuracy: Brier Score, Actual Vs. Expected Odds, Chi Square stats, R Square, MAPE etc.
- Discriminatory power: KS, Gini, Rank Ordering, Lift, capture Rate etc.
- Variable stability: PSI for each feature, Information Value for each feature, Mutual Information for each feature, Marginal predictive power of each feature.
A set of automated alerts and notifications are generated if the model performance is below one or more threshold as per the above tests.
Rules evaluation and tracking.
Decision automation rules are expected to provide the most optimal decision based on the inputs provided. The rules are created using predictive models, data-driven insights, and human judgment. Such rules must be tracked for efficacy to ensure that they remain relevant with the changing business environment, target segment etc. Hence, the solution must allow ways to track the effectiveness of the rules on an ongoing basis.
Decision consumption – real-time APIs and batch updates.
The final output of a decision automation solution is the recommended decision. Such recommendations are made available in real-time as APIs that can be integrated with multiple systems that will make use of the decision (example – loan origination system, Point-Of-Sale system etc.). For certain use cases, the decision may also be consumed as a database update to the backend database of the system that will consume the decision. A decision automation solution must provide the ability of consuming the final decision in both the forms as stated above.
Intelligence components.
The technology components described above are instrumental in implementing the intelligence derived using AI/ML solutions or codification of human intelligence. Primarily, two types of intelligence components are required for decision automation – 1) Machine Learning models 2) Business rules. Most of the machine learning models in turn require features as inputs and Business rules require decision keys as inputs. Hence, features and decision keys are additional intelligence components that are required. Machine Learning models are developed by internal data science teams or external experts. In some cases, the organization may use pre-built or consortium models provided by an external provider.
In most cases, business rules are derived using the judgement of business experts. Internal business analysis and data science teams often provide key data driven insights or validation of hypothesis which are also used to create decision rules. For example, in the context of a lending organization, the internal data science team may build a credit risk model and the risky policy team may use the model and insights form past data to design risk policy rules which are implemented as business rules to automate credit decisions.
