Top Challenges in Implementing a Learning Interface

The Right Approach

Training a machine learning model should not be too different from the human process of learning. It should be conceptually close to training a toddler to walk or talk, just like a toddler learns by observation, a machine learning model too, should learn to take certain actions based on the repeated exposure to certain actions by a human.

This type of learning based on a Human-Computer interaction (HCI) is becoming a preferred approach to train a machine learning model, since it can be done by business users or domain experts who can demonstrate and label the training data set by explicit user actions rather than a machine learning expert writing a code for it.

The Mortgage Challenge in Implementing a Machine Learning Interface

Business users like brokers, case handlers and underwriters in the mortgage origination domain are often stuck with mundane activities like arranging and classifying bulk documents received from borrowers, entering data into their systems by looking at the documents, validating the completeness, correctness and consistency of the documents and data for a loan application.

The HCI driven learning and automation approach, offered by Digilytics RevEL relies on a seamless and intuitive user experience offered as part of the software design, such as drag and drop documents to their respective categories with ease or training the models to extract the important and relevant data from documents through a point and click interface.

The UI/UX of Digilytics RevEl offering has been constructed with a basic but effective notion in mind, that the behavioural alterations for the business users should be minimum. Rather than business users bending their will to the technology offering, the technology should adapt and learn from the behavioural patterns and the specific user actions and in the background, allowing the machine learning algorithms to learn and improve the automation of tasks.

This approach however comes with its own set of challenges.

First and foremost, when the users are implicitly training a model based on their actions, it is important to consider that the users can be imprecise and inconsistent with their actions and can introduce their own biases which ultimately impacts the accuracy of the models especially in B2B systems where the data and processes are shared across multiple users.

In Digilytics RevEl applications, there are mechanisms built in place to control the overburdening of machine learning algorithm with data from every user action, the learning data instead is passed in bulk, periodically to the models only after certain logical checkpoints, for example only after a case is funded. This minimises the risk of the model getting trained on incorrect or intermittent data.

The second biggest challenge is the predictability of the learning response or output.

Even with the most refined training data definition, the response of a machine learning model on the actual data can only be probabilistically predicted and can never be absolutely determined, but when the training of the model itself is based on the data gathered from a variety of user actions and a wide array of data like in mortgage documents from different lenders, bank statements from different banks , following different types of layouts and structures, a predictable pattern or outcome is difficult to emerge.

This is where confidence score indicators comes into picture, like in Digilytics RevEl, anything predicted by the model with confidence less than an acceptable threshold is appropriately flagged and highlighted for the human in the loop to provide the correctional inputs.

The third most important factor for any learning model is validating the correctness of the learning and accuracy of the model.

It is not enough to just execute the automation experiment through machine learning, the ability to measure what is built, can often give much deeper insights to enable improvements along the way.

The Explainability reports on the learning models as offered by RevEl gives the business users insights into various data points such as the correct vs incorrect predictions by the model, the learning trends, or improvements that a model has made over time and much deeper insights like why certain prediction were incorrect using the Shap values.

As an increased focus and interest is put in by the industry into the designing of systems with an interactive learning interface which can be used by technical users and business domain experts alike, there is a need for some standard guidelines which can be referred, while designing such systems and Digilytics RevEl is certainly emerging as a pioneer in the field evolving up to a benchmark for the systems designed specifically for the mortgage industry.