As we approach the end of 2024, it’s a perfect time to share some updates since my last blog, which you can read here. While I’ll revisit some of the same topics, I’ll focus on different aspects this time, including:
- Recent updates and insights from our Data Quality Scanner Disputes Module
- More information about the CFPB’s inspection of furnishing files
- Additional thoughts on generative AI based on our AI Agent Assist development efforts
Further Insights from the Data Quality Scanner Disputes Module
The trends I discussed in my previous blog still hold true as we analyze more data from our clients. What’s new, and something that I’m very proud of, is that we can now see the results from the work of the early adopters of the Disputes module.
We have observed rapid improvements among disputes agents due to updated training and personalized feedback based on the module’s outputs. This has significantly increased the percentage of data issues being addressed and reduced the creation of new inconsistencies.
Additionally, we have seen clients experience an unexpected boost in productivity. By providing disputes agents with access to historical furnishing and disputes data, along with insights from our rulesets, agents are completing reasonable investigations more quickly while taking a more comprehensive approach to the data.
These new insights are promising and heighten my excitement about launching our Agent Assist tool in 2025. More on that next time!
CFPB Evaluation of Furnishing Files
In my last blog, I mentioned that the CFPB seemed to possess more advanced tools for inspecting Metro 2® files. Over the past few months, we have gained detailed insights that could benefit furnishers and the CFPB.
The issues raised by the CFPB indicate an effort to thoroughly review many months or even years of data, leading to the identification of several valid data issues in the furnishing files. However, these issues also highlight the significant problem of false positives in their tools. I can relate to this challenge, as we faced similar problems when developing our ruleset over a decade ago.
At some level, false positives are unavoidable due to limitations within the Metro 2® data. While this data is comprehensive, it cannot explain all the peculiarities that may be detected. Fortunately, our refinements over the years have significantly reduced this issue, enabling us to quickly identify the root causes of the false positives raised by the CFPB.
These experiences continue to create value for our clients, as the outputs from the Data Quality Scanner Furnishing Module allow them to identify and address issues before an examination. Our outputs and training support a deep, nuanced understanding of the data, empowering clients to engage in productive, data-driven discussions.
We will explore these findings further in a case study scheduled for release in 2025, so stay tuned!
Generative AI: Our Hands-On Experience
In my previous blog post, I introduced our efforts with Generative AI. This time, I want to delve deeper into our initiatives, the challenges we face, the potential we see, and how these relate to what I observed from other organizations.
Background on Our Agent Assist Tool
Need for the Tool
The need for our Agent Assist tool stems from insights gathered from our Disputes Module. Our data reveals that a significant volume of inconsistencies exists in ACDV requests. Additionally, we have noticed a wide variance in agents’ rates of correcting inconsistencies and the frequency of creating new ones.
This is not surprising, as our ruleset contains over 390 detailed rules developed over a decade of continuous improvement, drawing from internal efforts and client feedback. No human can retain all this information and efficiently evaluate all relevant data.
While procedural and training updates can have a positive impact, there is still a strong need for a real-time decision-support tool to assist disputes agents in comprehensively and efficiently addressing the tradeline data they review.
Solution
These needs prompted the creation of our Agent Assist tool. Its foundation is our Furnishing and Disputes modules, which include our extensive rulesets, routines for handling formatting inconsistencies, and routines for comparing Metro 2®, ACDV, and AUD data types. Our data infrastructure allows easy access to years of such data.
While this foundation offers a wealth of information for agents to process, it also presents an opportunity for Generative AI.
Since we are building this within the AWS ecosystem, while the challenges are universal, some solutions will be specific to the tools offered by AWS.
How Generative AI Helps
Our goal is to enable agents to conduct faster and more comprehensive investigations with the help of generative AI. This will involve utilizing insights from the aforementioned foundation and the borrower’s claimed issues, which include not just dispute codes but also free-form text. We will draw on data insights from our rulesets and the history of data changes related to furnishing and disputes over time.
Generative AI will help synthesize these insights, align them with detailed policies and procedures to inform required next steps, suggest appropriate data responses, guide agents to relevant information sources for their investigations, and summarize the investigation efforts at the conclusion.
Next Steps
The concept sounds promising, right? However, what are the realities? Our efforts have led to the development of prototypes that show significant promise, yet we face considerable challenges and have much to accomplish. Fortunately, some recent product announcements from AWS re:Invent appear to be very beneficial in addressing these challenges, which I address in a separate blog that you can read here.
Conclusion
Overall, I’m very excited as we head into 2025. We have been doing a good combination of investing in making updates to our infrastructure and working with the newest tools, so I’m feeling great about having the foundations in place for very valuable product launches and updates next year.
Thank you to all our colleagues, clients, and partners for making this all possible!
