Can ChatGPT with Explainable AI interpret explain black box models to non-financial professionals? If the answer is yes, the following question arises: Is it necessary to maintain our protective financial regulatory regime if these robo-advisors can effectively translate complicated financial knowledge to the public? Under the current regulatory framework, only accredited investors, defined as those persons “whose financial sophistication and ability to sustain the risk of loss of investment or ability to fend for themselves render the protections of the Securities Act’s registration process unnecessary.” This posed a significant barrier for non-accredited investors, who were often excluded from higher-yielding investments.
Thus, our working paper investigates whether ChatGPT is capable of providing non-accredited investors at different education levels with the pioneering use of Explainable AI (XAI) with a conversational AI model such as ChatGPT in finance. This could potentially democratize financial knowledge – to explain the predictions made by a nonlinear black box machine learning model in finance, thereby mitigating information asymmetries in the investment process and enabling non-accredited investors to “fend for themselves.”
Simple Model
The first test case showcases the ability of ChatGPT to explain financial definitions and metrics. With its vast training data, ChatGPT has the potential to demystify financial concepts and help make informed investment decisions. The impact of tone and target audience on language generated by ChatGPT is also being examined, based on the “mom’s test” principle that effective communication should use simple and understandable language. The study aims to determine if changing the tone and target audience leads to better-suited language for different audience.
For instance, one can prompt the reference to the paper and have GPT explain what illiquidity means to the target audience.
Prompt: Factor: ill, Illiquidity Amihud, 2002, JFM
Prompt: Explain to my mom.
ChatGPT: “Illiquidity” is a measure of how easy or difficult it is to buy or sell a stock. If a stock is considered illiquid, it means that there aren’t a lot of buyers or sellers, so it can be harder to find someone to trade with. This can make the price of the stock more volatile, and also make it more expensive to buy or sell. Amihud 2002 published this concept in the Journal of Financial Markets.
Complicated Model
Explainable AI is a rapidly growing area of research that focuses on creating AI systems that are transparent and can be understood by humans. This is particularly important in high-stakes domains such as finance, where the accountability of AI systems is crucial. In the context of investment, XAI has the potential to solve a significant problem known as the factor zoo problem. This refers to the difficulty in determining the underlying factors that drive the returns of hedge funds and other complex investment strategies.
Lundberg and Lee introduced a ground-breaking approach to XAI by using SHapley Additive exPlanations (SHAP) values. SHAP values provide a comprehensive and uniform method of attributing the predictions of machine learning models to their input features. Rooted in the principles of cooperative game theory, SHAP values ensure a fair distribution of value among a group of players. In the context of investment, SHAP values can be used to attribute the returns of a hedge fund or other investment strategy to its underlying factors. By combining XAI and NLP, investors can gain a more holistic view of their portfolio or stock-level factor attribution. This can help to identify any potential risks or opportunities that might be obscured by a more traditional, black box approach.
Figure 1
A waterfall plot generated by SHAP package by MechaniX Limited to explain the features importance of AI model for Norwegian Cruise Line Holdings.
The study employed the SHAP methodology to gain insight into the underlying factors contributing to the AI model’s prediction. The SHAP methodology generated a waterfall plot, where the X-axis displays the log-odds unit or the SHAP value, representing the contribution of each feature to the prediction. Nonetheless, such a waterfall plot can be difficult for individuals without a background in machine learning, explainable AI, or quantitative analysis to fully comprehend.
To make the results of the SHAP explanation more accessible to a wider audience, the output was transformed into ChatGPT prompts using a technique that involved automated conversion. The information from the waterfall plot was transformed into a “prompt” through a prompt template that can be filled in with the relevant information. For more complex templates, more advanced techniques like meta-language can also be used to ensure effective information transfer.
In the following, we demonstrate how ChatGPT explains the factor attributions to “my mom,” additional versions that imitates Barron’s, Financial Times, and the South China Morning Post can be found in the paper.
ChatGPT: The report is talking about a stock called Norwegian Cruise Line Holdings Ltd and why a model thinks it might do well in the next 3 months compared to other stocks. The model looks at different things about the stock, like how much money it makes and how big it is, to make this prediction. Out of all the things the model looks at, it thinks the most important one is how much money the stock makes compared to how much it costs. The next most important things are the size of the stock and how easy it is to buy and sell it. All these things combined make the model think this stock has a good chance of doing well in the next few months.
Discussion
Regarding the simple model, the quality of the output generated by ChatGPT was acceptable. It was able to effectively transform financial knowledge for non-financial professionals, providing explanations of concepts such as alpha and beta to non-finance professionals and even high school students.
However, we found that when the target audience is not specified, ChatGPT tends to provide responses that are more technical in nature and may include terms and concepts that may not be easily understood by non-finance professionals. This highlights the importance of specifying the target audience when utilizing ChatGPT as a tool for disseminating financial information.
For the complicated model, we combined XAI methods like SHAP and ChatGPT to generate a more accessible and understandable explanation, which can be presented in a format like that of a financial publication. By extracting the relevant information from the plot and inputting it into a “prompt” for ChatGPT, we were able to generate a more understandable explanation that could be presented in a manner like that of a financial news outlet such as Barron’s, Financial Times, and South China Morning Post. We also tested the extreme case of generating an explanation for my “mother” without using any professional terms and the results are quite impressive.
However, limitations in the utilization of ChatGPT for explaining model predictions were also noted. One such limitation is related to the bid-ask spread factor. ChatGPT has demonstrated an inadequate understanding of this factor and has even erroneously concluded that a lower bid-ask spread is a more favourable predictor of future stock returns. We believe that further research is needed to enhance the prompt engineering aspect of the model to overcome these obstacles. One promising avenue for improvement is the incorporation of more background knowledge for each factor.
The result of our research has substantial significance for non-accredited investors who possess limited financial literacy. In contrast to mutual funds, hedge funds are largely unregulated as unregistered private investment pools. They are governed by their partnership agreement and make use of exemptions within the legal framework to exempt themselves from several regulations that apply to registered investment vehicles. These exemptions include regulations pertaining to the composition of their portfolios and the nature of their investment strategies. Given hedge funds are subject to significantly weaker investor protection regulations, it is reasonable to implement measures aimed at limiting share ownership to accredited credited investors. However, the decision is based on an implied assumption that accredited investors “accredited investors” are sophisticated enough to “fend for themselves” in making investment decisions in highly complex unregistered financial securities, so that the information asymmetry between hedge funds and investors can be disregarded.
Dr. Thomas Yue and David Au Chi Chu are the Co-Founders of MechaniX Limited
Xavier Au Chi Chung is the Data Scientist of MechaniX Limited
Kwan Yuen IU is a Pupil Barrister at Albert Luk’s Chambers
This post is adapted from their paper, “Democratizing financial knowledge with ChatGPT by OpenAI: Unleashing the Power of Technology,” available on SSRN.
