Having introduced concepts in artificial intelligence and different types of machine learning (ML) models in our previous post, we will now discuss specific applications of machine learning in investment management, the benefits of using these models, and the challenges inherent in their use. (Chart 1)
A. Applications of Machine Learning in Investment Management
1. Identify and analyze new forms of data
It has become increasingly difficult to generate alpha using traditional investment strategies over the past decades. One of the reasons given for the lack of alpha generated by these strategies, is that they rely on the same financial data and investment factors in their construction. Realizing this, some investment managers have begun using machine learning algorithms to identify, analyze and incorporate alternative data sources into their investment strategies. Natural Language Processing (NLP) can be used to extract meaningful insights from news articles, social media, earnings calls etc. For example, NLP algorithms have been used to enhance the quantity and quality of data available to Environmental, Social and Governance (ESG) investors. In this case, NLP algorithms can extract unique information from public documents or earnings calls that companies have.1 This type of linguistic information can be particularly important for ESG investing considering the limited amount of ESG related information most companies currently disclose in their public filings. ESG rating organizations and other data accumulators such as TruValue Labs are using NLP, data scraping and other techniques to incorporate more timely company specific ESG related news and announcements, particularly relating to issues and controversies, to construct ESG metrics.2 Investors can use NLP to dive deeper into third-party social media platforms, such as Yelp and Indeed websites, to obtain more unbiased, real-time, and actionable insights on companies. ML algorithms can also be applied to conduct sentiment analysis and gain insights into the current environment facing a company or the real take that management teams have on their companies’ future prospects.3
2. Identify unique patterns and insights in models
Traditional fundamental factors from categories such as Value, Quality, and Momentum, and the linear relationships between them have been the basis for quantitative investment models for several decades. However, there is a high possibility for alpha decay as a result of crowded trades derived from using these factors and the relationships between them. Therefore, if investment managers wish to add alpha, it is important that they test alternative, more advanced factor research and model construction techniques. With that in mind, investment managers are now using ML algorithms such as Generalized Additive Models (GAM), Decision Tree models and Deep Learning to capture non-linear relationships between factors to improve the predictive ability of their models.
Machine Learning methods are now broadly used in factor research and to develop better models for the purposes of stock selection, asset allocation, and risk calculation. ML models can dynamically learn changing relationships between factors and predictors and handle multicollinearity among them. These algorithms can also take macroeconomic variables into account to detect market regime shifts.4 For example, multifactor models like Hidden Markov Models (HMM) classify market regimes using sophisticated time-series ML algorithms.5
3. Improve the accuracy and efficiency of trading
ML algorithms are particularly effective in optimizing trading execution. Traditionally, traders manually collected statistics to predict liquidity patterns and trading costs that were then used for trading decisions. ML algorithms can capture the microstructure of the market in a much quicker and more accurate way as well as efficiently process massive amounts of data for trade pattern analysis. More importantly, using scenario analysis, these techniques allow traders to evaluate the risks related to various trading activities. ML techniques can be used to build algorithmic trading models that reduce errors caused by human judgment and improve trading efficiency.
4. Other applications
Aside from their direct applications in investment activities, ML techniques can be used to analyze client behavior to improve support services and resolve queries. By analyzing current market trends and web activity responses to past advertising, sales and marketing departments can produce more effective strategies to attract new clients and increase the retention of current clients. ML can also increase operational efficiency by streamlining process automation for repetitive tasks, thus reducing costs for a business.
B. Challenges of Machine Learning Techniques
Incorporating ML techniques into their factor research and model building processes is viewed by investment managers as an opportunity for them to outperform competitors and improve returns. However, the use of ML techniques in an investment strategy does not guarantee strong future performance. It is important to highlight some of the challenges and limitations involved in integrating these algorithms into the investment management process.
1. Data quality
While ML algorithms are capable of analyzing much larger amounts of data than traditional models, there are data challenges when applying advanced techniques to investment strategies. The nature of financial data is mostly non-stationary and messy.6 Unlike collecting user data for customized ads on Instagram, stock market data is dynamic and volatile from one period to the next. The market has changed significantly in the past 20 years, due in part to the increase in popularity of passive and active quantitative strategies versus traditional fundamental investment strategies. Thus, simply feeding time-series data into a model may not yield ideal results.7 ML models require clean and well-processed data to generate reliable predictions. The labor intensive and time-consuming data processing step is a particularly important aspect of building ML strategies. In addition, the experience and expertise of the quantitative investment team is crucial in this stage for making judgments on how to process the data.
2. Overfitting and data mining bias
One of the issues highlighted with using ML algorithms in investment management is that while they may show strong historical results using training data, they may not have strong predictive ability using new data. This problem is known as overfitting. Machine Learning models have bias-variance trade-off. Bias refers to the difference between prediction of the model and the actual value the model tries to predict. A model with high bias always leads to high error on both training and test data.8 Variance refers to the variability or sensitivity of the prediction i.e., how the model can generalize the predictors using new data. Thus, more complex models usually come with higher variance while simple models result in higher bias given the difficulty in accurately estimating results through the assumption of a simple relationship.
Overfitting occurs when a model fits too well during back tests and captures the noise. Specifically, overfitting occurs when adopting over-complicated models, where the models often have low bias but high variance. As the models sometimes extract noise instead of true signals, it is frequently referred to as data-mining bias.9 Thus, it could be challenging to build a robust model for various investment strategies.
Overfitting can be addressed in several different ways. First, in the data collection step, it is crucial to obtain good quality financial data. Second, it is important to build a rigorous framework in the back testing process, such as cross-validation or applying regularization to penalize the complexity of the models. Lastly, adopting feature engineering and ensemble methods can produce stabler and cleaner outputs. To avoid data mining and obtain true insights from ML models, it is important to keep in mind that the value of ML applications lies in the investment philosophies and theories quantitative investors want to express instead of the algorithms themselves.10 In other words, these advanced techniques should explain true signals and not the noise.
3. Computing power/infrastructure costs
Another challenge for ML application is that it may take long to both back test the models and deploy them into production.11 Despite computing power increasing roughly according to Moore’s Law since the 1970s, more advanced ML algorithms still require a lot of computation and storage capability. The infrastructure requirements like GPU and high-intensity cores are expensive but essential for both back testing and production purpose. The data and infrastructure required can make it challenging for smaller investment firms to develop ML investment strategies. However, these challenges can be overcome by leveraging open-source resources, third-party vendors, and external technology services such as Amazon Web Services (AWS).
Machine Learning has become increasingly important in the asset management industry over the past decade. Managers who embrace these advanced technologies will have compelling opportunities to outperform in the long-term. BlackRock, the world’s largest asset manager, has established a new Artificial Intelligence (AI) lab in Palo Alto, California, to focus on the potential uses of AI in the asset management industry. Numerai, a new type of AI-powered hedge fund, has gathered tens of thousands of data scientists to create ML algorithms to drive open-source trades.
The key components of successful ML applications are good quality of data, robust models, infrastructure, and experienced teams. Asset management firms that use more innovative data, find unique alpha, and incorporate machine learning techniques to enhance their investment processes will be most likely to excel and adapt in an ever-evolving industry.
References:
- “How Can Ai Help Esg Investing?” Accelerating Progress, www.spglobal.com/en/research-insights/articles/how-can-ai-help-esg-investing.
- Aroomoogan, Kumesh. “Council Post: How Financial Organizations Can Use Ai to APPLY Esg Standards.” Forbes, Forbes Magazine, 30 Mar. 2021,
- forbes.com/sites/forbesfinancecouncil/2021/03/30/how-financial-organizations-can-use-ai-to-apply-esg-standards/.
- “How Financial Institutions Can Deal with Unstructured Data Overload.” InsideBIGDATA, 16 Mar. 2021, insidebigdata.com/2021/03/15/how-financial-institutions-can-deal-with-unstructured-data-overload/.
- “Machine Learning in Quant Investing: Revolution or Evolution?” Acadian Asset Management, www.acadian-asset.com/viewpoints/machine-learning-in-quant-investing-revolution-or-evolution.
- Jha, Osho. “When to ‘Buy the Dip’.” Medium, Medium, 24 Apr. 2019, medium.com/@oshojha/when-to-buy-the-dip-e2e128d737a7.
- “Challenges in Applying Machine Learning to Finance.” CFA UK, 10 Feb. 2020, www.cfauk.org/pi-listing/challenges-in-applying-machine-learning-to-finance#gsc.tab=0.
- “The Right Way to Use Machine Learning.” Essentia Analytics, 16 Aug. 2019, www.essentia-analytics.com/machine-learning-for-asset-managers/.
- Singh, Seema. “Understanding the Bias-Variance Tradeoff.” Medium, Towards Data Science, 9 Oct. 2018, towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229.
- Harris, Michael. “Impact of Artificial Intelligence and Machine Learning on Trading and Investing.” Medium, Towards Data Science, 2 June 2021, towardsdatascience.com/impact-of-artificial-intelligence-and-machine-learning-on-trading-and-investing-7175ef2ad64e.
- Prado, Marcos Lopez de. Advances in Financial Machine Learning. Wiley, 2018.
- Shah, Adarsh. “Challenges Deploying Machine Learning Models to Production.” Medium, Towards Data Science, 14 Oct. 2020, towardsdatascience.com/challenges-deploying-machine-learning-models-to-production-ded3f9009cb3.
This report is neither an offer to sell nor a solicitation to invest in any product offered by Xponance® and should not be considered as investment advice. This report was prepared for clients and prospective clients of Xponance® and is intended to be used solely by such clients and prospects for educational and illustrative purposes. The information contained herein is proprietary to Xponance® and may not be duplicated or used for any purpose other than the educational purpose for which it has been provided. Any unauthorized use, duplication or disclosure of this report is strictly prohibited.
This report is based on information believed to be correct, but is subject to revision. Although the information provided herein has been obtained from sources which Xponance® believes to be reliable, Xponance® does not guarantee its accuracy, and such information may be incomplete or condensed. Additional information is available from Xponance® upon request. All performance and other projections are historical and do not guarantee future performance. No assurance can be given that any particular investment objective or strategy will be achieved at a given time and actual investment results may vary over any given time.