DL /NLP Models and Frameworks: Current trends in Finance

As a promising field of Artificial Intelligence (AI), Deep Learning (DL) models and applications have revolutionized the financial and banking domains. Still, many practitioners and academics agree on the fact that a more comprehensive insight and state-of-the-art review that could capture the relationships between typical DL models and the financial arena is needed.

In this article, I will briefly address the recent trends in models applied in finance, what approaches are used or recommended by the expert community and what features are to take into consideration for DL/ NLP models.

In general terms, many of the traditional forecasting, machine learning and deep learning models are always evolving and providing new alternatives. Some notable DL techniques such as LSTM, CNN, Markov Models and Bayesian Stats are in constant evolution in perfecting a framework which is aimed to ascertain the optimal model in a specific condition. It is crucial, then, to identify the optimal DL model for various application scenarios. Another interesting approach where the community is advancing and developing strategies refers to Reinforcement Learning models for portfolio balancing and selection of financial instruments.

Meaningful Models: Comprehensive frameworks

In close-up, there is a strong tendency to study models that provide more information with descriptive rather than a predictive value. Explainable Models are a good example which allows analysts and investors to understand what is happening in the market, where to invest, which industries are better positioned, what are the market drivers, beyond having a model that tells you if A’Company shares are going up or down tomorrow.

Researchers and financial analysts should know the feasibilities of particular DL models toward a specified financial domain. Nevertheless, when doing so, they usually face difficulties due to the lack of connections between core financial domains and numerous DL models.

Beyond NLP: BERT

Natural Language Processing (NLP), is always evolving providing new tools and approaches. As we all know, NLP draws on machine learning and human-generated linguistic rules, to fill the gap between human communication and machine understanding.

BERT (Bidirectional Encoder Representations from Transformers), for instance, was a milestone towards democratisation of NLP models allowing a wide range of practical applications using pre-trained language models by fine-tuning them for specific tasks such as question answering, name entity recognition, classification, and also to produce state-of-the-art predictions with your own data. Research labs and organizations started to work on different projects that outperformed BERT on multiple NLP tasks.

One of the most interesting developments was FinBERT, which is a BERT-based language model with a deeper understanding of financial language and fine-tuned it for sentiment classification. The advantage of this approach is that you don’t need a huge dataset for fine-tuning, because the model learns about language during the original language model training. Nevertheless, as with most deep learning models, it is not very easy to intuit on failure modes of FinBERT. Of course, it is easier to differentiate between positive and negative. But it might be more challenging to decide whether a statement indicates a positive outlook or merely an objective observation.

There are some other developments that came after BERT: RoBERTa, DocBERT, DistillBERT, AlBERT, BETO (Spanish BERT), and the latest, exBERT.

“Readable” Variables

One of the key features that any (and I would stress out here) financial DL models require is that the input variables should be interpretable. Let’s consider some of them.

• The quality of data: When inputting the data, researchers should clarify effective variables and noise. Several financial features are likely to be created and added into the model. Which data? Corporate action event information, macro-economic data, market drivers, alt-data from trusted sources, what moves markets, among others that might be relevant for the purpose of each model.
• The number of input factors depends on the employment of the DL structure and the specific environment of the tasks.
• Selection on structures of DL models is directly related on problem domains and particular use cases in finance.
• Overfitting/underfitting is clearly unfavorable. A generated model performs perfectly well in one case but usually cannot replicate good performance with the same model and identical conditions. Can overfitting and underfitting be avoided? Some researchers agree that a good practice to mitigate overfitting is cross-validation. In the case of underfitting some techniques include: getting more training data, increasing the complexity of the model, the training time, just to mention a few.
• The sustainability of the model: in other words, how to justify the robustness of the DL models is one of the many problems to be solved. Moreover, the “survival” of DL model in dynamic environments must also be considered.
• The use of a trendy model: in financial markets the effectiveness for trading is subject to the popularity of the model. If traders in the same market use an identical model with limited information and adopt the same strategies, it is translated into loss of money.
  

What is the future direction for DL/NLP in finance?

NLP is still facing lot of challenges. The rising overall appetite for applied DL/NLP in areas that have promising potentials from an academic/industrial research perspective such as Discourse and Pragmatics, Linguistic Theories, Cognitive Modeling, Psycholinguistics, and Multimodality will keep increasing every year. With the advent of Quantum Computing, hybrid models will be the key outperforming DL methods. Frameworks and tools which provide built-in customizable libraries adaptable to the specific needs of the industry and which are available today due to significant interest shown by open source communities around the world will continue to grow. Multidisciplinarity is acquiring more relevance than ever: the figure of the MetaQuant with a holistic view is a shift, a step in the right direction.

In the end, NLP/DL real life applications and models take much more than a “good guess” or intuition.