Neural networks and trading. Practical implementation

The trading fashion is experiencing ups and downs along with the Bitcoin exchange rate. Now this topic is at a low point after incredible growth in 2018. Many during this time managed to get acquainted with crypto-exchanges - they delved into the topic, studied, traded, lost money, and even sometimes earned. As a result, the fashion passed, but the experience remained, albeit negative. The words “long”, “short”, “spread”, “diver” can be heard from those from whom you certainly do not expect this. But not only the “hands” trade attracted attention, there are also trading bots. What do we have in this area, what is the experience of the last 2-3 years talking about?

Unfortunately, there is nothing good in this area either. There are many tools for writing trading bots, but there are no stably profitable strategies. In the best case, a working solution needs to be constantly serviced by a pair of trader-programmer changing settings for the current market. The standard result after connecting a fully autonomous bot to the deposit is to drain the deposit.

The last hope remains - neural networks. This certainly should happen, because the neural network learns as a person and adapts itself to the market. How are you doing in this area? Well ... you guessed it. Everything is bad, there are many conversations, but there are no working solutions. Everything is very bad! A vivid example of this is the MQL5 community application store. Behind the MQL5 brand is a professional trading platform - MetaTrader5 plus a huge community of traders and programmers. So in this store there is a section for solutions based on neural networks, there are many bots, but not a single real one. Adding a bot using a neural network is prohibited by the rules. It's hard to believe, but it is. The ban on connecting an external API excludes the possibility of using neural networks, and the tools built into MQL5 do not work in practice. I discussed this issue with MQL5 tech support,I didn’t receive a clear answer. As a result, there is nothing really working out of the box on the market, the solutions from the “teach yourself” series laid out on the github are not taken into account.

Okay, I’m finishing with the introduction, I’m getting down to business. I managed to train a neural network, there are worthy results in the real market. Further without going into details, I will describe how this was done.

The first and main mistake of everyone who is trying to teach the neural network to trade is the trading thinking about the market. Usually, a trader tries to implement a solution based on neural networks together with a programmer who previously wrote bots. The trading bot algorithm solves the problem of finding the correct entry point to the position and determining the “take” and “stop”. If you have this task in mind when designing a neural network, then nothing will work. You can endlessly sort out ingenious options for input data, apply or not give indicators, try different types of neural networks, substitute crutches as training only in certain areas, or train showing only known patterns. Will not work.

With a neural network, you need to look like a child, look at the world through her eyes and start with simple tasks. The simplest question that can be asked is “where will the price go through X candles, up or down?”. It doesn’t matter how much the price changes and it doesn’t matter that the answer cannot be turned into a trading strategy. We forget about trading, now the main task is to educate the network at least something, just get the right answer.

I had this initial task solved after 100,500 input selections. Used TensoFlow plus Keras, a network of Sequental Dense. Input dataset for 200-300 thousand examples, input vector 250-350 feature. The form of the answer follows from the posed network of the question — the binary classification “up” or “down”. The input was prepared by the bot in MQL5. A bot running through a story forms a regular csv file, each line is a vector. At the end of each vector, the correct answer is 1: 0 if up, 0: 1 if down.

Here are some suggestions for those who try:

1. He studies well within the forecast from 15 minutes to 60 minutes. At a shorter period, the chaos of the price movement grows, at a larger period the external influence increases - news and so on, 15-60 minutes the most “technical” zone.
2. BTCUSD, EURUSD. — .
3. « ? ?». , .
4. , 50% . , .

You need to understand what material we are dealing with. The market is 95% chaos. When we submit training data to the network and indicate the correct answer, we are actually feeding it chaos. It’s like trying to train a neural network to distinguish a cat from a dog, and on training to show it butterflies, stars and zodiac signs, there will be no result. So it is here, but, fortunately, the market gives about 5% of the really working patterns that the network can catch and this will be reflected in its answers. Therefore, pay attention to the analysis of test results.

Using this approach, I got the following result:
in about 2% of the answers, the network guesses the further movement in the ratio of 2 correct answers to 1 incorrect. When testing in the real market, this is exactly what happens, but another problem arises. After all, we have only 2% of the questions answered, ignore the rest. Those. We launch a neuro in the real market on a timeframe of 5 minutes and wait ... at 2% - it’s only every 50th candle will be answered, one answer in 4 hours! And what to do with it? Well, if the answer was “buy / sell”, then 6 transactions a day is normal, and here the abstract “up / down” and then inaccurate, complete disappointment.

As a result, I solved this problem relatively easily - just every 5 minutes you need to interview not one model, but 20-30 models, as a result of answers there will be enough. Models are trained on different input data and, accordingly, are trained on different patterns. In practice, it turns out that the models honk on different candles, activating together only in obvious, predictable places and do not overlap each other.

To summarize, now there is something that can be launched in the real market and have up / down signals with average mining. Already more fun, but the practical sense is still zero.

A few words about the implementation. It worked for me on a bunch of MQL5 plus Keras. A bot launched in MetaTrader5 on each candlestick prepared data for a neural network and passed through a socket to a python script, which in turn polled all models and, when it passed an acceptable threshold, sent a signal to the Telegram channel.

So, the circuit works, but cannot be applied. It was not possible to add signals to some strategy. The main drawback is the discreteness of the answers. The answer is an event that you need to somehow react to - look at the market situation, think whether the network is right or not, etc. On one candle, one model could signal up, and another down, and which one to believe? As a result, the idea was born to abandon the threshold for passing the answer, and to begin to respect each answer of the network, albeit with a low degree of confidence. If you start to average all the answers into a single common opinion and consider this as a network response, then the answer becomes a completely different quality. In this case, the knowledge of all models begins to take shape, and this is a huge amount of joint training.

Whether it was long, short, but after all the alterations, I began to receive uniform responses of the neural network for each candle, expressed as a percentage of the expected working time from -100% to + 100%. The sign reflects the expected up / down direction. It became clear that now there is a sense in every answer. It works! I myself have trading experience and saw how the behavior of the network in front of my eyes became meaningful. Sometimes its logic was understandable, sometimes it wasn’t, but its answers always felt its own, often paradoxical, vision of the market. In addition to this, it turned out that the higher the network confidence, the closer the expected development to the right time and vice versa. Low confidence seemed to say "what will happen in 15 minutes I don’t know, but the general trend is up."

From this place, I realized that trying to formalize all this into buy / sell signals is like hammering nails with a microscope. What was needed was some kind of tool for visualizing the signals of the neural network — a graphic display of the level of “confidence” on each candle. A wide range of MQL5 tools made it possible to put all this into Expert for MetaTrader5. "Expert" through the API receives the responses of the neural network and is engaged only in rendering. Here is an example of his work on BTCUSD M1: The



colored area at the top is the forecast “down”, the area below is the forecast “up”, the thickness is the degree of confidence.

At this stage, the quality of the forecast does not matter, it is important that the neural network demonstrates a quite adequate opinion about the market situation. You can always upgrade the network even more, the main thing is that it works!

As a result, today there is Expert for MetaTrader5 with two types of forecasts - short-term and long-term. Statistics are gradually accumulating, there is feedback from traders. The result obtained inspires further work, now we need to get to the cherished "buy / sell" teams. This can be done by significantly increasing the quality of the forecast. Then I see this development path:

1. It is necessary to prepare a dozen forecasts in the interval between 15 and 60 minutes. Those. start predicting “up / down” for 20, 25, 30, 35 minutes and so on up to 60. Each forecast, I recall, is built from the answers of about 20 models.
2. Having such a volume of information on each minute candle, it is possible and necessary to analyze it with another neural network. The connection between forecasts at different time intervals can be completely nontrivial, therefore, a neural network here will be appropriate.
3. The dataset for this neural network will not be as noisy as that of its younger comrades, therefore, it must be trained not in a banal “up / down”, but in predicting the strength of an asset’s movement, and this is a direct exit to “buy / sell”.

That's all I wanted to tell you today. Work continues, I think there will be more articles.

By the way, this Expert for MetaTrader5 is distributed free of charge, if you want to see the work of a neural network in the real market, please contact.