⏩ 👩🏾‍🤝‍👩🏽 👐🏻 How we at Sportmaster chose a caching system. Part 1 👐🏼 📳 👨🏼‍🏭

Hello! My name is Alexey Pyankov, I am a developer at Sportmaster. In this post I talked about how work on the Sportmaster website began in 2012, what initiatives we managed to push through, and vice versa, which rake we collected.

Today I want to share the thoughts that follow another story - choosing a caching system for the java backend in the admin panel of the site. This plot is of particular importance to me - although the story took place only 2 months, but these 60 days we worked for 12-16 hours and without a day off. I had never thought and imagined that you can work so much.

Therefore, I break the text into 2 parts, so as not to download in full. On the contrary, the first part will be very easy - preparation, introduction, some considerations of what caching is. If you are already an experienced developer or have worked with caches - from the technical side, there is most likely nothing new in this article. But for a junior, a small such review can tell which way to look, if he finds himself at such a crossroads.

When the new version of the Sportmaster website was launched in production, the data came in a way, to put it mildly, not very convenient. The basis was the tables prepared for the previous version of the site (Bitrix), which had to be tightened up in ETL, brought to a new look and enriched with different little things from a dozen more systems. In order for a new picture or product description to appear on the site, you had to wait until the next day - update only at night, 1 time per day.

At first, there were so many worries from the first weeks of going into production that such inconveniences for content managers were a trifle. But, as soon as everything settled down, the development of the project continued - a few months later, at the beginning of 2015, we began to actively develop the admin panel. In 2015 and 2016, everything is going well, we will release it regularly, the admin area covers the greater part of the data preparation, and we are preparing for the fact that soon the most important and difficult thing will be entrusted to our team - the product line (complete preparation and maintenance of data for all products). But in the summer of 2017, just before the launch of the product line, the project will be in a very difficult situation - precisely because of caching problems. I want to talk about this episode in the second part of this two-part publication.

But in this post I’ll start from afar, like some thoughts - ideas about caching, which to scroll before a big project in advance would be a good step.

When a cache task arises

The caching task does not just appear. We are developers, we write a software product and we want it to be in demand. If the product is in demand and successful, users arrive. And more come and more. And so there are a lot of users and then the product becomes highly loaded.

At the first stages, we don’t think about optimization and code performance. The main thing is functionality, quickly roll out the pilot and test hypotheses. And if the load grows, we pump iron. We increase it by a factor of two, three, five, let it be 10 times. Somewhere here - finances will no longer allow. And how many times will the number of users grow? It will be not only 2-5-10, but if successful, it will be from 100-1000 and up to 100 thousand times. That is, sooner or later, but will have to do the optimization.

Let's say some part of the code (let's call this part a function) works indecently for a long time, and we want to reduce the execution time. Function - it can be access to the database, it can be the execution of some complex logic - the main thing is that it takes a long time. How much can you reduce lead time? In the limit - can be reduced to zero, no further. And how can you reduce the runtime to zero? Answer: generally exclude execution. Instead, immediately return the result. And how do you know the result? Answer: either calculate, or look somewhere. Calculate is a long time. And to peep is, for example, to remember the result that the function produced last time when called with the same parameters.

That is, the implementation of the function does not matter to us. It is enough to know what parameters the result depends on. Then, if the parameter values are presented in the form of an object that can be used as a key in some storage, then we can save the result of the calculation and read it the next time. If these write-read results are faster than performing the function, we have a profit in speed. The profit value can reach 100, 1000, and 100 thousand times (10 ^ 5 is more likely an exception, but in the case of a decent lag base it is quite possible).

Key Caching Requirements

The first thing that can become a requirement for a caching system is a fast read speed and, to a slightly lesser extent, a write speed. This is so, but only until we roll out the system in production.

Let's play such a case.

Suppose we provided the current load with iron and now we are gradually introducing caching. Users are growing a bit, the load is growing - we add caches a little, we fasten it here and there. This has been going on for some time, and now the heavy functions are practically not called up - all the main burden falls on the cache. The number of users during this time has grown N times.

And if the initial supply of iron could be 2-5 times, then with the help of the cache we could tighten the productivity every 10 or, in a good case, 100 times, in some places, possibly 1000. That is, we process on the same iron 100 times more requests. Great, deserve a gingerbread!

But now, at one point, by accident, the system crashed and the cache crashed. Nothing special - after all, the cache was selected on demand "high speed reading and writing, the rest does not matter."

Regarding the starting load, the iron reserve was 2-5 times, and the load during this time grew 10-100 times. With the help of the cache, we eliminated calls for heavy functions and therefore everything flew. And now, without a cache - how many times does our system sag? What will happen to us? The system will fall.

Even if our cache did not crash, but only cleared for a while, it will need to be warmed up, and this will take some time. And at this time - the main burden will fall on the functional.

Conclusion: high-load projects in the prod require from the caching system not only high speed of reading and writing, but also data safety and resistance to failures.

Flour of choice

In the project with the admin panel, the choice went like this: first they put Hazelcast, because were already familiar with this product from the experience of the main site. But, here this choice was unsuccessful - for our load profile, Hazelcast is working not only slowly, but terribly slowly. And at that time, we already signed up for the terms of withdrawal to the prod.

Spoiler: how exactly did the circumstances arise that we missed such a plop and got an acute and tense situation - I will tell in the second part - and how we turned out and how we got out. But now - I’ll just say that it was a lot of stress, and “think - somehow I don’t think, shake the bottle.” “Shaking the bottle” is also a spoiler, about this a little further.

What have we done:

We make a list of all the systems prompted by google and StackOverflow. A little over 30
, . , - — , .
, , , . , – , .
17- , . « », .

But this is an option when you need to choose a system that "crawls in speed" in pre-prepared tests. And if there are no such tests yet and would like to choose faster?

We will simulate such an option (it is difficult to imagine that the middle + developer lives in a vacuum, and at the time of selection he had not yet decided which product to try in the first place - therefore, further discussion is more likely a theoretician / philosophy / about a junior).

Having determined the requirements, we will begin to choose a solution from the box. Why reinvent the wheel: we will go and take a ready-made caching system.

If you are just starting and you will google, then plus or minus the order, but in general, the guidelines will be like that. First of all, you stumble upon Redis, it is everywhere heard. Then you will find out that there is EhCache as the oldest and most proven system. Then it will be written about Tarantool - a domestic development in which there is a unique aspect of the solution. And also Ignite, because it is now on the rise in popularity and enjoys the support of SberTech. In the end, there’s Hazelcast, because in the enterprise world it often flashes in the midst of large companies.

This list does not end there; there are dozens of systems. And we screw only one. Take the selected 5 systems for the "beauty contest" and conduct a selection. Who will be the winner?

Redis

We read what they write on the official website.
Redis is an open source project. It offers in-memory data storage, the ability to save on-disk, auto-partition into partitions, high availability and recovery from network breaks.

It seems that everything is fine, you can take it and screw it on - all he needs is what he does. But let's just look for the sake of interest in the other candidates.

Ehcache

EhCache - “the most widely used cache for Java” (translation of the slogan from the official site). Also opensource. And here we understand that Redis is not under java, but general, and to interact with it, you need a wrapper. And EhCache will be more convenient. What else does the system promise? Reliability, provenness, full functionality. Well, and she is the most common. And caches terabytes of data.

Redis is forgotten, I am ready to choose EhCache.

But a sense of patriotism pushes me to see what makes Tarantool good.

Tarantool

Tarantool - Meets the designation "Real-time Data Integration Platform." It sounds very difficult, so we read the page in detail and find a loud statement: "Caches 100% of the data in RAM." This should raise questions - after all, there can be much more data than memory. The decryption is that here it is implied that Tarantool does not run serialization to write data to a disk from memory. Instead, it uses the low-level features of the system when the memory simply maps onto a file system with very good I / O performance. In general, they did it somehow wonderful and cool.

Let's look at the implementation: Mail.ru corporate highway, Avito, Beeline, Megafon, Alfa-Bank, Gazprom ...

If there were still any doubts about Tarantool, then the introduction of the Mastercard implementation would kill me. I take Tarantool.

But anyway…

Ignite

... there is Ignite , declared as "an in-memory computing platform ... in-memory speeds on petabytes of data." There are also many advantages: distributed in-memory cache, the fastest key-value storage and cache, horizontal scaling, high availability, strict integrity. In general, it turns out that the fastest is Ignite.

Implementations: Sberbank, American Airlines, Yahoo! Japan And then I still find out that Ignite is not just implemented in Sberbank, but the SberTech team sends its people to the Ignite team to finalize the product. This is completely captivating and I am ready to take Ignite.

It is completely incomprehensible why, I look at the fifth point.

Hazelcast

I go to the Hazelcast website , read it. And it turns out that the fastest solution for distributed caching is Hazelcast. He is orders of magnitude faster than all other solutions and in general he is a leader in the field of in-memory data grid. Against this background, take something else - do not respect yourself. It also uses redundant data storage for continuous operation of the cluster without data loss.

Everything, I am ready to take Hazelcast.

Comparison

But if you look, then all five candidates are so painted that each of them is the best. How to choose? We can see which one is the most popular, look for comparisons, and the headache will pass.

We find such a review , choose our 5 systems.

Here they are sorted: at the top of Redis, in second place is Hazelcast, Tarantool and Ignite are gaining popularity, EhCache has been and remains.

But let's look at the calculation method : links to websites, general interest in the system, job offers - great! That is, when my system falls, I will say: “No, it is reliable! Here are a lot of job offers ... ". Such a simple comparison will not work.

All of these systems are not just caching systems. They still have a lot of functionality, including when not the data is transferred to the client for processing, but rather: the code that needs to be executed on the data moves to the server, it is executed there, and the result is returned. And as a separate system for caching, they are not so often considered.

Well, do not give up, we find a direct comparison of systems. Take the top two options - Redis and Hazelcast. We are interested in speed, we can compare them by this parameter.

Hz vs Redis

We find such a comparison :

Blue is Redis, red is Hazelcast. Hazelcast wins everywhere, and the rationale is given: it is multi-threaded, highly optimized, each thread works with its own partition, so there are no locks. And Redis is single-threaded, it does not take a gain from modern multi-core CPUs. Hazelcast has asynchronous I / O, Redis-Jedis has blocking sockets. In the end, Hazelcast uses a binary protocol, while Redis is text-oriented, meaning it is inefficient.

Just in case, we turn to another source of comparison. What will he show us?

Redis vs Hz

Another comparison :

Here, on the contrary, red is Redis. That is, Redis outperforms Hazelcast in performance. In the first comparison, Hazelcast won, in the second - Redis. Here, they explained very precisely why Hazelcast won in the previous comparison.

It turns out that the result of the first was actually rigged: Redis was taken in the base box, and Hazelcast was imprisoned for a test case. Then it turns out: firstly, no one can be trusted, and secondly, when we nevertheless choose a system, we still need to configure it correctly. These settings include dozens, almost hundreds of parameters.

Shaking a bottle

And the whole process that we have just done, I can explain with such a metaphor, "Shake the bottle." That is, now you can not program, now the main thing is to be able to read stackoverflow. And in my team there is a person, a professional, who works just like that at critical moments.

What is he doing? He sees a broken thing, sees a stack trace, takes some of his words (which ones are his expertise in the program), searches in Google, finds stackoverflow among the answers. Without reading, without thinking, among the answers to the question - he chooses something most similar to the sentence “do this and that” (choosing such an answer is his talent, as it’s not always the answer that collected more likes), he uses , looks: if something has changed, then fine. If it has not changed, we are rolling back. And we repeat start-check-search. And in such an intuitive way, he achieves that after some time the code works. He does not know why, he does not know what he has done, he cannot explain. But! This infection works. And the "fire extinguished." Now we understand what we have done. When the program works, it is much easier.And significantly saves time.

This method is very well explained by such an example.

It was once very popular to collect a sailboat in a bottle. At the same time, the sailboat is large and fragile, and the neck of the bottle is very narrow; you cannot push it inside. How to assemble it?

There is such a method, very fast and very effective.

The ship consists of a bunch of little things: sticks, ropes, sails, glue. We put all this in a bottle.
We take the bottle with both hands and start shaking. We are shaking, shaking. And usually - you get complete garbage, of course. But sometimes. Sometimes you get a ship! More precisely, something similar to a ship.

We are showing this to someone: “Serge, see !?”. And indeed, from afar - as if a ship. But then you can’t let this go.

There is another way. The guys are using more advanced ones, such hackers.

Gave such a guy a task, he did everything and left. And you look - it seems to be done. And after a while, when it is necessary to refine the code - there it begins because of it ... It is good that he has already managed to run far away. These are the guys who, using the example of a bottle, will do this: you see, where the bottom is - the glass bends. And it is not entirely clear whether it is transparent or not. Then the “hackers" cut off this bottom, insert the ship there, then paste the bottom again, and as if it were necessary.

From the point of view of setting the problem, everything seems to be correct. But here is an example of ships: why do this ship in general, who needs it at all? It does not carry any functionality. Usually such ships are gifts to very high-ranking people who put it on a shelf above themselves, as some kind of symbol, as a sign. And now, if such a person, the head of a large business or a high-ranking official, how does the flag stand such trash, in which the neck is cut? It would be better if he never knew about it. So, how do these ships ultimately be made that can be presented to an important person?

The only place, key, with which there really is nothing to be done, is the building. And the hull of the ship just passes through the neck. Whereas the ship is going outside the bottle. But it’s not just to assemble a ship, it is a real jewelry craft. Special levers are added to the components, which allow them to be lifted later. For example, the sails are folded, gently drifted in, and then with the help of tweezers it’s very jewelry, for sure, they are pulled and lifted. The result is a work of art that can be presented with a clear conscience and pride.

And if we want the project to be successful, there must be at least one jeweler in the team. Anyone who cares about the quality of the product and takes into account all aspects, without sacrificing a single one even in times of stress, when circumstances require urgent to the detriment of the important. All successful projects that are sustainable, that have stood the test of time, they are built on this principle. They have something very precise and unique, something that uses all the available features. In the example of a ship in a bottle, it is played out that the ship’s hull passes through the neck.

Returning to the task of choosing our caching server, how could this method be applied? I propose such an option from all the systems that there is - do not shake the bottle, do not choose, but see what, in principle, they have something to look for when choosing a system.

Where to look for bottle-neck

Let's try not to shake the bottle, not to sort through everything that is in turn, but let's see what tasks arise, if suddenly, for your task - to design such a system yourself. Of course, we will not assemble the bike, but we will use this scheme to orient ourselves on what points to pay attention to in the product descriptions. We outline such a scheme.

If the system is distributed, then we will have several servers (6). Let's say four (it’s convenient to place in the picture, but, of course, there can be any number of them). If the servers are on different nodes, it means that some code is spinning on them all, which is responsible for ensuring that these nodes form a cluster and, in the event of a break, connect, recognize each other.

Still need a code logic (2), which is actually about caching. Clients interact with this code through some API. Client code (1) can be both within the same JVM, and access it over the network. The logic implemented inside is the decision which objects to leave in the cache, which to throw. We use memory (3) to store the cache, but if necessary, we can also save part of the data on the disk (4).

Let's see in which parts the load will occur. Actually, each arrow and each node will be loaded. Firstly, between client code and api, if it is a network interaction, subsidence can be quite noticeable. Secondly, within the framework of api itself - having overridden with complex logic, we can run into the CPU. And it would be good if logic did not drive memory once again. And there remains interaction with the file system - in the usual version, it is serialized / restored and written / read.

Further interaction with the cluster. Most likely, it will be in the same system, but it can be separately. Here, you also need to consider the data transfer to it, the speed of data serialization and the interaction between the cluster.

Now, on the one hand, we can imagine “what gears will spin” in the cache system when processing requests from our code, and on the other hand, we can estimate what and how many requests our code will generate to this system. This is enough to make a more or less sober choice - to choose a system for our use case.

Hazelcast

Let's see how this decomposition applies to our list. For example, Hazelcast.

In order to put / take data from Hazelcast, the client code accesses (1) the api. Hz allows you to start the server as embedded, and in this case, accessing the api is a method call inside the JVM, you can read it for free.

To work out the logic in (2), Hz relies on a hash from the byte array of the serialized key - that is, key serialization will occur in any case. This is an inevitable overhead for Hz.
Eviction strategies are implemented well, but for special cases - you can connect your own. You don’t have to worry about this part.

Storage (4) can be connected. Fine. Interaction (5) for embedded can be considered instantaneous. The exchange of data between nodes in the cluster (6) - yes, it is. This contributes to resiliency at the cost of speed. The Hz feature of Near-cache allows lowering the price - data received from other nodes of the cluster will be cached.

What can be done in such conditions to increase speed?

For example, to avoid serializing the key in (2), on top of Hazelcast, screw another cache for the hottest data. At Sportmaster, Caffeine was chosen for this purpose.

For twisting at level (6), Hz offers two types of storage: IMap and ReplicatedMap.

It is worth saying how Hazelcast got into the Sportmaster technology stack.

In 2012, when we worked on the very first pilot of the future site, it was Hazelcast that turned out to be the first link that the search engine issued. The acquaintance started “the first time” - we were impressed by the fact that only two hours later, when we screwed the Hz into the system, it worked. And it worked well. Until the end of the day we added some tests, we were glad. And this supply of vigor was enough to overcome those surprises that Hz threw over time. Now the Sportmaster team has no reason to refuse from Hazelcast.

But such arguments as “the first link in the search engine” and “quickly assembled HelloWorld” - this, of course, is an exception and a feature of the moment in which the choice took place. These tests for the selected system begin with the release in the prod, and it is at this stage that you should pay attention when choosing any system, including cache. Actually, in our case, we can say that we chose Hazelcast by chance, but then it turned out that we chose the right one.

For production, it’s much more important: monitoring, processing failures on individual nodes, data replication, cost of scaling. That is, it is worth paying attention to the tasks that will arise just when the system is supported - when the load is ten times higher than planned, when we accidentally fill something in the wrong direction, when you need to roll out a new version of the code, replace the data and do it unnoticed for clients.

For all these requirements, Hazelcast is definitely suitable.

To be continued

But Hazelcast is not a panacea. In 2017, we chose Hazelcast for the cache in the admin panel, simply relying on the good impression of past experience. This played a key role in a very evil joke, which is why we were in a difficult situation and “heroically” got out of it for 60 days. But more about that in the next part.

In the meantime ... Happy New Code!

How we at Sportmaster chose a caching system. Part 1