Get best of the week

Alexey Naydenov. ITooLabs Case study on the Go (Golang) phone platform. Part 1

Alexey Naydenov, CEO of ITooLabs , talks about the development of a telecommunication platform for telecom operators in the Go programming language (Golang). Alexey also shares his experience of deploying and operating the platform in one of the largest Asian telecom operators, who used the platform to provide voice mail services (VoiceMail) and Virtual PBX (Cloud PBX).



Alexey Naydenov (hereinafter - AN): - Hello everyone! My name is Alexey Naydenov. I am the director of ITooLabs. First of all, I would like to answer what I am doing here and how I ended up here.

If you look at the Bitrix24 Marketplace (the “Telephony” section), then the 14 applications and 36 that are there (40%) are us:



More precisely, these are our partner operators, but behind all this is our platform (Platform as a Service ) - what we sell them for a small penny. Actually, I would like to talk about the development of this platform and how we came to Go.

Figures on our platform now:



44 operator partners, including Megafon. Generally speaking, we are very fond of embarking on various adventures, and we have actual access to 100 million subscribers of 44 operators here in Russia. Therefore, if someone has any business ideas, we will always listen to them with pleasure.

  • 5000 user companies.
  • 20,000 subscribers in total. This is all b2b - we only work with companies.
  • 300 calls per minute during the day.
  • 100 million minutes of calls last year (we celebrated this). This is without taking into account the internal negotiations that are on our platform.

How did it start?


How do the right dudes begin to make their platform? We must also take into account that we had a history of “hardcore-interprize” development, and even at the most precise time of the year for enterprise! It was that happy time when you come to the customer and say: "We need a couple more servers." And the customer: “Yes, no question! We have a dozen in the rack. ”

Therefore, we dealt with Oracle, Java, WebSphere, Db2 and all that. Therefore, we took, of course, the best vendor solutions, integrated them and tried to take off with this. We walked on their own. It would be such an internal startup.

It all started in 2009. Since 2006, we have been closely engaged in operator solutions, one way or another. We made several custom virtual PBXs (like what we now have on order): we looked, decided that it was good, and decided to stir up the internal startup.



Took VMWare. Since we walked on our own, I had to immediately abandon the cool vendor Storage. We all know about them: that promises must be divided by 3, and the cost multiplied by 10. Therefore, they did DirDB and so on.

Then it began to grow. To this was added a billing service, because the platform has ceased to cope. Then the MySQL billing server went to Mongo. The result is a working solution that processes all the calls that go there:



But somewhere inside, the same vendor’s product is spinning - the main, nuclear product that we once took. By the end of 2011, we realized for ourselves that this product will, of course, be the main bottleneck for us - we will run into it. We saw a wall in front of which we ran at full gallop, as customers walked, added.
Accordingly, we had to do something. Of course, we conducted quite a lot of research on various products - both open source and vendor ones. I will not dwell on this now - not about that. The most recent fallback we were thinking of is to create your own platform.

In the end, we came to this very option. Why? Because all vendor and open source products were made to solve the problems of 10 years ago. Well, if 10 year old, and some more! For us, the choice became obvious: either we say goodbye to our great idea of ​​an ideal service (for partners, operators and ourselves), or we do something our own.

We decided to do our own thing!

Platform requirements


If you do something for a long time (exploiting someone else's product), then the thought slowly develops in your head: how would I do it myself? Since we are all programmers in the company (except for sellers, there are no programmers), we have developed requirements for a long time, and they were clear:

  1. High development speed. The vendor’s product, which tormented us, didn’t suit in the first place with the fact that everything turned out long and slow. We wanted fast - we had a lot of ideas! We still have a lot of ideas, but then the list of ideas was such that it seemed as if it had been ten years ahead. Now only for a year.
  2. Maximum utilization of multicore iron. This was also important for us, since we saw that there would only be more and more nuclei.
  3. High reliability. What we also cried about.
  4. High fault tolerance.
  5. We wanted to end up with a daily release process. To do this, we needed a choice of language.



Accordingly, from the product requirements that we have made for ourselves, language requirements clearly and logically grow.

  1. If we want support for multi-core systems, then we need support for parallel execution.
  2. – , . , :
    • – , ;
    • , – , ( runtime, ) , .
  3. . , , , .



We didn’t have many options, in fact, if you recall. Firstly, Erlang - we love and know him, it was my personal, personal favorite. Secondly, Java is not even Java, but specifically Scala. Thirdly, a language that at that moment in time we did not know at all - Go. He then just appeared, or rather, for about two years already existed, but has not yet been released.

Defeated Go!

Go story


We made a platform on it. I’ll try to explain why.

A Brief History of Go. Started in 2007, opened in 2009, the first version was released in 2012 (that is, we started working even before the first release). The initiator was Google, who wanted to replace, as I suspect, Java.

The authors are very famous:

  • Ken Thomson, who was behind Unix, invented UTF-8, worked on the Plan 9 system;
  • , UTF-8, Plan 9, Inferno, Limbo Bell Labs;
  • , , Java HotSpot Compiler, , V8 ( Javascript’ Google);
  • 700 , .



Go:


We see that the language is more or less simple, understandable. We have obvious types: in some cases they need to be declared, in some they are not necessary (this means that the types are inferred, one way or another).



It can be seen that it is fashionable to describe structures. It can be seen that we have the concept of a pointer (where there is an asterisk). It can be seen that there is special support for declaring the initialization of arrays and associative arrays.

It’s approximately understandable - you can live. Trying to write Hello, world:



What do we see? This is a C-like syntax; a semicolon is optional. It can be a separator for two lines, but only if these are two constructions that are on the same line.

We see that the brackets in the control structures (in the 14th line) are optional, but curly ones are always required. We see that typing is static. Tim in most cases is displayed. This example is a bit more complicated than the usual Hello, world - just to show that there is a library.

What else is important? The code is organized into packages. And in order to use the package in your own code, you must import it using the import directive - this is also important. Launch - it works. Fine!

We try something more complicated later: Hello, world, but only now it is an http server. What is interesting here?



Firstly, the function acts as a parameter. This means that our function is a “first-class citizen” and with it you can do a lot of interesting things in a functional style. We see further unexpected: the import directive refers directly to the GitHub repository. That's right, it is - moreover, that’s how it should be done.

In Go, the universal identifier for a package is the url of its repository. There is a special Goget utility that goes over all the dependencies, downloads them, installs, compiles and prepares for use if necessary. At the same time, Goget knows about html-meta. Accordingly, you can keep an http-directory in which there will be links to your specific repository (as we, for example, do).

What else do we see? Http and Json in the native library. There is, obviously, an introspection - reflection, which should be used in encoding / json, because we are simply substituting some arbitrary object for it.

We start and see that we have put together useful code in 20 lines, which compiles, starts and gives the current average load of the machine (on the machine on which it is running).
What else is important from what we can immediately see here? It compiles into one static binary (buinary). This binary has no dependencies at all, no libraries! It can be copied to any system, run immediately, and it will work.

We move on.

Go: methods and interfaces


Go has methods. You can declare a method for any custom type. Moreover, this is not necessarily a structure, but it may be some kind of alias. You can declare alias for N32 and write methods for it to do something useful.

And here we go into a stupor for the first time ... It turns out that Go has no classes as such. Those who know Go might say that there is type inclusion, but that’s completely different. The sooner a developer stops thinking of it as inheritance, the better. There are no classes in Go, and no inheritance either.

Question! What did the Google-sponsored company give us in order to reflect the complexity of the world? We were given interfaces!



An interface is such a special type that allows you to write simply methods, method signatures. Further, any type for which these methods exist (are executed) will correspond to this interface. This means that you can simply describe the corresponding function for one type, for another (which corresponds to that type of interface). Next, declare a variable of the type of this interface and assign it any of these objects.

For hardcore fans, I can say that in this variable there will actually be two pointers: one for data, the other for a special descriptor table, which is typical for this particular type, for an interface of this type. That is, the compiler does such descriptor tables at the time of linking.

And there are Go pointers to void, of course. The word interface {} (with two curly braces) is a variable that allows you to point to any object in general.
So far, everything is in order, everything is familiar. No wonder.

Go: goroutines


Now we come to what interested: lightweight processes - goroutines in the Go terminology.



  1. Firstly, they are really lightweight (less than 2 Kb).
  2. Secondly, the costs of creating such goroutines are negligible: they can be created a thousand per second - nothing will happen.
  3. They are served by their own scheduler, which simply transfers control from one goroutine to another.
  4. In this case, control is transferred in the following cases:

    • if the expression go is found (if goroutine starts the next goroutine);
    • if the blocking call Input / Out is turned on;
    • if garbage collection starts;
    • if any operation with channels is started.

That is, whenever a Go program runs on a computer, it determines the number of cores in the system, starts as many threads as needed (how many cores in the system or how many you told her). Accordingly, the scheduler will run these lightweight threads of execution in all these threads of the operating system in each core.

It should be noted that this is the most effective way to utilize iron. In addition to what is shown, we are doing a lot more. We make, for example, DPI systems that allow 40 gigabits to be served in one unit (depending on what is happening on these lines).

There, before Go we used exactly the same scheme for this reason: because it allows you to maintain the processor cache locality, significantly reduce the number of OS context switches (which also takes a lot of time). I repeat: this is the most effective way to utilize iron.

This simple 21-line example is an example that does just an echo-server. Please note that the serve function is extremely simple, it is linear. There are no callbacks, no need to bother and think ... You just read and write!

At the same time, if you read and write, it should actually be blocked - this goroutine is simply put in the queue and is taken by the scheduler when execution becomes possible again. That is, this simple code can operate as an echo server for as many connections as the OS allows on this machine.

To be continued very soon ...


A bit of advertising :)


Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends cloud-based VPS for developers from $ 4.99 , a unique analog of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only we have 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands!Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - from $ 99! Read about How to Build Infrastructure Bldg. class c using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

More articles:


All Articles