👼🏼 🌦️ 👶🏿 Alexey Naydenov. ITooLabs Case study on the Go (Golang) phone platform. Part 2 🗨️ 😰 🖍️

Alexey Naydenov, CEO of ITooLabs , talks about the development of a telecommunication platform for telecom operators in the Go programming language (Golang). Alexey also shares his experience of deploying and operating the platform in one of the largest Asian telecom operators, who used the platform to provide voice mail services (VoiceMail) and Virtual PBX (Cloud PBX).

Alexey Naydenov. ITooLabs Case study on the Go (Golang) phone platform. Part 1

Go: channels

It is clear that when you have parallel execution, you need synchronization tools. Go provides all the synchronization tools that are needed: mutex's, shared memory ... But channels are a regular mechanism for it.

A channel can be considered as a unidirectional typed queue of n elements (the default is 1). The channel itself can be a parameter, a member of the structure. Moreover, a channel can be transmitted on a channel - at the same time this channel will have the type “channel of channels of this or that type”.

What does this example do in 20 lines? This is an http call counter. He simply counts where he starts the http server and shows how many times this server has been accessed. How is this usually done? Usually you make a variable, wrap it with mutex, and increment it. Or you make an atomic variable, increment atomically.

Go is different. A developer, having tried Go, very quickly begins to write this way. In Go, you run the goroutine function, which is called count (8th line) and go count (17th); make a variable; and then just increment the local variable - the next number is sent to the channel. Those of you who remembered Atmos Script Generators, of course, will be right, but in Go channels allow much wider use.

Go: select

Finally, there is the Select statement, which allows you to select from multiple channels at once. This is a typical template in Go, how to complete a goroutine:

you transmit two channels. Requests go to one channel, and signals go to the other that it is time to round off.

Go: C API (CGO)

For us, as for the developers of the telephony platform, it was important that Go has a C API, which allows us to write some pieces in C directly in the Go code, which will then be normally compiled, normally linked and then work normally.

That is, you can access all the C libraries that you have on the operating system. However, they will also be linked statically.

Go: tools

Go's toolkit is another reason why they love it. Everything is done with one command - (go). There is a go build command that builds, there is a go test that runs special test functions described in a particular package. There is a go run command that starts and builds right away. There is a command that updates dependencies (go get).

There are those that format the source: go fmt - we have it, for example, on push'ah. There is no more debate about where, in what way and what formatting style we use - tab or space: there is a go fmt command, there is a certain style that it formats, and there are no disputes. Don't like something? Run go fmt - it will be as it should be.

The go doc command allows you to read documentation and display it in html. There is a go tool pprof for profiling, a parsing generator (go tool yacc) and much more. In short, the Go toolkit is highly developed for such a young language. Many of the ideas in it were not that innovative, but very interesting, which later migrated to other systems - for example, the same Goget and the ability to specify the immediate repository as the name of the package.

Go: IDE

Generally, I prefer Emacs. If there are lovers, then we can fight. But for development, I prefer the "adult" IDE:

I tried them all, including VIM. I can say that at the moment the coolest IDE is the usual one, some kind of IDEA Edition with a plugin for Go. The best! Recommend. If anyone will use it, one must immediately begin and not suffer.

Go: libraries

Libraries - this was another very, very pleasant surprise, because literally on anyone we found something right away. Even when we needed a Javascript interpreter, we thought that we had to write ourselves - we just found it, it already was.

To date, more than 35,000 different repositories with at least one star have been registered in Git. I can repeat my thought: there are our libraries, and foreign ones, and there are a lot of companies that open some kind of internal libraries. In general, I can’t recall the case when I needed something, and I would not immediately find it on GitHub (in the case of Go).

Go: operation

Exploitation is a dream, especially for those who, for example, have worked with Java.

Static assembly. No run-time dependencies!
. , : , C++ , , . Garbage Collector’! , , .
– . .
, . .

Go:

Community - it also bribed. The Go community is just wonderful and it is flying a rising star. Unfortunately, the slide shows little and is not visible: on the left is the repository on GitHub, on the right are the Stack Overflow tags:

If you look at Go, then it is in the company of languages that are five times older than it!

Go vs Node.js

Battle! Compare with Node.js, for example. You can actually compare with anything. But why with Node?

Suppose Google planned that Go would supplant Java, compete with Java. Not really. Go turned out to be the main competitor for Node.js, Python, for interpreted languages. Why? Because on Go, due to type inference, due to many small details (such as the go run command), you get the feeling of an interpreted language, that is, an easy language on which you can make layouts, quickly build and prototype something. But at the same time, it really works: it really compiles, starts up and works quickly.

Here we have two simple servers - both there and there http:

The difference.

Compiled in Go, interpreted in Node.js. Launched - it immediately works (well, there too).
Go has a lot of threads (as many processors as there are in the machine). In Node.js, as you know, there is one thread (if you want a lot of threads, please start cluster).
Now we begin to complicate the logic - we complicate it in the Handler. In Go, you do not complicate anything, but simply take and do as you wrote a linear program that is blocked (send a request to the database - calculate - send a response to the client).
And questions immediately arise in Node.js ... What will you use now - callbacks, promise, Q, async.js or newfangled yield?

When you have a model like Erlang or Go, life becomes much easier.

Go: flaws?

Yes dofiga!

No parameterized polymorphism, patterns, generics. There are only special operations for working with array, arrays and associative arrays. All the rest is not.
. , .
cut’n’paste. Go , , – . , .
, . , , Haskell, , , . Go – ! , , - .
– , . – .
. panic/recover, .
, ( ).
There is no place for imagination - too easy!

Why is Go still?

Maybe because of the fast assembly or because of the deployment of one binary? Due to competitiveness or efficient iron utilization? Maybe due to the fact that he is developing rapidly or thanks to a cool community?

For a long time I tried to answer this question myself. In the end, I realized: no, none of these reasons! Everything is much simpler: Go works.

Go is an extremely practical language for engineers. For engineers who are engaged in the development of web services and distributed systems - to write interfaces on it, a lot of things ...

Is it possible to write "Bitrix" on it? I dont know. Most likely not, because there are too many interfaces. But to write on it a platform on which Bitrix will work faster (for example, a bus for sending messages) is what I need, I'm sure.

Who is using Go?

Who has already managed to jump into this car?

Google is understandable: he has many projects that use Go and have switched to it a long time ago.
Docker.
InfluxData If someone has not tried it, I recommend it. Also a rising star.
Twitter
Dropbox
Kodding.
Even Baidu. Go, by the way, is very popular in China.
A lot others.

There is a list of companies that have one way or another noted, doing some projects on Go: https://github.com/golang/go/wiki/GoUsers

What have we done on Go? ITooLabs Centrex

What did we write for the year?

A dynamic cluster that allows you to evenly distribute the load between all its nodes (by hash ring).
Uses Protobuffer for internal communication.
Uses SIP / HTTP / WebSocket, anything for external communication.
It has a media processor (let’s leave it aside, it is in C ++).
JavaScript, node ( ). , Go: , . Lua, , , – JavaScript Go ( ).
? , , , - , , , – . - / , 2 .
, – DevOps-friendly. , , , - , , , . , – .
1 .

As I said, around 2012, we realized that the wall was ahead of us. It was pretty scary. We didn’t make a big mistake in our forecasts (which is another two years and there will be cranks), because at the beginning of 2014 we really were very bad ...

And we quickly migrated a significant part of the generating functionality to our new platform, in particular, records, all kinds of external trunks - that generates a lot of load on the CPU, and with which the old vendor platform already categorically could not cope.

Case. Voice mail for 70 million subscribers

Now all this is already working successfully and almost the entire record that goes on the platform goes through our system on Go.

If you work for a long time for a reputation, at some point the reputation begins to work for me. And for a long time different people have come to us from operators with offers to do something.

At the end of last year, “something” turned out to be a voice mail system for 70 million subscribers for the Indonesian operator Indosat. We thought and made on the basis of our platform. About three weeks passed from the moment the decision was made (more precisely, from the moment the iron was placed because it took a long time) to the moment it was deployed.
16 , , , 7 – Application Service SIB- Go. – ++. 16 1800 50 000 . 6,5 ( – BHCA).
280 – , . 1 BHCA 36 , , 2 . , – 100 .
, , 600 . Ceph .
We checked and tested: the system fully withstands a system failure of up to two servers. That is, they can simply be turned off. This actually happened with us, because the Indonesian people who installed the cables made a little mistake.
Finally, we normally do Rolling updates.

Could we do it in some other language in such a short time? Let me remind you, from 2012 to 2015 - 3 years for a product that holds such a load and works for millions of Indonesia subscribers.

I don’t know, but I think we could do it on Erlang. Maybe we could do it in Java. Since I, besides the main work, also give lectures on real-time systems and parallel programming at Tula University, for me one of the final results, the reasons why we decided to use Go, was a very simple experiment.

I gave students a lab and made sure that the Go learning curve is steeper than any language I have ever come across (in this particular area). Students began to write more or less decent code, because indecent Go simply does not allow writing. Go gives boring code, and only boring code! But indecent does not work out - very hard.

I can boast that on December 31, 2015, just a month after deployment, 160 thousand messages were recorded on the system in one day, which subscribers send to each other.

Questions

Question from the audience (hereinafter - B): - Thank you very much for the report! Very interesting cases. Tell me, please, how does Go look now for error handling? For example, when I write in Node and I need to do 5 asynchronous operations in a row, my code looks in 5 lines: wait, wait, wait, wait, wait.

AN: - As I said, there are no exceptions. In any case, in the sense in which you can use them for control, management.

Q: - How do you manage errors?

AN: - ifs! I repeat that Golang allows you to write quite boring code and provokes cut'n'paste.

Q: - And how many of these ifs will be on 5 asynchronous operations, each of which can explode?

AN:- From my point of view, they will be synchronous (from the point of view of this goroutine). 5 ifs.

Q: - There is almost a doubling of the code.

AN: - Probably, if I paid the developers of the code, I would be upset. But since I pay for the fact that they make good, working code, I am glad of this in principle.

Q: - Is it possible to link some kind of C-library dynamically at all?

AN: - Yes.

Q: - That is, this functionality remains?

AN:- What I showed and talked about CGO - there it is linked with a static library, if there is one, and if not, a dynamic one is connected. Moreover, in Go 1.5 a dynamic linking appeared in principle, even now it allows you to create dynamic libraries. But we are so used to it, we have felt such a mechanism so much that we no longer feel the need. Of course, you can connect dynamic C libraries, C ++ - no.

Q: - And the binary as a result is not too gigantic?

AN: - No. The binary of our entire system that handles all these calls (along with the JavaScript interpreter inside) is just 6 megabytes. This is a binary that basically fits into the second cache.

AT:- I myself often heard comparisons: Go is often compared precisely with interpreted languages, and few people have the task of writing a full-fledged communication platform. What simpler practical cases are there?

AN: - A simpler case is excellent. For Megafon, we wrote pre-billing - something like a bus that collects platforms from various components inside (billing, “Multifon” - a large operator always has a lot of them), this whole thing integrates within itself, adds to the database, calculates what is needed and sends events to different places.

In the same Indonesia, a simple gateway (albeit simple, but distributed), which receives events from our system and passes on to the system that sends its SSD ... These are the kind of tasks - of course, all this is written very, very quickly, which attracts.

A bit of advertising :)

Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends, cloud VPS for developers from $ 4.99 , a unique analog of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only we have 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands!Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - from $ 99! Read about How to Build Infrastructure Bldg. class c using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

Alexey Naydenov. ITooLabs Case study on the Go (Golang) phone platform. Part 2