Get best of the week

Hosting and dedicated servers: answer questions. Part 2. Why such an expensive Internet in the data center?

In this series of articles, we want to consider the issues that people have when working with hosting providers and dedicated servers in particular. We conducted most of the discussions in English-language forums, trying to help users primarily with advice, rather than self-promotion, giving the most detailed and impartial answer, because our experience in the field has been over 14 years, hundreds of successful solutions and thousands of satisfied customers. Nevertheless, our answers should not be perceived as the only correct answers of the first instance, they may well contain inaccuracies and even errors, no one is perfect. We will be grateful if you add or correct them in the comments.

Hosting and dedicated servers: answer questions. Part 1

Hello hosters. I apologize if the question is stupid. But why is the Internet in the DC more expensive than the city provider? At home, 1 GB costs 500-600r. Hosters have over 20k. Why is that?

- Posted on 06/01/2020 at 17:21 - Although probably the price is formed by demand, there are a lot of home customers and the price is probably lower.

In fact, the question is very complex and extensive, and not at all silly. And it makes us think a little about what the Internet is in principle. Namely, that this is a multitude of separate devices, often combined into larger networks, which in turn are combined together into one worldwide network thanks to the TCP / IP protocol and wired or wireless communication lines that belong to different network participants. These devices and networks communicate with each other according to the BGP rules established by these network participants, and it is they who have the ability to determine which routes and at what speed access is possible between two separate segments.

Therefore, “1 gigabit” is a very conditional value. 1 gigabit where? Can you, using your home Internet, transfer data at such a speed to someone in Alaska? Or maybe this 1 gigabit is only possible within your city? Where is this gigabit provided and is it provided all the time? Or is the magic prefix “do” indicated in the tariff description? And in those moments that it is not provided, is it possible to get a much lower speed? Or you are far from constantly using your channel, and not 100% all the time, and due to this, the same physical or virtual channel can be used by hundreds of other subscribers and at the same time not interfere with each other, since everyone downloads and transfers data at different times and with different intensities?

Just think, if you download the 1 Gbit / s channel constantly, then over a month you can download up to 324,000 GB of data. How big is the hard drive or SSD in your computer? How much do you download per month? 300, 500, 1000 GB? And most importantly, how, when and where?

Let's deal with this issue in more detail. At the dawn of the evolution of the Internet, people, both in Ukraine and the Russian Federation, exchanged data mainly among themselves within small territories. Since access to foreign networks was quite expensive.

At the beginning, each of the smaller providers connected to the same, larger provider, and it was he who determined who and how, from the smaller providers - his clients, on what conditions and at what speeds would be communicated using the communication lines provided to him. Due to this, subscribers from different regions of one settlement could transfer data to each other and communicate relatively inexpensively, since the information passed through communication lines exclusively within that settlement. While communicating with subscribers from a neighboring city, it was not uncommon that traffic went through another country or even several countries due to the lack of a direct high-speed communication line between regional providers, due to the remoteness and high cost of building such a connection.But each of them was connected with a larger participant in the Internet, the first-level transit provider Tier I, owning physical lines of communication passing through several countries and connecting large providers from individual regions. The Internet was pretty fragmented.

What is the Internet in the 2000s? This is a lot of local networks in individual cities and regions, consisting of individual users who sought to connect to the same network for the purpose of exchanging files (videos, games and other information) almost for free, at a speed that is significantly higher than the modem one, while access outside these networks to what is called the World Wide Web was extremely expensive and slow.

In fact, it was then that a hierarchy of providers that ensured network connectivity emerged - Tier I, Tier II, and Tier III.

Tier III is a third-level provider that provides access to the Internet network of individual clients — you and I, home Internet users and legal entities. He always buys channels from larger providers, levels of Tier II and Tier I. That is, he pays for traffic to everyone, well, almost everyone :)

Tier II is a regional provider that has highways covering a certain region (city, country, or even several countries ), to which several Tier III providers are connected, which require both access to the global network and interconnectedness. He receives a fee for traffic that passes through his networks to Tier III providers (making money on local traffic on his networks), while paying for traffic from the global network, which he purchases from Tier I transit providers.

Tier I - does not pay anyone for traffic; both Tier III and Tier II providers can connect to it.



To make it clearer, let’s take an example. For example, there are Cogent and Telia, these are large I-level backbone providers that have points of presence in many cities and countries and their own communication lines between them. Smaller providers from these regions want to connect with them, in order to more efficiently exchange traffic between themselves and have a connection to the "global network", since each Tier II operator is necessarily connected to at least one Tier I operator, and on in fact, two (to ensure fault tolerance), and is not wasted on expensive trunk lines if there is not a lot of traffic in a certain direction. After all, why pull a direct channel from Kiev to Moscow, if you are a small provider, and your subscribers use only 1 Gbit / s in this direction, at that time,as a trunk channel, it can have a capacity of tens or even hundreds of thousands of gigabits and laying your own fiber is quite expensive.



At the same time, there are Tier I providers with a relatively small network. For example, if a provider provides fiber from London to New York, then it can also be a first-tier provider, if it is rented from it or part of the capacity (virtual channel) tier II and III providers located in the USA and / or Europe as in order to increase network bandwidth in a certain direction, and to increase fault tolerance due to an alternative route.

Nevertheless, since the laying of submarine highways is very expensive, Tier I backbone providers often have tens of thousands of kilometers of network, since few people have only one small trunk and do not want to buy access to other networks.

Topic article:Messages in depth: the amazing story of the underwater Internet

In order to provide global access to all networks, first-level operators enter into agreements for the exchange of traffic and use exchange points for this, as a result they do not pay anyone, but take money from everyone. Sometimes, in order to expand the sphere of influence and get as much traffic as possible, peer-to-peer wars are organized between large players, when subscribers of networks using Tier I of one supplier lose connection with subscribers of Tier I networks of another supplier. As a result, the Internet network is fragmented. But we will not be very distracted from the topic, since the question of peer-to-peer wars, perhaps, should be discussed separately.

Most of all in this story Tier III is “unlucky” for providers, since they pay everyone for traffic. As a result, back in the 2000s, providers began to lay networks to each other in order to save on traffic and exchange traffic between themselves for free or for a small fee. As a rule, the one who lays the fiber and uses it for free, the other side is forced to pay “rent”, as the first side invested funds and paid for the maintenance of the fiber-optic network.

For example, provider A consumes a lot of traffic from subscribers from provider B. At provider A, this is incoming traffic, while for provider B this traffic is outgoing. What do you think, which of the providers is more beneficial peering?

When considering this question, you need to remember where the sites are? On the servers. What about the servers? In data centers. Data centers generate a huge amount of traffic, since they download information from servers ... They have the main traffic - outgoing. At the same time as with providers that provide access to the Internet to us, on the contrary - incoming. The result, 1 Gbit / s capacity is sold immediately to two different types of subscribers, in order to use it effectively - Data Centers and Internet access providers. This is what Tier I operators do. Earn at all.

Often peering (direct communication between networks, bypassing other networks) will be beneficial primarily for that party whose subscribers generate a lot of traffic in a certain direction. And it is this side that will look for a solution to build it, thereby saving money, and selling a free channel to the data center or another provider that provides access to the Internet (depending on who you are, the data center or ISP).

The more traffic you generate, the greater the number of different other providers will want to bother with you. Peering conditions can be different - for example, you can allocate a port on your router for free if the provider builds the fiber for you at your own expense, or it can be 50/50 or another ratio, or even you can charge a port fee from the provider, if this peering is not so much profitable for you and you can gladly give traffic to the Tier I provider, providing it with a larger volume and a discount.

But very often, in order to simplify the scheme and share expenses more fairly, the so-called traffic exchange points are created. When everyone pays their contribution to the exchange point for participation (in fact, the exchange point is a Tier I provider) or there is some rule that you must generate so much traffic, otherwise a fee will be charged. At the same time, each of the participants connects to the point at his own expense, builds his own highway or rents fiber to the point.

For example, in 2004, the cost of participation at the UA-IX traffic exchange point (Ukrainian traffic exchange point located in Kiev) was $ 2530 for 1 Gbit / s for the first month, and $ 330 for the next month. We can see these conditions thanks to the Internet archive. At the same time, each of the providers provided the fiber from itself to the point independently, or rented it. And it was still much more profitable than paying for the same traffic to the same Ukrtelecom (second-level backbone provider), which sold gigabit on the Internet for $ 20,000 / month for 8 Mbps without taking into account traffic and provided rather expensive IP transit services between regions within the country.

More complete information can be found in the article: 20 years of the evolution of the Internet in Ukraine, and how do you remember the network 20 or 10 years ago?

At the same time, home networks sold access to the Ukrainian segment of the Internet to private users for only $ 20 per month without restrictions on the speed of 100 Mbit / s, when they themselves paid at least $ 33 + fiber maintenance or rent to the point of exchange. Alas, I can’t predict the oversell coefficient - but these are dozens of subscribers for sure, if not hundreds. It is interesting that today 100 gigabits in UA-IX can be purchased for less than $ 400 per month and you no longer need to get permission from other participants of the traffic exchange point, as it was before. Of course, this is only the port in the router; you still need to route your fibers to the exchange point.

We turn to the topic of oversell more tightly. How many subscribers can I sell a 1 Gbit / s channel so that everyone has high speed? Do not believe it, but at the beginning of 2009, one of the home network providers managed to sell high-speed Internet access at speeds up to 100 Mbit / s (in the foreign segment the speed was lower, but also pretty decent) at the rate of 15,000 customers per 1 Gbps foreign traffic. To my reasonable question, how is this possible - the answer was given that most of them are on social networks (the video was not so popular at that time) and in general everything is placed, because the traffic is not big + caching servers are used that help not to download one twice the same content from external networks. Plus, the fact that subscribers are not severely limited by the channel,allows you to make the periods of traffic consumption shorter in time and faster to free the channel for requests of other subscribers. It’s important to just maintain balance.

The total 1 Gbit / s offered to TS is most likely shared between several hundred customers, if not a thousand. Since even assuming that everyone will pump an average of 150-300 GB per month, the bandwidth will probably be enough, even taking into account the fact that the channel is less busy at night than during the day. Moreover, it is not a fact that the traffic is not local. Indeed, if you download the same torrent, and you are a subscriber of a large provider, it is highly likely that you will download from a subscriber from your own network or region, the traffic to which is not so expensive. Youtube and other popular traffic generators, in order to reduce costs and be closer to their audience, are building data centers and placing servers as close as possible, as a result of which the cost of traffic is also very low.

Plus there are channels of different quality. You may well be provided with gigabit in the Russian Federation, but abroad, it is unlikely that such a speed will be provided for one stream. Moreover, we must not forget that the bandwidth depends not only on the channel on your part, but also on the resource from which you download.

So why is traffic, when placed in data centers, more expensive? The fact is that servers statistically give much more traffic than home Internet users consume incoming traffic, since a single server can serve hundreds of thousands of requests and even millions. Among other things, data centers reserve channels to a much greater extent than Internet access providers for home users, since Sevrer should be online 24/7. And it also affects the price. For example, at a new site in Amsterdam, where we currently place subscribers, we use Tier I trunk lines from three “top-level” backbone providers at once - Level 3, GTT, NTT.

The cost of connectivity can be 3,000 euros for every 10 Gb / s and even more. Moreover, if you use only 2 trunk operators, and you need to provide 2N redundancy for the server, it turns out that half of the connectivity cannot be used. It turns out that for every 10 Gbit / s, you need to buy 20, spending 6,000 euros, and keep half the capacity in case of inadmissibility of the first 10, which increases the cost by 2 times. However, if there are 3 trunk operators, then it is sufficient to reserve only a third of the band. Since you can simultaneously lose a maximum of one third.

As a rule, channels that are so expensive provide incredibly high speed per stream, and these providers have a fairly widespread network around the world, which allows you to provide the minimum possible number of hop between you and the audience, and hence the minimum ping. Nevertheless, data centers always have the opportunity to save money by creating peer-to-peer connections to the necessary networks and exchange points. This does not always positively affect connectivity and therefore the quality of traffic can be very different.

So Cogent is considered one of the cheapest backbone providers. We sell a 10 gigabit channel for 1N individual customers at incredibly low prices, cheaper than buying it ourselves, since not everyone uses the bandwidth completely. Plus, there is no talk about 1 or 10 gigabits per stream (a point-to-point 1 or 10 gigabit channel with guaranteed QoS can cost incredible money), it is understood that the client can generate 10 gigabits in total for 1000 or 2000 subscribers, when the speed per stream is not it will exceed 10 Mbit / s, but this is enough to comfortably watch video streaming in HD quality from most regions of the Russian Federation and Ukraine from servers located in the Netherlands. Those who need a large guaranteed flow rate and QoS are required to buy even more expensive channels.

The same applies to servers with a traffic limit of 100 TB and a 1 Gbit / s channel (this is essentially a third or even half a gigabit). We give this traffic not for 20k, but not for 500-600 rubles. Having found the opportunity to provide a minimum price, since not everyone uses their limit. Do you know how many statistics show that 50 servers with a 1 Gbps channel and a limit of 100 TB generate traffic? 2-3 Gbit / s. That is why we have the opportunity to sell the channel to subscribers for less than it costs us. But of course, not in such a good ratio as for users of home networks, since hundreds, thousands and even thousands of people use servers simultaneously.

Among other things, we, as a hosting provider, have additional costs for routing this traffic, since a higher level of consumption (total) forces us to have more expensive equipment, and, as a result, a more expensive network. Just think about it! In 2014, we generated more traffic than all of Belarus:

“ua-hosting.company” or how to become a hosting provider from scratch and generate more traffic than all of Belarus

image

A bit of advertising :)


Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends, cloud VPS for developers from $ 4.99 , a unique analog of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only we have 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands!Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - from $ 99! Read about How to Build Infrastructure Bldg. class c using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

More articles:


All Articles