Get best of the week

ABP and everything, everything, everything: automatic reserve entry in the data center

In a previous post about PDUs, we said that in some racks ABP is installed - automatic input of a reserve. But in fact, in the data center, the ABP is installed not only in the rack, but also along the entire path of electricity. In different places they solve different problems:

  • in the main switchboards (main switchboards), the ATS switches the load between the input from the city and the backup power from the diesel generator sets (DGU); 
  • in uninterruptible power supplies (UPS), the ATS switches the load from the main input to bypass (more on that below); 
  • in racks, the ABP switches the load from one input to another in case of problems with one of the inputs. 


ATS in the standard power supply circuit of DataLine data centers.

We will talk about which ABPs and where are they used today. 

There are two main types of ATS: ATS (automatic transfer switch) and STS (static transfer switch) . They differ in principles of work and elemental base and are used for different tasks. In short, STS is a smarter ATS. It switches load faster and is more often used for large loads / currents. It is more flexible in configuration, but “with vagaries” to the network: it may refuse to work if 2 inputs are powered from different sources, for example: from a transformer and diesel generator set.  

ABP in main switchboard

 
The main ATS of the data center twenty years ago looked like a complex system of contactors and relays.


ABP sample of the early 2000s.

Now ABP is a compact multifunctional device.



The ABP system in the main switchboard controls input machines and gives commands to start and stop the diesel generator set. With a load of more than 2 MW at the main switchboard level, it is impractical to chase the speed. Even if it switches quickly, time will pass until the DGU starts. In this system, slower ATS are used and delays (settings) are set. It works like this: when the power of the data center from the transformers disappears, the ATS commands the devices: “Transformer, turn off. Now we wait 10 seconds (setting), DGU, turn on, wait another 10 seconds. " 

ABP to UPS  


Using the UPS as an example, let us see how the second type of ATS works - STS or static transfer switch.

In the UPS, the alternating current is converted to direct current on the rectifier. Then, on the inverter, it turns back into alternating current, but with stable parameters. This eliminates interference and improves energy quality. When the main power source is disconnected, the UPS switches to the batteries and feeds the data center while the diesel generator sets are in operation. 

But what if one of the elements fails: a rectifier, an inverter or batteries? In this case, each UPS has a bypass mechanism, or bypass. With it, the device continues to work around the main elements, immediately from the input voltage. Also, bypass is used when you need to turn off the UPS and take it out for repair. 

The STS in the UPS is needed to safely switch to bypass input. In short, the STS monitors the network parameters at the input and output, waits for them to match, and switches in safe conditions. 



ATS in a rack 


So, there are two power inputs to the rack. If your equipment has two power supplies, you safely connect it to different PDUs, and the loss of one input is not a problem for you. And if your server has one power supply? 
In the rack, ABP is used so that the profit from two inputs is not wasted. In case of problems with one of the inputs, the ATS switches the load to the other input.

Disclaimer: If you can, avoid equipment with a single power supply so as not to create a point of failure in the system. Further we will show what are the disadvantages of such a connection scheme. 




The task of the ABP in the rack is to switch the equipment to the working input so quickly that there is no interruption in its work. The speed necessary for this was found empirically: no more than 20 ms. Let's see how it was discovered.

Failures in the operation of server equipment occur due to voltage dips (due to work in substations, the connection of powerful loads or accidents). To illustrate how the equipment withstands different amplitudes and durations of voltage drops, CBEMA (Computer and Business Equipment Manufacturers Association) electrical safety curves have been developed. Now they are known as ITIC (Information Technology Industry Council) curves, their variants are included in the IEEE 446 ANSI standards (this is an analogue of our GOSTs).

Check the schedule. Our task is for the devices to work in the green zone. On the ITIC curve, we see that the equipment is ready to “tolerate” a failure of a maximum of 20 ms. Therefore, we focus on the ABP in the rack working out in 20 ms, or better, even faster.   


Source: meandr.ru .

ATS device . A typical ATS (ATS) in the rack of our data center occupies 1 unit and can withstand a load of 16 A. 

On the display we see from which input the ATS is fed, how many connected devices consume in amperes. A separate button selects, give priority to the first or second input. On the right are the ports for connecting to the ATS: 

  • Ethernet port - connect monitoring;
  • Serial port - go through the laptop and look in the logs what is happening; 
  • USB - insert a USB flash drive and update the firmware. 

Ports are interchangeable: you can perform all these operations if you have access to at least one of them. 



On the back there are plugs for connecting the main and backup inputs and a socket group for connecting IT equipment.



Detailed characteristics of the ATS we look through the web interface. There, the switching sensitivity is configured and the logs are visible. 


Web interface ABP.

Installation and connection of ATS . It is better to set the ABP in height in the middle of the rack. If we do not know in advance the configuration of the rack, then equipment with one power supply can reach the wires from both the bottom and the top.  

But then there are nuances: the depth of the standard rack is much greater than the depth of the ATS. We recommend installing it as close to the cold corridor as possible for two reasons:

  1. . , , . , .




    - , , — .
  2. . 45°. , .  : 

  • , ; 
  • , .

If you install the ABP from the side of the hot corridor and, in addition, pinch it with a pie from the servers, then we get a stove. In the best case, the ABP's brains will burn out and it will lose touch with the outside world, in the worst case, it will randomly switch the load or drop it.


ABP is steaming facing the hot corridor.

There was a case. The engineer on the detour heard uncharacteristic clicks.
In the bowels of the hot corridor, under the heap of servers, an ABP was detected, which constantly switched from the main input to the backup one. 

ABP replaced. Logs showed that for a whole week he switched every second - totaling more than half a million commutations. That's how it was

What other ABPs are in the rack?


Introductory ATS for a rack . In our data center, such an ABP acts as the only source of power distribution in the rack: it works as an ABP + PDU. It occupies several units, can withstand a load of 32 A, is connected by industrial connectors and can power up to 6 kW of equipment. It can be used when it is not possible to mount standard PDUs, and single-unit equipment in a rack does not handle critical loads. 



Rack STS . The STS in a rack is used for equipment that is sensitive to surges. This ATS switches faster than ATS. 
 

This particular STS takes up 6 units and has a slightly “vintage” interface.

Mini ATS . There are such babies, but this is not the case in our data center. This is a mini-ABP for a single server. 


This ATS is connected directly to the server power supply.

How we are looking for the perfect ABP


We test many different ABPs and test how they behave in high temperature environments.

Here's how to mock ABP to check it out: 

  • we connect to it a network quality registrar, a server and several more devices for the load;
  • isolate the rack with plugs or film to achieve high temperature;
  • heat to 50 ° C;
  • alternately disable inputs 20 times;
  • we look for power failures, how the server feels;
  • if the ABP passes the test - heat to 70 ° C.


Thermal imager photo from one of the tests.


The network analyzer captures the voltage over time. On the record we see how long the switching took: at this moment the sine wave was interrupted

By the way, we take the ABP for a test: we will check your device for strength and tell you what happened;) 

ATS in the rack: a hidden threat


The main problem with the ATS in the rack is that it can only switch the load from the main to the backup input, but it does not protect against short circuit or overload. If a short circuit occurs on the power supply, then the circuit breaker will trip a level of protection: on the PDU or in the distribution panel. As a result, one input is turned off, the ATS understands this and switches to the second input. If the short circuit still remains, the second input circuit breaker will trip. As a result, due to a problem on one equipment, the entire rack may be de-energized.

So again I repeat: think a thousand times before installing the ATS in a rack and using equipment with one power supply.

More articles:


All Articles