A loaded web project consumes terabytes of traffic. For large numbers, savings of 10-20% can significantly save money and help not to go beyond quotas.What should I do if traffic dangerously approaches the limits of your hosting tariff or even goes beyond them?In this article, we will analyze the basic techniques that help save traffic on the web server.Squeeze it!
The easiest way to save on traffic is to compress it. This loads the server processor, but allows you to send data to the client faster, reducing their size, so that connections can be closed faster. Mostly Deflate compatible algorithms are used, but there are also exotic ones.Gzip
The most common compression algorithm. Lossless compression, with a good compression ratio (configurable from 1 to 9, default 6) and fast unpacking. Simple and effective, suitable in most cases.
Nginxgzip on;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain application/xml;
ApacheAddOutputFilterByType DEFLATE text/plain
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/xml
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/xml
AddOutputFilterByType DEFLATE application/xhtml+xml
AddOutputFilterByType DEFLATE application/rss+xml
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE application/x-javascript
Zopfli
The modern alternative to gzip compresses 3-8% better, but much more slowly (it decompresses on the client at the same speed). It works on Deflate, therefore it is 100% compatible with zlib, support in browsers is also full.git clone https://code.google.com/p/zopfli/
cd zopfli
make
Nginxgzip_static on;
Brotli
Like Zopfli, developed in the bowels of Google. Able to compress not only in statics, but also on the fly, like gzip. Unlike previous algorithms, it not only searches for repetitions in the text, but also immediately maps its own dictionary, which has a lot of tags and standard code phrases, which is extremely effective for html / css / js compression: if Zopfli gives about 8% compression after gzip , then Brotli is able to throw another 10-15%, while someone else has 23%! But it is supported only in https and is incompatible with zlib / deflate. Caniuse is encouraging:
NginxSupport in the form of a standard module is available only in Plus, regular Nginx must be built with a third-party module ( --add-module = / path / to / ngx_brotli ):git clone https://github.com/google/ngx_brotli.git
git clone https://github.com/bagder/libbrotli.git
./autogen.sh
./configure
make
cd /path/to/nginx
./configure --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --with-threads --with-stream --with-stream_ssl_module --with-stream_geoip_module=dynamic --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_v2_module --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,--as-needed' --add-module=/path/to/ngx_brotli
make
Config:brotli_static on;
In dynamic mode:brotli on;
brotli_comp_level 6;
brotli_types text/plain text/css text/xml application/x-javascript;
ApacheEverything is simpler here, install mod_brotli and configure the module:<IfModule brotli_module>
BrotliCompressionLevel 10
BrotliWindowSize 22
AddOutputFilterByType BROTLI text/html text/plain text/css text/xml
AddOutputFilterByType BROTLI text/css
AddOutputFilterByType BROTLI application/x-javascript application/javascript
AddOutputFilterByType BROTLI application/rss+xml
AddOutputFilterByType BROTLI application/xml
AddOutputFilterByType BROTLI application/json
</IfModule>
Cache it!
You can also offload the channel between the user and the server, minimizing the need to reload resources. If the file was cached, then on the next request, the browser will receive its contents locally.The HTTP headers Cache-control, Expires and Vary allow you to design a very flexible caching policy, although you can put max-age = 2592000 everywhere on the forehead.Nginxlocation ~* ^.+\.(js|css)$ {
expires max;
}
Apache<ifModule mod_headers.c>
<FilesMatch "\.(html|htm)$">
Header set Cache-Control "max-age=43200"
</FilesMatch>
<FilesMatch "\.(js|css|txt)$">
Header set Cache-Control "max-age=604800"
</FilesMatch>
<FilesMatch "\.(flv|swf|ico|gif|jpg|jpeg|png)$">
Header set Cache-Control "max-age=2592000"
</FilesMatch>
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
Header unset Cache-Control
</FilesMatch>
</IfModule>
<ifModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 5 seconds"
ExpiresByType image/x-icon "access plus 2592000 seconds"
ExpiresByType image/jpeg "access plus 2592000 seconds"
ExpiresByType image/png "access plus 2592000 seconds"
ExpiresByType image/gif "access plus 2592000 seconds"
ExpiresByType application/x-shockwave-flash "access plus 2592000 seconds"
ExpiresByType text/css "access plus 604800 seconds"
ExpiresByType text/javascript "access plus 604800 seconds"
ExpiresByType application/javascript "access plus 604800 seconds"
ExpiresByType application/x-javascript "access plus 604800 seconds"
ExpiresByType text/html "access plus 43200 seconds"
ExpiresByType application/xhtml+xml "access plus 600 seconds"
</ifModule>
Distribute it!
A good CDN for a busy site usually costs a lot of money, but for the first time, free is enough. Downloading heavy resources via CDN can reduce traffic by several times! Do not neglect this opportunity, especially when it is shareware. There are a lot of one-time articles with tops of free networks on the Internet, but Cloudflare always comes first.
And they also have free Service Workers, on which you can manually raise your CDN with preference and courtesans. There are few examples, but the tutorial is on the official SW portal.Conclusion
If you are not using at least gzip, welcome to the Internet, here more than 80% of sites work with it. If you do not have enough standard compression and -9, use Brotli with a backup in the form of Zopfli (since Brotley does not yet have 100% coverage). This can save a lot of traffic:- gzip: 50-95% compression depending on the content. Web average 65-80%
- Zopfli: + 3-8% compression relative to gzip on average, but it happens 10%
- Brotli: + 10-15% compression relative to gzip depending on content with rare shots up to 20% and higher
Cache data on the client, this reduces traffic during repeated visits by 99% or lower, depending on the selected caching policy and changes on the site.Use a CDN for content delivery and basic balancing. The distributing server takes the brunt, while the main traffic will be reduced by several times. How much exactly depends on the network, the load and the selected operating mode.All of the above takes place in the shortest possible time and does not require a complex restructuring of the server architecture, which gives you time for its competent design. Compress, cache, distribute, and monitor your costs so you don’t get a lot of money.