Primário do projeto de sistema (parte 1 com adições de microsserviços)

Alguns anos atrás, para fechar um de seus itens, eu traduzi essa cartilha. De fato, essa opção se tornou uma tradução oficial. Mas o desenvolvimento desta tradução é dificultado pelo fato de ela ter sido escrita no Google Drive e fechada para edição. Hoje, reservei um tempo para reformatar todo esse texto no formato MD com a ajuda do editor Habr. É um prazer publicá-lo aqui e em breve o entregarei aos colaboradores atuais.

Dicionário

Node — —
Content — — e
Traffic — — /,
Hardware — —
Instance — — . API
Headers — — ( TCP , HTTP )

(Performance) vs (Scalability)

. , , .

• , .
• , , .

• A word on scalability
• Scalability, availability, stability, patterns

(Latency) vs (Throughput)

- .

, .

, .

• Understanding latency vs throughput

(Availability) vs () Consistency

CAP

, :

• — , .
• — , . , .
• — , , (node) .

, . .

CP — (consistency and partition tolerance) — +
, , timeout ( ). . . , -, .

AP — (availability and partition tolerance) — +
, . ( ), .

, . , .

• CAP theorem revisited
• A plain english introduction to CAP theorem
• CAP FAQ
• Eventual consistency

, . . CAP — .

, , . best effort (A best effort approach is taken)

memcached. real-time, VoIP, . , , , .

, , , ( ). .

DNS Email. .

, . , . .

RDBMSes. , .

• Transactions across data centers

: .

(Fail-over)

- (Active-passive)
(heartbeats) . , IP .

(downtime) , . (hot) (cold) .

— shoot the other node in the head (STONITH), . , , .

- master-slave — -.

-
- , , .

, DNS IP . , .

-, — master-master — -.

• .
• , .

Master-slave master-master
.

(Domain name system)

DNS — IP . www.example.com 127.0.0.1.

DNS . , DNS . DNS , , - . DNS .

• NS- (name server) — DNS /.
• MX- (mail exchange) — .
• A- (address) — IP .
• CNAME- (canonical) — (example.com www.example.com) A-.

CloudFlare Route 53 DNS . DNS , :

• 
  — ,
  —
  — A/B

DNS

• DNS , , .
• DNS , DNS, .
• DNS .

• DNS architecture
• Wikipedia
• DNS articles

(Content delivery network)

Content delivery network (CDN) — , . , CDN HTML, CSS, JS, .

CDN :

• CDN

Push CDN

Push CDNs . , CDN () CDN . .

Push CDN . CDN, .

Pull CDN

Pull CDN . , . CDN- .

TTL — time-to-live, . Pull CDN , CDN, , CDN , .

Pull CDN , .

: CDN.

• CDN , .
• .

:

• Globally distributed content delivery
• The differences between push and pull CDNs
• Wikipedia

(Load balancer)

(, ), , .

:

• ;
• ;
• ;

() HAProxy.

:

• SSL — .
  — .
  — X.509
• — . , , .

, - - .

:

• /
• Round robin or weighted round robin
• 4 OSI
• 7 OSI

Layer 4

, . IP , . Network Address Translation (NAT).

Layer 7

, , .

.

4- , , . , .

(Horizontal scaling)

, . ( ). - .

:

• ,
• (SQL, NoSQL) (Redis, Memcached)

:

• , .
• , .
• , .

• NGINX architecture
• HAProxy architecture guide
• Scalability
• Wikipedia
• Layer 4 load balancing
• Layer 7 load balancing
• ELB listener config

(-) — Reverse proxy (web server)

. , .

:

• — , IP , .
• — , .
• SSL — .
• —
• —
• —
  — HTML/CSS/JS
  —
  —
  —

vs

• . , .
• .
• NGINX HAProxy layer 7 .

:

• .
• . .

• Reverse proxy vs load balancer
• NGINX architecture
• HAProxy architecture guide
• Wikipedia

/ (Application layer)

. API, .

.

— Microservices

, , . API . Pinterest, , : , , , , .

«Microservice Architecture» . , : -, , (endpoints) .

«» — . , , . , , . , . , , , .

, — , , , HTTP. - . . .

, , (monolithic style): , . Enterprise : ( HTML javascript-), ( , ) . HTTP , , , HTML , . .

— . , , namespace-. . .

, , , . , , . , , . , .

: . , . .

, . , , Unix. , .

, , , , . , , , . , .

, , . , .

, , . : — , .

, — . , , — , - remote procedure call (RPS).

— . , , , . , , - , . , - , , , , — , .

— . Published Interface. . .

, . , , API (coarser-grained), . , - , .

, . , . , , .

, , UI , . , - - . , , . (Conway's Law) .

« , - ( ) , »
— Melvyn Conway, 1967

(Conway's Law)

. , -, , . - , : user-experience, project management.

,

, — www.comparethemarket.com. - , .

, . , . , . , - . , . .

?

«» , , «». , , . , « » (, ), .. 12 (. .: , ). , .
, , . , (1 12 1 1 ) , . (. .: , ; , , « », ).

,

, , , , . .

, , . , " , ", . , , , , .. .

. — , . , — -.

, , .

(Smart endpoints and dumb pipes)

, . Enterprise Service Bus (ESB). ESB- , , -.

: . , , (decoupled) (cohesive), : Unix- — , . , WS-* BPEL, REST- .

— HTTP API . Ian Robinson: «Be of the web, not behind the web».

, , , (, , Unix). IT-.

— . — RabbitMQ ZeroMQ . — , .

. . «» , . .

. , — . . , .

, , . Node.js ? . C++ real-time ? . , ? .

, -, . .

, , . , , -, , . , -. , git github - , - .

Netflix — , . , , , . , , .

, . Tolerant Reader Consumer-Driven Contracts , . Consumer-Driven . , . : , — YAGNI.

— , . , , 24/7. , , . Netflix — , . 3 — , .

SOA

, , Service Oriented Architecture (SOA), . , .. , SOA. , , , SOA , , , «SOA» , , - ESB, .

, SOA ( ESB, , ), .

, , , . Tolerant Reader — . — — , .

SOA , «SOA», SOA, , , SOA. , , SOA , , .

,

JVM — . , , . , « » .

, DSL- . , .

. , . , enterprise-: «» . «» . , .

, , . Bounded Context Domain-Driven Design (DDD). DDD . , , , .

, . , . , , . : , . Polyglot Persistence. Polyglot Persistence , .

, . , . .

, (temporal coupling), , , . , , , (eventual consistency) .

— , . , . , , .

, , vs

, , , . HTTP ATOM.

, . , IETF, , open-source .

, , .

. AWS , .

, , Continuous Delivery Continuous Integration. , , . .

Continuous Delivery, . , , . , (automated deployment).

, () . , Continuous Delivery — , .

, , — . , , , , .

(operations folk). , , . , , — open source Netflix, , Dropwizard, .

(Design for failure)

, . - . , . , .. . , , user experience. Simian Army Netflix () .

, . , , , .

, , , . , (, ), - (, ). , .

, .. . .

, . , . , , , , , . , , .

, , . , (/) -: , ..

, (downtime). . : , . , www.guardian.co.uk — . Netflix API .

, , () . : , , .

, , . - , , «» ? — , . : , , .

- Guardian — , , . , , API . , . — . . , .

— , , . , , . , , . , , .

(granular) . . (redeploy) , . . , , , . , , . , , .

?

, . , — , enterprise . , .

— Amazon, Netflix, The Guardian, the UK Government Digital Service, realestate.com.au, Forward comparethemarket.com. 2013 , , , , Travis CI. , , , , . ( SOA, , , SOA , , .)

, , — . , , , .

, . , , «» . , , .. . , , , , .

, - . , . . . , , , , . , . . .

, , . , , .

. , , , . , , , . , .

, , , . , . ( , .. .)

, . , . , , , , , .

— Service Discovery

Consul, Etcd, Zookeeper, (, ). Health checks ( ), , HTTP .

: application layer

• .
• .

• Intro to architecting systems for scale
• Crack the system design interview
• Service oriented architecture
• Introduction to Zookeeper
• Here's what you need to know about building microservices

(RDBMS)

SQL
ACID —

• Atomicity — .

  
  

• Consistency — .

  
  

• Isolation — .

  
  

• Durability — , .

  
  

. - , - , , , , SQL .

- — Master-slave replication
, , . , . , .

: master-slave:

• .

- — Master-master replication
. , .

: master-master replication

• , , .
• master-master ( ACID), - .
• , .
• . : , master-slave, master-master.

:

• .
• . , .
• , .
• .

• Scalability, availability, stability, patterns
• Multi-master replication

— Federation

. , . : , , . , . . , .

:

• .
• .
• .

Scaling up to your first 10 million users

// — Sharding

. . , , . .

: sharding

• SQL .
• . , . .
• .

• The coming of the shard
• Shard database architecture
• Consistent hashing

. , . (PostgreSQL, Oracle)
, . .

(1000:1), .

:

• , .
• . .

Denormalization

SQL

SQL .
.

• Benchmark — .
• Profile — , .

• MySQL .
• CHAR VARCHAR .
• CHAR , VARCHAR, , .
• TEXT . TEXT . TEXT .
• INT 2^32 4 .
• DECIMAL .
• BLOBS, .
• NOT NULL .

Use good indices

• (SELECT, GROUP BY, ORDER BY, JOIN) .
• B- , , .
• .
• , .
• , .

• 
  — .
• 
  — .

• Tips for optimizing MySQL queries
• Is there a good reason i see VARCHAR(255) used so often?
• How do null values affect performance?
  - Slow query log

NoSQL

NoSQL - , , . , . NoSQL ACID.

BASE NoSQL. CAP BASE , .

• Basically available — .
• Soft state — .
• Eventual consistency — .

, SQL NoSQL, , NoSQL . .

- — Key-value store

: hash table

Key-value store O(1),. , . - .

, in-memory . , , .

- .

• Key-value database
• Disadvantages of key-value stores
• Redis architecture
• Memcached architecture

/ — Document store

: key-value store

XML, JSON, binary . . API .

, , , . , .

MongoDB CouchDB SQL . DynamoDB - .

.

• Document-oriented database
• MongoDB architecture
• CouchDB architecture
• Elasticsearch architecture

— Wide column store

: map ColumnFamily<RowKey, Columns<ColKey, Value, Timestamp>>

— . ( SQL).

.

• SQL & NoSQL, a brief history
• Bigtable architecture
• HBase architecture
• Cassandra architecture

— Graph database

: graph

, — . , .

, .

• Graph database
• Neo4j
• FlockDB