🌭 🤞 🤹🏿 Modelo de servidor back-end Golang - parte 1 (servidor HTTP) 🙋🏾 ✡️ 🏓

O modelo de servidor Golang abaixo foi preparado para transferir conhecimento dentro de nossa equipe. O principal objetivo do modelo, além do treinamento, é reduzir o tempo para a criação de protótipos de pequenas tarefas de servidor no Go.

O modelo inclui:

Passando parâmetros do servidor pela linha de comando github.com/urfave/cli
Definindo as configurações do servidor através do arquivo de configuração github.com/sasbury/mini
Definir configurações do servidor HTTP TLS
Configurando um roteador registrando HTTP e manipuladores de prof github.com/gorilla/mux
Definindo níveis de log sem parar o servidor github.com/hashicorp/logutils
Configurando o Log de Tráfego HTTP sem Parar o Servidor
Configurando o Log de Erros na Resposta HTTP sem Parar o Servidor
Autenticação básica HTTP
Autenticação do MS AD gopkg.in/korylprince/go-ad-auth.v2
Token da Web JSON github.com/dgrijalva/jwt-go
Iniciando o servidor com a expectativa de um retorno ao canal de erro
Usando o contexto para parar corretamente o servidor e serviços relacionados
Configuração personalizada de tratamento de erros github.com/pkg/errors
Configurações de log personalizadas
Construir com versão, data de construção e confirmar

Link para o repositório do projeto .

HTTP (. HTTP Golang):

POST /echo — request HTTP body response
POST /signin — JSON Web Token Cookie
POST /refresh — JSON Web Token Cookie
POST /httplog — HTTP
POST /httperrlog — HTTP response
POST /loglevel — DEBUG, INFO, ERROR

2.2.
2.3.
,
3.1. daemon
3.2. daemon
3.3. daemon
4.1.
4.2.
4.3.
4.4.
5.1.
5.2.
5.3.
5.4. HTTP
7.1. go mod
7.2.

1.

1:ERP, — 1 IBM MQ.
IBM MQ :

IBM MQ (, , )
IBM MQ
IBM MQ (SYNCPOINT)

XML :

() RFC 3076
HMAC - -34.11.94
HMAC

1C IBM MQ . REST API IBM MQ .

1C IBM MQ Go IBM MQ. , , "" C . 2 [].

backend , .

1 IBM MQ .

REST_IBMMQ

2.

2.1.

, , . github.com/urfave/cli. :

   --httpconfig value, --httpcfg value    HTTP Config file name
   --listenstring value, -l value         Listen string in format <host>:<port>
   --httpuser value, --httpu value        User name for access to HTTP server
   --httppassword value, --httppwd value  User password for access to HTTP server
   --jwtkey value, --jwtk value           JSON web token secret key
   --debug value, -d value                Debug mode: DEBUG, INFO, ERROR
   --logfile value, --log value           Log file name

2.2.

github.com/sasbury/mini. , :

[HTTP_SERVER]
ReadTimeout = 6000          // HTTP read timeout duration in sec - default 60 sec
WriteTimeout = 6000         // HTTP write timeout duration in sec - default 60 sec
IdleTimeout = 6000          // HTTP idle timeout duration in sec - default 60 sec
MaxHeaderBytes = 262144     // HTTP max header bytes - default 1 MB
MaxBodyBytes = 1048576      // HTTP max body bytes - default 0 - unlimited
UseProfile = false          // use Go profiling
ShutdownTimeout = 30        // service shutdown timeout in sec - default 30 sec

[TLS]
UseTLS = false                  // use SSL
UseHSTS = false                 // use HTTP Strict Transport Security
TLSertFile = certs/server.pem  // TLS Certificate file name
TLSKeyFile = certs/server.key   // TLS Private key file name
TLSMinVersion = VersionTLS10    // TLS min version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30
TLSMaxVersion = VersionTLS12    // TLS max version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30

[JWT]
UseJWT = false                  // use JSON web token (JWT)
JWTExpiresAt = 20000            // JWT expiry time in seconds - 0 without restriction

[AUTHENTIFICATION]
AuthType = INTERNAL             // Autehtification type NONE | INTERNAL | MSAD
MSADServer = company.com        // MS Active Directory server
MSADPort = 389                  // MS Active Directory Port
MSADBaseDN = OU=, DC=, DC=      // MS Active Directory BaseDN
MSADSecurity = SecurityNone     // MS Active Directory Security: SecurityNone, SecurityTLS, SecurityStartTLS

[LOG]
HTTPLog = false                         // Log HTTP traffic
HTTPLogType = INREQ                     // HTTP trafic log mode INREQ | OUTREQ | INRESP | OUTRESP | BODY
HTTPLogFileName = ./httplog/http%s.log  // HTTP log file
HTTPErrLog = HEADER | BODY              // Log error into HTTP response header and body

3. ,

UML .

, daemon. :

context.Context
/

3.1. daemon

, daemon.
, daemon .
, daemon :

httpserverErrCh: make(chan error, 1), //   HTTP

, :

daemon —
— daemon

IBM MQ:
- IBM MQ, ,
- IBM MQ
PostgreSQL:
- , , SQL
JSON BoltDB:
- BoltDB
- ( PostgreSQL). , ( BoltDB 150 2 64 , BoltDB NVMe 0.03 ms).
HTTP :
- http.server
- TCP
- TLS
- HTTP
- pprof

3.2. daemon

daemon .
, — , , , .
, . , HTTP

go func() { httpserverErrCh <- d.httpserver.Run() }()

, daemon .
daemon , , - . , ( ) .
, IBM MQ , daemon IBM MQ .

syscalCh := make(chan os.Signal, 1) //   
signal.Notify(syscalCh, syscall.SIGINT, syscall.SIGTERM)

//       
select {
case s := <-syscalCh: //  
    mylog.PrintfInfoMsg("Exiting, got signal", s)
    d.Shutdown() //  daemon
    return nil
case err := <-d.httpserverErrCh: //   HTTP    
    mylog.PrintfErrorInfo(err) //  
    d.Shutdown() //  daemon
    return err
}

, , ( ). , daemon.

func (s *Server) Run() error {
    defer func() {
        var myerr error
        r := recover()
        if r != nil {
            msg := "Recover from panic"
            switch t := r.(type) {
            case string:
                myerr = myerror.New("8888", msg, t)
            case error:
                myerr = myerror.WithCause("8888", msg, t)
            default:
                myerr = myerror.New("8888", msg)
            }
            mylog.PrintfErrorInfo(myerr) //  
            s.errCh <- myerr             //       daemon
        }
    }()

    //  
}

3.3. daemon

daemon .

, , :

( ShutdownTimeout )
. IBM MQ :
- IBM MQ
stopCh

, :

. ,
, , , ,

for {
    select {
    case <-ctx.Done(): //    

        //  

        s.stopCh <- struct{}{} //     
        return
    default:
        //   
    }
}

HTTP :

//        ShutdownTimeout
cancelCtx, cancel := context.WithTimeout(s.ctx, time.Duration(s.cfg.ShutdownTimeout*int(time.Second)))
defer cancel()

//       ShutdownTimeout
if err := s.httpServer.Shutdown(cancelCtx); err != nil {
    return err
}

s.httpService.Shutdown() //   

//     HTTP 
s.stopCh <- struct{}{}

4.

4.1.

github.com/pkg/errors.
, , .

type Error struct {
    ID       uint64 //   
    Code     string //  
    Msg      string //  
    Caller   string // ,        
    Args     string //  
    CauseErr error  //  - 
    CauseMsg string //   - 
    Trace    string //      
}

, . , 1 IBM MQ, 4 . , "8" HTTP, "7" — IBM MQ.

Caller — , . , . :

httpserver.go:[209] - (*Server).Run()

Caller

func caller(depth int) string {
    pc := make([]uintptr, 15)
    n := runtime.Callers(depth+1, pc)
    frame, _ := runtime.CallersFrames(pc[:n]).Next()
    idxFile := strings.LastIndexByte(frame.File, '/')
    idx := strings.LastIndexByte(frame.Function, '/')
    idxName := strings.IndexByte(frame.Function[idx+1:], '.') + idx + 1

    return frame.File[idxFile+1:] + ":[" + strconv.Itoa(frame.Line) + "] - " + frame.Function[idxName+1:] + "()"
}

Args — , . .

CauseErr CauseMsg — . , .

Trace — . . github.com/pkg/errors '%+v'. .

fmt.Sprintf("'%+v'", pkgerr.New(""))

4.2.

Error, .

func (e *Error) Error() string {
    mes := fmt.Sprintf("ID=[%v], code=[%s], mes=[%s]", e.ID, e.Code, e.Msg)
    if e.Args != "" {
        mes = fmt.Sprintf("%s, args=[%s]", mes, e.Args)
    }
    if e.CauseMsg != "" {
        mes = fmt.Sprintf("%s, causemes=[%s]", mes, e.CauseMsg)
    }
    return mes
}

ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3']

// %s    print the error code, message, arguments, and cause message.
// %v    in addition to %s, print caller
// %+v   extended format. Each Frame of the error's StackTrace will be printed in detail.
func (e *Error) Format(s fmt.State, verb rune) {
    switch verb {
    case 'v':
        fmt.Fprint(s, e.Error())
        fmt.Fprintf(s, ", caller=[%s]", e.Caller)
        if s.Flag('+') {
            fmt.Fprintf(s, ", trace=%s", e.Trace)
            return
        }
    case 's':
        fmt.Fprint(s, e.Error())
    case 'q':
        fmt.Fprint(s, e.Error())
    }
}

'%+v'

ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3'], caller=[handler_echo.go:[31] - (*Service).EchoHandler.func1()], trace='
github.com/romapres2010/httpserver/error.New
        D:/golang/src/github.com/romapres2010/httpserver/error/error.go:72
github.com/romapres2010/httpserver/httpserver/httpservice.(*Service).EchoHandler.func1
        D:/golang/src/github.com/romapres2010/httpserver/httpserver/httpservice/handler_echo.go:31
        ...

. .
— Format.

4.3.

New(code string, msg string, args ...interface{}) error

WithCause(code string, msg string, causeErr error, args ...interface{}) error

, args ...interface{}.

. , :

myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)

4.4.

, INFO trace. , , , ,
WithCause()
INFO. .
, ERROR , trace.

HTTP :

requestBuf, err := ioutil.ReadAll(r.Body)
if err != nil {
    myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)
    mylog.PrintfErrorInfo(myerr) //    
    s.processError(myerr, w, http.StatusInternalServerError, reqID) //     HTTP response
    return myerr
}

processError HTTP / .
— HTTPErrLog, POST /httperrlog.

POST /httperrlog HTTP/1.1
HTTP-Err-Log: HEADER | BODY

HTTP header

// carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0)
headerReplacer := strings.NewReplacer("\x0a", " ", "\x0d", " ", "\x00", " ")

w.Header().Set("Err-Trace", headerReplacer.Replace(myerr.Trace))

5.

github.com/hashicorp/logutils.
RFC 5424 — The Syslog Protocol, :

debug —
info — , , /
error — ,

— 100

Dave Cheney Let’s talk about logging.
— A simple logging interface for Go.

5.1.

"log". github.com/hashicorp/logutils.

// logFilter represent a custom logger seting
var logFilter = &logutils.LevelFilter{
    Levels:   []logutils.LogLevel{"DEBUG", "INFO", "ERROR"},
    MinLevel: logutils.LogLevel("INFO"), // initial setting
    Writer:   os.Stderr,                 // initial setting
}

// InitLogger init custom logger
func InitLogger(wrt io.Writer) {
    logFilter.Writer = wrt   // custom logger
    log.SetOutput(logFilter) // set std logger to our custom
}

, , main MultiWriter

//     
if logFileFlag != "" {
    //        
    logFileFlag = strings.Replace(logFileFlag, "%s", time.Now().Format("2006_01_02_150405"), 1)

    //        APPEND
    logFile, err := os.OpenFile(logFileFlag, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        myerr := myerror.WithCause("6020", "Error open log file: Filename", err, logFileFlag)
        mylog.PrintfErrorMsg(fmt.Sprintf("%+v", myerr))
        return myerr
    }
    if logFile != nil {
        defer logFile.Close()
    }

    wrt := io.MultiWriter(os.Stderr, logFile) //    os.Stderr  

    mylog.InitLogger(wrt) //     
} else {
    mylog.InitLogger(os.Stderr)
}

5.2.

INFO DEBUG .

2020/03/10 17:56:39 [INFO] - httpserver.go:[61] - New() - Creating new HTTP server
2020/03/10 17:56:39 [INFO] - httpserver.go:[141] - New() - Created new TCP listener: network = 'tcp', address ['127.0.0.1:3000']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/echo', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/signin', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/refresh', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[179] - New() - HTTP server is created
2020/03/10 17:56:39 [INFO] - daemon.go:[121] - New() - New daemon is created
2020/03/10 17:56:39 [INFO] - daemon.go:[128] - (*Daemon).Run() - Starting daemon
2020/03/10 17:56:39 [INFO] - daemon.go:[133] - (*Daemon).Run() - Daemon is running. For exit <CTRL-c>
2020/03/10 17:56:39 [INFO] - httpserver.go:[209] - (*Server).Run() - Starting HTTP server
2020/03/10 17:57:22 [INFO] - daemon.go:[142] - (*Daemon).Run() - Exiting, got signal ['interrupt']
2020/03/10 17:57:22 [INFO] - daemon.go:[155] - (*Daemon).Shutdown() - Shutting down daemon
2020/03/10 17:57:22 [INFO] - httpserver.go:[215] - (*Server).Shutdown() - Waiting for shutdown HTTP Server: sec ['30']
2020/03/10 17:57:22 [INFO] - httpserver.go:[231] - (*Server).Shutdown() - HTTP Server shutdown successfuly
2020/03/10 17:57:22 [INFO] - daemon.go:[167] - (*Daemon).Shutdown() - Daemon is shutdown
2020/03/10 17:57:22 [INFO] - main.go:[163] - main.func1() - Server is shutdown

ERROR .

5.3.

func PrintfInfoMsg(mes string, args ...interface{}) {
    printfMsg("[INFO]", 0, mes, args...)
}
func PrintfDebugMsg(mes string, args ...interface{}) {
    printfMsg("[DEBUG]", 0, mes, args...)
}
func PrintfErrorMsg(mes string, args ...interface{}) {
    printfMsg("[ERROR]", 0, mes, args...)
}

, .
— DEBUG .

INFO DEBUG, .

mylog.PrintfInfoMsg(fmt.Sprintf("Create new TCP listener network='tcp', address='%s'", serverCfg.ListenSpec))

mylog.PrintfInfoMsg("Created new TCP listener: network = 'tcp', address", cfg.ListenSpec)

, , DEBUG INFO 1%.
— INFO DEBUG POST /loglevel.

POST /loglevel HTTP/1.1
Log-Level-Filter: DEBUG | ERROR | INFO

5.4. HTTP

, HTTP .
httplog HTTP header body:

— HttpLog HTTPLogType, POST /httplog.

POST /httplog HTTP/1.1
Http-Log: TRUE
Http-Log-Type: INREQ | OUTREQ | INRESP | OUTRESP | BODY

6.

: HTTP Basic Authentication MS AD Authentication.
MS AD Authentication gopkg.in/korylprince/go-ad-auth.v2.
, MS AD.

[AUTHENTIFICATION]
AuthType = INTERNAL | MSAD | NONE
MSADServer = company.com
MSADPort = 389
MSADBaseDN = OU=company, DC=dc, DC=corp
MSADSecurity = SecurityNone

HTTP Basic Authentication .

JSON Web Token (JWT) . JWTExpiresAt (JWTExpiresAt=0 — ). JWT .

[JWT]
UseJWT = true
JWTExpiresAt = 20000

JWT github.com/dgrijalva/jwt-go.
JWT: , , cookie jwt

JWT :

JWT , HTTP Basic Authentication MS AD Authentication
JWT , (StatusUnauthorized), JWT
JWT POST /signin
- JSON Web Token Claim ( Claim ),
- Claims JSON Web Token, HS256
- Token http Cookie "token". Cookie Token
, http Cookie JWT . , StatusUnauthorized
JWT POST /refresh
logout

7.

7.1. go mod

golang/dep/cmd/dep go mod, vendor . "" . .

go get -u ./../...
go mod vendor

go mod , . commit, :

go get github.com/ibm-messaging/mq-golang/ibmmq@19b946c

, vendor,

go build -v -mod vendor

7.2.

make ( - ):

rebuild —
build —
check - verifique o código usando github.com/golangci/golangci-lint

Ao compilar, a versão, a data de construção e a confirmação são incorporadas no arquivo executável. Esta informação é exibida em um arquivo de log - muito útil para analisar erros.
Para implementar essa implementação, adicionamos variáveis aos principais

var (
    version   = "0.0.2" //  ,  
    commit    = "unset" //  commit
    buildTime = "unset" //    
)

Ao compilar um makefile, solicitamos git para commit

COMMIT?=$(shell git rev-parse --short HEAD)
BUILD_TIME?=$(shell date -u '+%Y-%m-%d_%H:%M:%S')

Ao montar, vinculamos esses valores às variáveis declaradas anteriormente (opção -ldflags "-X" ), ao mesmo tempo em que indicam os sistemas operacionais sob os quais coletamos e o local de armazenamento do binário

GOOS=${GOOS} go build -v -a -mod vendor \
-ldflags "-X main.commit=${COMMIT} -X main.buildTime=${BUILD_TIME}" \
-o bin/${GOOS}/${APP}

Em main.main, escrevo a versão final, confirmo e construo as informações de data em uma variável

app.Version = fmt.Sprintf("%s, commit '%s', build time '%s'", version, commit, buildTime)

Modelo de servidor back-end Golang - parte 1 (servidor HTTP)

1.

2.