🛄 ⚗️ 🤦🏽 Modèle de serveur d'arrière-plan Golang

Le modèle de serveur Golang ci-dessous a été préparé pour transférer des connaissances au sein de notre équipe. L'objectif principal du modèle, en plus de la formation, est de réduire le temps de prototypage de petites tâches de serveur sur Go.

Le modèle comprend:

Passer les paramètres du serveur via la ligne de commande github.com/urfave/cli
Configuration des paramètres du serveur via le fichier de configuration github.com/sasbury/mini
Configurer les paramètres du serveur HTTP TLS
Configuration d'un routeur enregistrant HTTP et les gestionnaires de prof github.com/gorilla/mux
Définition des niveaux de journalisation sans arrêter le serveur github.com/hashicorp/logutils
Configuration de la journalisation du trafic HTTP sans arrêter le serveur
Définition de la journalisation des erreurs dans la réponse HTTP sans arrêter le serveur
Authentification de base HTTP
Authentification MS AD gopkg.in/korylprince/go-ad-auth.v2
Jeton Web JSON github.com/dgrijalva/jwt-go
Démarrage du serveur avec l'attente d'un retour au canal d'erreur
Utilisation du contexte pour arrêter correctement le serveur et les services associés
Configuration personnalisée de la gestion des erreurs github.com/pkg/errors
Paramètres de journalisation personnalisés
Construisez avec la version, la date de construction et validez

Lien vers le référentiel du projet .

HTTP (. HTTP Golang):

POST /echo — request HTTP body response
POST /signin — JSON Web Token Cookie
POST /refresh — JSON Web Token Cookie
POST /httplog — HTTP
POST /httperrlog — HTTP response
POST /loglevel — DEBUG, INFO, ERROR

2.2.
2.3.
,
3.1. daemon
3.2. daemon
3.3. daemon
4.1.
4.2.
4.3.
4.4.
5.1.
5.2.
5.3.
5.4. HTTP
7.1. go mod
7.2.

1.

1:ERP, — 1 IBM MQ.
IBM MQ :

IBM MQ (, , )
IBM MQ
IBM MQ (SYNCPOINT)

XML :

() RFC 3076
HMAC - -34.11.94
HMAC

1C IBM MQ . REST API IBM MQ .

1C IBM MQ Go IBM MQ. , , "" C . 2 [].

backend , .

1 IBM MQ .

REST_IBMMQ

2.

2.1.

, , . github.com/urfave/cli. :

   --httpconfig value, --httpcfg value    HTTP Config file name
   --listenstring value, -l value         Listen string in format <host>:<port>
   --httpuser value, --httpu value        User name for access to HTTP server
   --httppassword value, --httppwd value  User password for access to HTTP server
   --jwtkey value, --jwtk value           JSON web token secret key
   --debug value, -d value                Debug mode: DEBUG, INFO, ERROR
   --logfile value, --log value           Log file name

2.2.

github.com/sasbury/mini. , :

[HTTP_SERVER]
ReadTimeout = 6000          // HTTP read timeout duration in sec - default 60 sec
WriteTimeout = 6000         // HTTP write timeout duration in sec - default 60 sec
IdleTimeout = 6000          // HTTP idle timeout duration in sec - default 60 sec
MaxHeaderBytes = 262144     // HTTP max header bytes - default 1 MB
MaxBodyBytes = 1048576      // HTTP max body bytes - default 0 - unlimited
UseProfile = false          // use Go profiling
ShutdownTimeout = 30        // service shutdown timeout in sec - default 30 sec

[TLS]
UseTLS = false                  // use SSL
UseHSTS = false                 // use HTTP Strict Transport Security
TLSertFile = certs/server.pem  // TLS Certificate file name
TLSKeyFile = certs/server.key   // TLS Private key file name
TLSMinVersion = VersionTLS10    // TLS min version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30
TLSMaxVersion = VersionTLS12    // TLS max version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30

[JWT]
UseJWT = false                  // use JSON web token (JWT)
JWTExpiresAt = 20000            // JWT expiry time in seconds - 0 without restriction

[AUTHENTIFICATION]
AuthType = INTERNAL             // Autehtification type NONE | INTERNAL | MSAD
MSADServer = company.com        // MS Active Directory server
MSADPort = 389                  // MS Active Directory Port
MSADBaseDN = OU=, DC=, DC=      // MS Active Directory BaseDN
MSADSecurity = SecurityNone     // MS Active Directory Security: SecurityNone, SecurityTLS, SecurityStartTLS

[LOG]
HTTPLog = false                         // Log HTTP traffic
HTTPLogType = INREQ                     // HTTP trafic log mode INREQ | OUTREQ | INRESP | OUTRESP | BODY
HTTPLogFileName = ./httplog/http%s.log  // HTTP log file
HTTPErrLog = HEADER | BODY              // Log error into HTTP response header and body

3. ,

UML .

, daemon. :

context.Context
/

3.1. daemon

, daemon.
, daemon .
, daemon :

httpserverErrCh: make(chan error, 1), //   HTTP

, :

daemon —
— daemon

IBM MQ:
- IBM MQ, ,
- IBM MQ
PostgreSQL:
- , , SQL
JSON BoltDB:
- BoltDB
- ( PostgreSQL). , ( BoltDB 150 2 64 , BoltDB NVMe 0.03 ms).
HTTP :
- http.server
- TCP
- TLS
- HTTP
- pprof

3.2. daemon

daemon .
, — , , , .
, . , HTTP

go func() { httpserverErrCh <- d.httpserver.Run() }()

, daemon .
daemon , , - . , ( ) .
, IBM MQ , daemon IBM MQ .

syscalCh := make(chan os.Signal, 1) //   
signal.Notify(syscalCh, syscall.SIGINT, syscall.SIGTERM)

//       
select {
case s := <-syscalCh: //  
    mylog.PrintfInfoMsg("Exiting, got signal", s)
    d.Shutdown() //  daemon
    return nil
case err := <-d.httpserverErrCh: //   HTTP    
    mylog.PrintfErrorInfo(err) //  
    d.Shutdown() //  daemon
    return err
}

, , ( ). , daemon.

func (s *Server) Run() error {
    defer func() {
        var myerr error
        r := recover()
        if r != nil {
            msg := "Recover from panic"
            switch t := r.(type) {
            case string:
                myerr = myerror.New("8888", msg, t)
            case error:
                myerr = myerror.WithCause("8888", msg, t)
            default:
                myerr = myerror.New("8888", msg)
            }
            mylog.PrintfErrorInfo(myerr) //  
            s.errCh <- myerr             //       daemon
        }
    }()

    //  
}

3.3. daemon

daemon .

, , :

( ShutdownTimeout )
. IBM MQ :
- IBM MQ
stopCh

, :

. ,
, , , ,

for {
    select {
    case <-ctx.Done(): //    

        //  

        s.stopCh <- struct{}{} //     
        return
    default:
        //   
    }
}

HTTP :

//        ShutdownTimeout
cancelCtx, cancel := context.WithTimeout(s.ctx, time.Duration(s.cfg.ShutdownTimeout*int(time.Second)))
defer cancel()

//       ShutdownTimeout
if err := s.httpServer.Shutdown(cancelCtx); err != nil {
    return err
}

s.httpService.Shutdown() //   

//     HTTP 
s.stopCh <- struct{}{}

4.

4.1.

github.com/pkg/errors.
, , .

type Error struct {
    ID       uint64 //   
    Code     string //  
    Msg      string //  
    Caller   string // ,        
    Args     string //  
    CauseErr error  //  - 
    CauseMsg string //   - 
    Trace    string //      
}

, . , 1 IBM MQ, 4 . , "8" HTTP, "7" — IBM MQ.

Caller — , . , . :

httpserver.go:[209] - (*Server).Run()

Caller

func caller(depth int) string {
    pc := make([]uintptr, 15)
    n := runtime.Callers(depth+1, pc)
    frame, _ := runtime.CallersFrames(pc[:n]).Next()
    idxFile := strings.LastIndexByte(frame.File, '/')
    idx := strings.LastIndexByte(frame.Function, '/')
    idxName := strings.IndexByte(frame.Function[idx+1:], '.') + idx + 1

    return frame.File[idxFile+1:] + ":[" + strconv.Itoa(frame.Line) + "] - " + frame.Function[idxName+1:] + "()"
}

Args — , . .

CauseErr CauseMsg — . , .

Trace — . . github.com/pkg/errors '%+v'. .

fmt.Sprintf("'%+v'", pkgerr.New(""))

4.2.

Error, .

func (e *Error) Error() string {
    mes := fmt.Sprintf("ID=[%v], code=[%s], mes=[%s]", e.ID, e.Code, e.Msg)
    if e.Args != "" {
        mes = fmt.Sprintf("%s, args=[%s]", mes, e.Args)
    }
    if e.CauseMsg != "" {
        mes = fmt.Sprintf("%s, causemes=[%s]", mes, e.CauseMsg)
    }
    return mes
}

ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3']

// %s    print the error code, message, arguments, and cause message.
// %v    in addition to %s, print caller
// %+v   extended format. Each Frame of the error's StackTrace will be printed in detail.
func (e *Error) Format(s fmt.State, verb rune) {
    switch verb {
    case 'v':
        fmt.Fprint(s, e.Error())
        fmt.Fprintf(s, ", caller=[%s]", e.Caller)
        if s.Flag('+') {
            fmt.Fprintf(s, ", trace=%s", e.Trace)
            return
        }
    case 's':
        fmt.Fprint(s, e.Error())
    case 'q':
        fmt.Fprint(s, e.Error())
    }
}

'%+v'

ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3'], caller=[handler_echo.go:[31] - (*Service).EchoHandler.func1()], trace='
github.com/romapres2010/httpserver/error.New
        D:/golang/src/github.com/romapres2010/httpserver/error/error.go:72
github.com/romapres2010/httpserver/httpserver/httpservice.(*Service).EchoHandler.func1
        D:/golang/src/github.com/romapres2010/httpserver/httpserver/httpservice/handler_echo.go:31
        ...

. .
— Format.

4.3.

New(code string, msg string, args ...interface{}) error

WithCause(code string, msg string, causeErr error, args ...interface{}) error

, args ...interface{}.

. , :

myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)

4.4.

, INFO trace. , , , ,
WithCause()
INFO. .
, ERROR , trace.

HTTP :

requestBuf, err := ioutil.ReadAll(r.Body)
if err != nil {
    myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)
    mylog.PrintfErrorInfo(myerr) //    
    s.processError(myerr, w, http.StatusInternalServerError, reqID) //     HTTP response
    return myerr
}

processError HTTP / .
— HTTPErrLog, POST /httperrlog.

POST /httperrlog HTTP/1.1
HTTP-Err-Log: HEADER | BODY

HTTP header

// carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0)
headerReplacer := strings.NewReplacer("\x0a", " ", "\x0d", " ", "\x00", " ")

w.Header().Set("Err-Trace", headerReplacer.Replace(myerr.Trace))

5.

github.com/hashicorp/logutils.
RFC 5424 — The Syslog Protocol, :

debug —
info — , , /
error — ,

— 100

Dave Cheney Let’s talk about logging.
— A simple logging interface for Go.

5.1.

"log". github.com/hashicorp/logutils.

// logFilter represent a custom logger seting
var logFilter = &logutils.LevelFilter{
    Levels:   []logutils.LogLevel{"DEBUG", "INFO", "ERROR"},
    MinLevel: logutils.LogLevel("INFO"), // initial setting
    Writer:   os.Stderr,                 // initial setting
}

// InitLogger init custom logger
func InitLogger(wrt io.Writer) {
    logFilter.Writer = wrt   // custom logger
    log.SetOutput(logFilter) // set std logger to our custom
}

, , main MultiWriter

//     
if logFileFlag != "" {
    //        
    logFileFlag = strings.Replace(logFileFlag, "%s", time.Now().Format("2006_01_02_150405"), 1)

    //        APPEND
    logFile, err := os.OpenFile(logFileFlag, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        myerr := myerror.WithCause("6020", "Error open log file: Filename", err, logFileFlag)
        mylog.PrintfErrorMsg(fmt.Sprintf("%+v", myerr))
        return myerr
    }
    if logFile != nil {
        defer logFile.Close()
    }

    wrt := io.MultiWriter(os.Stderr, logFile) //    os.Stderr  

    mylog.InitLogger(wrt) //     
} else {
    mylog.InitLogger(os.Stderr)
}

5.2.

INFO DEBUG .

2020/03/10 17:56:39 [INFO] - httpserver.go:[61] - New() - Creating new HTTP server
2020/03/10 17:56:39 [INFO] - httpserver.go:[141] - New() - Created new TCP listener: network = 'tcp', address ['127.0.0.1:3000']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/echo', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/signin', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/refresh', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[179] - New() - HTTP server is created
2020/03/10 17:56:39 [INFO] - daemon.go:[121] - New() - New daemon is created
2020/03/10 17:56:39 [INFO] - daemon.go:[128] - (*Daemon).Run() - Starting daemon
2020/03/10 17:56:39 [INFO] - daemon.go:[133] - (*Daemon).Run() - Daemon is running. For exit <CTRL-c>
2020/03/10 17:56:39 [INFO] - httpserver.go:[209] - (*Server).Run() - Starting HTTP server
2020/03/10 17:57:22 [INFO] - daemon.go:[142] - (*Daemon).Run() - Exiting, got signal ['interrupt']
2020/03/10 17:57:22 [INFO] - daemon.go:[155] - (*Daemon).Shutdown() - Shutting down daemon
2020/03/10 17:57:22 [INFO] - httpserver.go:[215] - (*Server).Shutdown() - Waiting for shutdown HTTP Server: sec ['30']
2020/03/10 17:57:22 [INFO] - httpserver.go:[231] - (*Server).Shutdown() - HTTP Server shutdown successfuly
2020/03/10 17:57:22 [INFO] - daemon.go:[167] - (*Daemon).Shutdown() - Daemon is shutdown
2020/03/10 17:57:22 [INFO] - main.go:[163] - main.func1() - Server is shutdown

ERROR .

5.3.

func PrintfInfoMsg(mes string, args ...interface{}) {
    printfMsg("[INFO]", 0, mes, args...)
}
func PrintfDebugMsg(mes string, args ...interface{}) {
    printfMsg("[DEBUG]", 0, mes, args...)
}
func PrintfErrorMsg(mes string, args ...interface{}) {
    printfMsg("[ERROR]", 0, mes, args...)
}

, .
— DEBUG .

INFO DEBUG, .

mylog.PrintfInfoMsg(fmt.Sprintf("Create new TCP listener network='tcp', address='%s'", serverCfg.ListenSpec))

mylog.PrintfInfoMsg("Created new TCP listener: network = 'tcp', address", cfg.ListenSpec)

, , DEBUG INFO 1%.
— INFO DEBUG POST /loglevel.

POST /loglevel HTTP/1.1
Log-Level-Filter: DEBUG | ERROR | INFO

5.4. HTTP

, HTTP .
httplog HTTP header body:

— HttpLog HTTPLogType, POST /httplog.

POST /httplog HTTP/1.1
Http-Log: TRUE
Http-Log-Type: INREQ | OUTREQ | INRESP | OUTRESP | BODY

6.

: HTTP Basic Authentication MS AD Authentication.
MS AD Authentication gopkg.in/korylprince/go-ad-auth.v2.
, MS AD.

[AUTHENTIFICATION]
AuthType = INTERNAL | MSAD | NONE
MSADServer = company.com
MSADPort = 389
MSADBaseDN = OU=company, DC=dc, DC=corp
MSADSecurity = SecurityNone

HTTP Basic Authentication .

JSON Web Token (JWT) . JWTExpiresAt (JWTExpiresAt=0 — ). JWT .

[JWT]
UseJWT = true
JWTExpiresAt = 20000

JWT github.com/dgrijalva/jwt-go.
JWT: , , cookie jwt

JWT :

JWT , HTTP Basic Authentication MS AD Authentication
JWT , (StatusUnauthorized), JWT
JWT POST /signin
- JSON Web Token Claim ( Claim ),
- Claims JSON Web Token, HS256
- Token http Cookie "token". Cookie Token
, http Cookie JWT . , StatusUnauthorized
JWT POST /refresh
logout

7.

7.1. go mod

golang/dep/cmd/dep go mod, vendor . "" . .

go get -u ./../...
go mod vendor

go mod , . commit, :

go get github.com/ibm-messaging/mq-golang/ibmmq@19b946c

, vendor,

go build -v -mod vendor

7.2.

make ( - ):

rebuild —
build —
check - vérifiez le code en utilisant github.com/golangci/golangci-lint

Lors de la compilation, la version, la date de construction et la validation sont incorporées dans le fichier exécutable. Ces informations sont affichées dans un fichier journal - très utile pour analyser les erreurs.
Pour implémenter une telle implémentation, nous ajoutons des variables à main

var (
    version   = "0.0.2" //  ,  
    commit    = "unset" //  commit
    buildTime = "unset" //    
)

Lors de la compilation dans un makefile, nous demandons git pour commit

COMMIT?=$(shell git rev-parse --short HEAD)
BUILD_TIME?=$(shell date -u '+%Y-%m-%d_%H:%M:%S')

Lors de l'assemblage, nous lions ces valeurs aux variables précédemment déclarées (option -ldflags "-X" ), en indiquant en même temps les OS sous lesquels nous collectons et l'emplacement de stockage du binaire

GOOS=${GOOS} go build -v -a -mod vendor \
-ldflags "-X main.commit=${COMMIT} -X main.buildTime=${BUILD_TIME}" \
-o bin/${GOOS}/${APP}

Dans main.main, j'écris la version finale, valide et compile les informations de date dans une variable

app.Version = fmt.Sprintf("%s, commit '%s', build time '%s'", version, commit, buildTime)

Modèle de serveur d'arrière-plan Golang - partie 1 (serveur HTTP)

1.

2.