Get best of the week

Plantilla de servidor backend de Golang - parte 1 (servidor HTTP)

La siguiente plantilla de servidor de Golang se preparó para transferir conocimiento dentro de nuestro equipo. El objetivo principal de la plantilla, además de la capacitación, es reducir el tiempo de creación de prototipos de pequeñas tareas de servidor en Go.


La plantilla incluye:


  • Pasar parámetros del servidor a través de la línea de comando github.com/urfave/cli
  • Configuración del servidor a través del archivo de configuración github.com/sasbury/mini
  • Configure los ajustes del servidor HTTP TLS
  • Configuración de un enrutador que registre HTTP y controladores de perfil github.com/gorilla/mux
  • Establecer niveles de registro sin detener el servidor github.com/hashicorp/logutils
  • Configurar el registro de tráfico HTTP sin detener el servidor
  • Configurar el registro de errores en la respuesta HTTP sin detener el servidor
  • Autenticación básica HTTP
  • Autenticación de MS AD gopkg.in/korylprince/go-ad-auth.v2
  • JSON Web Token github.com/dgrijalva/jwt-go
  • Iniciar el servidor con la expectativa de un retorno al canal de error
  • Usar el contexto para detener correctamente el servidor y los servicios relacionados
  • Configuración personalizada de manejo de errores github.com/pkg/errors
  • Configuración de registro personalizada
  • Compilación con versión, fecha de compilación y confirmación

Enlace al repositorio del proyecto .


HTTP (. HTTP Golang):


  • POST /echo — request HTTP body response
  • POST /signin — JSON Web Token Cookie
  • POST /refresh — JSON Web Token Cookie
  • POST /httplog — HTTP
  • POST /httperrlog — HTTP response
  • POST /loglevel — DEBUG, INFO, ERROR



  2. 2.2.
    2.3.
  3. ,
    3.1. daemon
    3.2. daemon
    3.3. daemon

  4. 4.1.
    4.2.
    4.3.
    4.4.

  5. 5.1.
    5.2.
    5.3.
    5.4. HTTP

  7. 7.1. go mod
    7.2.

1.


1:ERP, — 1 IBM MQ.
IBM MQ :


  • IBM MQ (, , )
  • IBM MQ
  • IBM MQ (SYNCPOINT)

XML :



1C IBM MQ . REST API IBM MQ .


1C IBM MQ Go IBM MQ. , , "" C . 2 [].


backend , .


1 IBM MQ .


REST_IBMMQ


2.


2.1.


, , . github.com/urfave/cli. :


   --httpconfig value, --httpcfg value    HTTP Config file name
   --listenstring value, -l value         Listen string in format <host>:<port>
   --httpuser value, --httpu value        User name for access to HTTP server
   --httppassword value, --httppwd value  User password for access to HTTP server
   --jwtkey value, --jwtk value           JSON web token secret key
   --debug value, -d value                Debug mode: DEBUG, INFO, ERROR
   --logfile value, --log value           Log file name

2.2.


github.com/sasbury/mini. , :


[HTTP_SERVER]
ReadTimeout = 6000          // HTTP read timeout duration in sec - default 60 sec
WriteTimeout = 6000         // HTTP write timeout duration in sec - default 60 sec
IdleTimeout = 6000          // HTTP idle timeout duration in sec - default 60 sec
MaxHeaderBytes = 262144     // HTTP max header bytes - default 1 MB
MaxBodyBytes = 1048576      // HTTP max body bytes - default 0 - unlimited
UseProfile = false          // use Go profiling
ShutdownTimeout = 30        // service shutdown timeout in sec - default 30 sec

[TLS]
UseTLS = false                  // use SSL
UseHSTS = false                 // use HTTP Strict Transport Security
TLSertFile = certs/server.pem  // TLS Certificate file name
TLSKeyFile = certs/server.key   // TLS Private key file name
TLSMinVersion = VersionTLS10    // TLS min version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30
TLSMaxVersion = VersionTLS12    // TLS max version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30

[JWT]
UseJWT = false                  // use JSON web token (JWT)
JWTExpiresAt = 20000            // JWT expiry time in seconds - 0 without restriction

[AUTHENTIFICATION]
AuthType = INTERNAL             // Autehtification type NONE | INTERNAL | MSAD
MSADServer = company.com        // MS Active Directory server
MSADPort = 389                  // MS Active Directory Port
MSADBaseDN = OU=, DC=, DC=      // MS Active Directory BaseDN
MSADSecurity = SecurityNone     // MS Active Directory Security: SecurityNone, SecurityTLS, SecurityStartTLS

[LOG]
HTTPLog = false                         // Log HTTP traffic
HTTPLogType = INREQ                     // HTTP trafic log mode INREQ | OUTREQ | INRESP | OUTRESP | BODY
HTTPLogFileName = ./httplog/http%s.log  // HTTP log file
HTTPErrLog = HEADER | BODY              // Log error into HTTP response header and body

3. ,


UML .


http_server_run_stop


, daemon. :


  • context.Context
  • /

3.1. daemon


, daemon.
, daemon .
, daemon :


httpserverErrCh: make(chan error, 1), //   HTTP

, :


  • daemon —
  • — daemon

:


  • IBM MQ:
    • IBM MQ, ,
    • IBM MQ
  • PostgreSQL:
    • , , SQL
  • JSON BoltDB:
    • BoltDB
    • ( PostgreSQL). , ( BoltDB 150 2 64 , BoltDB NVMe 0.03 ms).
  • HTTP :
    • http.server
    • TCP
    • TLS
    • HTTP
    • pprof

3.2. daemon


daemon .
, — , , , .
, . , HTTP


go func() { httpserverErrCh <- d.httpserver.Run() }()

, daemon .
daemon , , - . , ( ) .
, IBM MQ , daemon IBM MQ .


syscalCh := make(chan os.Signal, 1) //   
signal.Notify(syscalCh, syscall.SIGINT, syscall.SIGTERM)

//       
select {
case s := <-syscalCh: //  
    mylog.PrintfInfoMsg("Exiting, got signal", s)
    d.Shutdown() //  daemon
    return nil
case err := <-d.httpserverErrCh: //   HTTP    
    mylog.PrintfErrorInfo(err) //  
    d.Shutdown() //  daemon
    return err
}

, , ( ). , daemon.


func (s *Server) Run() error {
    defer func() {
        var myerr error
        r := recover()
        if r != nil {
            msg := "Recover from panic"
            switch t := r.(type) {
            case string:
                myerr = myerror.New("8888", msg, t)
            case error:
                myerr = myerror.WithCause("8888", msg, t)
            default:
                myerr = myerror.New("8888", msg)
            }
            mylog.PrintfErrorInfo(myerr) //  
            s.errCh <- myerr             //       daemon
        }
    }()

    //  
}

3.3. daemon


daemon .


, , :


  • ( ShutdownTimeout )
  • . IBM MQ :
    • IBM MQ
  • stopCh

, :


  • . ,
  • , , , ,

for {
    select {
    case <-ctx.Done(): //    

        //  

        s.stopCh <- struct{}{} //     
        return
    default:
        //   
    }
}

HTTP :


//        ShutdownTimeout
cancelCtx, cancel := context.WithTimeout(s.ctx, time.Duration(s.cfg.ShutdownTimeout*int(time.Second)))
defer cancel()

//       ShutdownTimeout
if err := s.httpServer.Shutdown(cancelCtx); err != nil {
    return err
}

s.httpService.Shutdown() //   

//     HTTP 
s.stopCh <- struct{}{}

4.


4.1.


github.com/pkg/errors.
, , .


:


type Error struct {
    ID       uint64 //   
    Code     string //  
    Msg      string //  
    Caller   string // ,        
    Args     string //  
    CauseErr error  //  - 
    CauseMsg string //   - 
    Trace    string //      
}

, . , 1 IBM MQ, 4 . , "8" HTTP, "7" — IBM MQ.


Caller — , . , . :


httpserver.go:[209] - (*Server).Run()

Caller


func caller(depth int) string {
    pc := make([]uintptr, 15)
    n := runtime.Callers(depth+1, pc)
    frame, _ := runtime.CallersFrames(pc[:n]).Next()
    idxFile := strings.LastIndexByte(frame.File, '/')
    idx := strings.LastIndexByte(frame.Function, '/')
    idxName := strings.IndexByte(frame.Function[idx+1:], '.') + idx + 1

    return frame.File[idxFile+1:] + ":[" + strconv.Itoa(frame.Line) + "] - " + frame.Function[idxName+1:] + "()"
}

Args — , . .


CauseErr CauseMsg — . , .


Trace — . . github.com/pkg/errors '%+v'. .


fmt.Sprintf("'%+v'", pkgerr.New(""))

4.2.


Error, .


func (e *Error) Error() string {
    mes := fmt.Sprintf("ID=[%v], code=[%s], mes=[%s]", e.ID, e.Code, e.Msg)
    if e.Args != "" {
        mes = fmt.Sprintf("%s, args=[%s]", mes, e.Args)
    }
    if e.CauseMsg != "" {
        mes = fmt.Sprintf("%s, causemes=[%s]", mes, e.CauseMsg)
    }
    return mes
}


ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3']


// %s    print the error code, message, arguments, and cause message.
// %v    in addition to %s, print caller
// %+v   extended format. Each Frame of the error's StackTrace will be printed in detail.
func (e *Error) Format(s fmt.State, verb rune) {
    switch verb {
    case 'v':
        fmt.Fprint(s, e.Error())
        fmt.Fprintf(s, ", caller=[%s]", e.Caller)
        if s.Flag('+') {
            fmt.Fprintf(s, ", trace=%s", e.Trace)
            return
        }
    case 's':
        fmt.Fprint(s, e.Error())
    case 'q':
        fmt.Fprint(s, e.Error())
    }
}

'%+v'


ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3'], caller=[handler_echo.go:[31] - (*Service).EchoHandler.func1()], trace='
github.com/romapres2010/httpserver/error.New
        D:/golang/src/github.com/romapres2010/httpserver/error/error.go:72
github.com/romapres2010/httpserver/httpserver/httpservice.(*Service).EchoHandler.func1
        D:/golang/src/github.com/romapres2010/httpserver/httpserver/httpservice/handler_echo.go:31
        ...

. .
— Format.


4.3.


:



New(code string, msg string, args ...interface{}) error


WithCause(code string, msg string, causeErr error, args ...interface{}) error

, args ...interface{}.


. , :


myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)

4.4.


:


  • , INFO trace. , , , ,
  • WithCause()
  • INFO. .
  • , ERROR , trace.

HTTP :


requestBuf, err := ioutil.ReadAll(r.Body)
if err != nil {
    myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)
    mylog.PrintfErrorInfo(myerr) //    
    s.processError(myerr, w, http.StatusInternalServerError, reqID) //     HTTP response
    return myerr
}

processError HTTP / .
— HTTPErrLog, POST /httperrlog.


POST /httperrlog HTTP/1.1
HTTP-Err-Log: HEADER | BODY

HTTP header


// carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0)
headerReplacer := strings.NewReplacer("\x0a", " ", "\x0d", " ", "\x00", " ")

w.Header().Set("Err-Trace", headerReplacer.Replace(myerr.Trace))

5.


github.com/hashicorp/logutils.
RFC 5424 — The Syslog Protocol, :


  • debug —
  • info — , , /
  • error — ,

— 100


Dave Cheney Let’s talk about logging.
— A simple logging interface for Go.


5.1.


"log". github.com/hashicorp/logutils.


// logFilter represent a custom logger seting
var logFilter = &logutils.LevelFilter{
    Levels:   []logutils.LogLevel{"DEBUG", "INFO", "ERROR"},
    MinLevel: logutils.LogLevel("INFO"), // initial setting
    Writer:   os.Stderr,                 // initial setting
}

// InitLogger init custom logger
func InitLogger(wrt io.Writer) {
    logFilter.Writer = wrt   // custom logger
    log.SetOutput(logFilter) // set std logger to our custom
}

, , main MultiWriter


//     
if logFileFlag != "" {
    //        
    logFileFlag = strings.Replace(logFileFlag, "%s", time.Now().Format("2006_01_02_150405"), 1)

    //        APPEND
    logFile, err := os.OpenFile(logFileFlag, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        myerr := myerror.WithCause("6020", "Error open log file: Filename", err, logFileFlag)
        mylog.PrintfErrorMsg(fmt.Sprintf("%+v", myerr))
        return myerr
    }
    if logFile != nil {
        defer logFile.Close()
    }

    wrt := io.MultiWriter(os.Stderr, logFile) //    os.Stderr  

    mylog.InitLogger(wrt) //     
} else {
    mylog.InitLogger(os.Stderr)
}

5.2.


INFO DEBUG .


2020/03/10 17:56:39 [INFO] - httpserver.go:[61] - New() - Creating new HTTP server
2020/03/10 17:56:39 [INFO] - httpserver.go:[141] - New() - Created new TCP listener: network = 'tcp', address ['127.0.0.1:3000']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/echo', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/signin', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/refresh', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[179] - New() - HTTP server is created
2020/03/10 17:56:39 [INFO] - daemon.go:[121] - New() - New daemon is created
2020/03/10 17:56:39 [INFO] - daemon.go:[128] - (*Daemon).Run() - Starting daemon
2020/03/10 17:56:39 [INFO] - daemon.go:[133] - (*Daemon).Run() - Daemon is running. For exit <CTRL-c>
2020/03/10 17:56:39 [INFO] - httpserver.go:[209] - (*Server).Run() - Starting HTTP server
2020/03/10 17:57:22 [INFO] - daemon.go:[142] - (*Daemon).Run() - Exiting, got signal ['interrupt']
2020/03/10 17:57:22 [INFO] - daemon.go:[155] - (*Daemon).Shutdown() - Shutting down daemon
2020/03/10 17:57:22 [INFO] - httpserver.go:[215] - (*Server).Shutdown() - Waiting for shutdown HTTP Server: sec ['30']
2020/03/10 17:57:22 [INFO] - httpserver.go:[231] - (*Server).Shutdown() - HTTP Server shutdown successfuly
2020/03/10 17:57:22 [INFO] - daemon.go:[167] - (*Daemon).Shutdown() - Daemon is shutdown
2020/03/10 17:57:22 [INFO] - main.go:[163] - main.func1() - Server is shutdown

ERROR .


5.3.


:


func PrintfInfoMsg(mes string, args ...interface{}) {
    printfMsg("[INFO]", 0, mes, args...)
}
func PrintfDebugMsg(mes string, args ...interface{}) {
    printfMsg("[DEBUG]", 0, mes, args...)
}
func PrintfErrorMsg(mes string, args ...interface{}) {
    printfMsg("[ERROR]", 0, mes, args...)
}

, .
— DEBUG .


INFO DEBUG, .


mylog.PrintfInfoMsg(fmt.Sprintf("Create new TCP listener network='tcp', address='%s'", serverCfg.ListenSpec))


mylog.PrintfInfoMsg("Created new TCP listener: network = 'tcp', address", cfg.ListenSpec)

, , DEBUG INFO 1%.
— INFO DEBUG POST /loglevel.


POST /loglevel HTTP/1.1
Log-Level-Filter: DEBUG | ERROR | INFO

5.4. HTTP


, HTTP .
httplog HTTP header body:



— HttpLog HTTPLogType, POST /httplog.


POST /httplog HTTP/1.1
Http-Log: TRUE
Http-Log-Type: INREQ | OUTREQ | INRESP | OUTRESP | BODY

6.


: HTTP Basic Authentication MS AD Authentication.
MS AD Authentication gopkg.in/korylprince/go-ad-auth.v2.
, MS AD.


[AUTHENTIFICATION]
AuthType = INTERNAL | MSAD | NONE
MSADServer = company.com
MSADPort = 389
MSADBaseDN = OU=company, DC=dc, DC=corp
MSADSecurity = SecurityNone

HTTP Basic Authentication .


JSON Web Token (JWT) . JWTExpiresAt (JWTExpiresAt=0 — ). JWT .


[JWT]
UseJWT = true
JWTExpiresAt = 20000

JWT github.com/dgrijalva/jwt-go.
JWT: , , cookie jwt


JWT :


  • JWT , HTTP Basic Authentication MS AD Authentication
  • JWT , (StatusUnauthorized), JWT
  • JWT POST /signin
    • JSON Web Token Claim ( Claim ),
    • Claims JSON Web Token, HS256
    • Token http Cookie "token". Cookie Token
  • , http Cookie JWT . , StatusUnauthorized
  • JWT POST /refresh
  • logout

7.


7.1. go mod


golang/dep/cmd/dep go mod, vendor . "" . .


go get -u ./../...
go mod vendor

go mod , . commit, :


go get github.com/ibm-messaging/mq-golang/ibmmq@19b946c

, vendor,


go build -v -mod vendor

7.2.


make ( - ):


  • rebuild —
  • build —
  • check - verifica el código usando github.com/golangci/golangci-lint

Al compilar, la versión, la fecha de compilación y la confirmación se incrustan en el archivo ejecutable. Esta información se muestra en un archivo de registro, muy útil para analizar errores.
Para implementar dicha implementación, agregamos variables a main


var (
    version   = "0.0.2" //  ,  
    commit    = "unset" //  commit
    buildTime = "unset" //    
)

Al compilar en un archivo MAKE, solicitamos git para commit


COMMIT?=$(shell git rev-parse --short HEAD)
BUILD_TIME?=$(shell date -u '+%Y-%m-%d_%H:%M:%S')

Al ensamblar, vinculamos estos valores a variables previamente declaradas (opción -ldflags "-X" ), al mismo tiempo especificamos los sistemas operativos bajo los cuales recopilamos y la ubicación de almacenamiento del binario


GOOS=${GOOS} go build -v -a -mod vendor \
-ldflags "-X main.commit=${COMMIT} -X main.buildTime=${BUILD_TIME}" \
-o bin/${GOOS}/${APP}

En main.main, escribo la versión final, la información de confirmación y la fecha de compilación en una variable


app.Version = fmt.Sprintf("%s, commit '%s', build time '%s'", version, commit, buildTime)

More articles:


All Articles