🌥️ ⏏️ 🦀 Templat server backend Golang - bagian 1 (server HTTP) 📰 ☺️ 🏯

Templat server Golang di bawah ini disiapkan untuk mentransfer pengetahuan dalam tim kami. Tujuan utama dari template, selain pelatihan, adalah untuk mengurangi waktu untuk membuat prototipe tugas server kecil di Go.

Template termasuk:

Melewati parameter server melalui baris perintah github.com/urfave/cli
Mengonfigurasi pengaturan server melalui file konfigurasi github.com/sasbury/mini
Konfigurasikan pengaturan server HTTP TLS
Mengkonfigurasi router yang mendaftarkan HTTP dan prof-handler github.com/gorilla/mux
Mengatur level logging tanpa menghentikan server github.com/hashicorp/logutils
Mengkonfigurasi logging traffic HTTP tanpa menghentikan server
Menyetel kesalahan saat masuk dalam respons HTTP tanpa menghentikan server
Otentikasi Dasar HTTP
Otentikasi MS AD gopkg.in/korylprince/go-ad-auth.v2
Token Web JSON github.com/dgrijalva/jwt-go
Memulai server dengan harapan kembali ke saluran kesalahan
Menggunakan konteks untuk menghentikan server dan layanan terkait dengan benar
Penyiapan penanganan kesalahan khusus github.com/pkg/errors
Pengaturan Logging Kustom
Bangun dengan versi, bangun tanggal, dan komit

Tautan ke repositori proyek .

HTTP (. HTTP Golang):

POST /echo — request HTTP body response
POST /signin — JSON Web Token Cookie
POST /refresh — JSON Web Token Cookie
POST /httplog — HTTP
POST /httperrlog — HTTP response
POST /loglevel — DEBUG, INFO, ERROR

2.2.
2.3.
,
3.1. daemon
3.2. daemon
3.3. daemon
4.1.
4.2.
4.3.
4.4.
5.1.
5.2.
5.3.
5.4. HTTP
7.1. go mod
7.2.

1.

1:ERP, — 1 IBM MQ.
IBM MQ :

IBM MQ (, , )
IBM MQ
IBM MQ (SYNCPOINT)

XML :

() RFC 3076
HMAC - -34.11.94
HMAC

1C IBM MQ . REST API IBM MQ .

1C IBM MQ Go IBM MQ. , , "" C . 2 [].

backend , .

1 IBM MQ .

REST_IBMMQ

2.

2.1.

, , . github.com/urfave/cli. :

   --httpconfig value, --httpcfg value    HTTP Config file name
   --listenstring value, -l value         Listen string in format <host>:<port>
   --httpuser value, --httpu value        User name for access to HTTP server
   --httppassword value, --httppwd value  User password for access to HTTP server
   --jwtkey value, --jwtk value           JSON web token secret key
   --debug value, -d value                Debug mode: DEBUG, INFO, ERROR
   --logfile value, --log value           Log file name

2.2.

github.com/sasbury/mini. , :

[HTTP_SERVER]
ReadTimeout = 6000          // HTTP read timeout duration in sec - default 60 sec
WriteTimeout = 6000         // HTTP write timeout duration in sec - default 60 sec
IdleTimeout = 6000          // HTTP idle timeout duration in sec - default 60 sec
MaxHeaderBytes = 262144     // HTTP max header bytes - default 1 MB
MaxBodyBytes = 1048576      // HTTP max body bytes - default 0 - unlimited
UseProfile = false          // use Go profiling
ShutdownTimeout = 30        // service shutdown timeout in sec - default 30 sec

[TLS]
UseTLS = false                  // use SSL
UseHSTS = false                 // use HTTP Strict Transport Security
TLSertFile = certs/server.pem  // TLS Certificate file name
TLSKeyFile = certs/server.key   // TLS Private key file name
TLSMinVersion = VersionTLS10    // TLS min version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30
TLSMaxVersion = VersionTLS12    // TLS max version VersionTLS13, VersionTLS12, VersionTLS11, VersionTLS10, VersionSSL30

[JWT]
UseJWT = false                  // use JSON web token (JWT)
JWTExpiresAt = 20000            // JWT expiry time in seconds - 0 without restriction

[AUTHENTIFICATION]
AuthType = INTERNAL             // Autehtification type NONE | INTERNAL | MSAD
MSADServer = company.com        // MS Active Directory server
MSADPort = 389                  // MS Active Directory Port
MSADBaseDN = OU=, DC=, DC=      // MS Active Directory BaseDN
MSADSecurity = SecurityNone     // MS Active Directory Security: SecurityNone, SecurityTLS, SecurityStartTLS

[LOG]
HTTPLog = false                         // Log HTTP traffic
HTTPLogType = INREQ                     // HTTP trafic log mode INREQ | OUTREQ | INRESP | OUTRESP | BODY
HTTPLogFileName = ./httplog/http%s.log  // HTTP log file
HTTPErrLog = HEADER | BODY              // Log error into HTTP response header and body

3. ,

UML .

, daemon. :

context.Context
/

3.1. daemon

, daemon.
, daemon .
, daemon :

httpserverErrCh: make(chan error, 1), //   HTTP

, :

daemon —
— daemon

IBM MQ:
- IBM MQ, ,
- IBM MQ
PostgreSQL:
- , , SQL
JSON BoltDB:
- BoltDB
- ( PostgreSQL). , ( BoltDB 150 2 64 , BoltDB NVMe 0.03 ms).
HTTP :
- http.server
- TCP
- TLS
- HTTP
- pprof

3.2. daemon

daemon .
, — , , , .
, . , HTTP

go func() { httpserverErrCh <- d.httpserver.Run() }()

, daemon .
daemon , , - . , ( ) .
, IBM MQ , daemon IBM MQ .

syscalCh := make(chan os.Signal, 1) //   
signal.Notify(syscalCh, syscall.SIGINT, syscall.SIGTERM)

//       
select {
case s := <-syscalCh: //  
    mylog.PrintfInfoMsg("Exiting, got signal", s)
    d.Shutdown() //  daemon
    return nil
case err := <-d.httpserverErrCh: //   HTTP    
    mylog.PrintfErrorInfo(err) //  
    d.Shutdown() //  daemon
    return err
}

, , ( ). , daemon.

func (s *Server) Run() error {
    defer func() {
        var myerr error
        r := recover()
        if r != nil {
            msg := "Recover from panic"
            switch t := r.(type) {
            case string:
                myerr = myerror.New("8888", msg, t)
            case error:
                myerr = myerror.WithCause("8888", msg, t)
            default:
                myerr = myerror.New("8888", msg)
            }
            mylog.PrintfErrorInfo(myerr) //  
            s.errCh <- myerr             //       daemon
        }
    }()

    //  
}

3.3. daemon

daemon .

, , :

( ShutdownTimeout )
. IBM MQ :
- IBM MQ
stopCh

, :

. ,
, , , ,

for {
    select {
    case <-ctx.Done(): //    

        //  

        s.stopCh <- struct{}{} //     
        return
    default:
        //   
    }
}

HTTP :

//        ShutdownTimeout
cancelCtx, cancel := context.WithTimeout(s.ctx, time.Duration(s.cfg.ShutdownTimeout*int(time.Second)))
defer cancel()

//       ShutdownTimeout
if err := s.httpServer.Shutdown(cancelCtx); err != nil {
    return err
}

s.httpService.Shutdown() //   

//     HTTP 
s.stopCh <- struct{}{}

4.

4.1.

github.com/pkg/errors.
, , .

type Error struct {
    ID       uint64 //   
    Code     string //  
    Msg      string //  
    Caller   string // ,        
    Args     string //  
    CauseErr error  //  - 
    CauseMsg string //   - 
    Trace    string //      
}

, . , 1 IBM MQ, 4 . , "8" HTTP, "7" — IBM MQ.

Caller — , . , . :

httpserver.go:[209] - (*Server).Run()

Caller

func caller(depth int) string {
    pc := make([]uintptr, 15)
    n := runtime.Callers(depth+1, pc)
    frame, _ := runtime.CallersFrames(pc[:n]).Next()
    idxFile := strings.LastIndexByte(frame.File, '/')
    idx := strings.LastIndexByte(frame.Function, '/')
    idxName := strings.IndexByte(frame.Function[idx+1:], '.') + idx + 1

    return frame.File[idxFile+1:] + ":[" + strconv.Itoa(frame.Line) + "] - " + frame.Function[idxName+1:] + "()"
}

Args — , . .

CauseErr CauseMsg — . , .

Trace — . . github.com/pkg/errors '%+v'. .

fmt.Sprintf("'%+v'", pkgerr.New(""))

4.2.

Error, .

func (e *Error) Error() string {
    mes := fmt.Sprintf("ID=[%v], code=[%s], mes=[%s]", e.ID, e.Code, e.Msg)
    if e.Args != "" {
        mes = fmt.Sprintf("%s, args=[%s]", mes, e.Args)
    }
    if e.CauseMsg != "" {
        mes = fmt.Sprintf("%s, causemes=[%s]", mes, e.CauseMsg)
    }
    return mes
}

ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3']

// %s    print the error code, message, arguments, and cause message.
// %v    in addition to %s, print caller
// %+v   extended format. Each Frame of the error's StackTrace will be printed in detail.
func (e *Error) Format(s fmt.State, verb rune) {
    switch verb {
    case 'v':
        fmt.Fprint(s, e.Error())
        fmt.Fprintf(s, ", caller=[%s]", e.Caller)
        if s.Flag('+') {
            fmt.Fprintf(s, ", trace=%s", e.Trace)
            return
        }
    case 's':
        fmt.Fprint(s, e.Error())
    case 'q':
        fmt.Fprint(s, e.Error())
    }
}

'%+v'

ID=[1], code=[8004], mes=[Error message], args=['arg1', 'arg2', 'arg3'], caller=[handler_echo.go:[31] - (*Service).EchoHandler.func1()], trace='
github.com/romapres2010/httpserver/error.New
        D:/golang/src/github.com/romapres2010/httpserver/error/error.go:72
github.com/romapres2010/httpserver/httpserver/httpservice.(*Service).EchoHandler.func1
        D:/golang/src/github.com/romapres2010/httpserver/httpserver/httpservice/handler_echo.go:31
        ...

. .
— Format.

4.3.

New(code string, msg string, args ...interface{}) error

WithCause(code string, msg string, causeErr error, args ...interface{}) error

, args ...interface{}.

. , :

myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)

4.4.

, INFO trace. , , , ,
WithCause()
INFO. .
, ERROR , trace.

HTTP :

requestBuf, err := ioutil.ReadAll(r.Body)
if err != nil {
    myerr = myerror.WithCause("8001", "Failed to read HTTP body: reqID", err, reqID)
    mylog.PrintfErrorInfo(myerr) //    
    s.processError(myerr, w, http.StatusInternalServerError, reqID) //     HTTP response
    return myerr
}

processError HTTP / .
— HTTPErrLog, POST /httperrlog.

POST /httperrlog HTTP/1.1
HTTP-Err-Log: HEADER | BODY

HTTP header

// carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0)
headerReplacer := strings.NewReplacer("\x0a", " ", "\x0d", " ", "\x00", " ")

w.Header().Set("Err-Trace", headerReplacer.Replace(myerr.Trace))

5.

github.com/hashicorp/logutils.
RFC 5424 — The Syslog Protocol, :

debug —
info — , , /
error — ,

— 100

Dave Cheney Let’s talk about logging.
— A simple logging interface for Go.

5.1.

"log". github.com/hashicorp/logutils.

// logFilter represent a custom logger seting
var logFilter = &logutils.LevelFilter{
    Levels:   []logutils.LogLevel{"DEBUG", "INFO", "ERROR"},
    MinLevel: logutils.LogLevel("INFO"), // initial setting
    Writer:   os.Stderr,                 // initial setting
}

// InitLogger init custom logger
func InitLogger(wrt io.Writer) {
    logFilter.Writer = wrt   // custom logger
    log.SetOutput(logFilter) // set std logger to our custom
}

, , main MultiWriter

//     
if logFileFlag != "" {
    //        
    logFileFlag = strings.Replace(logFileFlag, "%s", time.Now().Format("2006_01_02_150405"), 1)

    //        APPEND
    logFile, err := os.OpenFile(logFileFlag, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
    if err != nil {
        myerr := myerror.WithCause("6020", "Error open log file: Filename", err, logFileFlag)
        mylog.PrintfErrorMsg(fmt.Sprintf("%+v", myerr))
        return myerr
    }
    if logFile != nil {
        defer logFile.Close()
    }

    wrt := io.MultiWriter(os.Stderr, logFile) //    os.Stderr  

    mylog.InitLogger(wrt) //     
} else {
    mylog.InitLogger(os.Stderr)
}

5.2.

INFO DEBUG .

2020/03/10 17:56:39 [INFO] - httpserver.go:[61] - New() - Creating new HTTP server
2020/03/10 17:56:39 [INFO] - httpserver.go:[141] - New() - Created new TCP listener: network = 'tcp', address ['127.0.0.1:3000']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/echo', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/signin', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[155] - New() - Handler is registered: Path, Method ['/refresh', 'POST']
2020/03/10 17:56:39 [INFO] - httpserver.go:[179] - New() - HTTP server is created
2020/03/10 17:56:39 [INFO] - daemon.go:[121] - New() - New daemon is created
2020/03/10 17:56:39 [INFO] - daemon.go:[128] - (*Daemon).Run() - Starting daemon
2020/03/10 17:56:39 [INFO] - daemon.go:[133] - (*Daemon).Run() - Daemon is running. For exit <CTRL-c>
2020/03/10 17:56:39 [INFO] - httpserver.go:[209] - (*Server).Run() - Starting HTTP server
2020/03/10 17:57:22 [INFO] - daemon.go:[142] - (*Daemon).Run() - Exiting, got signal ['interrupt']
2020/03/10 17:57:22 [INFO] - daemon.go:[155] - (*Daemon).Shutdown() - Shutting down daemon
2020/03/10 17:57:22 [INFO] - httpserver.go:[215] - (*Server).Shutdown() - Waiting for shutdown HTTP Server: sec ['30']
2020/03/10 17:57:22 [INFO] - httpserver.go:[231] - (*Server).Shutdown() - HTTP Server shutdown successfuly
2020/03/10 17:57:22 [INFO] - daemon.go:[167] - (*Daemon).Shutdown() - Daemon is shutdown
2020/03/10 17:57:22 [INFO] - main.go:[163] - main.func1() - Server is shutdown

ERROR .

5.3.

func PrintfInfoMsg(mes string, args ...interface{}) {
    printfMsg("[INFO]", 0, mes, args...)
}
func PrintfDebugMsg(mes string, args ...interface{}) {
    printfMsg("[DEBUG]", 0, mes, args...)
}
func PrintfErrorMsg(mes string, args ...interface{}) {
    printfMsg("[ERROR]", 0, mes, args...)
}

, .
— DEBUG .

INFO DEBUG, .

mylog.PrintfInfoMsg(fmt.Sprintf("Create new TCP listener network='tcp', address='%s'", serverCfg.ListenSpec))

mylog.PrintfInfoMsg("Created new TCP listener: network = 'tcp', address", cfg.ListenSpec)

, , DEBUG INFO 1%.
— INFO DEBUG POST /loglevel.

POST /loglevel HTTP/1.1
Log-Level-Filter: DEBUG | ERROR | INFO

5.4. HTTP

, HTTP .
httplog HTTP header body:

— HttpLog HTTPLogType, POST /httplog.

POST /httplog HTTP/1.1
Http-Log: TRUE
Http-Log-Type: INREQ | OUTREQ | INRESP | OUTRESP | BODY

6.

: HTTP Basic Authentication MS AD Authentication.
MS AD Authentication gopkg.in/korylprince/go-ad-auth.v2.
, MS AD.

[AUTHENTIFICATION]
AuthType = INTERNAL | MSAD | NONE
MSADServer = company.com
MSADPort = 389
MSADBaseDN = OU=company, DC=dc, DC=corp
MSADSecurity = SecurityNone

HTTP Basic Authentication .

JSON Web Token (JWT) . JWTExpiresAt (JWTExpiresAt=0 — ). JWT .

[JWT]
UseJWT = true
JWTExpiresAt = 20000

JWT github.com/dgrijalva/jwt-go.
JWT: , , cookie jwt

JWT :

JWT , HTTP Basic Authentication MS AD Authentication
JWT , (StatusUnauthorized), JWT
JWT POST /signin
- JSON Web Token Claim ( Claim ),
- Claims JSON Web Token, HS256
- Token http Cookie "token". Cookie Token
, http Cookie JWT . , StatusUnauthorized
JWT POST /refresh
logout

7.

7.1. go mod

golang/dep/cmd/dep go mod, vendor . "" . .

go get -u ./../...
go mod vendor

go mod , . commit, :

go get github.com/ibm-messaging/mq-golang/ibmmq@19b946c

, vendor,

go build -v -mod vendor

7.2.

make ( - ):

rebuild —
build —
check - periksa kode menggunakan github.com/golangci/golangci-lint

Saat kompilasi, versi, tanggal pembuatan, dan komit tertanam dalam file yang dapat dieksekusi. Informasi ini ditampilkan dalam file log - sangat berguna untuk kesalahan parsing.
Untuk mengimplementasikan implementasi seperti itu, kami menambahkan variabel ke main

var (
    version   = "0.0.2" //  ,  
    commit    = "unset" //  commit
    buildTime = "unset" //    
)

Ketika mengkompilasi dalam makefile, kami meminta git untuk melakukan

COMMIT?=$(shell git rev-parse --short HEAD)
BUILD_TIME?=$(shell date -u '+%Y-%m-%d_%H:%M:%S')

Saat merakit, kami mengikat nilai-nilai ini ke variabel yang dideklarasikan sebelumnya (option -ldflags "-X" ), pada saat yang sama menentukan OS di mana kami mengumpulkan dan lokasi penyimpanan biner

GOOS=${GOOS} go build -v -a -mod vendor \
-ldflags "-X main.commit=${COMMIT} -X main.buildTime=${BUILD_TIME}" \
-o bin/${GOOS}/${APP}

Di main.main, saya menulis versi final, mengkomit dan membangun informasi tanggal menjadi variabel

app.Version = fmt.Sprintf("%s, commit '%s', build time '%s'", version, commit, buildTime)

Templat server backend Golang - bagian 1 (server HTTP)

1.

2.