Get best of the week

Blockchain development for industry on Go. Part 1

For four months now, I have been working on a project called “Development of blockchain-based security and data management tools in government and industry”.
Now I would like to talk about how I started this project, and now I will describe the program code in detail.

image

This is the first article in a series of articles. Here I describe the server and protocol. In fact, the reader can even write his own versions of these blockchain elements.

But the second part is about the data structures of the blockchain and transactions, as well as about the package that implements interaction with the database.

Last year, at the Digital Break hackathon, they threw up the idea that for industry and the digital economy to make a useful system for technology of distributed registries, a grant was also issued for development by the Innovation Assistance Fund (about the grant, I should write a separate article for those who are just starting to do startups ), and now in order.

Development takes place in Go, and the database in which the blocks are stored is LevelDB.
The main parts are the protocol, the server (which runs TCP and WebSocket - the first to synchronize the blockchain, the second to connect clients, send transactions and commands from JavaScript, for example.

As mentioned, this blockchain is primarily needed to automate and protect the exchange of products between suppliers and customers, or both in one person. They are in no hurry to trust each other. But the task is not only to create a “checkbook” with a built-in calculator, but a system with automation of most of the routine tasks that arise when working with the product life cycle. The byte code that is responsible for this business, as is customary for blockchains, is stored in the inputs and outputs of transactions (the transactions themselves are in blocks, the blocks in LevelDB are pre-encoded in the GOB format). First, let's talk about the protocol and server (aka node).

The protocol does not work hard, its whole point is that in response to a special command line, switch to the load mode of some data, usually a block or transaction, and it is also needed for exchanging inventory so that the node knows who it is connected to and how they have things to do (the nodes connected for the synchronization session are also called “neighboring” because their IP is known and their state data is stored in memory).

Folders (directories as Linux calls them) are called packages in the understanding of Go-programmers, so at the beginning of each file with Go-code from this directory at the beginning they write package folder_name_where_sit_this_file. Otherwise, the package cannot be fed to the compiler. Well, this is not a secret for those who know this language. These packages are:

  • Networking (server, client, protocol)
  • Structures of stored and transmitted data (block, transaction)
  • (blockchain)
  • (consensus)
  • (xvm)
  • (crypto, types) .

Here is a link to github.

This is a training version, it lacks interprocess communication and several experimental components, for which the structure corresponds to the one under development. If you have something to tell in the comments, I will gladly take into account the further development. And now for the server and protocol packages .

First, take a look at server.

The server routine acts as a data server running on top of the TCP protocol using data structures from the protocol package.

The routine uses the following packages: server , protocol , types . The package itself contains tcp_server.go data structure Serve.

type Serve struct {
	Port string
	BufSize int
	ST *types.Settings
}

It can take the following parameters:

  • Network port through which data will be exchanged
  • JSON server configuration file
  • Run flag in debug mode (private blockchain)

Progress:

  • Read configuration from JSON file
  • The debug mode flag is checked: if it is set, then the network synchronization scheduler is not launched and the blockchain does not load
  • Initializing the configuration data structure and starting the server

Server


  • Starts the TCP server and network communication in accordance with the protocol.
  • It has a Serve data structure consisting of a port number, a buffer size, and a pointer to the types.Settings structure
  • Run ( , handle )
  • handle , protocol.Choice
  • protocol.Choice result . result protocol.Interprete, intrpr — InterpreteData,
  • Then the switch is executed by intrpr.Commands [0] in which one of the following is checked: result, inv, error and there is a default section
  • In the result section, there is a switch by the value of intrpr.Commands [1] which checks the values ​​of bufferlength and version (in each case the corresponding function is called)

The GetVersion and BufferLength functions are located in the srvlib.go file of the server package.

GetVersion(conn net.Conn, version string)

just prints to the console and sends the client the version passed in the parameter:

conn.Write([]byte("result:" + version))
.
Function

BufferLength(conn net.Conn, intrpr *protocol.InterpreteData)

performs loading of a block, transaction or other specific data as follows:

  • Prints on the console the type of data specified in the protocol that needs to be accepted:

    fmt.Println("DataType:", intrpr.Commands[2])
  • Reads the intrpr.Body in numeric variable buf_len
  • Creates a newbuf buffer of the specified size:

    make([]byte, buf_len)
  • Sends ok response:

    conn.Write([]byte("result:ok"))
  • Produces a full buffer from the read stream:

    io.ReadFull(conn, newbuf)
    .
  • Prints the contents of the buffer to the console

    fmt.Println(string(newbuf))

    and the number of bytes read

    fmt.Println("Bytes length:", n)
  • Sends ok response:

    conn.Write([]byte("result:ok"))

The methods from the server package are configured in such a way that they process the received data with functions from the protocol package .

Protocol


The protocol serves as a tool that presents data during network communication.

Choice (str string) (string, error) performs the initial processing of the data received by the server, receives a string representation of the data and returns the string prepared for Interprete :

  • The input string is split into head and body using ReqParseN2 (str)
  • head is broken into elements and placed in the commands slice using ReqParseHead (head)
  • In switch (commands [0]) we select the received command ( cmd, key, address or the default section is triggered )
  • In cmd, 2 switch commands (commands [1]) are checked - length and getversion .
  • length checks the data type in commands [2] and stores it in datatype
  • Checks that body contains a string value

    len(body) < 1
  • Returns a response string:

    "result:bufferlength:" + datatype + "/" + body
  • getversion returns a string

    return "result:version/auto"

Interprete


It contains the InterpreteData structure and performs secondary processing of the string returned from Choice and the formation of the InterpreteData object .

type InterpreteData struct {
	Head string
	Commands []string
	Body string
	IsErr bool
	ErrCode int 
	ErrMessage string
}

Function

Interprete(str string) (*InterpreteData, error)

accepts the string result and creates a reference to the InterpreteData object .

Progress:

  • Similarly, Choice retrieves head and body using ReqParseN2 (str)
  • head is broken into elements using ReqParseHead (head)
  • The InterpreteData object is initialized and a pointer to it is returned:

res := &InterpreteData{
	Head: head,
	Commands: commands,
	Body: body,
}
return res, nil

This object is used in server.go of the main package.

Client


The client package contains the TCPConnect and TCPResponseData functions .

Function

TCPConnect(s *types.Settings, data []byte, payload []byte)

works as follows:

  • A connection is made to the connection specified in the transferred settings object

    net.Dial("tcp", s.Host + ":" + s.Port)
  • The data passed in the data parameter is transferred:

    conn.Write(data)
  • Read answer

    resp, n, _ := TCPResponseData(conn, s.BufSize)

    and printed on the console

    fmt.Println(string(resp[:n]))
  • If payload is transmitted then it is transmitted

    conn.Write(payload)

    and also reads the server response, printing it on the console

Function

 TCPResponseData(conn net.Conn, bufsiz int) ([]byte, int, error)

creates a buffer of the specified size, reads the server response there and returns this buffer and the number of bytes read, as well as an error object.

Client routine


Serves to transmit commands to the node servers, as well as to obtain brief statistics and testing.

It can take the following parameters: a configuration file in JSON format, data to be transmitted to the server as a string, a path to the file to send to payload, a node scheduler emulation flag, the type of data to be transferred as a numerical value.

  • Get the configuration

    st := types.ParseConfig(*config)
  • If you are given an emu flag starts sheduler
  • If the flag f is specified indicating the path to the file, then load its data in fdb and send the contents to the server

    client.TCPConnect(st, []byte(CMD_BUFFER_LENGTH + ":" + strconv.Itoa(*t) + "/" + strconv.Itoa(fdblen)), fdb)
  • If the file is not specified, then data from the -d flag is simply sent :

    client.TCPConnect(st, []byte(*data), nil)

All of this is a simplified view showing the structure of the protocol. During development, the necessary functionality is added to its structure.

In the second part I will talk about data structures for blocks and transactions, in 3 about a WebSocket server for connecting from JavaScript, in 4 we will consider a synchronization scheduler, then a stack machine that processes bytecode from inputs and outputs, cryptography and output pools.

More articles:


All Articles