Hyperledger Fabric is an open source blockchain platform from the Linux Foundation. With it, you can create blockchain applications and control access and permissions for data in the blockchain. Hyperledger Fabric should be used if you want to create a private blockchain network or set restrictions on what transactions each side can see.In the documentation of Hyperledger Fabric, the creation of a blockchain network on one machine using Docker is well-planned, where each network member is a separate container. But the process of setting up a network on several physical or virtual machines is not described. This will be discussed in this article.Our blockchain application is a service for storing some medical data of school and preschool children. Data must be unchanged. They will be presented in the form of information about vaccination and the patient’s agreement with a particular doctor. The network will include members of Parents (an organization that represents the interests of parents), Hospital (a hospital with whose pediatricians parents can enter into an agreement to care for their child / children) and Kindergarten (a kindergarten that can request health reports and vaccinations for or another child at the hospital).Amazon managed blockchain
AWS has an Amazon Managed Blockchain service. It allows you to deploy blockchain networks using the AWS console user interfaces. But, having gained some experience working with Amazon Managed Blockchain, we had to abandon the idea of ​​using the service in commercial projects. There were several reasons for this. The main ones are:- Only the version of Hyperledger Fabric 1.2 was available in the Amazon Managed Blockchain, and at the time of this writing, version 1.4 is used to build enterprise blockchain networks. It also does not allow the use of modern versions of the Node.js SDK tools;
- Amazon Managed Blockchain does not support the CouchDB database, making it difficult to create applications that index stored data.
Therefore, our opinion is that it is better to use virtual or physical servers to build networks using the Hyperledger Fabric framework.About app channels
Hyperledger Fabric has the concept of "channel." A channel is a subnet for conducting sensitive transactions. There will be two channels in our application.- "Parentshospital". It is intended for transactions between members of Parents and Hospital organizations: creating and modifying medical records, creating an agreement with a pediatrician, and signing a pediatrician agreement.
- "Parentshospitalkindergarten." Within this channel, members of Parents and Hospital will interact with members of Kindergarten. Also, reports will be generated based on data from medical records.
Work environment
We will deploy our blockchain network using Docker Swarm - this is the native Docker system for deploying a host cluster. If you have never worked with Docker (and especially with Docker Swarm), then first check out this product and how to orchestrate containers using Docker Swarm. The documentation describes the basics of working with these tools well.Our entire network will be located on AWS EC2 servers. Information within the blockchain network will be stored in CouchDB. Hyperledger Fabric also supports the LevelDB database, but CouchDB, unlike the latter, makes it possible to search by any criterion in a document collection. We will also use Fabric Certificate Authority, a modular component for managing network identifiers of all organizations and their users. It provides registration of network participants and the issuance of certificates.We will have four EC2 servers:- Orderer, MySQL DB (for storing application information that does not apply to medical records)
- Parents organization (Peer0, Peer1), CouchDB, Fabric-CA, CLI
- Hospital organization (Peer0, Peer1), CouchDB, Fabric-CA
- Kindergarten organization (Peer0, Peer1), CouchDB, Fabric-CA
Creating EC2 Servers on AWS
We need t2.medium services to work. This type of EC2 server has sufficient computing power to install all the necessary software and work with the Hyperledger Fabric tool. In the Configure Security Group, set All traffic, and Source - 0.0.0.0/0, :: / 0.
Fig. 1. Configure Security Group in the AWS panel.We got four EC2 servers:- ec2-18-232-164-119.compute-1.amazonaws.com
- ec2-54-145-203-186.compute-1.amazonaws.com
- ec2-54-80-241-117.compute-1.amazonaws.com
- ec2-52-87-193-235.compute-1.amazonaws.com
You will naturally have your own paths to EC2 servers.
Fig. 2. Prepared for operation EC2 servers.Installing prerequisite software on EC2 servers
Now on each EC2 server we need to install Docker and Git. For convenience, open four tabs in the terminal. Let's go to each EC2 service via SSH:Terminal 1:HOST1=ec2-18-232-164-119.compute-1.amazonaws.com
ssh ec2-user@$HOST1 -i ./key-poc.pem
Terminal 2:HOST2=ec2-54-145-203-186.compute-1.amazonaws.com
ssh ec2-user@$HOST2 -i ./key-poc.pem
Terminal 3:HOST3=ec2-54-80-241-117.compute-1.amazonaws.com
ssh ec2-user@$HOST3 -i ./key-poc.pem
Terminal 4:HOST4=ec2-52-87-193-235.compute-1.amazonaws.com
ssh ec2-user@$HOST4 -i ./key-poc.pem
In each terminal, execute the following commands:sudo yum update -y
sudo yum install -y docker
sudo service docker start
sudo docker info
sudo yum install git
sudo usermod -a -G docker ec2-user
newgrp docker
Now we have Docker and Git installed.Network
We need to clone elements of our network from the repository. To do this, execute the command in each terminal:git clone https://github.com/nixsolutions/blockchain-poc-network.git network
We also need to clone smart contracts from the repository that will regulate the relationships between network participants. Smart contracts are written in the Golang language.We perform these steps on each of the four servers:cd network && mkdir chaincode && cd chaincode
git clone https://github.com/nixsolutions/blockchain-poc-medical-contract.git medical-contract
git clone https://github.com/nixsolutions/blockchain-poc-kindergarten-contract.git kindergarten-contract
In the first terminal, we need to generate channel configuration files and certificates for connecting organizations to the network:cd network/poc-network
./1_generate_connection_files.sh
./2_generating_channel_configuration.sh
Now copy the channel-artifacts and crypto-config folders to the other three hosts in the poc-network folder of the network parent directory. Please note: when copying organizations to the EC2 server, you need to remove third-party certificates from the crypto-config / peerOrganizations folder that relate to the work of other organizations. For example, when copying to the Parents host, we need to leave the parents.poc-network.com folder in this folder, but delete hospital.poc-network.com and kindergarten.poc-network.com.Creating a Docker Swarm Stack
In order for containers that belong to different organizations and are located on different EC2 servers to be able to access each other, we need to combine them into a stack. Let's look at our file network / poc-network / docker-compose-general.yaml. It contains the configuration of services, indicates which hosts a particular service (key node.hostname) will be deployed to , aliases (key aliases) are indicated .We need to initialize swarm in the first terminal:docker swarm init
docker swarm join-token manager
A token will appear to attach another EC2 server to the stack. Something like: docker swarm join --token SWMTKN-1-42ml0ohnnbidg8kflgp8xp9dkkus6mn1lslqc15hrxj4tk9e3q-5h4vbzbfk8p90n83oe08gbltf 172.31.46.214:2377.Now we can execute in the remaining terminals:docker swarm join --token SWMTKN-1-2xzco7t7txohnzd09318eczpbgmm8woex80byxptpt1jl5i2ar-bsg37h40xze1gaabg80i96gw2 172.31.38.245:2377
A message appears in the console:This node joined a swarm as a manager.
After all three hosts are connected to Swarm, we can see them in the terminal of the first host by running the command:docker node ls
Fig. 3. List of servers in Docker Swarm.In the network / poc-network folder there is a .env.template file in which you need to specify a hostname for each of the hosts:ORDERER=ip-172-31-38-245PARENTS=ip-172-31-43-64HOSPITAL=ip-172-31-38-130KINDERGARTEN=ip-172-31-40-157To generate a .env file, you need to run the ./3_env_gen.sh file.Now create a network with a driver overlay:docker network create --driver overlay --attachable stage_byfn
Run Swarm to create a stack based on the docker-compose-general.yaml file:env $(cat .env | grep ^[A-Z] | xargs) docker stack deploy -c docker-compose-general.yaml stage 2>&1
With this command, we also set environment variables, which are described in the .env file and which are necessary for the network to work correctly.You can execute the command docker service lsin the first terminal. You will see a list of all the services that are running on all our EC2 instances.Fig. 4. List of all services that were running on all EC2 servers.If you see the same thing in your terminal as in the screenshot, then everything is fine. Now run our network. In the second terminal, you must enter the CLI container (after all, this container is deployed on the second host):docker exec -ti stage_cli.1.owni217t53m53efjtikb5oa2f /bin/bash
The container name can be seen by executing the command in the terminal docker ps.In the container, we execute all our commands from the 4_create_channels.sh file, copying them one by one to the terminal. In the bin folder, we have the binaries for creating the network:- peer channel create - create channels;
- peer channel join - joining a peer to a channel;
- peer channel update - update configurations;
- peer chaincode install - install the codecode;
- peer chaincode instantiate - deploy the specified chain code on the network.
Now you can leave the container by running the command exit.Parents Back Application
Install the application on the second host:cd ~
git clone https://github.com/nixsolutions/blockchain-poc-backend.git back
We also need to install NodeJS and gcc-c ++ (compiler):curl -sL https://rpm.nodesource.com/setup_10.x | sudo bash -
sudo yum install nodejs
cd back
sudo yum install gcc-c++ -y
npm install
We can also enter our database through adminer with the following authorization data:<HOST_1>:3306(where <HOST_1>is the address of the first EC2 server on which the database is stored)devdevpasspoc_databaseIt is also necessary to connect our API application to the database:vim ~/back/config/config.json
Assign the correct value to the key host. Our back-application has prepared migrations and seeds. You need to run them:npx sequelize-cli db:migrate
npx sequelize-cli db:seed:all
Tables should appear in the database. And in the table of organizations we will see three rows with our organizations - parents, hospital, kindergarten.When generating certificates at the very beginning, we received files for connecting to the network for each organization on the first host. Copy the connection-parents.json file from the first host to the second. To do this, open another terminal from the folder where ours keyis located and execute the commands:HOST1=ec2-18-232-164-119.compute-1.amazonaws.com
HOST2=ec2-54-145-203-186.compute-1.amazonaws.com
scp -i ./key-poc.pem -r ec2-user@$HOST1:~/network/poc-network/connection-parents.json ~
scp -i ./key-poc.pem ~/connection-parents.json ec2-user@$HOST2:~/network/poc-network
Fix exports.hostNamein constants.js (value for the current host):vim ~/back/constants.js
Run our application in the second terminal:npm start
In the userGenerator.sh file, assign the HOST_2correct value to the variable :vim ~/back/generators/userGenerator.sh
Next, you need to run the script that will create the users of the Parents organization:./back/generators/userGenerator.sh parents
Run the script to generate medical records:node ~/back/generators/setCards.js
Two cards will be created in the blockchain, which will also be written into the SQL database in the cards table.Hospital back application
By analogy with the previous back-application, we need to place the file to connect to the network on the third host. To do this, in the fifth terminal, execute:scp -i ./key-poc.pem -r ec2-user@$HOST1:~/network/poc-network/connection-hospital.json ~
scp -i ./key-poc.pem ~/connection-hospital.json ec2-user@$HOST3:~/network/poc-network
We will clone our application to the third host:cd ~
git clone https://github.com/nixsolutions/blockchain-poc-backend.git back
Install NodeJS and gcc-c ++ (compiler):curl -sL https://rpm.nodesource.com/setup_10.x | sudo bash -
sudo yum install nodejs
cd back
sudo yum install gcc-c++ -y
npm install
Fix exports.hostNamein constants.js (value for the current host):vim ~/back/constants.js
Run our application in the third terminal:npm start
In the userGenerator.sh file, assign the HOST_3correct value to the variable :vim ~/back/generators/userGenerator.sh
Next, we need to run a script that will create users of the Hospital organization:./back/generators/userGenerator.sh hospital
We now have a second user in the Users table in the database.Kindergarten back application
By analogy with the previous back-application, we need to place the file to connect to the network on the third host. To do this, in the fifth terminal, execute:scp -i ./key-poc.pem -r ec2-user@$HOST1:~/network/poc-network/connection-kindergarten.json ~
scp -i ./key-poc.pem ~/connection-kindergarten.json ec2-user@$HOST4:~/network/poc-network
We will clone our application to the third host:cd ~
git clone https://github.com/nixsolutions/blockchain-poc-backend.git back
Install NodeJS and Gcc-c ++ (compiler):curl -sL https://rpm.nodesource.com/setup_10.x | sudo bash -
sudo yum install nodejs
cd back
sudo yum install gcc-c++ -y
npm install
Fix exports.hostNamein constants.js (value for the current host):vim ~/back/constants.js
Run our application in the third terminal:npm start
In the userGenerator.sh file, assign the HOST_4correct value to the variable :vim ~/back/generators/userGenerator.sh
Next, we need to run a script that will create users of the Hospital organization:./back/generators/userGenerator.sh kindergarten
We now have a third user in the Users table in the database.Install wkhtmltopdf on the second, third and fourth EC2 servers
The back-application uses the wkhtmltopdf library to generate a PDF report on the child’s health status.We need to install libpng - the official library for working with raster graphics in PNG format. The library is platform independent and consists of functions written in C language.The first team will install wkhtmltox - the open source LGPLv3 library for rendering HTML to PDF and various image formats using the QtWebKit rendering engine:wget https://downloads.wkhtmltopdf.org/0.12/0.12.5/wkhtmltox-0.12.5-1.centos7.x86_64.rpm
sudo yum install -y wkhtmltox-0.12.5-1.centos7.x86_64.rpm
The second block of commands will download and deploy the libpng15 library:wget https://sourceforge.net/projects/libpng/files/libpng15/older-releases/1.5.15/libpng-1.5.15.tar.gz/download -O libpng-1.5.15.tar.gz
tar -zxvf libpng-1.5.15.tar.gz
cd libpng-1.5.15
The third block of commands is needed to install the compiler:sudo yum groupinstall -y "Development Tools"
./configure --prefix=/usr
sudo make install
You still need to install LibTIFF:sudo yum install -y libtiff
We can test:wkhtmltopdf https://majestic.cloud majestic.pdf
Now you can generate and view PDF documents.A little bit about Node SDK
NodeJS-tools for working with the Hyperledger Fabric network are presented in the form of packages "fabric-ca-client -v 1.4.0" and "fabric-network" v 1.4.0 ". All services of the backend application that interact with the network using the Node SDK are located in the Services folder, at the root of the project.You can read about these tools and their methods in the official documentation .Application business processes
Now we’ll talk about how network members interact with each other in this application. All processes are described in a UML diagram. Let's consider each of them in more detail.Fig.5. A graphical description of the business processes in the application.So, there are several organizations on the network: Parents, Hospital and Kindergarten.A member of Parents creates electronic health records for two of their children. This data will be stored (and supplemented) in the blockchain network. Next, the parent sends a request for an agreement with the pediatrician, who will observe his children until they reach adulthood (in the diagram - request for an agreement with a doctor ).A member of the hospital organization signs an agreement (in the diagram - doctor signs an agreement ).Next, the parent sends a request to kindergarten with a request to enroll his child (in the diagram - request for kindergarten to receive a child ).Fig. 6. A page with a list of available educational institutions in the parent's account.Kindergarten sees the request in the list of new requests:Fig. 7. A page with a list of applications for admission to the personal account of the educational institution.Kindergarten before deciding on the admission of the child must make sure his satisfactory state of health. To do this, the kindergarten sends a request to the clinic in which this child is observed (in the diagram - child health report request ). The request to the pediatrician is executed automatically after the request for admission to the garden is moved to the "Accepted" tab.Fig. 8. A page with a list of approved applications for admission to the personal account of the educational institution.The pediatrician sees the request and generates a report that contains data on the child’s health (in the diagram - child health report create ).Fig. 9. Page with a list of applications for creating a report on the state of the child’s health in the personal account of the hospital.The parent sees that the pediatrician has generated a report:Fig. 10. The page in the parent’s personal account, which displays the status of reports on the child’s health... and makes a decision on granting access to the preschool educational institution to view and download the report (in the diagram - give permission to view the report ).Fig. 11. A pop-up window for choosing an educational institution that can view the generated medical report.Parent clicks "Confirm":Fig. 12. Notification that the selected educational institution has received permission to view the medical report.In the list of reports in the application of the educational institution, information appears that the report has been generated and is available for viewing and downloading in PDF format.Fig. 13. Link to download the report in PDF format in the line with the approved application.Here is a report you will see in a new tab on your computer:Fig. 14. A health report generated by a hospital employee.Creating medical records and agreements with a pediatrician, signing an agreement
Since the application is designed more to demonstrate the deployment of the network on separate EC2 servers, we will execute some commands from the prepared scripts.First EC2 Server:node ~/back/generators/setCards.js
We have previously connected to the database. After executing the command, you will see two entries in the Cards table. Also, information about the medical data of these two children was recorded on the blockchain.From this EC2 server we need to create an agreement with a pediatrician who will observe the children:node ~/back/generators/createAgreement.js
Now this agreement must be signed by the user whom we created for the organization Hospital. We execute the command from the second host:node ~/back/generators/signAgreement.js
Our agreement is signed by the doctor and we can continue to work on our application. Next, we need to install the frontend application on the second, third and fourth EC2 servers.PS: You can find all the commands for generating users, cards, agreements and signing agreements in the ~ / back / generators folder.Parents Frontend Application
Run on the second host (where the Parents organization is located):cd ~
git clone https://github.com/nixsolutions/blockchain-poc-frontend.git front
We create the .env file and specify the host name on which our frontend application will be located:vim ~ /front/.env:
VUE_APP_API_BASE_URL='http://ec2-107-22-75-46.compute-1.amazonaws.com:3006/'
It is necessary to establish the dependencies:cd front
npm install
You must also build the application and run it on port 8080:docker build . -t my-app
docker run -d -p 8080:80 my-app
Our application will be available at: http://<HOST_2>:8080where <HOST_2>is the address of the second EC2 server on which our application is located.Login: parentspassword: password.This is what the frontend will look like. Here you will see information about the medical cards that we created from the terminal:Fig. 15. Information about the medical records of children in the personal account of the parent.Hospital Frontend Application
Run on the second host (where the Hospital organization is located):cd ~
git clone https://github.com/nixsolutions/blockchain-poc-frontend.git front
We create the .env file and specify the host name on which our frontend application will be located:vim ~ /front/.env:
VUE_APP_API_BASE_URL='http://ec2-3-91-238-161.compute-1.amazonaws.com:3006/'
It is necessary to establish the dependencies:cd front
npm install
You must also build the application and run it on port 8080:docker build . -t my-app
docker run -d -p 8080:80 my-app
Our application will be available at: http://<HOST_3>:8080where <HOST_3>is the address of the third EC2 server.Login: hospitalpassword: password.Kindergarten frontend application
Run on the second host (where the Kindergarten organization is located):cd ~
git clone https://github.com/nixsolutions/blockchain-poc-frontend.git front
We create the .env file and specify the host name on which our frontend application will be located:vim ~ /front/.env:
VUE_APP_API_BASE_URL='http://ec2-107-22-12-137.compute-1.amazonaws.com:3006/'
It is necessary to establish the dependencies:cd front
npm install
You must also build the application and run it on port 8080:docker build . -t my-app
docker run -d -p 8080:80 my-app
Our application will be available at: http://<HOST_4>:8080where <HOST_4>is the address of the fourth EC2 server.Login: kindergartenpassword: password.Conclusion
We examined how to deploy a network of blockchain applications on several virtual or physical servers. The article is designed to be familiar with the basics of blockchain technology, know what smart contracts are, and at least superficially familiar with the Hyperledger Fabric framework and its standard fabric-samples application.