Implementierung von Docker Pull- und Docker Push-Befehlen ohne Docker-Client über HTTP-Anforderungen

Wir hatten 2 Säcke Gras, 75 Tabletten Meskalin Unix-Umgebung, Docker-Repository und Aufgabe zum Implementieren von Docker-Pull- und Docker-Push-Befehlen ohne Docker-Client.

UPD:
Frage: Wofür ist das alles?
Antwort: Lasttest des Produkts (NICHT mittels Bash, Skripte werden zu Bildungszwecken bereitgestellt). Es wurde beschlossen, den Docker-Client nicht zu verwenden, um zusätzliche Ebenen (innerhalb angemessener Grenzen) zu reduzieren und dementsprechend eine höhere Last zu emulieren. Infolgedessen wurden alle Systemverzögerungen des Docker-Clients entfernt. Erhielt eine relativ saubere Ladung direkt auf dem Produkt

Lassen Sie uns zunächst sehen, was diese Teams tun.

Wofür wird Docker Pull verwendet? Laut Dokumentation :

"Ziehen Sie ein Image oder ein Repository aus einer Registrierung".

Dort finden wir einen Link zum Verständnis von Bildern, Containern und Speichertreibern .

docker image layers, , . registry API.

:

"An “image” is a combination of a JSON manifest and individual layer files. The process of pulling an > image centers around retrieving these two components."

“Pulling an Image Manifest”.

, . : GET /v2/{name}/manifests/{reference}

"The name and reference parameter identify the image and are required. The reference may include a tag or digest."

, :

curl -s -X GET "http://localhost:8081/link/to/docker/registry/v2/centos-11-10/manifests/1.1.1" -H "header_if_needed"

json , . , : "GET /v2/{name}/blobs/{digest}"

“Access to a layer will be gated by the name of the repository but is identified uniquely in the registry by digest.”

digest , .

curl -s -X GET "http://localhost:8081/link/to/docker/registry/v2/centos-11-10/blobs/sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f" -H "header_if_needed" --output firstLayer

.

file firstLayer

.. tar , .

#!/bin/bash -eu

downloadDir=$1
# url as http://localhost:8081/link/to/docker/registry
url=$2
imageName=$3
tag=$4

# array of layers
layers=($(curl -s -X GET "$url/v2/$imageName/manifests/$tag" | grep -oP '(?<=blobSum" : ").+(?=")'))

# download each layer from array
for layer in "${layers[@]}"; do
    echo "Downloading ${layer}"
    curl -v -X GET "$url/v2/$imageName/blobs/$layer" --output "$downloadDir/$layer.tar"
done

# find all layers, untar them and remove source .tar files
cd "$downloadDir" && find . -name "sha256:*" -exec tar xvf {} \;
rm sha256:*.tar
exit 0

./script.sh dirName “http://localhost:8081/link/to/docker/registry” myAwesomeImage 1.0

2 — docker push

.

. , . .

:

• — "POST /v2/{repoName}/blobs/uploads/"
• ( , .. ) — "PUT /v2/{repoName}/blobs/uploads/{uuid}?digest={digest}
  Content-Length: {size of layer}
  Content-Type: application/octet-stream
  Layer Binary Data".
• — "PUT /v2/{repoName}/manifests/{reference}".

, . (chunked) PATCH :

"PATCH /v2/{repoName}/blobs/uploads/{uuid}
Content-Length: {size of chunk}
Content-Type: application/octet-stream
{Layer Chunk Binary Data}".

, .. 202 4.

:

• 
  2 3 , .

. archlinux:latest

docker pull archlinux

docker save c24fe13d37b9 -o savedArch

tar xvf savedArch

. ,

cat manifest.json | json_pp

. , .

, .

config . 2 ( ), mediaType :

echo ‘{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": config_size,
      "digest": "config_hash"
   },
   "layers": [
      ’ > manifest.json

. json :

{
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": ${layersSizes[$i]},
         "digest": \"sha256:${layersNames[$i]}\"
      },

.

sed -i "s/config_size/$configSize/g; s/config_hash/$configName/g" $manifestFile

Jetzt können Sie den Startvorgang starten und sich die UUID speichern, die von allen nachfolgenden Anforderungen begleitet werden sollte.

Das vollständige Skript sieht ungefähr so ​​aus:

#!/bin/bash -eux

imageDir=$1
# url as http://localhost:8081/link/to/docker/registry
url=$2
repoName=$3
tag=$4
manifestFile=$(readlink -f ${imageDir}/manifestCopy)
configFile=$(readlink -f $(find $imageDir -name "*.json" ! -name "manifest.json"))

# calc layers sha 256 sum, rename them accordingly, and add info about each to manifest file
function prepareLayersForUpload() {
  info_file=$imageDir/info
  # lets calculate layers sha256 and use it as layers names further
  layersNames=($(find $imageDir -name "layer.tar" -exec shasum -a 256 {} \; | cut -d" " -f1))

  # rename layers according to shasums. !!!Set required amount of fields for cut command!!!
  # this part definitely can be done easier but i didn't found another way, sry
  find $imageDir -name "layer.tar" -exec bash -c 'mv {} "$(echo {} | cut -d"/" -f1,2)/$(shasum -a 256 {} | cut -d" " -f1)"' \;

  layersSizes=($(find $imageDir -name "*.tar" -exec ls -l {} \; | awk '{print $5}'))

  for i in "${!layersNames[@]}"; do
    echo "{
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": ${layersSizes[$i]},
         "digest": \"sha256:${layersNames[$i]}\"
      }," >> $manifestFile
  done
  # remove last ','
  truncate -s-2 $manifestFile
  # add closing brakets to keep json consistent
  printf "\n\t]\n}" >> $manifestFile
}

# calc config sha 256 sum and add info about it to manifest
function setConfigProps() {
  configSize=$(ls -l $configFile | awk '{print $5}')
  configName=$(basename $configFile | cut -d"." -f1)

  sed -i "s/config_size/$configSize/g; s/config_hash/$configName/g" $manifestFile
}

#prepare manifest file
prepareLayersForUpload
setConfigProps
cat $manifestFile

# initiate upload and get uuid
uuid=$(curl -s -X POST -I "$url/v2/$repoName/blobs/uploads/" | grep -oP "(?<=Docker-Upload-Uuid: ).+")

# patch layers
# in data-binary we're getting absolute path to layer file
for l in "${!layersNames[@]}"; do
  pathToLayer=$(find $imageDir -name ${layersNames[$l]} -exec readlink -f {} \;)
    curl -v -X PATCH "$url/v2/$repoName/blobs/uploads/$uuid" \
  -H "Content-Length: ${layersSizes[$i]}" \
  -H "Content-Type: application/octet-stream" \
  --data-binary "@$pathToLayer"

# put layer
  curl -v -X PUT "$url/v2/$repoName/blobs/uploads/$uuid?digest=sha256:${layersNames[$i]}" \
  -H 'Content-Type: application/octet-stream' \
  -H "Content-Length: ${layersSizes[$i]}" \
  --data-binary "@$pathToLayer"
done

# patch and put config after all layers
curl -v -X PATCH "$url/v2/$repoName/blobs/uploads/$uuid" \
  -H "Content-Length: $configSize" \
  -H "Content-Type: application/octet-stream" \
  --data-binary "@$configFile"

  curl -v -X PUT "$url/v2/$repoName/blobs/uploads/$uuid?digest=sha256:$configName" \
  -H 'Content-Type: application/octet-stream' \
  -H "Content-Length: $configSize" \
  --data-binary "@$configFile"

# put manifest
curl -v -X PUT "$url/v2/$repoName/manifests/$tag" \
  -H 'Content-Type: application/vnd.docker.distribution.manifest.v2+json' \
  --data-binary "@$manifestFile"

exit 0

Wir können ein vorgefertigtes Skript verwenden:

./uploadImage.sh "~/path/to/saved/image" "http://localhost:8081/link/to/docker/registry" myRepoName 1.0

Der Artikel verwendete GNU-Versionswerkzeuge.

Im Allgemeinen entdeckt der Artikel Amerika nicht, sondern strukturiert die unterschiedlichen Daten geringfügig und ergänzt die Lücken in der Dokumentation. Danke fürs Lesen.