Implementación de los comandos docker pull y docker push sin el cliente docker a través de solicitudes HTTP

Tuvimos 2 bolsas de hierba, 75 tabletas de mescalina entorno Unix, repositorio de Docker y tarea para implementar los comandos Docker Pull y Docker Push sin el cliente Docker.

UPD:
Pregunta: ¿Para qué es todo esto?
Respuesta: Prueba de carga del producto (NO mediante bash, los scripts se proporcionan con fines educativos). Se decidió no utilizar el cliente de Docker para reducir capas adicionales (dentro de límites razonables) y, en consecuencia, emular una carga más alta. Como resultado, se eliminaron todos los retrasos del sistema del cliente de Docker. Recibió una carga relativamente limpia directamente en el producto

Primero, veamos qué hacen estos equipos.

Entonces, ¿para qué se utiliza Docker Pull? De acuerdo con la documentación :

"Extraer una imagen o un repositorio de un registro".

Allí encontramos un enlace para comprender imágenes, contenedores y controladores de almacenamiento .

docker image layers, , . registry API.

:

"An “image” is a combination of a JSON manifest and individual layer files. The process of pulling an > image centers around retrieving these two components."

“Pulling an Image Manifest”.

, . : GET /v2/{name}/manifests/{reference}

"The name and reference parameter identify the image and are required. The reference may include a tag or digest."

, :

curl -s -X GET "http://localhost:8081/link/to/docker/registry/v2/centos-11-10/manifests/1.1.1" -H "header_if_needed"

json , . , : "GET /v2/{name}/blobs/{digest}"

“Access to a layer will be gated by the name of the repository but is identified uniquely in the registry by digest.”

digest , .

curl -s -X GET "http://localhost:8081/link/to/docker/registry/v2/centos-11-10/blobs/sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f" -H "header_if_needed" --output firstLayer

.

file firstLayer

.. tar , .

#!/bin/bash -eu

downloadDir=$1
# url as http://localhost:8081/link/to/docker/registry
url=$2
imageName=$3
tag=$4

# array of layers
layers=($(curl -s -X GET "$url/v2/$imageName/manifests/$tag" | grep -oP '(?<=blobSum" : ").+(?=")'))

# download each layer from array
for layer in "${layers[@]}"; do
    echo "Downloading ${layer}"
    curl -v -X GET "$url/v2/$imageName/blobs/$layer" --output "$downloadDir/$layer.tar"
done

# find all layers, untar them and remove source .tar files
cd "$downloadDir" && find . -name "sha256:*" -exec tar xvf {} \;
rm sha256:*.tar
exit 0

./script.sh dirName “http://localhost:8081/link/to/docker/registry” myAwesomeImage 1.0

2 — docker push

.

. , . .

:

• — "POST /v2/{repoName}/blobs/uploads/"
• ( , .. ) — "PUT /v2/{repoName}/blobs/uploads/{uuid}?digest={digest}
  Content-Length: {size of layer}
  Content-Type: application/octet-stream
  Layer Binary Data".
• — "PUT /v2/{repoName}/manifests/{reference}".

, . (chunked) PATCH :

"PATCH /v2/{repoName}/blobs/uploads/{uuid}
Content-Length: {size of chunk}
Content-Type: application/octet-stream
{Layer Chunk Binary Data}".

, .. 202 4.

:

• 
  2 3 , .

. archlinux:latest

docker pull archlinux

docker save c24fe13d37b9 -o savedArch

tar xvf savedArch

. ,

cat manifest.json | json_pp

. , .

, .

config . 2 ( ), mediaType :

echo ‘{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": config_size,
      "digest": "config_hash"
   },
   "layers": [
      ’ > manifest.json

. json :

{
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": ${layersSizes[$i]},
         "digest": \"sha256:${layersNames[$i]}\"
      },

.

sed -i "s/config_size/$configSize/g; s/config_hash/$configName/g" $manifestFile

Ahora puede iniciar el proceso de arranque y guardarse uuid, que debe ir acompañado de todas las solicitudes posteriores.

El guión completo se parece a esto:

#!/bin/bash -eux

imageDir=$1
# url as http://localhost:8081/link/to/docker/registry
url=$2
repoName=$3
tag=$4
manifestFile=$(readlink -f ${imageDir}/manifestCopy)
configFile=$(readlink -f $(find $imageDir -name "*.json" ! -name "manifest.json"))

# calc layers sha 256 sum, rename them accordingly, and add info about each to manifest file
function prepareLayersForUpload() {
  info_file=$imageDir/info
  # lets calculate layers sha256 and use it as layers names further
  layersNames=($(find $imageDir -name "layer.tar" -exec shasum -a 256 {} \; | cut -d" " -f1))

  # rename layers according to shasums. !!!Set required amount of fields for cut command!!!
  # this part definitely can be done easier but i didn't found another way, sry
  find $imageDir -name "layer.tar" -exec bash -c 'mv {} "$(echo {} | cut -d"/" -f1,2)/$(shasum -a 256 {} | cut -d" " -f1)"' \;

  layersSizes=($(find $imageDir -name "*.tar" -exec ls -l {} \; | awk '{print $5}'))

  for i in "${!layersNames[@]}"; do
    echo "{
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": ${layersSizes[$i]},
         "digest": \"sha256:${layersNames[$i]}\"
      }," >> $manifestFile
  done
  # remove last ','
  truncate -s-2 $manifestFile
  # add closing brakets to keep json consistent
  printf "\n\t]\n}" >> $manifestFile
}

# calc config sha 256 sum and add info about it to manifest
function setConfigProps() {
  configSize=$(ls -l $configFile | awk '{print $5}')
  configName=$(basename $configFile | cut -d"." -f1)

  sed -i "s/config_size/$configSize/g; s/config_hash/$configName/g" $manifestFile
}

#prepare manifest file
prepareLayersForUpload
setConfigProps
cat $manifestFile

# initiate upload and get uuid
uuid=$(curl -s -X POST -I "$url/v2/$repoName/blobs/uploads/" | grep -oP "(?<=Docker-Upload-Uuid: ).+")

# patch layers
# in data-binary we're getting absolute path to layer file
for l in "${!layersNames[@]}"; do
  pathToLayer=$(find $imageDir -name ${layersNames[$l]} -exec readlink -f {} \;)
    curl -v -X PATCH "$url/v2/$repoName/blobs/uploads/$uuid" \
  -H "Content-Length: ${layersSizes[$i]}" \
  -H "Content-Type: application/octet-stream" \
  --data-binary "@$pathToLayer"

# put layer
  curl -v -X PUT "$url/v2/$repoName/blobs/uploads/$uuid?digest=sha256:${layersNames[$i]}" \
  -H 'Content-Type: application/octet-stream' \
  -H "Content-Length: ${layersSizes[$i]}" \
  --data-binary "@$pathToLayer"
done

# patch and put config after all layers
curl -v -X PATCH "$url/v2/$repoName/blobs/uploads/$uuid" \
  -H "Content-Length: $configSize" \
  -H "Content-Type: application/octet-stream" \
  --data-binary "@$configFile"

  curl -v -X PUT "$url/v2/$repoName/blobs/uploads/$uuid?digest=sha256:$configName" \
  -H 'Content-Type: application/octet-stream' \
  -H "Content-Length: $configSize" \
  --data-binary "@$configFile"

# put manifest
curl -v -X PUT "$url/v2/$repoName/manifests/$tag" \
  -H 'Content-Type: application/vnd.docker.distribution.manifest.v2+json' \
  --data-binary "@$manifestFile"

exit 0

podemos usar un script listo para usar:

./uploadImage.sh "~/path/to/saved/image" "http://localhost:8081/link/to/docker/registry" myRepoName 1.0

El artículo usaba herramientas de versión GNU.

En general, el artículo no descubre Estados Unidos, sino que estructura ligeramente los datos dispares y complementa los vacíos en la documentación. Gracias por leer.