How to trust an insecure image registry?

Problem description

The image registry hosting platform component images may not provide HTTPS service or may not have a valid TLS certificate issued by a public certificate authority. The steps below configure Containerd to skip TLS verification for one registry address. They do not install a CA certificate, and they do not save a username or password on the node.

Use the real host[:port] from the import procedure, including the port when the image reference includes one. Do not use cpaas-registry.cpaas-system. Skipping verification for that logical name does not cover pulls from the real endpoint. If the cluster is already imported, use the registry host in the Sentry Pod image reference when it differs from the address printed on the global cluster.

Platform components authenticate with an image pull secret created from the credentials entered during import. The secret's authentication key is the same real address. These steps do not create that secret.

Configure Containerd to skip TLS verification

Notes:

  • All nodes that need to use images, including newly added nodes, must be configured and have Containerd restarted.
  • The configuration differs slightly between Containerd v1.4/v1.5 and v1.6. Follow the appropriate steps for your version.
  • After Containerd restarts, pull the Sentry image on an imported node. A successful curl on the global cluster does not prove that the imported node can pull the image.
  1. Run the following on every node in the import cluster:

    • Back up the configuration file

      mkdir -p '/var/backup-containerd-confs/'
      if ! [ -f /etc/containerd/config.toml ]; then
          echo 'Containerd config not found. Please check if containerd is correctly installed. If you still cannot resolve the issue, contact technical support.'
          exit 1
      else
          cp /etc/containerd/config.toml /var/backup-containerd-confs/config.toml_$(date +%F_%T)
      fi
    • Get the Containerd runtime version

      # Get the containerd version
      # Compare this version to v1.6. Choose steps accordingly
      ctr --version | grep -Eo 'v[0-9]+\.[0-9]+\.[0-9]+'
    Containerd v1.4 v1.5 configuration for insecure registries
  2. Run the following on every node in the import cluster:

    • Edit /etc/containerd/config.toml

      # Example content to add to the config file
      # Lines in brackets are sections. If the file already has sections with the same name, merge their contents.
      [plugins."io.containerd.grpc.v1.cri".registry]
          [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
              [plugins."io.containerd.grpc.v1.cri".registry.mirrors."<registry-address>"]
                  endpoint = ["https://<registry-address>", "http://<registry-address>"]
              [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.134.43"]
                  endpoint = ["https://192.168.134.43", "http://192.168.134.43"]
          [plugins."io.containerd.grpc.v1.cri".registry.configs]
              [plugins."io.containerd.grpc.v1.cri".registry.configs."<registry-address>".tls]
                  insecure_skip_verify = true
              [plugins."io.containerd.grpc.v1.cri".registry.configs."192.168.134.43".tls]
                  insecure_skip_verify = true
    • Restart Containerd.

      systemctl daemon-reload && systemctl restart containerd
Containerd v1.6 configuration for insecure registries
  1. Run the following on every node in the import cluster:

    • Check whether config_path exists in the config.

      if ! grep -qF 'config_path' /etc/containerd/config.toml; then
          if grep -qE '\[plugins."io.containerd.grpc.v1.cri".registry.(mirrors|configs)(\.|\])' /etc/containerd/config.toml; then
              echo 'Follow the steps in "Containerd v1.4 v1.5 configuration for insecure registries".'
          else
              cat >> /etc/containerd/config.toml << 'EOF'
      [plugins."io.containerd.grpc.v1.cri".registry]
          config_path = "/etc/containerd/certs.d/"
      EOF
          fi
      fi
      
      config_path_var=$(grep -F '/etc/containerd/certs.d' /etc/containerd/config.toml)
      if [ -z "$config_path_var" ]; then
         echo 'The value of config_path in the file is unexpected. Please check!'
         exit 1
      fi
    • Create the hosts.toml file.

      If the previous command printed Follow the steps in "Containerd v1.4 v1.5 configuration for insecure registries"., see Containerd v1.4 v1.5 configuration for insecure registries.

      REGISTRY='<real host[:port] from the import procedure>'
      
      mkdir -p "/etc/containerd/certs.d/$REGISTRY/"
      cat > "/etc/containerd/certs.d/$REGISTRY/hosts.toml" << EOF
      server = "$REGISTRY"
      [host."http://$REGISTRY"]
        capabilities = ["pull", "resolve", "push"]
        skip_verify = true
      [host."https://$REGISTRY"]
        capabilities = ["pull", "resolve", "push"]
        skip_verify = true
      EOF
    • Restart Containerd.

      systemctl daemon-reload && systemctl restart containerd