---
id: d001e79b-6b29-46e8-b422-4937c07d92f6
---

# Container Image Vulnerability on Kubernetes Cluster
---

This incident type refers to the discovery of a security vulnerability in a container image running on a Kubernetes cluster. Container images are commonly used to package and deploy software applications in a lightweight and portable way. Kubernetes is a popular platform for managing containerized applications. A vulnerability in a container image can potentially expose the application and the underlying infrastructure to security threats. This incident requires immediate attention to identify and remediate the vulnerability to prevent any further security breaches.

### Parameters
```shell
export CLUSTER_NAME="PLACEHOLDER"

export POD_NAME="PLACEHOLDER"

export IMAGE_NAME="PLACEHOLDER"

export DEPLOYMENT="PLACEHOLDER"

export NAMESPACE="PLACEHOLDER"

export NEW_IMAGE="PLACEHOLDER"
```

## Debug


### Check the status of the nodes in the cluster
```shell
kubectl get nodes
```

### Check the status of the pods running in the cluster
```shell
kubectl get pods
```

### Check the events related to a particular pod
```shell
kubectl describe pod ${POD_NAME}
```

### Check the container image used by a pod
```shell
kubectl describe pod ${POD_NAME} | grep Image
```

### Check the vulnerabilities in a container image
```shell
# Once inside the container, run:

apt-get update && apt-get install -y curl

curl -s https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh

trivy image ${IMAGE_NAME}
```

## Repair

### Upgrade the container image to the latest version with the vulnerability patched or fixed.
```shell

#!/bin/bash

# Set variables

NAMESPACE=${NAMESPACE}

DEPLOYMENT=${DEPLOYMENT}

NEW_IMAGE=${NEW_IMAGE}

# Upgrade container image to latest version

kubectl set image deployment/$DEPLOYMENT *=$NEW_IMAGE -n $NAMESPACE

```

This incident type refers to the discovery of a security vulnerability in a container image running on a Kubernetes cluster. Container images are commonly used to package and deploy software applications in a lightweight and portable way. Kubernetes is a popular platform for managing containerized applications. A vulnerability in a container image can potentially expose the application and the underlying infrastructure to security threats. This incident requires immediate attention to identify and remediate the vulnerability to prevent any further security breaches.


The Vault cluster health incident is related to the health of a Vault cluster instance. This incident type is triggered when the cluster instance is not healthy and requires attention to ensure it is functioning properly. The incident typically involves evaluating the current state of the cluster instance, diagnosing the issue, and taking corrective action to restore the health of the instance.


Vault cluster health incident on kubernetes

The Vault Sealed incident type occurs when an instance of the Vault has been sealed due to an error or issue. This can cause disruption to the system and requires immediate attention to resolve the issue and unseal the Vault instance. It is important to identify the root cause of the issue in order to prevent it from happening again in the future.


Vault Sealed Incident

This incident type is characterized by the detection of unauthorized access to the Kubernetes API server. This unauthorized access potentially enables attackers to manipulate cluster resources. Such incidents can lead to serious consequences, as unauthorized access allows attackers to make changes to the Kubernetes cluster, which may compromise the security and integrity of the entire system. Therefore, it is crucial to promptly detect and address such incidents to ensure the security and proper functioning of the Kubernetes cluster.


Unauthorized Access to Kubernetes API Server Detected

This incident type involves monitoring the replicas of a Kubernetes Statefulset, which is a type of workload in Kubernetes used for stateful applications. The incident is triggered when more than one replica's pods are down, creating an unsafe situation for manual operations. This incident is critical and requires immediate attention to resolve the issue and ensure the smooth functioning of the stateful applications.


Kubernetes Statefulset Replicas Monitoring Incident

A Kubernetes Pod ImagePullBackOff incident occurs when a pod in a Kubernetes cluster is unable to pull its container image. This can happen due to various reasons, such as incorrect image path or tag, or misconfigured image pulling credentials. This incident can cause the pod to fail to start and impact the availability of the application running in the pod. It requires investigation and resolution to ensure the pod can pull its container image and restart successfully.


Kubernetes Pod ImagePullBackOff Incident

```shell
export CLUSTER_NAME="PLACEHOLDER"

export POD_NAME="PLACEHOLDER"

export IMAGE_NAME="PLACEHOLDER"

export DEPLOYMENT="PLACEHOLDER"

export NAMESPACE="PLACEHOLDER"

export NEW_IMAGE="PLACEHOLDER"
```


### Check the status of the nodes in the cluster

```shell
kubectl get nodes
```

### Check the status of the pods running in the cluster

```shell
kubectl get pods
```

### Check the events related to a particular pod

```shell
kubectl describe pod ${POD_NAME}
```

### Check the container image used by a pod

```shell
kubectl describe pod ${POD_NAME} | grep Image
```

### Check the vulnerabilities in a container image

```shell
# Once inside the container, run:

apt-get update && apt-get install -y curl

curl -s https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh

trivy image ${IMAGE_NAME}
```


### Upgrade the container image to the latest version with the vulnerability patched or fixed.

```shell

#!/bin/bash

# Set variables

NAMESPACE=${NAMESPACE}

DEPLOYMENT=${DEPLOYMENT}

NEW_IMAGE=${NEW_IMAGE}

# Upgrade container image to latest version

kubectl set image deployment/$DEPLOYMENT *=$NEW_IMAGE -n $NAMESPACE

```


Container Image Vulnerability on Kubernetes Cluster

Overview

Parameters

Debug

Check the status of the nodes in the cluster

Check the status of the pods running in the cluster

Check the container image used by a pod

Check the vulnerabilities in a container image

Repair

Upgrade the container image to the latest version with the vulnerability patched or fixed.

Learn more

Related Runbooks

Vault cluster health incident on kubernetes

Vault Sealed Incident

Unauthorized Access to Kubernetes API Server Detected

Kubernetes Statefulset Replicas Monitoring Incident

Support