---
id: baac3424-2ff5-4849-8e13-804e37a2bcbc
---

# Pod Rejection Due to Pod Security Policy (PSP) Violation
---

This incident type occurs when a Pod in Kubernetes is rejected due to a violation of the Pod Security Policy (PSP). A Pod Security Policy is a set of rules that specify the conditions that a pod must meet to be accepted and run in a Kubernetes cluster. These rules are designed to prevent security risks and ensure that pods run with the least privilege necessary. When a pod violates the PSP, it is rejected, and the incident is triggered.

### Parameters
```shell
export NAMESPACE="PLACEHOLDER"

export POD_NAME="PLACEHOLDER"

export PSP_NAME="PLACEHOLDER"

export SERVICEACCOUNT_NAME="PLACEHOLDER"

export ROLE_NAME="PLACEHOLDER"

export CONTAINER_NAME="PLACEHOLDER"
```

## Debug

### List all Pods in the Namespace that are in the Pending state
```shell
kubectl get pods -n ${NAMESPACE} --field-selector status.phase=Pending
```

### Get the detailed status of a Pod that was rejected due to a PSP violation
```shell
kubectl describe pod ${POD_NAME} -n ${NAMESPACE}
```

### Check the Pod Security Policy that was violated
```shell
kubectl describe psp ${PSP_NAME}
```

### List all ServiceAccounts in the Namespace that are allowed to use the PSP
```shell
kubectl get psp ${PSP_NAME} -o=jsonpath='{.spec.serviceAccountName}'
```

### Get the detailed status of the ServiceAccount that the Pod is using
```shell
kubectl describe sa ${SERVICEACCOUNT_NAME} -n ${NAMESPACE}
```

### List all Roles that are bound to the ServiceAccount
```shell
kubectl get roles -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.namespace}{"\n"}{end}' --field-selector metadata.annotations.'rbac.authorization.kubernetes.io/autoupdate'='true',metadata.annotations.'rbac.authorization.kubernetes.io/autoupdate-after-anyclean'=true | xargs -n2 sh -c 'kubectl get role $0 -n $1 -o=jsonpath="{.metadata.name}\t{.metadata.namespace}\t{.metadata.annotations}\n"'
```

### Get the detailed status of the Role that the ServiceAccount is bound to
```shell
kubectl describe role ${ROLE_NAME} -n ${NAMESPACE}
```

## Repair

### Check if the pod has any privileged containers running. If yes, remove the privilege from the container, and try to deploy the pod again.
```shell

#!/bin/bash

# Get pod name

POD_NAME=${POD_NAME}

# Get container name

CONTAINER_NAME=${CONTAINER_NAME}

# Check if container is privileged

PRIVILEGED=$(kubectl get pod $POD_NAME -o jsonpath="{.spec.containers[?(@.name==\"$CONTAINER_NAME\")].securityContext.privileged}")

# If container is privileged, remove privilege

if [[ $PRIVILEGED == "true" ]]; then

    kubectl patch pod $POD_NAME -p '{"spec":{"containers":[{"name": "'"$CONTAINER_NAME"'", "securityContext":{"privileged":false}}]}}'

fi


```

This incident type occurs when a Pod in Kubernetes is rejected due to a violation of the Pod Security Policy (PSP). A Pod Security Policy is a set of rules that specify the conditions that a pod must meet to be accepted and run in a Kubernetes cluster. These rules are designed to prevent security risks and ensure that pods run with the least privilege necessary. When a pod violates the PSP, it is rejected, and the incident is triggered.


The Vault cluster health incident is related to the health of a Vault cluster instance. This incident type is triggered when the cluster instance is not healthy and requires attention to ensure it is functioning properly. The incident typically involves evaluating the current state of the cluster instance, diagnosing the issue, and taking corrective action to restore the health of the instance.


Vault cluster health incident on kubernetes

The Vault Sealed incident type occurs when an instance of the Vault has been sealed due to an error or issue. This can cause disruption to the system and requires immediate attention to resolve the issue and unseal the Vault instance. It is important to identify the root cause of the issue in order to prevent it from happening again in the future.


Vault Sealed Incident

This incident type is characterized by the detection of unauthorized access to the Kubernetes API server. This unauthorized access potentially enables attackers to manipulate cluster resources. Such incidents can lead to serious consequences, as unauthorized access allows attackers to make changes to the Kubernetes cluster, which may compromise the security and integrity of the entire system. Therefore, it is crucial to promptly detect and address such incidents to ensure the security and proper functioning of the Kubernetes cluster.


Unauthorized Access to Kubernetes API Server Detected

Nodes with PID Pressure in Kubernetes is an incident type that occurs when a Kubernetes cluster node experiences PID pressure, meaning that it may not be able to start more containers. This is a rare condition where a pod or container spawns too many processes and starves the node of available process IDs. Each node has a limited number of process IDs to distribute amongst running processes; and if it runs out of IDs, no other processes can be started. Kubernetes lets you set PID thresholds for pods to limit their ability to perform runaway process-spawning, and a PID pressure condition means that one or more pods are using up their allocated PIDs and need to be examined.


Nodes with PID Pressure in Kubernetes

This incident type involves an issue with Kubernetes deployments where the expected number of pods to run is not matching the actual number of pods running. This can lead to alerts being triggered and potential disruptions in the system.


Kubernetes pods not starting - Deployment issue

```shell
export NAMESPACE="PLACEHOLDER"

export POD_NAME="PLACEHOLDER"

export PSP_NAME="PLACEHOLDER"

export SERVICEACCOUNT_NAME="PLACEHOLDER"

export ROLE_NAME="PLACEHOLDER"

export CONTAINER_NAME="PLACEHOLDER"
```


### List all Pods in the Namespace that are in the Pending state

```shell
kubectl get pods -n ${NAMESPACE} --field-selector status.phase=Pending
```

### Get the detailed status of a Pod that was rejected due to a PSP violation

```shell
kubectl describe pod ${POD_NAME} -n ${NAMESPACE}
```

### Check the Pod Security Policy that was violated

```shell
kubectl describe psp ${PSP_NAME}
```

### List all ServiceAccounts in the Namespace that are allowed to use the PSP

```shell
kubectl get psp ${PSP_NAME} -o=jsonpath='{.spec.serviceAccountName}'
```

### Get the detailed status of the ServiceAccount that the Pod is using

```shell
kubectl describe sa ${SERVICEACCOUNT_NAME} -n ${NAMESPACE}
```

### List all Roles that are bound to the ServiceAccount

```shell
kubectl get roles -o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.namespace}{"\n"}{end}' --field-selector metadata.annotations.'rbac.authorization.kubernetes.io/autoupdate'='true',metadata.annotations.'rbac.authorization.kubernetes.io/autoupdate-after-anyclean'=true | xargs -n2 sh -c 'kubectl get role $0 -n $1 -o=jsonpath="{.metadata.name}\t{.metadata.namespace}\t{.metadata.annotations}\n"'
```

### Get the detailed status of the Role that the ServiceAccount is bound to

```shell
kubectl describe role ${ROLE_NAME} -n ${NAMESPACE}
```


### Check if the pod has any privileged containers running. If yes, remove the privilege from the container, and try to deploy the pod again.

```shell

#!/bin/bash

# Get pod name

POD_NAME=${POD_NAME}

# Get container name

CONTAINER_NAME=${CONTAINER_NAME}

# Check if container is privileged

PRIVILEGED=$(kubectl get pod $POD_NAME -o jsonpath="{.spec.containers[?(@.name==\"$CONTAINER_NAME\")].securityContext.privileged}")

# If container is privileged, remove privilege

if [[ $PRIVILEGED == "true" ]]; then

    kubectl patch pod $POD_NAME -p '{"spec":{"containers":[{"name": "'"$CONTAINER_NAME"'", "securityContext":{"privileged":false}}]}}'

fi


```


Pod Rejection Due to Pod Security Policy (PSP) Violation

Overview

Parameters

Debug

List all Pods in the Namespace that are in the Pending state

Get the detailed status of a Pod that was rejected due to a PSP violation

Check the Pod Security Policy that was violated

List all ServiceAccounts in the Namespace that are allowed to use the PSP

Get the detailed status of the ServiceAccount that the Pod is using

List all Roles that are bound to the ServiceAccount

Get the detailed status of the Role that the ServiceAccount is bound to

Repair

Check if the pod has any privileged containers running. If yes, remove the privilege from the container, and try to deploy the pod again.

Learn more

Related Runbooks

Vault cluster health incident on kubernetes

Vault Sealed Incident

Unauthorized Access to Kubernetes API Server Detected

Nodes with PID Pressure in Kubernetes

Support