---
id: f76accfa-e3c6-4762-8913-72ffe3797743
---

# Low cache hits count in CoreDNS
---

This incident type occurs when the cache hits count in a CoreDNS test environment is lower than expected. This could indicate a problem with caching functionality or performance issues. It may require investigation and troubleshooting to determine the root cause of the low cache hits count and resolve the issue.

### Parameters
```shell
# Environment Variables

export NAMESPACE="PLACEHOLDER"

export POD_NAME="PLACEHOLDER"

export SERVICE_NAME="PLACEHOLDER"

export COREDNS_CONFIGMAP_NAME="PLACEHOLDER"

export NEW_CACHE_SIZE="PLACEHOLDER"

```

## Debug

### Check if the CoreDNS pods are running
```shell
kubectl get pods -n ${NAMESPACE}
```

### If the pods are not running, check the pod events for any errors
```shell
kubectl describe pod ${POD_NAME} -n ${NAMESPACE}
```

### Check the CoreDNS logs for any errors
```shell
kubectl logs ${POD_NAME} -n ${NAMESPACE}
```

### Check the CoreDNS configuration for any issues
```shell
kubectl get configmap coredns -n ${NAMESPACE}
```

### Check if the Kubernetes DNS service is properly configured with CoreDNS
```shell
kubectl get svc kube-dns  -n ${NAMESPACE}
```

### Check the DNS resolution from a pod
```shell
kubectl run -it --rm check_dns --image=busybox:1.28 --restart=Never -- nslookup ${SERVICE_NAME}.${NAMESPACE}.svc.cluster.local
```

## Repair

### Get the current CoreDNS cache size
```shell
CURRENT_CACHE_SIZE=$(kubectl exec -it $DEPLOYMENT -c $CONTAINER -- cat /etc/coredns/Corefile | grep -oP 'cache\s\{\s*success\s+([\d]+)' | awk '{print $3}')
```

### Increase the CoreDNS cache size by 50%
```shell
NEW_CACHE_SIZE=$(echo "$CURRENT_CACHE_SIZE * 1.5" | bc | awk '{print int($1+0.5)}')
```

### Update the CoreDNS cache size
 with the new cache size
```shell
kubectl patch configmap ${CONFIGMAP_NAME} -n ${NAMESPACE} --type merge --patch '{"data": {"Corefile": "cache ${NEW_CACHE_SIZE} \n"}}'
```

### Restart the CoreDNS pod to apply the changes
```shell
kubectl rollout restart deployment $DEPLOYMENT
```


This incident type occurs when the cache hits count in a CoreDNS test environment is lower than expected. This could indicate a problem with caching functionality or performance issues. It may require investigation and troubleshooting to determine the root cause of the low cache hits count and resolve the issue.


This incident type occurs when the number of 4xx errors on Traffic Server is at an anomalous level, higher than usual. It could be an indicator of an issue with the server or an increase in traffic that is causing errors. It requires investigation and resolution to ensure the server is functioning correctly and not impacting users.


High 4xx Errors on Traffic Server

The "High error rate on NGINX" incident type refers to a situation where the error rate on the NGINX server is above 1% for the last 5 minutes. This can result in degraded performance or downtime of the affected service, impacting user experience and potentially leading to lost revenue. The incident requires immediate attention and resolution to minimize the impact on users and prevent further damage.


High error rate on NGINX incident

This incident type refers to the failure of a systemd service on a particular host instance. The incident could be triggered by various causes such as a software bug, hardware failure, or system overload. This type of incident can cause downtime or service disruption to the affected host instance, which may require immediate resolution to restore normal operations.


Host systemd service crashed (instance) incident.

This incident type refers to a situation where the host memory of a system is not being fully utilized and may be causing performance issues. It may trigger when the memory load is below the expected level for a certain period of time. The incident may require investigation to determine the root cause and may involve reducing the amount of memory or other memory management strategies to optimize system performance.


"Host Memory is underutilized incident"

A Host CPU high iowait incident occurs when the CPU iowait is greater than 10%, indicating that the system is disk or network bound. This can cause performance issues and may require investigation to identify the root cause and resolve the issue.


Host CPU high iowait incident.

```shell
# Environment Variables

export NAMESPACE="PLACEHOLDER"

export POD_NAME="PLACEHOLDER"

export SERVICE_NAME="PLACEHOLDER"

export COREDNS_CONFIGMAP_NAME="PLACEHOLDER"

export NEW_CACHE_SIZE="PLACEHOLDER"

```


### Check if the CoreDNS pods are running

```shell
kubectl get pods -n ${NAMESPACE}
```

### If the pods are not running, check the pod events for any errors

```shell
kubectl describe pod ${POD_NAME} -n ${NAMESPACE}
```

### Check the CoreDNS logs for any errors

```shell
kubectl logs ${POD_NAME} -n ${NAMESPACE}
```

### Check the CoreDNS configuration for any issues

```shell
kubectl get configmap coredns -n ${NAMESPACE}
```

### Check if the Kubernetes DNS service is properly configured with CoreDNS

```shell
kubectl get svc kube-dns  -n ${NAMESPACE}
```

### Check the DNS resolution from a pod

```shell
kubectl run -it --rm check_dns --image=busybox:1.28 --restart=Never -- nslookup ${SERVICE_NAME}.${NAMESPACE}.svc.cluster.local
```


### Get the current CoreDNS cache size

```shell
CURRENT_CACHE_SIZE=$(kubectl exec -it $DEPLOYMENT -c $CONTAINER -- cat /etc/coredns/Corefile | grep -oP 'cache\s\{\s*success\s+([\d]+)' | awk '{print $3}')
```

### Increase the CoreDNS cache size by 50%

```shell
NEW_CACHE_SIZE=$(echo "$CURRENT_CACHE_SIZE * 1.5" | bc | awk '{print int($1+0.5)}')
```

### Update the CoreDNS cache size

with the new cache size

```shell
kubectl patch configmap ${CONFIGMAP_NAME} -n ${NAMESPACE} --type merge --patch '{"data": {"Corefile": "cache ${NEW_CACHE_SIZE} \n"}}'
```

### Restart the CoreDNS pod to apply the changes

```shell
kubectl rollout restart deployment $DEPLOYMENT
```


Low cache hits count in CoreDNS

Overview

Parameters

Debug

Check if the CoreDNS pods are running

If the pods are not running, check the pod events for any errors

Check the CoreDNS logs for any errors

Check the CoreDNS configuration for any issues

Check if the Kubernetes DNS service is properly configured with CoreDNS

Check the DNS resolution from a pod

Repair

Get the current CoreDNS cache size

Increase the CoreDNS cache size by 50%

Update the CoreDNS cache size

Restart the CoreDNS pod to apply the changes

Learn more

Related Runbooks

High 4xx Errors on Traffic Server

High error rate on NGINX incident

Host systemd service crashed (instance) incident.

"Host Memory is underutilized incident"

Support