---
id: d09880c0-2d38-4f41-bfb9-7f28e58904ea
---

# Redis Rejected Connections Incident
---

This incident type refers to a situation where Redis, a popular in-memory data structure store, has rejected connections. This could happen due to various reasons such as server overload, network issues, or misconfiguration. When this occurs, it can cause disruptions in the performance of applications that rely on Redis for data storage or caching. Thus, it requires immediate attention and resolution to minimize the impact on the applications and users.

### Parameters
```shell
# Environment Variables

export REDIS_PORT="PLACEHOLDER"

export REDIS_HOST="PLACEHOLDER"

export PATH_TO_REDIS_CONF="PLACEHOLDER"

```

## Debug

### Check if Redis is running
```shell
systemctl status redis
```

### Check if Redis is listening on the correct port
```shell
sudo netstat -tuln | grep ${REDIS_PORT}
```

### Check Redis logs for any error messages
```shell
sudo tail -f /var/log/redis/redis-server.log
```

### Check the Redis client connections
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} info clients
```

### Check the Redis memory usage
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} info memory
```

### Check the Redis CPU usage
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} info cpu
```

### Check the Redis configuration
```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} config get *
```

### Check the system load and resource usage
```shell
top -c
```

### Check the network connectivity to Redis
```shell
ping ${REDIS_HOST}
```

### Check the firewall rules and open ports
```shell
sudo iptables -L -n
```

## Repair

### Update the firewall rules and open ports
```shell


#!/bin/bash



# Set variables

${PORT_NUMBER}="PLACEHOLDER"

${PROTOCOL}="PLACEHOLDER"

${FIREWALL_RULE_NAME}="PLACEHOLDER"



# Add firewall rule to open port

sudo firewall-cmd --permanent --add-port=$PORT_NUMBER/$PROTOCOL --zone=public --permanent --name=$FIREWALL_RULE_NAME

sudo firewall-cmd --reload



# Verify firewall rule is added

sudo firewall-cmd --list-all


```

### Optimizing Redis configuration settings such as increasing the max number of connections or tuning the eviction policies to avoid overload.
```shell


#!/bin/bash



# Set the Redis configuration file path

REDIS_CONF_PATH=${PATH_TO_REDIS_CONF}



# Increase the max number of connections

sed -i 's/maxclients ${VALUE}/maxclients 10000/g' $REDIS_CONF_PATH



# Tune the eviction policies

sed -i 's/eviction-policy ${VALUE}/eviction-policy allkeys-lru/g' $REDIS_CONF_PATH



# Restart the Redis service

systemctl restart redis


```

This incident type refers to a situation where Redis, a popular in-memory data structure store, has rejected connections. This could happen due to various reasons such as server overload, network issues, or misconfiguration. When this occurs, it can cause disruptions in the performance of applications that rely on Redis for data storage or caching. Thus, it requires immediate attention and resolution to minimize the impact on the applications and users.


This incident type refers to a situation where the Tomcat server's JDBC connection pool usage surpasses its configured limits. This can lead to a shortage of available connections for incoming requests, resulting in slow response times or even server crashes. It is important to monitor and manage the JDBC connection pool usage to prevent such incidents from occurring.


Tomcat High JDBC Connection Pool Usage Incident.

Tomcat Connection Pool Exhaustion Incident occurs when the maximum number of connections that a Tomcat server can handle has been reached. This can happen due to an increase in traffic or a sudden surge in user activity. When the connection pool is exhausted, new requests cannot be handled, and the application may become unavailable. This type of incident requires immediate attention to ensure that the application is restored to its normal functioning state.


Tomcat Connection Pool Exhaustion Incident

The Redis too many masters incident occurs when there are too many master nodes in a Redis cluster, leading to connection issues and potential data loss. This can happen due to misconfiguration, network issues, or other factors, and requires immediate attention to prevent further damage.


Redis too many masters incident

This incident type refers to an issue with Redis replication, which means that there is a problem with the synchronization of data between Redis instances. This issue could impact the availability and performance of the system and may require immediate attention to restore the replication and ensure data consistency. The incident could be caused by various factors, such as network problems, hardware failures, or configuration issues. The incident must be investigated and resolved as soon as possible to avoid any data loss or downtime.


Redis replication broken incident.

The Redis missing master incident occurs when the Redis cluster has no node marked as master. This can cause problems with the Redis service and may result in service disruptions or failures. It is important to address this issue promptly to ensure the smooth operation of the Redis service.


Redis missing master incident

```shell
# Environment Variables

export REDIS_PORT="PLACEHOLDER"

export REDIS_HOST="PLACEHOLDER"

export PATH_TO_REDIS_CONF="PLACEHOLDER"

```


### Check if Redis is running

```shell
systemctl status redis
```

### Check if Redis is listening on the correct port

```shell
sudo netstat -tuln | grep ${REDIS_PORT}
```

### Check Redis logs for any error messages

```shell
sudo tail -f /var/log/redis/redis-server.log
```

### Check the Redis client connections

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} info clients
```

### Check the Redis memory usage

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} info memory
```

### Check the Redis CPU usage

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} info cpu
```

### Check the Redis configuration

```shell
redis-cli -h ${REDIS_HOST} -p ${REDIS_PORT} config get *
```

### Check the system load and resource usage

```shell
top -c
```

### Check the network connectivity to Redis

```shell
ping ${REDIS_HOST}
```

### Check the firewall rules and open ports

```shell
sudo iptables -L -n
```


### Update the firewall rules and open ports

```shell


#!/bin/bash



# Set variables

${PORT_NUMBER}="PLACEHOLDER"

${PROTOCOL}="PLACEHOLDER"

${FIREWALL_RULE_NAME}="PLACEHOLDER"



# Add firewall rule to open port

sudo firewall-cmd --permanent --add-port=$PORT_NUMBER/$PROTOCOL --zone=public --permanent --name=$FIREWALL_RULE_NAME

sudo firewall-cmd --reload



# Verify firewall rule is added

sudo firewall-cmd --list-all


```

### Optimizing Redis configuration settings such as increasing the max number of connections or tuning the eviction policies to avoid overload.

```shell


#!/bin/bash



# Set the Redis configuration file path

REDIS_CONF_PATH=${PATH_TO_REDIS_CONF}



# Increase the max number of connections

sed -i 's/maxclients ${VALUE}/maxclients 10000/g' $REDIS_CONF_PATH



# Tune the eviction policies

sed -i 's/eviction-policy ${VALUE}/eviction-policy allkeys-lru/g' $REDIS_CONF_PATH



# Restart the Redis service

systemctl restart redis


```


Redis Rejected Connections Incident

Overview

Parameters

Debug

Check if Redis is running

Check if Redis is listening on the correct port

Check Redis logs for any error messages

Check the Redis client connections

Check the Redis memory usage

Check the Redis CPU usage

Check the Redis configuration

Check the system load and resource usage

Check the network connectivity to Redis

Check the firewall rules and open ports

Repair

Update the firewall rules and open ports

Optimizing Redis configuration settings such as increasing the max number of connections or tuning the eviction policies to avoid overload.

Learn more

Related Runbooks

Tomcat High JDBC Connection Pool Usage Incident.

Tomcat Connection Pool Exhaustion Incident

Redis too many masters incident

Redis replication broken incident.

Support