---
id: db6d79fb-9de9-4ca8-b8ea-396ff677fa33
---

# Frequent Redis Sentinel Elections Incident
---

This incident type refers to a situation where Redis Sentinel, a high-availability solution for Redis, experiences frequent elections. Elections happen when a new Sentinel is elected as the leader, responsible for managing Redis instances. Frequent elections can be a sign of network instability or misconfiguration, which can result in downtime or reduced performance. It is important to investigate the root cause of the frequent elections and take prompt action to prevent service disruption.

### Parameters
```shell
export SENTINEL_PORT="PLACEHOLDER"

export SENTINEL_HOST="PLACEHOLDER"

export MASTER_NAME="PLACEHOLDER"

export REDIS_INSTANCE_HOST="PLACEHOLDER"

export REDIS_INSTANCE_PORT="PLACEHOLDER"

export NEW_QUORUM_SIZE="PLACEHOLDER"

export NEW_FAILOVER_TIMEOUT="PLACEHOLDER"

export PATH_TO_REDIS_CONF_FILE="PLACEHOLDER"
```

## Debug

### Check if Redis Sentinel is running
```shell
systemctl status redis-sentinel
```

### Check the current leader Sentinel
```shell
redis-cli -h ${SENTINEL_HOST} -p ${SENTINEL_PORT} SENTINEL get-master-addr-by-name ${MASTER_NAME}
```

### Check the Sentinel log for election events
```shell
journalctl -u redis-sentinel | grep "Elected leader"
```

### Check the network connectivity between Sentinels and Redis instances
```shell
ping ${REDIS_INSTANCE_HOST}
```

### Check the Redis instance log for connection errors
```shell
journalctl -u redis | grep "connection refused"
```

### Check the Redis instance configuration for Sentinel settings
```shell
redis-cli -h ${REDIS_INSTANCE_HOST} -p ${REDIS_INSTANCE_PORT} CONFIG GET sentinel
```

## Repair

### Adjust the quorum size or failover timeout settings to better suit the network conditions and workload demands.
```shell


#!/bin/bash



# Define variables

REDIS_CONF=${PATH_TO_REDIS_CONF_FILE}

QUORUM_SIZE=${NEW_QUORUM_SIZE}

FAILOVER_TIMEOUT=${NEW_FAILOVER_TIMEOUT}



# Backup the original configuration file

cp $REDIS_CONF $REDIS_CONF.bak



# Update the quorum size and failover timeout settings

sed -i "s/^quorum .*/quorum $QUORUM_SIZE/" $REDIS_CONF

sed -i "s/^failover-timeout .*/failover-timeout $FAILOVER_TIMEOUT/" $REDIS_CONF



# Restart Redis Sentinel to apply the changes

systemctl restart redis-sentinel



echo "Quorum size and failover timeout settings have been updated successfully."


```

This incident type refers to a situation where Redis Sentinel, a high-availability solution for Redis, experiences frequent elections. Elections happen when a new Sentinel is elected as the leader, responsible for managing Redis instances. Frequent elections can be a sign of network instability or misconfiguration, which can result in downtime or reduced performance. It is important to investigate the root cause of the frequent elections and take prompt action to prevent service disruption.


```shell
export SENTINEL_PORT="PLACEHOLDER"

export SENTINEL_HOST="PLACEHOLDER"

export MASTER_NAME="PLACEHOLDER"

export REDIS_INSTANCE_HOST="PLACEHOLDER"

export REDIS_INSTANCE_PORT="PLACEHOLDER"

export NEW_QUORUM_SIZE="PLACEHOLDER"

export NEW_FAILOVER_TIMEOUT="PLACEHOLDER"

export PATH_TO_REDIS_CONF_FILE="PLACEHOLDER"
```


### Check if Redis Sentinel is running

```shell
systemctl status redis-sentinel
```

### Check the current leader Sentinel

```shell
redis-cli -h ${SENTINEL_HOST} -p ${SENTINEL_PORT} SENTINEL get-master-addr-by-name ${MASTER_NAME}
```

### Check the Sentinel log for election events

```shell
journalctl -u redis-sentinel | grep "Elected leader"
```

### Check the network connectivity between Sentinels and Redis instances

```shell
ping ${REDIS_INSTANCE_HOST}
```

### Check the Redis instance log for connection errors

```shell
journalctl -u redis | grep "connection refused"
```

### Check the Redis instance configuration for Sentinel settings

```shell
redis-cli -h ${REDIS_INSTANCE_HOST} -p ${REDIS_INSTANCE_PORT} CONFIG GET sentinel
```


### Adjust the quorum size or failover timeout settings to better suit the network conditions and workload demands.

```shell


#!/bin/bash



# Define variables

REDIS_CONF=${PATH_TO_REDIS_CONF_FILE}

QUORUM_SIZE=${NEW_QUORUM_SIZE}

FAILOVER_TIMEOUT=${NEW_FAILOVER_TIMEOUT}



# Backup the original configuration file

cp $REDIS_CONF $REDIS_CONF.bak



# Update the quorum size and failover timeout settings

sed -i "s/^quorum .*/quorum $QUORUM_SIZE/" $REDIS_CONF

sed -i "s/^failover-timeout .*/failover-timeout $FAILOVER_TIMEOUT/" $REDIS_CONF



# Restart Redis Sentinel to apply the changes

systemctl restart redis-sentinel



echo "Quorum size and failover timeout settings have been updated successfully."


```


Frequent Redis Sentinel Elections Incident

Overview

Parameters

Debug

Check if Redis Sentinel is running

Check the current leader Sentinel

Check the Sentinel log for election events

Check the network connectivity between Sentinels and Redis instances

Check the Redis instance log for connection errors

Check the Redis instance configuration for Sentinel settings

Repair

Adjust the quorum size or failover timeout settings to better suit the network conditions and workload demands.

Learn more

Related Runbooks

Support