---
id: 253921c4-6d28-4391-999c-50406b4b1aaa
---

# MongoDB number cursors open incident.
---

A MongoDB number cursors open incident occurs when there are too many cursors opened by MongoDB for clients, usually exceeding 10,000, which can cause performance issues and impact the overall stability of the system. This incident needs to be addressed promptly to ensure that the number of cursors is reduced to a manageable level, and that the issue does not recur.

### Parameters
```shell
# Environment Variables

export MONGODB_OWNER="PLACEHOLDER"

export MONGODB_USER="PLACEHOLDER"

export MONGODB_PASSWORD="PLACEHOLDER"

export MONGODB_INSTANCE="PLACEHOLDER"

export POOL_SIZE="PLACEHOLDER"

```

## Debug

### 1. Check MongoDB version
```shell
mongo --version
```

### 2. Check number of open cursors for each database
```shell
mongo --eval "db._adminCommand( { serverStatus: 1 } ).cursors"
```

### 3. Check the number of clients and connections for MongoDB
```shell
netstat -an | grep 27017 | grep ESTABLISHED | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr
```

### 4. Check the current open file limit for the MongoDB process
```shell
cat /proc/$(pidof mongod)/limits | grep "open files"
```

### 5. Check the ulimit settings for the MongoDB process owner
```shell
sudo su - ${MONGODB_OWNER} -c "ulimit -a"
```

### 6. Check the MongoDB logs for any errors related to open cursors
```shell
tail -n 1000 /var/log/mongodb/mongod.log | grep "Too many open cursors"
```

### 7. Check the MongoDB configuration file for cursor-related settings
```shell
cat /etc/mongod.conf | grep cursor
```

## Repair

### Reduce the number of open cursors in MongoDB

```shell


#!/bin/bash



# Step 1: Identify the MongoDB instance and connect to it

MONGODB_INSTANCE=${MONGODB_INSTANCE}

MONGODB_USER=${MONGODB_USER}

MONGODB_PASSWORD=${MONGODB_PASSWORD}

mongo -u $MONGODB_USER -p $MONGODB_PASSWORD $MONGODB_INSTANCE



# Step 2: Check the number of open cursors and identify idle or long-running ones

db.serverStatus().cursors



# Step 3: Terminate idle or long-running cursors
CURSOR_ID="PLACEHOLDER"

db.killCursors(${CURSOR_ID})



# Step 4: Implement connection pooling to manage the number of open connections

# Add the following line to the MongoDB driver configuration file

connectionPoolSize=${POOL_SIZE}



# Step 5: Monitor and analyze the MongoDB logs to identify any recurring issues

tail -f /var/log/mongodb/mongodb.log


```

A MongoDB number cursors open incident occurs when there are too many cursors opened by MongoDB for clients, usually exceeding 10,000, which can cause performance issues and impact the overall stability of the system. This incident needs to be addressed promptly to ensure that the number of cursors is reduced to a manageable level, and that the issue does not recur.


This incident type refers to a situation where the Tomcat server's JDBC connection pool usage surpasses its configured limits. This can lead to a shortage of available connections for incoming requests, resulting in slow response times or even server crashes. It is important to monitor and manage the JDBC connection pool usage to prevent such incidents from occurring.


Tomcat High JDBC Connection Pool Usage Incident.

The Redis too many masters incident occurs when there are too many master nodes in a Redis cluster, leading to connection issues and potential data loss. This can happen due to misconfiguration, network issues, or other factors, and requires immediate attention to prevent further damage.


Redis too many masters incident

This incident type refers to a situation where Redis, a popular in-memory data structure store, has rejected connections. This could happen due to various reasons such as server overload, network issues, or misconfiguration. When this occurs, it can cause disruptions in the performance of applications that rely on Redis for data storage or caching. Thus, it requires immediate attention and resolution to minimize the impact on the applications and users.


Redis Rejected Connections Incident

This incident type occurs when the number of concurrent connections to a MySQL database server exceeds the configured limit of 80%. This can lead to performance degradation or even database server crashes, making the application or system that relies on the database server unavailable for the end-users. This type of incident typically requires immediate attention and resolution to restore normal operation of the affected service.


MySQL too many connections incident

This incident type involves an alert triggered when read latency is higher than usual for a MongoDB host. The alert is triggered when at least 100% of the average read latency values have been more than 2 deviations above the predicted values during the last 15 minutes. The incident requires monitoring and investigation to identify the root cause of the increased read latency and take appropriate actions to resolve it.


Read latency higher than average for MongoDB host.

```shell
# Environment Variables

export MONGODB_OWNER="PLACEHOLDER"

export MONGODB_USER="PLACEHOLDER"

export MONGODB_PASSWORD="PLACEHOLDER"

export MONGODB_INSTANCE="PLACEHOLDER"

export POOL_SIZE="PLACEHOLDER"

```


### 1. Check MongoDB version

```shell
mongo --version
```

### 2. Check number of open cursors for each database

```shell
mongo --eval "db._adminCommand( { serverStatus: 1 } ).cursors"
```

### 3. Check the number of clients and connections for MongoDB

```shell
netstat -an | grep 27017 | grep ESTABLISHED | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr
```

### 4. Check the current open file limit for the MongoDB process

```shell
cat /proc/$(pidof mongod)/limits | grep "open files"
```

### 5. Check the ulimit settings for the MongoDB process owner

```shell
sudo su - ${MONGODB_OWNER} -c "ulimit -a"
```

### 6. Check the MongoDB logs for any errors related to open cursors

```shell
tail -n 1000 /var/log/mongodb/mongod.log | grep "Too many open cursors"
```

### 7. Check the MongoDB configuration file for cursor-related settings

```shell
cat /etc/mongod.conf | grep cursor
```


### Reduce the number of open cursors in MongoDB

```shell


#!/bin/bash



# Step 1: Identify the MongoDB instance and connect to it

MONGODB_INSTANCE=${MONGODB_INSTANCE}

MONGODB_USER=${MONGODB_USER}

MONGODB_PASSWORD=${MONGODB_PASSWORD}

mongo -u $MONGODB_USER -p $MONGODB_PASSWORD $MONGODB_INSTANCE



# Step 2: Check the number of open cursors and identify idle or long-running ones

db.serverStatus().cursors



# Step 3: Terminate idle or long-running cursors
CURSOR_ID="PLACEHOLDER"

db.killCursors(${CURSOR_ID})



# Step 4: Implement connection pooling to manage the number of open connections

# Add the following line to the MongoDB driver configuration file

connectionPoolSize=${POOL_SIZE}



# Step 5: Monitor and analyze the MongoDB logs to identify any recurring issues

tail -f /var/log/mongodb/mongodb.log


```


MongoDB number cursors open incident.

Overview

Parameters

Debug

1. Check MongoDB version

2. Check number of open cursors for each database

3. Check the number of clients and connections for MongoDB

4. Check the current open file limit for the MongoDB process

5. Check the ulimit settings for the MongoDB process owner

Repair

Reduce the number of open cursors in MongoDB

Learn more

Related Runbooks

Tomcat High JDBC Connection Pool Usage Incident.

Redis too many masters incident

Redis Rejected Connections Incident

MySQL too many connections incident

Support