---
id: ab235621-3dc1-436c-a068-073a866dc3b8
---

# MongoDB Write Latency Higher Than Average
---

This incident type refers to a situation where the write latency for a MongoDB database is higher than the average. This might cause delays in write operations and can negatively impact the performance of the system. The incident might be triggered by an alert that indicates that the write latency has exceeded a certain threshold, and it requires immediate attention from a software engineer to identify and resolve the underlying issue. The incident can be resolved by identifying and fixing the root cause of the latency issue, and ensuring that the system is functioning within acceptable performance parameters.

### Parameters
```shell
# Environment Variables

export MONGODB_PORT="PLACEHOLDER"

export MONGODB_HOST="PLACEHOLDER"

export MONGODB_PASSWORD="PLACEHOLDER"

export IP_ADDRESS="PLACEHOLDER"

export MONGODB_USERNAME="PLACEHOLDER"

export SERVER_ADDRESS="PLACEHOLDER"
```

## Debug

### Check if MongoDB is running
```shell
systemctl status mongodb
```

### Check if MongoDB is listening on the correct port
```shell
netstat -tln | grep ${MONGODB_PORT}
```

### Check for any errors in the MongoDB log file
```shell
tail -n 100 /var/log/mongodb/mongodb.log
```

### Check the status of the MongoDB replica set
```shell
mongo --host ${MONGODB_HOST} --port ${MONGODB_PORT} --eval "rs.status()"
```

### Check the latency of MongoDB writes
```shell
mongo --host ${MONGODB_HOST} --port ${MONGODB_PORT} --eval "db.getSiblingDB('admin').runCommand({whatsmyuri: 1}).you" # get the IP address of the current connection

mongostat --host ${MONGODB_HOST} --port ${MONGODB_PORT} --discover --authenticationDatabase admin --username ${MONGODB_USERNAME} --password ${MONGODB_PASSWORD} --rowcount 1 --noheaders | grep ${IP_ADDRESS} | awk '{print $NF}' # get the write latency of the current connection
```

### Check the disk space usage of the MongoDB server
```shell
df -h /var/lib/mongodb
```

### Check the CPU and memory usage of the MongoDB process
```shell
top -p $(pidof mongod)
```


## Repair

### Check if there are any long-running queries or operations that are causing the high write latency, optimize or terminate them if possible.
```shell

#!/bin/bash

OP_ID="PLACEHOLDER"

# Connect to the MongoDB server using the ${USERNAME}, ${PASSWORD}, and ${SERVER_ADDRESS} parameters.

mongo --username ${USERNAME} --password ${PASSWORD} ${SERVER_ADDRESS}

# Check for any long-running queries or operations using the "currentOp" command.

db.currentOp({"secs_running": {"$gte": 60}})

# If there are any long-running queries or operations, terminate them using the "killOp" command.

db.killOp(${OP_ID})

# Close the MongoDB connection.

exit

```


This incident type refers to a situation where the write latency for a MongoDB database is higher than the average. This might cause delays in write operations and can negatively impact the performance of the system. The incident might be triggered by an alert that indicates that the write latency has exceeded a certain threshold, and it requires immediate attention from a software engineer to identify and resolve the underlying issue. The incident can be resolved by identifying and fixing the root cause of the latency issue, and ensuring that the system is functioning within acceptable performance parameters.


This incident type indicates that there is a high latency issue in the execution of a Spark job. Spark is a distributed computing framework that is used for processing large datasets. High latency in this context means that the time taken to execute the Spark job is significantly longer than expected or normal. This can result in delays in processing data and can impact the performance of the application or system that is utilizing Spark.


High Latency Incident for Spark Job Execution.

This incident type refers to a scenario where there is a high rate of statement timeouts in a Postgresql database instance. This can lead to degraded performance and potentially impact the availability of the database. It is important to quickly identify and address the underlying cause of the timeouts to ensure the stability of the system.


Postgresql high rate statement timeout incident.

This incident type involves an alert triggered when read latency is higher than usual for a MongoDB host. The alert is triggered when at least 100% of the average read latency values have been more than 2 deviations above the predicted values during the last 15 minutes. The incident requires monitoring and investigation to identify the root cause of the increased read latency and take appropriate actions to resolve it.


Read latency higher than average for MongoDB host.

This incident type describes an issue where the efficiency of queries in MongoDB is degrading. The incident may be triggered by an alert monitoring system that detects an anomaly in the performance of queries executed in the database. The incident may result in a decrease in system performance and may require the attention of a software engineer to troubleshoot and resolve the issue.


MongoDB Efficiency of queries is degrading

This incident type refers to an issue with high virtual memory usage in a MongoDB instance. It can lead to performance degradation or even service downtime. It requires immediate attention from the responsible team to investigate the root cause and take corrective measures to resolve the issue.


MongoDB virtual memory usage incident.

```shell
# Environment Variables

export MONGODB_PORT="PLACEHOLDER"

export MONGODB_HOST="PLACEHOLDER"

export MONGODB_PASSWORD="PLACEHOLDER"

export IP_ADDRESS="PLACEHOLDER"

export MONGODB_USERNAME="PLACEHOLDER"

export SERVER_ADDRESS="PLACEHOLDER"
```


### Check if MongoDB is running

```shell
systemctl status mongodb
```

### Check if MongoDB is listening on the correct port

```shell
netstat -tln | grep ${MONGODB_PORT}
```

### Check for any errors in the MongoDB log file

```shell
tail -n 100 /var/log/mongodb/mongodb.log
```

### Check the status of the MongoDB replica set

```shell
mongo --host ${MONGODB_HOST} --port ${MONGODB_PORT} --eval "rs.status()"
```

### Check the latency of MongoDB writes

```shell
mongo --host ${MONGODB_HOST} --port ${MONGODB_PORT} --eval "db.getSiblingDB('admin').runCommand({whatsmyuri: 1}).you" # get the IP address of the current connection

mongostat --host ${MONGODB_HOST} --port ${MONGODB_PORT} --discover --authenticationDatabase admin --username ${MONGODB_USERNAME} --password ${MONGODB_PASSWORD} --rowcount 1 --noheaders | grep ${IP_ADDRESS} | awk '{print $NF}' # get the write latency of the current connection
```

### Check the disk space usage of the MongoDB server

```shell
df -h /var/lib/mongodb
```

### Check the CPU and memory usage of the MongoDB process

```shell
top -p $(pidof mongod)
```


### Check if there are any long-running queries or operations that are causing the high write latency, optimize or terminate them if possible.

```shell

#!/bin/bash

OP_ID="PLACEHOLDER"

# Connect to the MongoDB server using the ${USERNAME}, ${PASSWORD}, and ${SERVER_ADDRESS} parameters.

mongo --username ${USERNAME} --password ${PASSWORD} ${SERVER_ADDRESS}

# Check for any long-running queries or operations using the "currentOp" command.

db.currentOp({"secs_running": {"$gte": 60}})

# If there are any long-running queries or operations, terminate them using the "killOp" command.

db.killOp(${OP_ID})

# Close the MongoDB connection.

exit

```


MongoDB Write Latency Higher Than Average

Overview

Parameters

Debug

Check if MongoDB is running

Check if MongoDB is listening on the correct port

Check for any errors in the MongoDB log file

Check the status of the MongoDB replica set

Check the latency of MongoDB writes

Check the disk space usage of the MongoDB server

Check the CPU and memory usage of the MongoDB process

Repair

Check if there are any long-running queries or operations that are causing the high write latency, optimize or terminate them if possible.

Learn more

Related Runbooks

High Latency Incident for Spark Job Execution.

Postgresql high rate statement timeout incident.

Read latency higher than average for MongoDB host.

MongoDB Efficiency of queries is degrading

Support