---
id: 754a365a-616d-11ee-8c99-0242ac120002
---

# Missing Cassandra JMX Metrics on Dashboard
---

This incident type refers to the issue of missing JMX metrics for Cassandra on a dashboard. JMX metrics are important for monitoring the performance of Cassandra, and their absence on a dashboard can lead to difficulties in identifying and resolving issues that may arise. This incident requires investigation to determine the cause of the issue and to restore the missing metrics to the dashboard.

### Parameters
```shell
export JMX_PORT="PLACEHOLDER"

export JMX_USERNAME="PLACEHOLDER"

export JMX_PASSWORD="PLACEHOLDER"

export CASSANDRA_HOST="PLACEHOLDER"

export DASHBOARD_CONFIG_FILE="PLACEHOLDER"

export CASSANDRA_PORT="PLACEHOLDER"

export IP_ADDRESS="PLACEHOLDER"

export IP_ADDRESS_OF_CASSANDRA_NODE="PLACEHOLDER"

export JMX_PORT_NUMBER="PLACEHOLDER"
```

## Debug

### Step 1: Check if Cassandra is running
```shell
sudo service cassandra status
```

### Step 2: Check if JMX is enabled in Cassandra's configuration file
```shell
sudo cat /etc/cassandra/cassandra.yaml | grep -A 1 "jmx:"
```

### Step 3: Check if JMX is reachable from the local machine
```shell
sudo apt-get install -y jmxterm

java -jar /usr/share/jmxterm/jmxterm.jar -l localhost:${JMX_PORT} -u ${JMX_USERNAME} -p ${JMX_PASSWORD} -n -v silent -c "domains"
```

### Step 4: Check if JMX is reachable from the remote machine where the dashboard is hosted
```shell
java -jar /usr/share/jmxterm/jmxterm.jar -l ${CASSANDRA_HOST}:${JMX_PORT} -u ${JMX_USERNAME} -p ${JMX_PASSWORD} -n -v silent -c "domains"
```

### Step 5: Check if the metrics are being collected in Cassandra
```shell
nodetool sjk mxdump ${CASSANDRA_HOST}:${JMX_PORT} | grep -i "CassandraMetrics"
```

### Step 6: Check if the dashboard is properly configured to display the JMX metrics
```shell
sudo cat ${DASHBOARD_CONFIG_FILE} | grep -i "CassandraMetrics"
```

## Repair

### Verify that the JMX interface is enabled on the Cassandra nodes and that the appropriate JMX ports are open and accessible.
```shell


#!/bin/bash



# Define variables

${IP_ADDRESS}="${IP_ADDRESS_OF_CASSANDRA_NODE}"

${JMX_PORT}="${JMX_PORT_NUMBER}"



# Verify that the JMX interface is enabled on the Cassandra node

jmx_enabled=$(sudo grep "com.sun.management.jmxremote" /etc/cassandra/cassandra-env.sh | wc -l)



if [ $jmx_enabled -eq 0 ]; then

  echo "JMX interface is not enabled on the Cassandra node."

  exit 1

else

  echo "JMX interface is enabled on the Cassandra node."

fi



# Verify that the JMX port is open and accessible

jmx_open=$(sudo nc -zv $IP_ADDRESS $JMX_PORT 2>&1 | grep succeeded | wc -l)



if [ $jmx_open -eq 0 ]; then

  echo "JMX port is not open or accessible on the Cassandra node."

  exit 1

else

  echo "JMX port is open and accessible on the Cassandra node."

fi



exit 0


```


This incident type refers to the issue of missing JMX metrics for Cassandra on a dashboard. JMX metrics are important for monitoring the performance of Cassandra, and their absence on a dashboard can lead to difficulties in identifying and resolving issues that may arise. This incident requires investigation to determine the cause of the issue and to restore the missing metrics to the dashboard.


This incident type refers to the failure of authentication in a Kafka cluster that is using the SASL\_SSL protocol for authentication. This could be caused by various reasons such as invalid credentials, expired certificates, or configuration issues. As a result, users may not be able to access the Kafka cluster, or may experience delays or errors when attempting to do so. This can have a significant impact on the overall functionality of the software system that relies on the Kafka cluster for data processing and communication.


Authentication Failure in Kafka Cluster with SASL_SSL protocol.

This incident type refers to a situation where there is a significant delay in the execution of queries on a Cassandra cluster. This delay can cause the system to become unresponsive and result in slower performance. It may be caused by a variety of factors such as an increase in traffic, inefficient queries, or hardware issues. The issue can impact the functionality of the system and requires immediate attention to prevent further disruption.


Slow Query Performance on Cassandra Cluster.

In this incident type, there is an issue with a Cassandra cluster where one or more disks are running slow. This can cause performance issues and potentially lead to data loss or downtime. The goal is to identify and address the specific disk(s) causing the problem in order to restore normal cluster operations.


Slow Disk in Cassandra Cluster

This incident type refers to the report of overheating by Prometheus exporter at a specific instance for the Cassandra service. It could be caused by a high volume of traffic or some issues with the server, and it requires immediate attention to avoid any potential downtime or data loss. The incident is assigned to an engineer who will investigate and resolve the issue as quickly as possible.


Overheating reported by Prometheus exporter at {{ $labels.instance }} for Cassandra service.

This incident type refers to a situation where a high number of mutations are being dropped on a Cassandra database. Mutations are changes made to the database, such as inserting new data or updating existing data. When mutations are dropped, it means that they were not successfully recorded in the database. This can be caused by a variety of factors, such as hardware or network issues, configuration problems, or bugs in the software. When this occurs, it can result in data inconsistencies or loss, and can impact the performance and reliability of the application that relies on the database.


High Number of Dropped Mutations on Cassandra Database

```shell
export JMX_PORT="PLACEHOLDER"

export JMX_USERNAME="PLACEHOLDER"

export JMX_PASSWORD="PLACEHOLDER"

export CASSANDRA_HOST="PLACEHOLDER"

export DASHBOARD_CONFIG_FILE="PLACEHOLDER"

export CASSANDRA_PORT="PLACEHOLDER"

export IP_ADDRESS="PLACEHOLDER"

export IP_ADDRESS_OF_CASSANDRA_NODE="PLACEHOLDER"

export JMX_PORT_NUMBER="PLACEHOLDER"
```


### Step 1: Check if Cassandra is running

```shell
sudo service cassandra status
```

### Step 2: Check if JMX is enabled in Cassandra's configuration file

```shell
sudo cat /etc/cassandra/cassandra.yaml | grep -A 1 "jmx:"
```

### Step 3: Check if JMX is reachable from the local machine

```shell
sudo apt-get install -y jmxterm

java -jar /usr/share/jmxterm/jmxterm.jar -l localhost:${JMX_PORT} -u ${JMX_USERNAME} -p ${JMX_PASSWORD} -n -v silent -c "domains"
```

### Step 4: Check if JMX is reachable from the remote machine where the dashboard is hosted

```shell
java -jar /usr/share/jmxterm/jmxterm.jar -l ${CASSANDRA_HOST}:${JMX_PORT} -u ${JMX_USERNAME} -p ${JMX_PASSWORD} -n -v silent -c "domains"
```

### Step 5: Check if the metrics are being collected in Cassandra

```shell
nodetool sjk mxdump ${CASSANDRA_HOST}:${JMX_PORT} | grep -i "CassandraMetrics"
```

### Step 6: Check if the dashboard is properly configured to display the JMX metrics

```shell
sudo cat ${DASHBOARD_CONFIG_FILE} | grep -i "CassandraMetrics"
```


### Verify that the JMX interface is enabled on the Cassandra nodes and that the appropriate JMX ports are open and accessible.

```shell


#!/bin/bash



# Define variables

${IP_ADDRESS}="${IP_ADDRESS_OF_CASSANDRA_NODE}"

${JMX_PORT}="${JMX_PORT_NUMBER}"



# Verify that the JMX interface is enabled on the Cassandra node

jmx_enabled=$(sudo grep "com.sun.management.jmxremote" /etc/cassandra/cassandra-env.sh | wc -l)



if [ $jmx_enabled -eq 0 ]; then

  echo "JMX interface is not enabled on the Cassandra node."

  exit 1

else

  echo "JMX interface is enabled on the Cassandra node."

fi



# Verify that the JMX port is open and accessible

jmx_open=$(sudo nc -zv $IP_ADDRESS $JMX_PORT 2>&1 | grep succeeded | wc -l)



if [ $jmx_open -eq 0 ]; then

  echo "JMX port is not open or accessible on the Cassandra node."

  exit 1

else

  echo "JMX port is open and accessible on the Cassandra node."

fi



exit 0


```


Missing Cassandra JMX Metrics on Dashboard

Overview

Parameters

Debug

Step 1: Check if Cassandra is running

Step 2: Check if JMX is enabled in Cassandra's configuration file

Step 3: Check if JMX is reachable from the local machine

Step 4: Check if JMX is reachable from the remote machine where the dashboard is hosted

Step 5: Check if the metrics are being collected in Cassandra

Step 6: Check if the dashboard is properly configured to display the JMX metrics

Repair

Verify that the JMX interface is enabled on the Cassandra nodes and that the appropriate JMX ports are open and accessible.

Learn more

Related Runbooks

Authentication Failure in Kafka Cluster with SASL_SSL protocol.

Slow Query Performance on Cassandra Cluster.

Slow Disk in Cassandra Cluster

Overheating reported by Prometheus exporter at {{ $labels.instance }} for Cassandra service.

Support