kafka-get-lag

In order to get a topic lag we may execute the following command:

master cluster

## process callbacks
JMX_PORT=5557 && kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group worker | grep message_callbacks_raw | awk '{sum += $6} END {print sum}'
 
## reach extractor
JMX_PORT=5557 && kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group reach_extractor | grep message_responses | awk '{sum += $6} END {print sum}'
 
## reach detector
JMX_PORT=5557 && kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group deliveries | grep message_callbacks_raw | awk '{sum += $6} END {print sum}'
 
## subscriber segments observer
JMX_PORT=5557 && kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group directory_segments | awk '{sum += $6} END {print sum}'
 
## subscriber reach detector
```bash
JMX_PORT=5557 && kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group deliveries | grep message_callbacks_raw | awk '{sum += $6} END {print sum}'

statistics lag (ClickHouse)

This will show all lags for statistics group used only by ClickHouse.

JMX_PORT=5557 && kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group statistics

slave cluster

kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group worker | grep message_requests | awk '{sum += $6} END {print sum}'