database-replica

Urgency and Impact

As of 2024-11-08, database replica is the only way our master is synchronized with slave clusters. It goes without saying that any interruption in the replication process will worsen how the system operates and might lead to unpredictable results.

Alerts

• triggers on source and target tables schema mismatch (e.g. after a migration on master that was not applied on slave)
• triggers when subscriber table last ID is much lower than the master one

On zabbix we have a trigger that checks the growth of the subscribers_subscriber table by MAX(id) and triggers if the growth is less than 10 records in the last 1 hour. Here you can find the Latest data example in Zabbix and the Zabbix Graph example.

Known cases

Affects

• subscribers-tasks
• manager
• directory-messages