matrixbridge/purge/README.md

# Clean up postgres synapse database

Shell script that cleans up the matrix postgresql database:
 - removes push notifications that aren't needed
 - kicks out inactive users from rooms
 - cleans up states with synapse_auto_compressor
 - deletes messages that are older than 14 days from rooms

## Running

1. Set SQL credentials, URL and homeserver to local matrix in the script
2. build [synapse_auto_compressor](https://github.com/matrix-org/rust-synapse-compress-state) and set its path in the script
3. make sure that the bridge did start at least once (it creates rooms and adds an admin user that we need)
4. make sure that you do NOT have a [Message Retention Policy](https://matrix-org.github.io/synapse/latest/message_retention_policies.html) set, because this script does it for you, however, media_retention is still needed. This script will not delete any media.
5. add it as a cron job, like:

```
0 2,8,14,23 * * * root /etc/matrix-synapse/matrixpurge.sh
12 11 * * 0 root /etc/matrix-synapse/matrixpurge.sh reset
```

The `"reset"` argument is for resetting the synapse_auto_compressor, it shouldn't be run often, but might come in hany if the compressor ends up in a weird state:

```
/etc/matrix-synapse/matrixpurge.sh reset
```

## Further resources

- [Shrink Synapse Database](https://levans.fr/shrink-synapse-database.html)
- [Message retention policies](https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md)
- [Purge History API](https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md)
- [Find unreferenced state groups](https://github.com/erikjohnston/synapse-find-unreferenced-state-groups)
- [matrix-synapse purge_events.py](https://github.com/matrix-org/synapse/blob/develop/synapse/storage/databases/main/purge_events.py)
- [remove traces of rooms from the db](https://github.com/matrix-org/synapse/issues/14539)

## Useful commands

check currently active queries:

```sql
SELECT pid, query, NOW() - query_start AS elapsed FROM pg_stat_activity WHERE query != '<IDLE>';
```

check events of room sorted by time:

```sql
select content, type, received_ts from events where room_id = '!scTbMproDsaihhGesQ:pixelplanet.fun' and type = 'm.room.message' order by topological_ordering limit 100;
```

largest room by state_groups_state

```sql
SELECT s.room_id, COUNT(s.room_id)
  FROM state_groups_state s
  GROUP BY s.room_id 
  ORDER BY COUNT(s.room_id) DESC
  LIMIT 20;
```

show tables by size:

```sql
SELECT nspname || '.' || relname AS "relation",
    pg_size_pretty(pg_total_relation_size(c.oid)) AS "total_size"
  FROM pg_class c
  LEFT JOIN pg_namespace n ON (n.oid = c.relnamespace)
  WHERE nspname NOT IN ('pg_catalog', 'information_schema')
    AND c.relkind <> 'i'
    AND nspname !~ '^pg_toast'
  ORDER BY pg_total_relation_size(c.oid) DESC
  LIMIT 20;
```

show complexity of room:

```sql
SELECT COUNT(*)/500.0 FROM current_state_events WHERE room_id='...';
```