Cassandra marks a data to be deleted by writing a "tombstone". The marked data defaults to 10 days (gc_grace_seconds in the configuration file) and is actually deleted from the disk when it is executed by compaction or cleanup to the corresponding SSTable, because if the delete operation was only on 2 of the 3 nodes at that time If the execution is successful, then once the two nodes with tombstone delete the data, only the node without tombstone remains on the cluster. The next time the key is read, the corresponding data will be returned, resulting in the resurrection of the deleted data. The Repair operation can synchronize the data of all nodes to ensure that tombstone exists in all three nodes. Therefore, if you want to ensure that the deletion is 100% successful, it will not be revived. You need to periodically perform repair operations with a period of less than gc_grace_seconds (so the official recommendation is "weekly")
However, when selecting data, all tombstones that meet the query conditions encountered in each SSTable should be placed in memory so that when the data of each SSTable file is merged, which column data has not been deleted can be finally returned, too many tombstones will cause The heap is heavily occupied and various GCs are needed to affect performance, so cassandra reads 100,000 (configurable) tombstones by default and forcibly cancels this reading, because in this case you must be "doing it wrong". Therefore, if you often delete the column key under a row key, and you need to select multiple column keys under a row key at a time, it is easy to encounter this situation. Tombstone is often read successfully even if it is less than 10w. This time the reading will be very slow, and it will soon lead to triggering the young generation GC and slow down the response speed of the entire node.
The best solution is definitely to avoid it when designing. However, if there is no need to modify the table structure, there are two solutions: if you cannot accept a column key to be deleted and resurrected, first change gc_grace_seconds to 0, and then use the ALL mode when deleting, of course, as long as one node times out / Failed to delete will fail; if you can accept the resurrection of the deleted data, the deletion is only to reduce the use of disk space, then directly set gc_grace_seconds to 0, and resurrect it.
