State cache tweaks (#7095)

Backport of:

- https://github.com/sigp/lighthouse/pull/7067

For:

- https://github.com/sigp/lighthouse/issues/7039


  - Prevent writing to state cache when migrating the database
- Add `state-cache-headroom` flag to control pruning
- Prune old epoch boundary states ahead of mid-epoch states
- Never prune head block's state
- Avoid caching ancestor states unless they are on an epoch boundary
- Log when states enter/exit the cache

Co-authored-by: Eitan Seri-Levi <eserilev@ucsc.edu>

beacon_node/beacon_chain/src/attester_cache.rs

@@ -325,8 +325,10 @@ impl AttesterCache {
             return Ok(value);
         }
 
+        // We use `cache_state = true` here because if we are attesting to the state it's likely
+        // to be recent and useful for other things.
         let mut state: BeaconState<T::EthSpec> = chain
-            .get_state(&state_root, None)?
+            .get_state(&state_root, None, true)?
             .ok_or(Error::MissingBeaconState(state_root))?;
 
         if state.slot() > slot {