MaintenancePolicy ================================ Overview -------- The **MaintenancePolicy** is a central component of the Quake indexing system that automates index upkeep by monitoring query “hit” patterns and deciding when to perform maintenance actions such as splitting or deleting partitions. It ties together several key components: - **PartitionManager**: Manages the set of index partitions. The maintenance policy uses it to query partition sizes, delete outdated partitions, and trigger splits. - **MaintenanceCostEstimator**: Provides latency-based estimates for how much a particular maintenance action (split or delete) will cost. It uses measured or extrapolated scan latencies as a function of partition size and search parameters. - **HitCountTracker**: Records per-query “hit” counts (i.e. which partitions were scanned) over a sliding window. It computes the average scan fraction and maintains history of split and delete events for later analysis. Key Methods ------------------------- 1. **Record Query Hits:** Each time a query is processed, the `record_query_hits()` method is called with the partition IDs that were “hit” during the search. The *HitCountTracker* accumulates these events to compute the current scan fraction across the sliding window. 2. **Perform Maintenance:** When the window is full, `perform_maintenance()` is invoked. This method: - Aggregates hit counts from the *HitCountTracker*. - For each partition, uses the *MaintenanceCostEstimator* to compute a “delta” (change in cost) for both deletion and splitting. The decision is based on whether the estimated cost delta exceeds configured thresholds. - Determines which partitions should be split (to improve query efficiency) or deleted (if underutilized). - Triggers the corresponding operations through the *PartitionManager* and then, if needed, calls local refinement on newly split partitions. - Returns timing information via a *MaintenanceTimingInfo* structure. 3. **Reset:** After maintenance operations complete, the policy can be reset (via `reset()`) to clear the hit history and start a fresh monitoring window. Configuration and Parameters ------------------------------ Maintenance behavior is governed by a set of parameters (encapsulated in the **MaintenancePolicyParams** structure): - **window_size**: Number of queries over which hits are aggregated. - **refinement_radius** and **refinement_iterations**: Control local refinement of new partitions. - **delete_threshold_ns** and **split_threshold_ns**: Latency thresholds (in nanoseconds) that trigger deletion or splitting. - **alpha**: Scaling factor applied to cost estimates. - **enable_split_rejection / enable_delete_rejection**: Flags to allow rejecting an otherwise triggered action if additional checks (such as vector reassignments) suggest it may not be beneficial.