The compactor is a hidden engine that handles post-ingestion and pre-query workloads in the background, enabling low latency for data ingestion and high performance for queries. The tasks of post-ingestion and pre-query include data file merging, delete application, and data deduplication, which are performed on a separate server to avoid sharing resources with servers handling data loading and reading. Data compaction is a critical process that reorganizes data into smaller files, reduces I/O operations, and improves query performance. The system uses a combination of techniques such as data overlapping, compaction levels, and isolated compactors to optimize resource utilization and improve efficiency. By separating the tasks of ingestion, querying, and compacting workloads into independent servers, the system can maximize resource utilization and minimize out-of-memory incidents.