HiCache host pool: check hugepages, not regular RAM, for the hugetlbfs-backed (CP shared-L2) pool

The host-pool size check used psutil.virtual_memory().available -- regular RAM -- unconditionally. But
hugepages are carved OUT of regular RAM, so reserving e.g. 1 TB of 2M hugepages drops psutil.available
by ~1 TB, and the check then spuriously fails ("Not enough host memory ... only have 897 GB free") even
though the slab is allocated from the hugepages, not regular RAM. It was inspecting the wrong pool.

Two fixes:
- memory_pool_host.py: gate the psutil check on the DEFAULT (regular-malloc) allocator. A custom
  host_tensor_allocator (the CP shared-L2 slab over hugetlbfs) owns its own capacity check, so skip the
  regular-RAM check for it.
- cp_shared_l2_pool.py: add cp_shared_l2_hugetlbfs_free_bytes() + check_cp_shared_l2_hugetlbfs_capacity()
  (statvfs the hugetlbfs mount: f_frsize = hugepage size, f_bavail = free hugepages) and call it in
  create_cp_shared_host_slab before ftruncate/mmap. Now insufficient hugepages fail LOUD with a clean,
  actionable message ("insufficient hugepages in /dev/hugepages: need X GB (N x 2M pages) but only Y GB
  free; reserve more or reduce --hicache-size") instead of a cryptic mmap ENOMEM. Per-slab + sequential,
  so each payload's slab sees the free remaining after the prior ones.

Unit-tested (capacity check: tiny fits, over-capacity fails loud). Pre-existing 2b L2-pooling code (not L3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-23 10:17:33 +00:00
co-authored by Claude Opus 4.8
parent 9dd83bcb88
commit 6114c3e3e5
3 changed files with 77 additions and 12 deletions
@@ -2836,5 +2836,26 @@ class TestCpSharedL2EightRankReserveDeterminism(unittest.TestCase):
self.assertIsNotNone(a.get_range("cp_hicache_node:9", PAYLOAD_TARGET_KV))
class TestCpSharedL2HugetlbfsCapacity(unittest.TestCase):
"""The hugetlbfs capacity check fails LOUD on insufficient hugepages (clean message vs a cryptic mmap
ENOMEM). statvfs works on any mount, so the logic is exercised here against a tmpdir; the real hugetlbfs
semantics (f_frsize = hugepage size, f_bavail = free hugepages) validate on the live server."""
def test_free_bytes_and_capacity_check(self):
import os
import tempfile
free_bytes_fn = _cp_shared_l2_pool.cp_shared_l2_hugetlbfs_free_bytes
check_fn = _cp_shared_l2_pool.check_cp_shared_l2_hugetlbfs_capacity
with tempfile.TemporaryDirectory() as d:
free, pages, page_bytes = free_bytes_fn(d)
self.assertGreater(free, 0)
self.assertGreater(page_bytes, 0)
self.assertEqual(free, pages * page_bytes)
check_fn(d, 4096) # tiny request fits -> no raise
with self.assertRaises(ValueError): # exceeds free capacity -> fail loud
check_fn(d, free + 10**12)
if __name__ == "__main__":
unittest.main()