Repository navigation
v3: advise MADV_HUGEPAGE independently of allow_large - #1504
Closed
Matthew Charles Vladislav Busel (Mattbusel) wants to merge 1 commit into
Closed
Matthew Charles Vladislav Busel (Mattbusel) wants to merge 1 commit into
Matthew Charles Vladislav Busel (Mattbusel) wants to merge 1 commit into
Conversation
Port of microsoft#1328 to v3. The aligned page meta data change made the OS fallback in mi_arenas_page_alloc_from_os allocate with allow_large=false (it commits lazily), and unix_mmap only advised MADV_HUGEPAGE when allow_large was set. With the distro default THP setting (madvise), huge objects therefore stopped getting transparent huge pages. allow_large still gates MAP_HUGETLB. The THP advise now depends only on mi_option_allow_thp and _mi_os_canuse_large_page, as in v2. Fixes microsoft#1502. Assisted-by: Claude:claude-opus-5-5
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Collaborator
Author
|
Works for me, thanks Daan. Glad it's fixed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1502.
Port of #1328 to v3. Since the aligned page meta data change (82fd55f, 67b324e),
mi_arenas_page_alloc_from_osallocates withallow_large=falsebecause it commits lazily (_mi_prim_allocassertscommit || !allow_large).unix_mmaponly advisedMADV_HUGEPAGEwhenallow_largewas set, so on Linux with the default THP setting (madvise) huge objects no longer get transparent huge pages.This drops
allow_largefrom the THP advise condition insrc/prim/unix/prim.c, as #1328 did for v2.allow_largestill gatesMAP_HUGETLB.MIMALLOC_ALLOW_THP=0still disables the advise.Measured
WSL2 Ubuntu 24.04, kernel 6.18, THP
[madvise], static Release builds of dev3 (0bba5cc) and this branch. The test doesmi_malloc(n),memsetover it, then reads the mapping'sVmFlagsandAnonHugePagesfrom/proc/self/smapsand the minor fault count fromgetrusage:hg)hg)hg)hg)The 64 MiB object comes from an arena and was already advised. With
MIMALLOC_ALLOW_THP=0the 256 MiB case is back to 65542 faults and nohg.ctestpasses (16/16) in Release and Debug on the same machine.Written with help from Claude (Assisted-by trailer on the commit).