We recently encountered this problem when debugging 32-bit products: using ddr virtual disk as a hard disk to start the system (because the maximum address space that 32-bit products can access is only 4G, the address space of the kernel space is also relatively tight. )
The implementation of our virtual disk is to use a large piece of continuous physical memory as the disk, and map this physical memory to the virtual address space of vmalloc through vm_map_ram. Then a problem will arise at this time. When we When the mapped area is large and exceeds VMALLOC_END-VMALLOC_START, you will find that such a large address space cannot be mapped, and an overflow log will be reported during the mapping process.
It is easy to see from the above code that it is because start+size>end, so how much address space can vmaloc map? We have to start with the kernel address space. So for a 32-bit processor, the kernel space should What does it look like? As shown below
We can see that the kernel space is only 1G, and the linear mapping area occupies 896M. Starting from 3G+896M is the high-end memory. The high-end memory includes the Vmalloc area and some temporary mapping areas and permanent mapping areas.
So how much space can be given to Vmalloc by the kernel? We have to look at it in conjunction with the code: There is the following code in arch/arm/mm/mmu.c:
From the above code, we can analyze that: when there is no vmalloc=size command in cmdline, early_vmalloc will not be executed, so the size of vmalloc is determined by vmalloc_min on line 1081. Obviously, when we do not set vmalloc size, vmalloc The area is only 240M in size, hence the problem we started with. So since it is too small, of course we have to look at how big vmalloc can be set (how big can it be, and how small can it be??). From early_vmalloc, we can easily draw the following conclusions: The maximum size of vmalloc can only be 984. The minimum is 16M, so we can set the size of vmalloc to be larger than the required size.
The problem is solved here, but we definitely have a question. The entire kernel space is only 1G. If more than 900M is given to Vmalloc, then where does so much space come from? ? Here we need to look at the code analysis. For this, we need to look at sanity_check_meminfo (I don’t like to post so much code, but I have to post it to illustrate the problem):
Note: The concept of high-end memory will not be introduced here, just educate yourself (this concept exists for 32-bit machines, but not for 64-bit machines)
void __init sanity_check_meminfo(void)
{
phys_addr_t memblock_limit = 0;
int highmem = 0;
phys_addr_t vmalloc_limit = __pa(vmalloc_min - 1) + 1;// Get the physical address from the vmalloc_min-PAGE_OFFSET obtained above
struct memblock_region *reg;
bool should_use_highmem = false;
/*This is mainly to get the physical address block passed in*/
for_each_memblock(memory, reg) {
phys_addr_t block_start = reg->base;
phys_addr_t block_end = reg->base + reg->size;
phys_addr_t size_limit = reg->size;
if (reg->base >= vmalloc_limit)//Determine whether there is a corresponding block that exceeds this vmalloc_min. If so, this physical address needs to be mapped through high-end memory, and the linear mapping area cannot be used.
highmem = 1;
else
size_limit = vmalloc_limit - reg->base;// If it does not exceed the size limit between the starting address of this physical block and vmalloc_min!
if (!IS_ENABLED(CONFIG_HIGHMEM) || cache_is_vipt_aliasing()) {
if (highmem) {
pr_notice("Ignoring RAM at %pa-%pa (!CONFIG_HIGHMEM)\n",
&block_start, &block_end);
memblock_remove(reg->base, reg->size);// If you use high-end memory, then we need to remove this memory from memblock
should_use_highmem = true;
continue;
}
if (reg->size > size_limit) {/ /If part of the memory block exceeds vmalloc_min, then the exceeded area needs to be mapped through high-end memory, so it needs to be removed using memblock_remove
phys_addr_t overlap_size = reg->size - size_limit;
pr_notice("Truncating RAM at %pa-%pa to -%pa",
&block_start, &block_end, &vmalloc_limit);
memblock_remove(vmalloc_limit, overlap_size);
block_end = vmalloc_limit;
should_use_highmem = true;
}
}
if (!highmem) {/ /Here is the key to the source of vmalloc
if (block_end > arm_lowmem_limit) {
if (reg->size > size_limit)
arm_lowmem_limit = vmalloc_limit;// Update arm_lowmem_limit through vmalloc_limit
else
arm_lowmem_limit = block_end;
}
/*
* Find the first non-pmd-aligned page, and point
* memblock_limit at it. This relies on rounding the
* limit down to be pmd-aligned, which happens at the
* end of this function.
*
* With this algorithm, the start or end of almost any
* bank can be non-pmd-aligned. The only exception is
* that the start of the bank 0 must be section-
* aligned, since otherwise memory would need to be
* allocated when mapping the start of bank 0, which
* occurs before any free memory is mapped.
*/
if (!memblock_limit) {
if (!IS_ALIGNED(block_start, PMD_SIZE))
memblock_limit = block_start;
else if (!IS_ALIGNED(block_end, PMD_SIZE))
memblock_limit = arm_lowmem_limit;
}
}
}
if (should_use_highmem)
pr_notice("Consider using a HIGHMEM enabled kernel.\n");
high_memory = __va(arm_lowmem_limit - 1) + 1;// The address of high_memory will be updated by arm_lowmem_init. We know that VMALLOC_START=high_memory+8M
/*So here we can know that when vmalloc is larger, the linear mapping area (original 896M) will be compressed to a small size*/
/*
* Round the memblock limit down to a pmd size. This
* helps to ensure that we will allocate memory from the
* last full pmd, which should be mapped.
*/
if (memblock_limit)
memblock_limit = round_down(memblock_limit, PMD_SIZE);
if (!memblock_limit)
memblock_limit = arm_lowmem_limit;
blocking_set_current_limit(memblock_limit);
}
Ok, the problem is solved, and we now know where vmalloc comes from and where it goes! !
Please correct me if I am wrong! ! !