How to extend win32 program virtual memory to 3GB?

This article is participating in the "Golden Stone Project. Share 60,000 cash prize"

In the summary of Windows memory management knowledge , I mentioned

32-bit processes can only use 2GB of virtual memory in Windows

Windows provides us with a way to expand virtual memory to 3GB

This article will continue to introduce

How to expand virtual memory

How should we implement this optimization for online products

Operation process introduction

BCDEdit \set increaseuserva 3072

In systems above XP, you need to open cmd in administrator mode, enter BCDEdit \set increaseuserva 3072, and the system parameter increaseuserva 3072 will be set, where 3072 means 3GB
In XP system, you need to modify boot.ini
For details, please refer to learn.microsoft.com/zh-cn/windo…

Verify that the settings are in effect

Open cmd again, enterbcdedit

Open our application and use vmmap to view the memory situation

It can be seen from the picture that Total + Free has not reached 3GB, but is still 2GB. What's going on? In addition to setting system parameters, we also need to modify the linker parameters of the application. Let's see how to do it

SET CONNECTOR PARAMETER /LARGEADDRESSAWARE

I have an older version of Visual Studio, newer versions should be in a similar location

dumpbin.exe in the Microsoft Visual Studio xx\VC\bin\directory
Open cmd, enter dumpbin /HEADERS {your exe path}, you can see Application can handle large (>2GB) addressthat the connector parameter setting is successful

double check

As you can see, Total + Free has reached 3GB

Let's do the math carefully~ Total + Free is actually equal to 4GB, so what's going on?

是因为我的 windows 操作系统为 64 位，在 64 位系统上运行 LargeAddress 模式的 32 位程序，虚拟内存会被扩展到 4GB，为什么会有这样的结果，后文会具体介绍。

64 位 Windows 操作系统运行 win32 程序

在 64 位系统上除了上面提到的虚拟内存会被扩展到 4GB，还有额外一点要注意： 不需要像 32 位系统一样设置 bcdedit，只需设置连接器参数 /LARGEADDRESSAWARE 即可。

32位进程虚拟内存地址空间不是一共只有 4GB，为什么在 64 位操作系统下能扩展到 4GB？这不是占用了内核空间么？

实际上，并没有占用内核空间地址。原因是 32 位进程跑在 64 位系统时，实际是运行的 64 位进程，所以虚拟内存地址空间为 2^64，低地址 4GB 远达不到内核空间的位置。下面附上一张 64 位进程的虚拟地址空间图

如何落地 LargeAddress 优化？

前面提到过，32 位系统下，需要通过 cmd 设置系统参数，我们总不能要求用户去设置吧？那么有没有能让用户花最小代价设置的方案呢？

我们能想到最直接的方案是，在代码中运行 bcdedit 命令，代码如下

ShellExecute(0, L"runas", L"BCDEdit.exe", L"/set increaseuserva 3072", NULL, SW_SHOWNORMAL);
复制代码

遗憾的是，这样做并不能成功，会报找不到 BCDEdit.exe的错误，但直接在 cmd 中却能设置成功，这又是为什么呢？

因为我测试代码还是运行在 64 位系统上，在 64 位跑 32 位进程时，由于文件系统重定向机制（File System Redirection）程序无法访问 SysWOW64 文件夹，所以需要我们做些额外的操作

BOOL bWow64 = false;
IsWow64Process(GetCurrentProcess(), &bWow64);
if (bWow64)
{
  PVOID OldValue = NULL;
  if (Wow64DisableWow64FsRedirection(&OldValue))
  {
    ShellExecute(0, L"runas", L"BCDEdit.exe", L"/set increaseuserva 3072", NULL, SW_SHOWNORMAL);
    Wow64RevertWow64FsRedirection(&OldValue);
  }
}

复制代码

使用上述代码后，可以成功在 64 位系统上设置 bcdedit 系统参数了。更详细可以参考微软社区大神回答我的内容 learn.microsoft.com/en-us/answe…

But wait! Did n't we just say that the 64-bit system does not need to be bcdeditset ? Well, I did take a detour in this step, but I also learned more knowledge in the process of solving the problem. In addition, the masters of the Microsoft community also hope that the questioner can learn more about some mechanisms of the Windows system

It should also be mentioned that when executing cmd through code, an interface will still pop up allowing the user to choose whether to allow running cmd in administrator mode. Only when the user agrees to open it can the system parameters be set successfully.

RISKS OF LARGEADRRESSAWARE AND HOW TO AVOID

Earlier we introduced in detail how to set the LargeAddress mode, so is there any risk in doing so?

The conclusion is that when the code that is not compatible with the LargeAddress mode allocates a high address exceeding 2GB, it may cause pointer truncation, or the newly allocated address is 0

I haven't researched specific examples. My guess is that some programs will allocate fixed addresses. When the address exceeds 2GB, the address that may be fetched becomes 0, or it may be a specific type of pointer that can only address 2GB. Portions exceeding 2GB are truncated.

So how can we resolve the risk?

We need to use the Top Down memory allocation mode to conduct a comprehensive test . Under normal circumstances, the memory address allocation is from low to high. If it is changed from high to low, we can use the addresses above 2GB first, and we can verify the program. Is there a problem with pointer truncation in the .

So what to do?

Need to modify the registry [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management]

Added AllocationPreference=00100000, see the following figure for details

After the setting is successful, you must restart the computer to take effect!

For details, please refer to stackoverflow.com/questions/2…