Table of contents
1. The desktop image captured by the amplifier is incomplete when the display ratio is not 100%.
1.1. Prohibit the system from scaling the software through the manifest file
1.3. Use the winver command to view the year and month version of Windows
2. The memory leak problem encountered when using the amplifier mode
2.3. Check the cause of the memory leak of the desktop sharing module
VC++ common function development summary (column article list, welcome to subscribe, continuous update...) 
https://blog.csdn.net/chenlycly/article/details/124272585 C++ software exception troubleshooting series tutorial from entry to mastery (column article list , Welcome to subscribe, keep updating...) https://blog.csdn.net/chenlycly/article/details/125529931 C++ software analysis tool case collection (column article is being updated...) https://blog.csdn .net/chenlycly/category_12279968.html There are many ways to achieve desktop image acquisition in the WebRTC open source library. In order to support the function of filtering some windows, we adopted the magnification amplifier method, but encountered some problems when using the amplifier acquisition method , Here is a general summary, to provide you with a reference or reference.
1. The desktop image captured by the amplifier is incomplete when the display ratio is not 100%.
In order to support the filter window, our software adopts the magnifying glass acquisition mode supported in the open source WebRTC library, but after using the magnifier mode, the test found that when the display ratio of the system is adjusted to a non-100% display ratio (such as 150%, 200%, etc.) , the desktop image captured by the amplifier component is incomplete, and only a part of the desktop is captured . It should be caused by system DPI display scaling. By default, the system will automatically scale the program according to the current display ratio.
1.1. Prohibit the system from scaling the software through the manifest file
Generally, on a computer with high resolution, it is necessary to set the display ratio of the system to a ratio of more than 100%. The setting entry is to right-click on a blank space on the desktop, click Display Settings in the pop-up right-click menu, and then find the Zoom and Layout bar in the opened window to change the display ratio of the system, as shown below:
By default, the system will automatically scale the program according to the current display ratio , unless we want to disable system scaling, so that the program always maintains a 100% display effect, and check whether the desktop image collected by the amplifier component is complete. Go directly to the properties of our program desktop shortcut and set to prohibit the system from zooming our program. The setting entry is as follows:
That is, no matter what display ratio is set by the system, our program always maintains a 100% display effect. Rerun the program and find that the collected desktop image is complete.
The above settings are manually modified, is there a way to set it through code? What about telling the system not to scale my program? We have studied the system API function SetProcessDPIAware before, this function can prohibit the system from scaling our program, but running the program in the Win10 system has no effect, the system still scales the program. Later, I checked the description of the SetProcessDPIAware function on MSDN, and it reminded me not to use this interface anymore, but to use the method of embedding the manifest file . The relevant instructions are as follows:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0" xmlns:asmv3="urn:schemas-microsoft-com:asm.v3">
<asmv3:application>
<asmv3:windowsSettings>
<dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true</dpiAware>
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness>
</asmv3:windowsSettings>
</asmv3:application>
</assembly>We create a new txt file, then copy the above content into the file, and then rename the file to .mannifest file (note that the name of the manifest file must be consistent with the process name). Then directly add the manifest file to the project like adding a file. After compiling, the startup program will take effect, and the system scaling can be prohibited in the Win10 system.
1.2. Call the SetThreadDpiAwarenessContext function to prohibit the system from scaling the window in the target thread
If you want the magnifier component to capture the full desktop image, you need to disable the system's scaling of the program. But to disable the zooming of the system and let the program always display the 100% size effect, it will cause the program to display too small on some high-resolution computers (such as 2K or 4K Microsoft Surface tablets), and it is impossible to see it at all. Operated. Many software of the Tencent series have disabled the system scaling, and the software has realized the scaling according to the display ratio of the system itself, so they do not have such troubles. However, it is difficult for the program to achieve scaling by itself. We cannot do it at present, so we still need to rely on system scaling.
Later, I found an API function SetThreadDpiAwarenessContext that sets the DPI scaling attribute for the thread , which can prohibit the system from scaling the window created in a certain thread . We can put the operation of the magnifier component into a thread, and then call this function to prohibit the system from zooming the magnifier window in the thread, so that the magnifier component can capture the complete desktop image.
This API interface is placed in the system library user32.dll:
Versions before Win10 are not supported, and even Win10 systems are supported only after version 1607 (released in July 2016). Therefore, to dynamically load from the user32.dll library, if the SetThreadDpiAwarenessContext interface cannot be found, it will return directly; if the interface is found and then called, the relevant code is as follows:
bool SetThreadDpiAware()
{
//设置本线程为DPI感知模式，解决缩放时屏幕采集不全的问题
HMODULE user32_module = GetModuleHandle(TEXT("user32.dll"));
if (nullptr == user32_module)
{
return false;
}
decltype(
&SetThreadDpiAwarenessContext) set_thread_dpi_awareness_context_func =
(decltype(&SetThreadDpiAwarenessContext))GetProcAddress(
user32_module, "SetThreadDpiAwarenessContext");
if (nullptr == set_thread_dpi_awareness_context_func)
{
return false;
}
DPI_AWARENESS_CONTEXT original_dpi_awareness_context = set_thread_dpi_awareness_context_func(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2);
return true;
}1.3. Use the winver command to view the year and month version of Windows
The SetThreadDpiAwarenessContext function mentioned above is only supported by the Win10 system, and it is only supported by versions after "Windows 10, version 1607". The version number 1607 here means that it was released in July 2016. This version 1607 is the year and month version , and the version viewed by using the systeminfo command on the command line is the system internal version:
If you want to view the year and month version released by Windows, you need to use the winver command . After executing this command, the following version window will open:
As shown in the figure, the year and month version of the current machine is 1909, which is the version released in September 2019.
2. The memory leak problem encountered when using the amplifier mode
The test colleagues reported that after launching desktop sharing many times, the software would crash, and the exception capture module did not capture it, and no dump file was generated.
2.1. Using Windbg dynamic debugging, it is found that the software throws a bad_alloc exception due to failure to apply for memory, causing the program to crash
For this kind of scene where the exception is not caught, it is necessary to use Windbg for dynamic debugging. So I asked my colleagues to restart the software, then attach Windbg to it, let Wingdbg run with the software, and then follow the steps to reproduce the flashback problem. When the problem reappears, Windbg senses it immediately and interrupts it. Use the kn command to view the function call stack of the function at this time, as shown below:
It can be seen from the stack that a bad_alloc exception is thrown when using new to dynamically apply for memory, so an abnormal crash is triggered.
2.2. Further analysis found that the memory leak caused insufficient memory in the process, which caused the failure to apply for memory and threw a bad_alloc exception
It should be that the memory application fails due to insufficient memory, and then a bad_alloc exception is thrown. At this time, the process is still there. Use the Process Explorer tool to view the virtual memory usage of the process (the virtual memory in the user mode is displayed) , and you can see that the virtual memory of the process has already occupied 1.7GB. Note that the Windows task manager cannot see the virtual memory occupied by the process, but can only see the content related to the physical memory. You need to use the Process Explorer tool to view it.
The Process Explorer tool does not display the Virtual Size virtual memory by default. You need to right-click the title bar of the process list, click "Select Columns", click the "Process Memory" tab in the pop-up window, and then select the "Virtual Size" option:
Our program is 32-bit, and the system allocates 4GB of virtual memory to the process, of which the user mode virtual memory occupies 2GB, and the kernel mode virtual memory occupies 2GB. The user-mode virtual memory usage of the current program process reaches 1.7GB:
There is still 300MB free from the upper limit of 2GB. Why does it fail when using new to apply for memory? It is possible to apply for a long buffer, and the free 300MB virtual memory is scattered small pieces of memory scattered in different places (this is what we often talk about memory fragmentation), and a large continuous memory cannot be found To allocate , so the memory allocation failed. Why does the virtual memory usage of the program reach as much as 1.7GB? It is estimated that there is a memory leak in the program .
In view of the current operating scenario, there may be a memory leak in the desktop sharing function module, so let the test record the total virtual memory size before initiating desktop sharing, then initiate desktop sharing, and then turn off desktop sharing to see if there is any significant increase in memory. After testing, it is found that the memory will be requested when the sharing is initiated, but the requested memory will not be released after the sharing is stopped, so this operation has a memory leak. Later, Windbg was used to analyze the memory leak, and it was found that the function call stack where the leaked memory occurred pointed to the module code of the desktop sharing. For details on how to use Windbg to analyze memory leaks, please refer to my previous article:
Use Windbg to Locate Memory Leaks in Windows C++ Programs https://blog.csdn.net/chenlycly/article/details/121295720
2.3. Check the cause of the memory leak of the desktop sharing module
2.3.1. It is suspected that the buffer sent by the amplifier component callback has not been released, resulting in a memory leak
In order to achieve window filtering, the desktop sharing module uses the magnification amplifier mode mentioned above, so check the relevant code of the operation amplifier in detail to see why there is a memory leak. The code mainly uses the system API functions MagInitialize, MagUninitialize, MagSetWindowSource, MagSetWindowFilterList, and MagSetImageScalingCallback. I checked the detailed descriptions of these functions on MSDN, but the descriptions are relatively few, and no clues were found, nor did they say that additional Which resources are freed. We also call MagUninitialize to release resources when we end desktop sharing, but there is still a memory leak.
So far, the investigation has reached a deadlock, and it is impossible to proceed. Later, I thought, could it be that the buffer called back by the callback function set by the amplifier component needs to be released externally? This callback function is set by calling the MagSetImageScalingCallback function, which is declared as follows:
ypedef BOOL (CALLBACK* MagImageScalingCallback)(HWND hwnd, void * srcdata, MAGIMAGEHEADER srcheader, void * destdata, MAGIMAGEHEADER destheader, RECT unclipped, RECT clipped, HRGN dirty );The buffer address that is called back is stored in the void * srcdata pointer variable. By printing, it is found that the first address of the buffer that is called back is an unchanging address. It is estimated that a section of buffer that was applied for in the amplifier component at the beginning is used to store the captured desktop sharing. For image data, the image data captured multiple times is stored in the buffer, and then the address of the buffer is called back each time.
It is suspected that the buffer memory allocated inside the amplifier component has not been released, resulting in a memory leak . So the buffer address called back by the callback function is saved in the member variable void* m_srcdata, and the upper layer actively releases the buffer memory when the result is shared. At first, I tried to use delete to release, but an exception occurred when the delete was executed.
2.3.2. Referring to the description of the Remarks section of the API function GetAdaptersAddresses, it is decided to use HeapFree to release
There are many ways to dynamically apply for memory, such as using new (to be released by delete), such as using malloc (to be released by free), or calling the system API function HeapCreate or HeapAlloc (to be released by HeapFree), and You can call the API function VirtualAlloc (use VirtualFree to release), and other API functions.
Which method is used for this buffer? In the past, when writing and reading multiple network card information, it was necessary to call the system API function GetAdaptersAddresses . When calling this interface, the buffer used to store the network card information needs to be applied externally. See the Remarks section of the GetAdaptersAddresses function on MSDN:
It is recommended to use HeapAlloc to apply for memory, and then use HeapFree to release the memory after use. The sample code for calling the system API function GetAdaptersAddresses is as follows:
#include <winsock2.h>
#include <iphlpapi.h>
#include <stdio.h>
#include <stdlib.h>
// Link with Iphlpapi.lib
#pragma comment(lib, "IPHLPAPI.lib")
void PrintAdapterInfo()
{
// Declare and initialize variables
DWORD dwSize = 0;
DWORD dwRetVal = 0;
unsigned int i = 0;
// Set the flags to pass to GetAdaptersAddresses
ULONG flags = GAA_FLAG_INCLUDE_PREFIX;
// default to unspecified address family (both)
ULONG family = AF_UNSPEC;// AF_INET - ipv4, AF_INET6 - ipv6
LPVOID lpMsgBuf = NULL;
PIP_ADAPTER_ADDRESSES pAddresses = NULL;
ULONG outBufLen = 0;
ULONG Iterations = 0;
PIP_ADAPTER_ADDRESSES pCurrAddresses = NULL;
PIP_ADAPTER_UNICAST_ADDRESS pUnicast = NULL;
PIP_ADAPTER_ANYCAST_ADDRESS pAnycast = NULL;
PIP_ADAPTER_MULTICAST_ADDRESS pMulticast = NULL;
IP_ADAPTER_DNS_SERVER_ADDRESS *pDnServer = NULL;
IP_ADAPTER_PREFIX *pPrefix = NULL;
// Allocate a 15 KB buffer to start with.
outBufLen = 15000;
do
{
pAddresses = (IP_ADAPTER_ADDRESSES *)HeapAlloc(GetProcessHeap(), 0, outBufLen);
if (pAddresses == NULL)
{
printf("Memory allocation failed for IP_ADAPTER_ADDRESSES struct\n");
return;
}
dwRetVal = GetAdaptersAddresses( family, flags, NULL, pAddresses, &outBufLen );
if (dwRetVal == ERROR_BUFFER_OVERFLOW)
{
HeapFree( GetProcessHeap(), 0, pAddresses );
pAddresses = NULL;
}
else
{
break;
}
Iterations++;
} while ((dwRetVal == ERROR_BUFFER_OVERFLOW) && (Iterations < 3));
if (dwRetVal == NO_ERROR)
{
// If successful, output some information from the data we received
pCurrAddresses = pAddresses;
while (pCurrAddresses)
{
printf("\tLength of the IP_ADAPTER_ADDRESS struct: %ld\n",
pCurrAddresses->Length);
printf("\tIfIndex (IPv4 interface): %u\n", pCurrAddresses->IfIndex);
printf("\tAdapter name: %s\n", pCurrAddresses->AdapterName);
pUnicast = pCurrAddresses->FirstUnicastAddress;
if (pUnicast != NULL)
{
for (i = 0; pUnicast != NULL; i++)
pUnicast = pUnicast->Next;
printf("\tNumber of Unicast Addresses: %d\n", i);
}
else
printf("\tNo Unicast Addresses\n");
pAnycast = pCurrAddresses->FirstAnycastAddress;
if (pAnycast)
{
for (i = 0; pAnycast != NULL; i++)
pAnycast = pAnycast->Next;
printf("\tNumber of Anycast Addresses: %d\n", i);
}
else
printf("\tNo Anycast Addresses\n");
pMulticast = pCurrAddresses->FirstMulticastAddress;
if (pMulticast)
{
for (i = 0; pMulticast != NULL; i++)
pMulticast = pMulticast->Next;
printf("\tNumber of Multicast Addresses: %d\n", i);
}
else
printf("\tNo Multicast Addresses\n");
pDnServer = pCurrAddresses->FirstDnsServerAddress;
if (pDnServer)
{
for (i = 0; pDnServer != NULL; i++)
pDnServer = pDnServer->Next;
printf("\tNumber of DNS Server Addresses: %d\n", i);
}
else
printf("\tNo DNS Server Addresses\n");
printf("\tDNS Suffix: %wS\n", pCurrAddresses->DnsSuffix);
printf("\tDescription: %wS\n", pCurrAddresses->Description);
printf("\tFriendly name: %wS\n", pCurrAddresses->FriendlyName);
if (pCurrAddresses->PhysicalAddressLength != 0)
{
printf("\tPhysical address: ");
for (i = 0; i < (int)pCurrAddresses->PhysicalAddressLength;i++)
{
if (i == (pCurrAddresses->PhysicalAddressLength - 1))
printf("%.2X\n",
(int)pCurrAddresses->PhysicalAddress[i]);
else
printf("%.2X-",
(int)pCurrAddresses->PhysicalAddress[i]);
}
}
printf("\tFlags: %ld\n", pCurrAddresses->Flags);
printf("\tMtu: %lu\n", pCurrAddresses->Mtu);
printf("\tIfType: %ld\n", pCurrAddresses->IfType);
printf("\tOperStatus: %ld\n", pCurrAddresses->OperStatus);
printf("\tIpv6IfIndex (IPv6 interface): %u\n",
pCurrAddresses->Ipv6IfIndex);
printf("\tZoneIndices (hex): ");
for (i = 0; i < 16; i++)
printf("%lx ", pCurrAddresses->ZoneIndices[i]);
printf("\n");
printf("\tTransmit link speed: %I64u\n", pCurrAddresses->TransmitLinkSpeed);
printf("\tReceive link speed: %I64u\n", pCurrAddresses->ReceiveLinkSpeed);
pPrefix = pCurrAddresses->FirstPrefix;
if (pPrefix)
{
for (i = 0; pPrefix != NULL; i++)
pPrefix = pPrefix->Next;
printf("\tNumber of IP Adapter Prefix entries: %d\n", i);
}
else
printf("\tNumber of IP Adapter Prefix entries: 0\n");
printf("\n");
pCurrAddresses = pCurrAddresses->Next;
}
}
else
{
printf("Call to GetAdaptersAddresses failed with error: %d\n",
dwRetVal);
if (dwRetVal == ERROR_NO_DATA)
printf("\tNo addresses were found for the requested parameters\n");
else
{
if (FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
NULL, dwRetVal, MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
// Default language
(LPTSTR)& lpMsgBuf, 0, NULL))
{
printf("\tError: %s", lpMsgBuf);
LocalFree(lpMsgBuf);
if (pAddresses)
HeapFree(GetProcessHeap(), 0, pAddresses);
}
}
}
if (pAddresses)
{
HeapFree(GetProcessHeap(), 0, pAddresses);
}
return;
}According to the tips here, it is estimated that the Windows API prefers to use HeapAlloc to dynamically apply for heap memory, so here I decided to try HeapFree to release the buffer memory that the amplifier component called back. After testing, the use of HeapFree is effective. Using the new version of the test, there is no obvious memory leak after the desktop sharing is stopped every time. It is basically determined that the memory leak is caused by not releasing the buffer memory from the callback.
2.4. Why does the buffer sent by the callback function set by the MagSetImageScalingCallback interface need to be released externally?
In MSDN's MagSetImageScalingCallback function description page:
It has been shown that the MagSetImageScalingCallback function is abandoned by Microsoft, and the application should not use it any more. But for desktop image collection, this interface must be used to set the callback, and the collected desktop image will be called back through the set callback. As for the problem that the buffer sent by the callback function needs to be released externally, I don’t know if it has something to do with it.
2.5. Solved the memory leak in amplifier mode, but WebRTC related modules still have a small memory leak
However, after detailed testing, there is still a slight memory leak after desktop sharing is initiated and stopped, about 5MB each time. The same problem exists with the previous old version test, about 5MB each time. This should be a memory leak in the open source WebRTC library, and it should have nothing to do with using the amplifier mode, because the old version of desktop sharing does not use the desktop sharing mode. This 5MB leak can be tolerated and will not directly cause problems, but it is a hidden danger. For example, if the customer does not shut down the computer for a long time and the software runs for a long time, if the desktop sharing is initiated many times, the memory leak will continue to accumulate. For example, if the desktop sharing is initiated 100 times, 100*5=500MB of memory will be leaked, and the impact will be relatively large.
However, the internal code of the open source WebRTC is more complicated, and it is more difficult to troubleshoot. I will study it in detail later when I have time. At present, the obvious memory leak caused by this amplifier mode is over.
3. Finally
When encountering the above problems before, I have not found any useful information on the Internet. Here I will make a detailed summary of these two typical problems encountered by using the magnification amplifier method, so as to provide you with a reference or reference.