I encountered an error while indexing Sphinx today: WARNING: collect_hits: mem_limit=-2097152 kb too low, increasing to -118752 kb. After troubleshooting, it was found that there was a problem with the mem_limit configuration item. The configuration item mem_limit in the indexer index was when the index was built Index memory limit value, this value is not as big as possible. When modifying the configuration today, considering that the server memory is large enough, I changed this configuration to 2G or 2048M, and then encountered the following when indexing The error is reported as follows:
[kermit@kermit sphinxdata]$ sudo bin/indexer -c /usr/local/coreseek4.1/etc/test.conf base
Coreseek Fulltext 4.1 [Sphinx 2.0.2-dev (r2922)]
Copyright © 2007 -2011,
Beijing Choice Software Technologies Inc (http://www.coreseek.com)
using config file'/usr /local/coreseek4.1/etc/test.conf
'… indexing
index'base '… WARNING: collect_hits: mem_limit =-2097152 kb too low, increasing to -118752 kb
*** Oops, indexer crashed! Please send the following report to developers.
Sphinx 2.0.2-dev (r2922)
-------------- report begins here ---------------
Current document: docid=14844312, hits=0
Current batch: minid=0, maxid=0
Hit pool start: docid=0, hit=0
-------------- backtrace begins here ---------------
Program compiled with gcc 4.4.7
Host OS is Linux BFG-OSER-4435 2.6.32-642.15.1.el6.x86_64 #1 SMP Fri Feb 24 14:31:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Stack bottom = 0x0, thread stack size = 0x14000
begin of system backtrace:
begin of system symbols:
bin/indexer(_Z12sphBacktraceib+0x95)[0x521045]
bin/indexer(_Z7sigsegvi+0xbe)[0x45cdfe]
/lib64/libpthread.so.0[0x317000f7e0]
bin/indexer(_ZN13CSphIndex_VLN5BuildERK10CSphVectorIP10CSphSource16CSphVectorPolicyIS2_EEii+0x33f9)[0x4aea19]
bin/indexer(_Z7DoIndexRK17CSphConfigSectionPKcRK17SmallStringHash_TIS_EbP8_IO_FILE+0x1fb6)[0x464e66]
bin/indexer(main+0x21fe)[0x46733e]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x316fc1ed1d]
bin/indexer[0x45c5a9]
Backtrace looks OK. Now you have to do following steps:
- Run the command over the crashed binary (for example, ‘indexer’):
nm -n indexer > indexer.sym - Attach the binary, generated .sym and the text of backtrace (see above) to the bug report.
Also you can read the section about resolving backtraces in the documentation.
-------------- backtrace ends here ---------------#The following is the indexer configuration in sphinx's .conf configuration file.
indexer
{ mem_limit = 2048M # Generally speaking, this configuration can be 256M to 1024M. } From the error report, you can see that mem_limit=-2097152 kb too low, the value of 2097152 is 2048M. But the configuration file thinks that this configuration is too low, which shows that this configuration is not recognized by Sphinx. I found such a paragraph in the official document. The original text and translation are as follows: mem_limit-----------------
You can resolve that either by raisingtimeouts on SQL server side or by lowering mem_limit. Example:| mem_limit = 256M| # mem_limit = 262144K # same, but in KB| # mem_limit = 268435456 # same, but in bytes. Index RAM usage limit. Optional, the default is 32M. The indexer will not exceed the memory usage limit after mandatory. This configuration can be specified in units of bytes or kilobytes (using K postfix) or megabytes (using M postfix); see this example. If set to a very low value that causes the I/O buffer to be less than 8KB, this limit will be automatically increased; to be precise, it depends on the data size of the index. If the buffer does not exceed 256KB, a warning will be generated. The maximum possible limit is 2047M. Too low a value will hurt the indexing speed, but if it is not the entire data set, 256M to 1024M is sufficient. Setting this value too high may cause SQL Server to time out. During the document collection phase, there will be a period of time when the memory buffer is partially allocated and no communication with the database will take place. The database server may time out. You can solve this problem by increasing the time on the SQL server side or by lowering the mem_limit mem_limit = 256M | #mem_limit = 262144K# the same, but in KB | # mem_limit = 268435456# the same, but the unit is bytes Index RAM usage limit. Optional, the default is 32M. The indexer will not exceed the memory usage limit after being forced. This configuration can be specified in units of bytes or kilobytes (using K postfix) or megabytes (using M postfix); see this example. If set to a very low value that causes the I/O buffer to be less than 8KB, this limit will be automatically increased; to be precise, it depends on the data size of the index. If the buffer does not exceed 256KB, a warning will be generated. The maximum possible limit is 2047M. Too low a value will hurt the indexing speed, but if it is not the entire data set, 256M to 1024M is sufficient. Setting this value too high may cause SQL Server to time out. During the document collection phase, there will be a period of time when the memory buffer is partially allocated and no communication with the database will take place. The database server may time out. You can solve this problem by increasing the time on the SQL server side or by lowering the mem_limit mem_limit = 256M | #mem_limit = 262144K# the same, but in KB | # mem_limit = 268435456# the same, but the unit is bytes Index RAM usage limit. Optional, the default is 32M. The indexer will not exceed the memory usage limit after mandatory. This configuration can be specified in units of bytes or kilobytes (using K postfix) or megabytes (using M postfix); see this example. If set to a very low value that causes the I/O buffer to be less than 8KB, this limit will be automatically increased; to be precise, it depends on the data size of the index. If the buffer does not exceed 256KB, a warning will be generated. The maximum possible limit is 2047M. Too low a value will hurt the indexing speed, but if it is not the entire data set, 256M to 1024M is sufficient. Setting this value too high may cause SQL Server to time out. During the document collection phase, there will be a period of time when the memory buffer is partially allocated and no communication with the database will take place. The database server may time out. You can solve this problem by increasing the time on the SQL server side or by lowering the mem_limit mem_limit = 256M | #mem_limit = 262144K# the same, but in KB | # mem_limit = 268435456# the same, but the unit is bytes