oracle@WWJD-DB1:~> $ORACLE_HOME/bin/srvctl start database -d ndscdb
PRCR-1079 : Failed to start resource ora.ndscdb.db
CRS-5017: The resource action "ora.ndscdb.db start" encountered the following error:
ORA-00443: background process "LMD1" did not start
. For details refer to "(:CLSN00107:)" in "/oracle/app/grid/diag/crs/WWJD-db1/crs/trace/crsd_oraagent_oracle.trc".
CRS-2674: Start of 'ora.ndscdb.db' on 'WWJD-db1' failed
CRS-2632: There are no more servers to try to place resource 'ora.ndscdb.db' on that would satisfy its placement policy--alert.log
Starting background process LMD1
2019-01-28T01:29:19.181299-08:00
Process startup failed, error stack:
2019-01-28T01:29:19.181569-08:00
Errors in file /oracle/app/oracle/diag/rdbms/ndscdb/NDSCDB1/trace/NDSCDB1_psp0_148992.trc:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3
2019-01-28T01:29:19.181989-08:00
* Load Monitor used for high load check
* New Low - High Load Threshold Range = [61440 - 81920]
2019-01-28T01:29:19.184921-08:00
LMS0 started with pid=24, OS id=149071 at elevated (RT) priority
2019-01-28T01:29:19.186634-08:00
LMS2 started with pid=26, OS id=149075 at elevated (RT) priority
2019-01-28T01:29:19.187214-08:00
LMS1 started with pid=25, OS id=149073 at elevated (RT) priority
2019-01-28T01:29:19.187425-08:00
LMS3 started with pid=27, OS id=149077 at elevated (RT) priority
2019-01-28T01:29:20.180278-08:00
Process LMD1 died, see its trace file
2019-01-28T01:29:20.182144-08:00
USER (ospid: 148922): terminating the instance due to error 443
2019-01-28T01:29:21.228555-08:00
Instance terminated by USER, pid = 148922--trace
oracle@WWJD-DB1:~> more /oracle/app/oracle/diag/rdbms/ndscdb/NDSCDB1/trace/NDSCDB1_psp0_148992.trc
Trace file /oracle/app/oracle/diag/rdbms/ndscdb/NDSCDB1/trace/NDSCDB1_psp0_148992.trc
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
Build label: RDBMS_12.2.0.1.0_LINUX.X64_170125
ORACLE_HOME: /oracle/app/oracle/product/12.2.0/db_1
System name: Linux
Node name: WWJD-DB1
Release: 4.4.162-94.69-default
Version: #1 SMP Mon Nov 5 18:58:52 UTC 2018 (9e06c56)
Machine: x86_64
Instance name: NDSCDB1
Redo thread mounted by this instance: 0 <none>
Oracle process number: 4
Unix process pid: 148992, image: oracle@WWJD-DB1 (PSP0)
*** 2019-01-28T01:29:17.870458-08:00
*** CLIENT ID:() 2019-01-28T01:29:17.870482-08:00
*** SERVICE NAME:() 2019-01-28T01:29:17.870488-08:00
*** MODULE NAME:() 2019-01-28T01:29:17.870493-08:00
*** ACTION NAME:() 2019-01-28T01:29:17.870501-08:00
*** CLIENT DRIVER:() 2019-01-28T01:29:17.870506-08:00
[kji_attach][LMON !running, !call kjual]
*** SESSION ID:(2065.45615) 2019-01-28T01:29:19.181339-08:00
*** SERVICE NAME:() 2019-01-28T01:29:19.181353-08:00
*** 2019-01-28T01:29:19.181318-08:00
Process startup failed, error stack:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3
OS - DIAGNOSTICS
----------------
loadavg : 0.32 0.51 0.63
Memory (Avail / Total) = 497616.35M / 515669.59M
Swap (Avail / Total) = 65536.00M / 65536.00M
Max user processes limits(s / h) = 65536 / 65536
----------------
oracle@WWJD-DB1:~> - 诊断 ORA-27300 ORA-27301 ORA-27302 错误 (文档 ID 2179478.1)
- ORA-27300 OS system dependent operation:fork failed with status: 11 (文档 ID 392006.1)
- 在 Unix AIX，HP-UX，Linux，Solaris 和 MS Windows 操作系统上安装和配置 Oracle 数据库（RDBMS）的要求的快速参考(12.1/12.2) (文档 ID 2226599.1)
--Oracle要求
Shell limits
Soft File Descriptors 最少 1024
Hard File Descriptors 最少 65536
Soft FILE size -1 (Unlimited)
Soft CPU time -1 (Unlimited)
Soft DATA segment -1 (Unlimited)
Soft STACK size -1 (Unlimited)
Number of processes (Soft) 最少 2047
Number of processes (Hard) 最少 16384
maxuprocs 16384
oracle@WWJD-DB1:~> cat /etc/security/limits.conf
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
grid hard stack 32768
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
oracle hard stack 32768
# End of file
* - nofile 102400
* - nproc 148270
#Oracle recommended value for memlock is set to 3145728 for user oracle
oracle hard memlock 3145728
#Oracle recommended value for memlock is set to 3145728 for user oracle
oracle soft memlock 3145728
oracle@WWJD-DB1:~> ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 2062592
max locked memory (kbytes, -l) 3145728
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 2047
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
oracle@WWJD-DB1:~> 重启集群,好了...
下班了，先回家...