Author | JiekeXu
Source|Public Account JiekeXu DBA Road (ID: JiekeXu_IT)
If you need to reprint, please contact for authorization | (Personal WeChat ID: JiekeXu_DBA)
Hello everyone, I am JiekeXu, I am very glad to meet you again, today I will take a look at the Oracle Log4j vulnerability repair and the simple use of AHF, please click on the blue word "JiekeXu DBA Road" above to follow my official account, mark Star or stick to the top, more dry goods will arrive as soon as possible!
Introduction to Log4j
Apache Log4j is an open source Java logging framework, which is widely used in middleware, development frameworks and Web applications to record log information.
Vulnerability description:
1. CVE-2021-44228, Apache Log4j Remote Code Execution Vulnerability:
2.0-beta9 <= Apache Log4j 2.x < 2.15.0, the vulnerability is low in difficulty to exploit, high in severity, and has a very large impact.
2. CVE-2021-45046, Apache Log4j denial of service and remote code execution vulnerability: 2.0-beta9 <= Apache Log4j 2.x < 2.15.0.
3. CVE-2021-4104, ApacheLog4j 1.2 JMSAppender Remote Code Execution Vulnerability: Apache Log4j = 1.2.
4. CVE-2021-45105, Apache Log4j2 denial of service vulnerability: 2.0-beta9 <= Apache Log4j <= 2.16.0.
5. CVE-2021-44832, JDBCAppender code execution vulnerability: 2.0-beta7 <= Apache Log4j <= 2.17.0 (Note: only log4j-core jar files are affected by this vulnerability).
Affected version
2.0 <= Apache Log4j 2.x <= 2.17.0。
unaffected version
Apache Log4j 2.17.1 and above (supports Java 8 and above)
Apache Log4j 2.12.4 (supports Java 7)
Apache Log4j 2.3.2 (supports Java 6)
Troubleshooting method
Check if log4j is used, find / -name log4j*.jar
(1), check the deployment package, whether there is a log4j-core-2.x.jar package in the deployment environment, and check the version
Implementation-Version property value under META-INF/MANIFEST.MF file.
(2) For third-party packages, whether org/apache/logging/log4j-related path structure exists after Java JAR decompression is used to determine whether a component with a vulnerability is used. If so, the vulnerability is likely to exist.
(3) For the source code project, check the dependencies and check whether there is log4j-core of org.apache.logging.log4j.
(4) Open source missing scan deployment package.
repair suggestion
1) Security version
JDK8 and above environment: upgrade to high-end version:
The above-mentioned vulnerabilities have been fixed in the official Apache log4j 2.17.1 and above versions.
https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core/2.17.1JDK7 environment: upgrade to Apache Log4j 2.12.4
JDK6 environment: upgrade to Apache Log4j 2.3.2
2) Sustained release measures
For the most severe CVE-2021-44228 vulnerability,
In case upgrade is not possible, use mitigations:
(1) Add jvm startup parameter -Dlog4j2.formatMsgNoLookups=true
(2) Add the log4j2.component.properties configuration file under the classpath of the application, the file content: log4j2.formatMsgNoLookups=True
(3) Remove the JndiLookup class file in the log4j-core package and restart the service. Specific commands:
zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.classNext, let's see where the log4j vulnerabilities exist in the database, let's find them.
[root@test19crac1 ~]# find /u01 -name log4j*.jar
/u01/app/19c/grid/suptools/tfa/release/tfa_home/jlib/log4j-api-2.9.1.jar
/u01/app/19c/grid/suptools/tfa/release/tfa_home/jlib/log4j-core-2.9.1.jar
/u01/app/19c/grid/tfa/test19crac1/tfa_home/jlib/log4j-api-2.9.1.jar
/u01/app/19c/grid/tfa/test19crac1/tfa_home/jlib/log4j-core-2.9.1.jar
/u01/app/oracle/product/19c/db_1/suptools/tfa/release/tfa_home/jlib/log4j-api-2.9.1.jar
/u01/app/oracle/product/19c/db_1/suptools/tfa/release/tfa_home/jlib/log4j-core-2.9.1.jar
/u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-jcl-2.11.0.jar
/u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-slf4j-impl-2.11.0.jar
/u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-api-2.11.0.jar
/u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-core-2.11.0.jar
As shown below, the database has not been patched, the DB version is 19.3, and there are two log4j versions, one is the 2.9.1 version used by TFA, and the other is the 2.11.0 version in the database instance directory.
[oracle@test19crac2 ~]$ $ORACLE_HOME/OPatch/opatch version
OPatch Version: 12.2.0.1.17
OPatch succeeded.
[oracle@test19crac2 ~]$ $ORACLE_HOME/OPatch/opatch lspatches
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)
29517242;Database Release Update : 19.3.0.0.190416 (29517242)
OPatch succeeded.Then it can be found that there are three monitoring and inspection tools, orachk, oratop, and tfa, in the suptools directory under the home directory of the grid.
[root@test19crac1 suptools]# cd /u01/app/19c/grid/suptools
[root@test19crac1 suptools]# ls
orachk oratop tfa
If our database has been patched, here we take 19c RAC as an example, please test it yourself in the 11g environment. If the patch version is too low, then the log4j in the GI directory no longer exists, but the affected AHF version 2.13.3 still exists under the AHF framework, then in this case, you can continue to update the higher version 19c RU15 - RU20 Yes, you can also upgrade AHF separately, and how to upgrade will be mentioned later.
Then, what about the Oracle 19c of the stand-alone file system (19.3 is not patched), there is also an affected version of log4j-core-2.9.1.jar in the tfa_home directory, the production environment is recommended to be patched, and the patch is upgraded to RU19.12 Or RU19.15 or above, the stand-alone environment patch update downtime is only about 30 minutes. For the specific patch upgrade method, please refer to my previous article: "RU15 Patch Update" . Of course, you can also try to download a higher version of the log4j file and replace it in the form of a soft link. As mentioned below, you can test it yourself in the test environment.
/u01/app/oracle/product/19.0.0/dbhome_1/suptools/tfa/release/tfa_home/jlibLet's take a look at the simple usage of TFA.
Introduction to using TFA
View TFA: Trace File Analyzer Status and Version.
TFA more help commands
[grid@test19crac2 ~]$ tfactl --help
WARNING - TFA Software is older than 180 days. Please consider upgrading TFA to the latest version.
Unknown option: -help
Usage : /u01/app/19c/grid/bin/tfactl[options]
commands:diagcollect|collection|analyze|ips|run|start|stop|print|directory|toolstatus|setupmos|upload|events|search|changes
For detailed help on each command use:
/u01/app/19c/grid/bin/tfactl -help
tfactl>It can be found that the tfactl command can perform operations such as |start|stop|print |toolstatus.
To delete the log and trace files of the automatic diagnostic repository 180 days ago, the parameter dryrun first estimates the files to be deleted and the space to be released, and then removes the parameter dryrun to perform the deletion.
/u01/app/19c/grid/bin/tfactl managelogs -purge -older 180d -dryrunOf course, you can also delete them separately according to gi and database
$ tfactl managelogs -purge -older 30d
$ tfactl managelogs -purge -older 30d –gi
$ tfactl managelogs -purge -older 30d -databaseView directory usage
$ tfactl managelogs -show usage
$ tfactl managelogs -show usage –gi
$ tfactl managelogs -show usage -database
/u01/app/19c/grid/bin/tfactl managelogs -purge -older 30d -gi
/u01/app/19c/grid/bin/tfactl managelogs -show usage -gi
For more practical operations of TFA, please check the official documentation, which is not demonstrated here.
Reference document: https://docs.oracle.com/en/engineered-systems/health-diagnostics/autonomous-health-framework/ahfug/managing-and-configuring-tfa.html#GUID-CBF85753-9DCC-48BC-AA83 -5CA2982ED0EB
TFA upgrade
Download the latest AHF (Autonomous Health Framework) Oracle autonomous health framework. In later versions of 19c, the official recommendation is to use AHF instead of the TFA framework, so here we directly download the latest AHF, MOS document Doc ID: 1513912.1 can be downloaded, or You can get this software package by replying [AHF] in the background of this official account.
1) Upload this software package to the server RAC1 and decompress it
unzip AHF-LINUX_v23.6.0.zip -d AHF_23.6
2) Install AHF
The installation command is given in the appendix of the official document, we can refer to it, specify the relevant parameters to run, or directly run the ahf_setup command (upgrade is also run this command), only run on node 1, you need to enter the node during the installation or upgrade process 2 root password.
./ahf_setup
[-ahf_loc AHF Location]
[-data_dir AHF Repository]
[-nodes node1,node2]
[-extract[orachk|exachk|-notfasetup]]
[-force]
[-local]
[-silent]
[-tmp_loc directory]
[-debug [-level 1-6]]
[root@test19crac1 AHF_23.6]# ./ahf_setup
AHF Installer for Platform Linux Architecture x86_64
AHF Installation Log : /tmp/ahf_install_236000_26649_2023_07_31-10_51_26.log
Starting Autonomous Health Framework (AHF) Installation
AHF Version: 23.6.0 Build Date: 202306301433
TFA is already installed at : /u01/app/19c/grid/tfa/test19crac1/tfa_home
Installed TFA Version : 183300 Build ID : 20190315044534
Default AHF Location : /opt/oracle.ahf
Do you want to install AHF at [/opt/oracle.ahf] ? [Y]|N : Y
AHF Location : /opt/oracle.ahf
AHF Data Directory stores diagnostic collections and metadata.
AHF Data Directory requires at least 5GB (Recommended 10GB) of free space.
Choose Data Directory from below options :
1. /u01/app/grid [Free Space : 5701 MB]
2. Enter a different Location
Choose Option [1 - 2] : 1
AHF Data Directory : /u01/app/grid/oracle.ahf/data
Shutting down TFA : /u01/app/19c/grid/tfa/test19crac1/tfa_home
Copying TFA Data Files from /u01/app/19c/grid/tfa/test19crac1/tfa_home
Uninstalling TFA : /u01/app/19c/grid/tfa/test19crac1/tfa_home
Do you want to add AHF Notification Email IDs ? [Y]|N : N
AHF will also be installed/upgraded on these Cluster Nodes :
1. test19crac2
The AHF Location and AHF Data Directory must exist on the above nodes
AHF Location : /opt/oracle.ahf
AHF Data Directory : /u01/app/grid/oracle.ahf/data
Do you want to install/upgrade AHF on Cluster Nodes ? [Y]|N : Y
Extracting AHF to /opt/oracle.ahf
Configuring TFA Services
Copying TFA Data Files to AHF
Discovering Nodes and Oracle Resources
Starting TFA Services
Created symlink from /etc/systemd/system/multi-user.target.wants/oracle-tfa.service to /etc/systemd/system/oracle-tfa.service.
Created symlink from /etc/systemd/system/graphical.target.wants/oracle-tfa.service to /etc/systemd/system/oracle-tfa.service.
.--------------------------------------------------------------------------------.
| Host | Status of TFA | PID | Port | Version | Build ID |
+-------------+---------------+-------+------+------------+----------------------+
| test19crac1 | RUNNING | 34212 | 5000 | 23.6.0.0.0 | 23600020230630143300 |
| test19crac2 | RUNNING | 10468 | 5000 | 18.3.3.0.0 | 18330020190315044534 |
'-------------+---------------+-------+------+------------+----------------------'
Running TFA Inventory...
Adding default users to TFA Access list...
.------------------------------------------------------------------.
| Summary of AHF Configuration |
+-----------------+------------------------------------------------+
| Parameter | Value |
+-----------------+------------------------------------------------+
| AHF Location | /opt/oracle.ahf |
| TFA Location | /opt/oracle.ahf/tfa |
| Orachk Location | /opt/oracle.ahf/orachk |
| Data Directory | /u01/app/grid/oracle.ahf/data |
| Repository | /u01/app/grid/oracle.ahf/data/repository |
| Diag Directory | /u01/app/grid/oracle.ahf/data/test19crac1/diag |
'-----------------+------------------------------------------------'
Starting Orachk Scheduler from AHF
Setting up AHF CLI and SDK
AHF install completed on test19crac1
Installing AHF on Remote Nodes :
AHF will be installed on test19crac2, Please wait.
AHF will prompt twice to install/upgrade per Remote Node. So total 2 prompts
Do you want to continue Y|[N] : Y
AHF will continue with Installing on remote nodes
Installing AHF on test19crac2 :
[test19crac2] Copying AHF Installer
root@test19crac2's password:
[test19crac2] Running AHF Installer
root@test19crac2's password:
AHF binaries are available in /opt/oracle.ahf/bin
AHF is successfully installed
Do you want AHF to store your My Oracle Support Credentials for Automatic Upload ? Y|[N] : N
Moving /tmp/ahf_install_236000_26649_2023_07_31-10_51_26.log to /u01/app/grid/oracle.ahf/data/test19crac1/diag/ahf/
[root@test19crac1 AHF_23.6]#3) Check version
[root@test19crac1 AHF_23.6]# ahfctl version
[root@test19crac1 AHF_23.6]# tfactl version
[root@test19crac1 AHF_23.6]# tfactl status
4) Delete all log4j*.jar in the suptools directory of the grid user (if it exists)
The log4j under the grid here has been automatically deleted. If it still exists, it can be deleted manually. The log4j-core.jar displayed by /opt already meets the requirements. As for the one under the db directory property_graph, it needs to be patched, and then this file will be automatically deleted. Here I make a bold attempt to directly delete this file and restart the host to see if the database can start normally (PS: this is not recommended for production environments, and RU patch updates are recommended). Of course, you can also delete these two files and use high The version of log4j-core makes a soft link pointing to the lower version of log4j.
#cp /u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j* /home/oracle
#rm -rf /u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-api-2.11.0.jar
#rm -rf /u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-core-2.11.0.jar
#cp /opt/oracle.ahf/common/jlib/log4j-api-2.17.1.jar /u01/app/oracle/product/19c/db_1/md/property_graph/lib/
#cp /opt/oracle.ahf/common/jlib/log4j-core-2.17.1.jar /u01/app/oracle/product/19c/db_1/md/property_graph/lib/
#chown oracle:oinstall /u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j*
#su - oracle
$ ln -s /u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-core-2.17.1.jar log4j-core-2.11.0.jar
$ ln -s /u01/app/oracle/product/19c/db_1/md/property_graph/lib/log4j-api-2.17.1.jar log4j-api-2.11.0.jar
#reboot
After restarting, check the status of the database, and everything in the database is normal here.
AHF uninstall
Use the root user or the installer user to execute the tfactl or ahfctl uninstall command to uninstall, first stop ORAchk, then stop TFA, and finally delete the AHF installation directory /opt/oracle.ahf.
Running the command:
Stops Oracle ORAchk
Stops Oracle Trace File Analyzer
Deletes the Oracle Autonomous Health Framework installation directory
ahfctl uninstall -deleterepo -silentAHF common commands
ahfctl statusahf -all
ahfctl showrepo -all
Get CPU memory resource limit
ahfctl getresourcelimit
Limit the number of CPU cores to 2
ahfctl setresourcelimit -value 2
Limit memory usage to 500M
ahfctl setresourcelimit -resource kmem -value 500
Limit the sum of memory and SWAP memory to 1024M
ahfctl setresourcelimit -resource swmem -value 1024
Analyze all error messages (including database ORA errors and GI errors)
tfactl analyze -type error
Similarly, you can also use tfactl to clean up the diagnostic log
tfactl purge -older 30d
For more commands, please refer to the official documentation, which is not demonstrated here.
OGG log4j vulnerability
Finally, take a moment to briefly talk about the OGG log4j vulnerability. Normally, this vulnerability does not exist in versions above ogg19c, but I also have Oracle GoldenGate for Big Data Version 12.3.2.1.1, which also uses The package of log4j-core-2.9.1.jar. You can also fix this vulnerability by patching, or upgrading OGG, or using a higher version of OGG, but these three methods are not commonly used, and generally few people use it. I am bold here, and I deleted it directly in the test environment. The log4j-core-2.9.1.jar package and services can be used normally, and has been running safely and stably for more than a year. In the production environment, I replaced this package with a high-version soft link, and it has been running safely and stably for a week. The following is the operation process:
[root@cs-ogg lib]# cd /goldengate/ggjava/resources/lib/optional
[root@cs-ogg optional]# ll log*
-rwxr-xr-x 1 oracle oracle 239856 5月 1 2018 log4j-api-2.9.1.jar
-rwxr-xr-x 1 oracle oracle 1549865 5月 1 2018 log4j-core-2.9.1.jar
-rwxr-xr-x 1 oracle oracle 23153 5月 1 2018 log4j-slf4j-impl-2.9.1.jar
[root@cs-ogg lib]# su - oracle
[oracle@cs-ogg optional]$ cd /goldengate/ggjava/resources/lib/optional/
[oracle@cs-ogg optional]$ mv log4j-core-2.9.1.jar log4j-core-2.9.1.jar_bak20230728
[oracle@cs-ogg optional]$ ln -s /goldengate/ggjava/resources/lib/optional/log4j-core-2.17.1.jar log4j-core-2.9.1.jar
[oracle@cs-ogg optional]$ ll
总用量 3560
-rwxr-xr-x 1 oracle oracle 239856 5月 1 2018 log4j-api-2.9.1.jar
-rw-r--r-- 1 oracle oracle 1770490 3月 9 2022 log4j-core-2.17.1.jar
lrwxrwxrwx 1 oracle oracle 63 7月 28 16:00 log4j-core-2.9.1.jar -> /goldengate/ggjava/resources/lib/optional/log4j-core-2.17.1.jar
-rwxr-xr-x 1 oracle oracle 1549865 5月 1 2018 log4j-core-2.9.1.jar_bak20230728
-rwxr-xr-x 1 oracle oracle 23153 5月 1 2018 log4j-slf4j-impl-2.9.1.jar
drwxr-xr-x 2 oracle oracle 32 7月 14 2018 logback
-rwxr-xr-x 1 oracle oracle 6983 5月 1 2018 slf4j-jcl-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 8460 5月 1 2018 slf4j-jdk14-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 12244 5月 1 2018 slf4j-log4j12-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 4007 5月 1 2018 slf4j-nop-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 15257 5月 1 2018 slf4j-simple-1.7.25.jar
[oracle@cs-ogg optional]$ chmod a+x log4j-core-2.17.1.jar
[oracle@cs-ogg optional]$ ll
总用量 3560
-rwxr-xr-x 1 oracle oracle 239856 5月 1 2018 log4j-api-2.9.1.jar
-rwxr-xr-x 1 oracle oracle 1770490 3月 9 2022 log4j-core-2.17.1.jar
lrwxrwxrwx 1 oracle oracle 63 7月 28 16:00 log4j-core-2.9.1.jar -> /goldengate/ggjava/resources/lib/optional/log4j-core-2.17.1.jar
-rwxr-xr-x 1 oracle oracle 1549865 5月 1 2018 log4j-core-2.9.1.jar_bak20230728
-rwxr-xr-x 1 oracle oracle 23153 5月 1 2018 log4j-slf4j-impl-2.9.1.jar
drwxr-xr-x 2 oracle oracle 32 7月 14 2018 logback
-rwxr-xr-x 1 oracle oracle 6983 5月 1 2018 slf4j-jcl-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 8460 5月 1 2018 slf4j-jdk14-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 12244 5月 1 2018 slf4j-log4j12-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 4007 5月 1 2018 slf4j-nop-1.7.25.jar
-rwxr-xr-x 1 oracle oracle 15257 5月 1 2018 slf4j-simple-1.7.25.jar
[oracle@cs-ogg goldengate]$ ./ggsci
Oracle GoldenGate for Big Data
Version 12.3.2.1.1 (Build 005)
Oracle GoldenGate Command Interpreter
Version 12.3.0.1.2 OGGCORE_OGGADP.12.3.0.1.2_PLATFORMS_180712.2305
Linux, x64, 64bit (optimized), Generic on Jul 13 2018 00:46:09
Operating system character set identified as UTF-8.
Copyright (C) 1995, 2018, Oracle and/or its affiliates. All rights reserved.
GGSCI (cs-ogg) 1> stop *
GGSCI (cs-ogg) 2> sart *
GGSCI (cs-ogg) 3> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
REPLICAT RUNNING REP01 00:00:13 00:00:01
REPLICAT RUNNING REP02 00:00:00 00:00:09The full text is finished, I hope it can help you who are reading, if you think this article is helpful to you, you can share it with your friends, colleagues, whoever you care about, and share it with whoever you care about, let's learn together and make progress~~~
Welcome to follow my official account [JiekeXu DBA Road], and learn new knowledge together as soon as possible! You can find me at the following three addresses. The other addresses are pirated copyright infringement crawling my articles, and the code format, pictures, etc. are all messed up, which is inconvenient to read. Welcome to my official account or Motianlun address to follow me, the first time Get the latest news.
Welcome to follow my official account [JiekeXu DBA Road], and learn new knowledge together as soon as possible!
————————————————————————————————
Public account: JiekeXu DBA Road
CSDN: https://blog.csdn.net/JiekeXu
Motian Wheel: https://www.modb.pro/u/4347
———————————————————————————————
Share several database backup scripts
Oracle table fragmentation check and defragmentation scheme
OGG|Oracle GoldenGate 基础2022 年公众号历史文章合集整理Several problems encountered by Oracle 19c RAC
OGG|Oracle 数据迁移后比对一致性OGG|Oracle GoldenGate Microservice Architecture
Oracle query table space usage is extremely slow
Domestic database|TiDB 5.4 stand-alone quick installation first experience
Oracle ADG standby database shutdown maintenance process and incremental recovery
Linux environment to build MySQL8.0.28 master-slave synchronization environment
What information can you learn from the domestic database research report and my summary suggestions
