The Log4Shell (Log4j) situation is rapidly evolving. The purpose of this page is to provide a jump-off point to resources, with an emphasis on Bureau-wide remediation activities. Questions should be directed to GS Security Assurance.
Log4j Version 1.x Vulnerabilities: EVSS has new Nessus plugins that identify Log4j version 1.x instances as vulnerable. (Plugin IDs: 156860, 156032, 156103, 156240). Initial Log4Shell guidance was to avoid upgrading end-of-life version 1.x instances until such time that the version 2.x train stabilized. Version 2.17.1 was published 6 weeks ago, thus we believe the needed stability is now available. In order to remediate these newer Nessus findings, remediators should begin upgrading any/all version 1.x log4j instances where an upgrade path to version 2.17.1 is possible.
External Informational Resources:
(Links to vendor-specific remediation guidance is a new section at the bottom of this page.)
- Emergency Directive 22-02: Mitigate Apache Log4J Vulnerability
- CISA Vulnerability Guidance for Apache Log4j Vulnerability CVE-2021-44228
- CISA Ongoing List of Impacted Products and Services
- Apache: Log4j Security Vulnerabilities
- Log4Shell: RCE 0-day exploit found in log4j2, a popular Java logging package
- NVD – CVE-2021-44228 (nist.gov)
Internal Informational Resources:
IT All Memos are archived and in the ACIO Communications Library. Use the IT Security View or the Log4Shell View to filter the list to emails marked with the IT Security Category.
USGS Log4Shell Report contains Log4Shell vulnerability information across 432 409 329 123 fewer than 30 computers. Many computers are listed in the report multiple times, once per vulnerable file instance. Mitigated Log4j version 2.x instances, and instances at 2.16 or higher, are not listed in this report. Computers with a mitigated Log4j file will be included, but computers that no longer have log4j files on them will not. Actionable devices from EVSS are listed as well. Every effort has been made to remove entries from the EVSS report where mitigated jar files exist. If you discover any concerns with the data in the report, please email GS Security Assurance. Going forward, DOI CIRC will distribute Log4j vulnerability status reports weekly on Tuesdays.
DOI: Log4j found (Web Report) – This report contains ALL results from both types of Department-initiated Log4j scans. This report DOES NOT focus on actionable information and DOES NOT identify A&A boundary information.
GS: Log4j found (Web Report) – This report contains ONLY ACTIONABLE INFORMATION from both Log4j scans from the previous weeks. ACIO recommends using this report when focusing on remediation activities.
As of 12/28: the web reports above no longer have current data from CSV scans.
Scan Schedule as of 1/25/2022:
BigFix scans are scheduled as follows: Full scans run on Thurs starting at 1pm and will run anytime Friday if needed. Scans on vulnerable hosts or those without a recent scan file will run anytime Sat – Wed. EVSS scans will continue to occur twice per week, though the specific scan day will depend on the IP address space a device sits in.
Scan Fixlets as of 12/28/2021:
The Department is running the following fixlets, starting at 12am on 12/29/2021, repeating every two days. Users with BigFix Console access may choose to re-deploy these fixlets locally to help validate remediation actions:
- DOI: Logpresso v2.7.1 log4j2-scan – Windows x64 no 1.x version -CSV
- DOI: Logpresso v2.7.1 log4j2-scan – Linux (x64) no 1.x version -CSV
- DOI: Logpresso v2.7.1 log4j2-scan – MacOS (x64) no 1.x version -CSV
The log file that the scan creates is found at “{pathname of storage folder of client}/BPS-Scans”. For example on Windows, it would be: C:\Program Files (x86)\BigFix Enterprise\BES Client\BPS-Scans. Initial scans generated a .txt file – the current CSV scans generate a .csv file.
The following properties may be viewed in BigFix Web Reports to check the Scan Completion time and the scan output:
- DOI-CVE-2021-44228-Log4j-Logpresso-Scan-Completion-Time-CSV
- DOI-CVE-2021-44228-Log4j-Logpresso-Scan-CSV
The data in the USGS Log4Shell spreadsheet was generated from the CSV scanning activity, and filtered down to VULNERABLE files that are less than version 2.16.0.
Interpreting BigFix Consolidated Properties - Data In These Properties is Not Current
Interpreting BigFix Scan Reports (Info provided by OCIO)
By default, if properties return:
- <not reported> then the property has not evaluated on the endpoint.
- <error> then there was an error evaluating the property for whatever reason.
Property: DOI-CVE-2021-44228-Log4j-Consolidated-Time Description: This property will return the time stamp from the most recent scan result file as well as the filename.
| Scan | Filename |
|---|---|
| JNDI | CCVE-2021-44228-NEW.txt |
| Logpresso | results-log4j-scan.txt |
| File Name | CCVE-2021-44228.txt |
Property: DOI-CVE-2021-44228-Log4j-Consolidated-Results Description: This property will read the most recent scan result file in the format provided by the script. Scan files are in the BigFix client folder under BPS-Scans. If the property shows <none> there were no scan results. See Below for additional details:
JNDI Scan Results
These will include lines such as:
- WARNING: C:\ForescoutLABConsole\GuiManager\current\lib\java\log4j-core-2.15.0.jar contains org/apache/logging/log4j/core/lookup/JndiLookup.class
- C:\Users\bathow\Forescout Console 8.1.3\GuiManager\current\lib\java\log4j-core-2.16.0.jar contains org/apache/logging/log4j/core/lookup/JndiLookup.class ** BUT APPEARS TO BE PATCHED **
- No Data Found
The WARNING lines will need to be looked at/remediated.
** BUT APPEARS TO BE PATCHED ** – self explanatory
No vulnerable components found = scan ran but nothing was found
No data found = scan ran but nothing was found
Logpresso Scan Results
These will include lines such as:
- C:\ForescoutLABConsole\GuiManager\current\lib\java\log4j-core-2.15.0.jar
- C:\Users\bathow\Forescout Console 8.1.3\GuiManager\current\lib\java\log4j-core-2.16.0.jar
This provides the pathname of files that are NOT mitigated
File Name Scan Results
These will include lines such as:
- C:\ForescoutLABConsole\GuiManager\current\lib\java\log4j-core-2.15.0.jar, ba55c13d7ac2fd44df9cc8074455719a33f375b9, Not Matched
- C:\Users\bathow\Forescout Console 8.1.3\GuiManager\current\lib\java\log4j-core-2.16.0.jar, ca12fb3902ecfcba1e1357ebfc55407acec30ede, Not Matched
This provides the name of *log4j*.jar files, their sha values, and if they matched known bad sha values for out of the box vendor files.
A file with not found instead of a sha value indicates a *log4j*.jar file that does not match any of the vendor named files.
BOD 22-01 Monitoring of log4j
EVSS Dashboard: As Log4j is listed on the BOD 22-01 Known Exploited Vulnerabilities Catalog, the dashboard named: “USGS DHS BOD 22-01 (ITSOT Maintained)” can be used for monitoring from EVSS. Queries include “USGS BOD 22-01 Past Due” and “USGS BOD 22-01 Future Due Dates” are also available. All of these objects are owned by usgsmgr.
The DOI maintains two BigFix WebReport filters for BOD 22-01: Future Due Date and Past Due. The Due date for log4j is currently listed as December 24th, 2021. USGS has created web reports using these filters, which are linked on the Web Reports Page under the Patch Status tab.
Information Sharing within the USGS IT Community
The spreadsheet: SoftwareDependencies-log4j.xlsx is available for adding information learned at local sites about locally procured and managed software remediation activities.
Vendor-Specific Remediation Documentation
- Apache: Log4j Security Vulnerabilities
- Apple: XCode 13.2.1 Release Notes *Additional Guidance for USGS is below
- ArcGIS and Apache Log4j Vulnerabilities
- Avaya Impacted Products
- Dell Response to Apache Log4j Remote Code Execution Vulnerability
- ManageEngine Log4j impact
- ManageEngine: Upgrade to the latest version of EventLog Analyzer – Download service packs!
- Onset (Hobo) Log4j Response
- Oxygen XML Log4Shell Vulnerability Analysis FAQ, oxygen-log4j-patcher (GitHub)
- SAS Log4j Response, SAS 9.4 Instructions for the SAS Response to Log4j Vulnerabilities
- Tableau: An update on the Apache Log4j2 vulnerability
- VMware Impacted Products Response Matrix
Interim USGS Guidance for XCode and TexLive: It is recommended to remove Xcode where not needed. A list of current XCode installations, (data pulled from JAMF), is available here.
Remediation for XCode and TexLive Installs
If the full Xcode app is not needed, please remove. This may require reinstalling Xcode Command Line Tools if the user needs. The script located here will remove all instances of Xcode, including Command Line Tools, but will reinstall the CLI at the end of running: https://code.chs.usgs.gov/tst/butst/xcode-removal/-/tree/main
If Xcode is needed by the user, run the script located here. Instructions are found in the script, but you will need the path to where the offending Xcode log4j file is located. See the ReadMe for most common location.
This script can also be used to remediate the TexLive vulnerability since there is no updated version.
https://code.chs.usgs.gov/tst/butst/fix-jndi/-/tree/main
USGS Enterprise Actions Taken to Remediate log4j
- MECM Baselines applied to all USGS Windows Workstations
- GS-ENT log4j Removal – Matlab 2015b Runtime. Deletes the file “C:\Program Files\MATLAB\MATLAB Runtime\v90\java\jarext\log4j.jar”
- GS-ENT log4j Removal – Oracle 11g. Deletes the file “C:\oracle\product\11.2.0\client_1\oui\jlib\jlib\log4j-core.jar”
- MECM Applications built and deployed to all USGS computers with previous versions installed. Systems running Tableau Public (for demo purposes) will need to remove the Public software, and install the MECM applications below, if licensed to use Tableau:
4-GS – Tableau Desktop 2021.4.14-GS – Tableau Desktop 2021.4.2- 4-GS – Tableau Desktop 21.4.1708 (log4j files updated to version 2.17.1)
4-GS – Tableau Prep Builder 2021.4.24-GS – Tableau Prep Builder 2021.4.3- 4-GS – Tableau Prep Builder 21.4.18090 (log4j files updated to version 2.17.1)