UPDATE 2 (11/18/2022):
Microsoft has published Out-of-Band (OOB) Cumulative Updates for Server OSs, including KB5021654, KB5021655, KB5021656. These non-security updates address the Kerberos authentication issue that were introduced and discovered during the November Monthly Fast Ring.
No new patches have been published for client operating systems.
The November Monthly BigFix Baseline has been updated with the new server patch content.
eAD Servers will be Rebooted for Patch Updates: Tuesday, November 29, 2022 and Wednesday, November 30, 2022
- Starting at 6 PM (local time), GS.DOI.NET eAD Hyper-V host servers will be patched with the required Microsoft updates and rebooted. This will begin with servers in the Eastern Time zone.
- Starting at 8 PM (local time), all virtual GS.DOI.NET DC’s, eAD servers, services, and eAD Distribution Point servers will be patched with the required updates and rebooted.
Fast Ring deployments will be reinitiated, starting today. If no issues are reported, Production will begin next Wednesday, November 23rd. The updated schedule is below:
WHEN IT WILL HAPPEN:
Fast Ring Testing Schedule:
- Offers Available: Friday, November 18, 2022
- Installation Deadline: Tuesday, November 22, 2022 at 2:00am
- Please send Fast Ring feedback using the Service Desk Form
Production Patching Schedule:
- Offers Available: Wednesday, November 23, 2022 at 12:30am
- Installation Deadline: Wednesday, November 30, 2022 at 2:00am
UPDATE 1 (11/15/2022):
During Fast Ring patching, a Kerberos authentication issue was reported on patched computers that are communicating with a patched DC. The Domain Administrators are taking action to roll back the November cumulative update from Fast Ring Domain Controllers.
Microsoft has communicated that an updated patch is under development. When new content is released, the ePatching team will restart the November patch cycle.
Fast Ring will be repeated.
An updated patch schedule will be communicated in a future memo.
IT Staff who made local deployments of the November MECM Software Update group, the November monthly BigFix baseline, or individual November cumulative updates, should remove those deployments. No additional action is needed by local IT Staff at this time.
Please Note: Microsoft addressed 6 zero-day vulnerabilities with this month’s security updates. CISA has added the following CVE’s to the Known Exploited Vulnerabilities (KEV) Catalog: CVE-2022-41091, CVE-2022-41073, CVE-2022-41125, and CVE-2022-41128. As the due date for these vulnerabilities is 11-29-2022, there will be no change to this month’s ePatching schedule.
To: GS IT All
Subject: November 2022 ePatching for Windows – Deadline 11/23/2022
WHEN IT WILL HAPPEN:
Fast Ring Testing Schedule:
- Offers Available: Thursday, November 10, 2022 at 2:30pm
- Installation Deadline: Tuesday, November 15, 2022 at 2:00am
- Please send Fast Ring feedback using the Service Desk Form
Production Patching Schedule:
- Offers Available: Wednesday, November 16, 2022 at 12:30am
- Installation Deadline: Wednesday, November 23, 2022 at 2:00am
WHAT WILL HAPPEN: ePatching for Windows – November 2022
This Month’s Patch Cycle:
- eAD Server Patch Schedule (Informational)
- Dell Command Monitor
- Software Update Group Rebuild and Consolidation
WHAT YOU NEED TO KNOW:
The ePatching Team has posted this month’s products and versions on the TST ePatching page. An archive of previous notifications can be found here.
Additional information regarding vulnerability management actions can be found below.
Important Reminders –
- Use the report showing systems not properly labeled with Keyfiles and take action to repair them following guidance on the TST site to ensure patching activities can complete as scheduled. As a reminder – actions will be taken on BigFix Endpoints missing correct FISMA tagging.
- Servicing Stack Updates are not released monthly, but when Microsoft does release Servicing Stack Updates, systems that require them may require multiple reboots.
- MS SQL Patches: Sites that manage SQL servers should review the monthly SQL patching baseline and ensure installation of SQL patches are scheduled and completed before the monthly deadline. This will help minimize unscheduled service interruptions.
- MacOS Patching: MacOS patching cycles are now independent of the Windows patch schedule and will be announced in a separate memo.
eAD Servers will be Rebooted for Patch Updates: Friday, November 19, 2022 and Saturday, November 20, 2022
- Starting at 6 PM (local time), GS.DOI.NET physical Domain Controllers (DCs) and eAD Hyper-V host servers will be patched with the required Microsoft updates and rebooted. This will begin with servers in the Eastern Time zone.
- Starting at 8 PM (local time), all virtual GS.DOI.NET DC’s, eAD servers, services, and eAD Distribution Point servers will be patched with the required updates and rebooted.
- Patching will be staggered throughout the evening to minimize the impact of downtime.
- During the 3-5 minutes that each Domain Controller is rebooting, clients will fail-over to other Domain Controllers for authentication and DNS resolution if configured to do so.
Dell Command Monitor – During the production deployment of the Dell Command Monitor application, a bug was discovered with how failed installations were handled, sending a small number of computers into a cycle of reinstalling, failing, then rebooting, every two hours. The USGS has made a new version of the application, which resolves the bug: “4-GS – Dell Command Monitor | 10.8.0.284”. This application will be deployed to all Dell Non-Server hardware during the November ePatching cycle.
Software Update Group Rebuild and Consolidation – As a result of the MECM outage and restoration, all Software Update groups had to be re-created. The ePatching team has consolidated relevant Software Updates into two new groups, which will be deployed alongside of the November updates during the November ePatching cycle:
- GS-ENT-2022-Jan_to_Oct – Updates
- GS-ENT-Pre-2022 – Updates
WHAT YOU NEED TO DO:
Local system administrators are responsible for testing the required patches and reporting any issues to the ePatching team.
Specific instructions regarding MECM, BigFix, and JAMF patching can be found on the TST website at: https://tst.usgs.gov/security/epatching/
