Update rings: how a patch reaches every machine

Inventory, test group, scheduled release, and out-of-band patching: how the update process for operating systems and applications is built.

Topic
IT security
Author
ARLITECH expert team
Published

Weeks can pass between the release of a patch and the moment it lands on the last affected machine. Attackers use that window. A published patch often reveals, once reverse engineered, where the flaw was, and machines that have not been updated stay exactly as vulnerable as before.

Update management is therefore largely an organizational job. You need to know which machines and software you have, which patch reaches which group and when, what happens if an update breaks something, and how you find out that the process actually ran to completion. The sections below follow that chain from inventory to reporting. Which of the many vulnerabilities is urgent is covered in the article on prioritizing vulnerabilities; this one is about the path a patch takes.

Why third-party applications fall behind

Operating system updates work in most places. The built-in update service downloads the patches, and this is also what central management tools have handled the longest. The other programs on the machine are a different story. Browsers, PDF readers, archivers, video conferencing clients, runtimes and line-of-business applications each follow their own release schedule, and each updates in its own way.

  • Own updater without the rights. Some applications have an updater that looks for new versions on its own but asks for local administrator rights to install them. Whether it does depends on whether an updater service or scheduled task running with system rights was installed alongside the program. If not, and the user has no administrator rights, the update silently never happens.
  • Per-user installation. Some programs install into the user's profile without administrator rights. These often do not even show up in the central software list.
  • Several versions side by side. An old version of a runtime can stay on the machine next to the new one if an older application depends on it. The vulnerable version then remains available after the update.
  • Manual downloads. For anything that is not in a software catalog or package manager, updating depends on someone noticing the new version.

From an attacker's point of view these are exactly the interesting applications, because they handle content coming from outside directly: web pages, attachments, downloaded files. A process that only deals with the operating system covers just part of the machine's attack surface.

Device and software inventory

Update policies only apply to the machines in the inventory. If the inventory has gaps, the report shows everything in order while nothing changes on the machines that were left out.

The inventory has two parts. The device inventory lists the machines on the network, the software inventory lists what runs on them. For devices, the management tool's own list is not enough, because it only sees the machines where its agent runs. It is worth cross-checking against other sources: computer accounts in the directory, DHCP leases, network discovery results and the endpoint protection console's list. Anything that appears in one of these but not in the management tool is probably running without updates.

What to record for each device

  • the exact operating system version and edition, because end of support differs by version,
  • the installed software with version numbers, including software installed in user profiles,
  • the machine's role (workstation, server, kiosk, machine supporting production) and its business owner,
  • the update group it belongs to,
  • the time of the last successful update and the last logon.

Unsupported versions need a separate flag. No update policy can fix an operating system or application for which the vendor no longer releases patches. They need a decision: replace them, move them to an isolated network segment, or have someone accept the risk in writing until a set date.

Test group, pilot and broad release

An update ring means the patch does not go to every machine at once. A small group gets it first, and the next group only gets it if no problem appeared in the previous one. If an update clashes with a line-of-business application or a driver, the problem shows up on a few machines, and the release can be stopped before it affects the whole organization.

A typical split:

RingWho belongs hereWhat to check
TestIT staff machines and a few test serverswhether the patch installs, the machine boots and the basic applications work
Pilota few users from every major department, with their typical applicationswhether line-of-business programs, printers and connected devices work
Broad releasemost workstationsthe share of successful installations and the incoming problem reports
Special machinesservers, kiosks, machines supporting productionmaintenance window and rollback plan

A pilot group is only worth as much as the share of the organization's applications it covers. If the accounting or design office software is not in it, the problem only surfaces at broad release. Pilot members need to know that they get updates first and where to report it if something stops working.

Deferral and deadline

The wait between rings can be set in days since release: the pilot gets the patch a few days after the test ring, the broad release a few days after that. Every ring should also have a deadline by which the update must be installed. After it, the machine asks for a restart and finally forces one. Without a deadline the ring is only a recommendation, and because of postponed restarts some machines run in the old state for weeks.

Rolling back is part of the plan as well. Before the update moves to the next ring, it must be clear how the release can be paused and how the patch can be removed where it has already caused problems.

Maintenance windows and restarts on servers

Restarting a workstation bothers one person, restarting a server bothers everyone who uses the services on it. Servers are therefore updated in a fixed maintenance window, at a time agreed in advance with the service owner.

  • Dependency order. The application server should stop before and start after the database it relies on. If the order is not written down, the application comes up with errors after a night-time restart, and nobody notices until morning.
  • Clusters and redundant pairs. Where a service runs on several nodes, the nodes are updated one after another, so the service stays available in the meantime. Check beforehand that the remaining node really takes over the load.
  • Backup and restore point. Before the update there should be a recent backup, and it should be clear how long it takes to restore the previous state. A virtual machine snapshot is fine for short-term rollback but does not replace a backup.
  • Checks afterwards. After the restart it is not enough that the machine answers on the network. Check whether the services have started and whether the things users rely on work.

A common mistake is that the update is installed but the restart never happens. Many patches only take effect after a restart; until then the machine runs the old, vulnerable code and the management tool shows the status “installed, pending restart”, so the report counts these machines separately.

Different rules apply to machines in OT environments. There, downtime affects production, and updates often have to wait for approval from the equipment vendor, so these machines get their own schedule.

Out-of-band patching for actively exploited vulnerabilities

A ring-based release takes days or weeks, and for routine patches that is as it should be. But if a vulnerability is already being actively exploited, or the exploitation method became public together with the patch, there is no time for that. This situation needs a procedure written in advance, otherwise it has to be worked out in the middle of the alarm.

The signal can come from several places: the vendor's security advisory, a warning from the national CSIRT, or the vulnerability being added to the US CISA KEV catalog. The catalog gives a remediation deadline for each entry, which is mandatory for US federal civilian executive branch agencies under CISA BOD 22-01 and can serve others as a reference point.

What changes in an out-of-band release

  1. The decision. Designate in advance who can order an out-of-band release and under what conditions, for example when the vulnerability is being exploited and the affected system is reachable from the internet.
  2. Shorter rings. The test ring stays, because a faulty update can still cause an outage, but the wait drops from days to hours. Exposed systems get the patch first: internet-facing servers, systems that provide remote access, email and browsers.
  3. Temporary protection. If the patch has not been released yet or cannot be installed right away, a vendor-published workaround, disabling a service, restricting access or network-based virtual patching reduces the risk until the update.
  4. Follow-up investigation. If the vulnerability was exploited before the patch, the update does not remove what the attacker left on the machine in the meantime. Vendor advisories often describe which traces to look for.
  5. Return to the normal schedule. The machines go back into their rings, and the temporary exceptions are closed.
If a patch cannot be installed on a machine, record the exception in writing: which machine it is, why it cannot get the update, what temporary protection is in place, who approved it, and on which date it must be reviewed again.

What the patch status report should show

The report should show where the process stands and where it got stuck. A single percentage of machines that are “compliant” says little, because it hides which machines make up the missing part and how long they have been missing updates.

What is worth seeing

  • Coverage: which machines in the inventory report to the management tool, and which have not checked in for a while.
  • Status per ring: installed, pending restart, failed, not started yet.
  • The oldest missing patch per machine. One machine that has been missing an update for months calls for different action than fifty machines waiting for yesterday's release.
  • Versions of third-party applications, separately from the operating system.
  • Patch progress for exploited vulnerabilities, measured against the out-of-band deadline.
  • Unsupported versions and open exceptions, with expiry dates.

The report has two audiences. Operations staff need a per-machine list that turns into tasks. Management cares about the trend: whether the number of long-missing patches is going down and whether out-of-band deadlines are being met.

Regulation also asks about the process. Article 21 of the NIS2 Directive lists vulnerability handling among the risk management measures, and Annex A of ISO/IEC 27001:2022 has a separate control on the management of technical vulnerabilities. In both cases you have to show that patches reach the machines regularly and that the gaps are known.

Related service: Patch and update management →

Contact

Ready to secure your critical systems?

Our experts will assess where your security stands and build a roadmap that fits your organization.

ARLITECH's office in Gödöllő

Visit us

You are welcome to visit our Gödöllő office in person.

Address
Centrum Office Building, 2100 Gödöllő, Dózsa György út 13, 3rd floor, office 309
Office hours
Mon–Fri, 9 a.m. to 5 p.m.
Get directions (opens in a new tab)