Note to self (and anyone else with similar curiosities) — If I take another, bigger look at WordPress patch uptake, I’ll start before a security release and follow the same sites more closely afterwards. October’s three checks showed me what to keep, what to change, and what the public evidence can’t tell us. This is what I’ll do differently next time, if there is a next time.

The October study followed the same 607 sites on October 6, 8, and 15: roughly ten hours after WordPress 7.1.3 was announced, and then about two and nine days after release. It measured movement between those checks. It could not tell exactly when a site updated or what installed the update.

The October per-site files were deleted on October 15, after the final check. That was for security reasons and general cleaning up, but I should’ve kept them a bit longer. The aggregate counts published in the findings post survive. They support arithmetic checks and the reported group comparisons, but they can’t reconstruct individual transitions or support new checks of the same sites and networks. The evidence note explains the limits. A future study would need a fresh sample.

What I’d want to learn

• Speed: what sample share shows the latest release at about 12 hours, 3 days and 2 weeks after its announcement? Each observation provides an interval in which the visible version changed, not an exact installation time.
• Backports versus upgrades: how many sites reach their branch’s security backport, how many move to a newer branch containing the fix, and how many remain behind? A version increase within an old branch is not enough to count as patched.
• Clusters: do groups of sites change version in the same observation interval? Shared network information may help identify patterns, but cannot establish who installed an update.
• Coverage: how many additional WordPress installations can a documented set of checks find beyond the homepage on a single domain?

Start before the release

Security release dates are not reliably known in advance, so you can’t arrange a baseline exactly one week before one, but you can often expect they’re going to follow after a major release. I’d prepare a sample and take dated baseline scans while waiting. The last completed scan before the announcement would become the baseline, with its age reported.

I’d aim for follow-ups 12 hours, 3 days and 14 days after the public announcement, recording each fetch’s actual time and the start and end of each scan. A scan takes time; it is not a snapshot of every site at one instant. If the setup wasn’t ready before a release, I’d wait for a later one.

Release announcements, package availability and the update offers received by sites are different events. I’d use the announcement as the common clock and record the public release and backport timestamps I could verify. I cannot observe when every site was first offered a patch.

Sample and detection

I’d consider a draw of 20,000 domains from a dated Tranco list. Applying October’s overall detection rates suggests this would yield about 2,800 WordPress sites and 1,500 readable versions. Those are planning estimates, not a promised sample size.

I’d like to compare three bands from each rank: 1–10,000, 10,001–100,000 and 100,001–1,000,000. Equal-sized draws would make those groups easier to compare, but they would overrepresent the highest ranks in a combined total. Any overall estimate would need the sampling weights, and readable-version results would still have selection limits.

I’d record the list ID, draw method and final allocation before scanning. The sampled domains and anything that could reconstruct them, including a reproducible random seed, would stay private and be covered by the standard deletion policy. With the October domain files gone, I cannot deliberately exclude every previous site measured; overlap would be unknown, but that’s not going to bias the results.

• Version evidence: I’d use the generator tag or a checked list of core assets whose version strings follow WordPress releases. I wouldn’t treat every js/dist file or bundled library version as a WordPress version. I’d mark conflicting or implausible signals as uncertain.
• Redirects and extra paths: I’d document redirect handling and separately report additional detections from /blog/, blog. and news. checks. I’d account for duplicate destinations rather than count every redirect as a separate site.
• API evidence: a recognizably WordPress REST API response can support a WordPress detection. It does not establish a readable version or prove the site is headless.
• Consistent requests: I’d keep timeout, user agent and retry rules fixed, and record failures. Consistent settings reduce measurement changes; they do not make every change in reachability a change at the site.

What counts as patched

For the release being studied, I’d record which fixes and backports were actually published. I’d use WordPress.org’s stable-check status alongside the release record, retaining dated copies so a later API change does not change how the earlier result is interpreted. I’d check which older branches received fixes at the time, rather than assume today’s coverage still applied.

I’d keep separate counts for sites that reach the same-branch backport, sites that upgrade to a newer patched branch, sites that advance but remain behind, sites with no observed change, and sites whose versions can no longer be read. I’d report starting groups and denominators explicitly. A disappearing version is neither a successful update nor proof of failure.

Predictions and network patterns

October’s results included more movement among lagging sites than I initially expected, much of it through major upgrades. Next time, I’d make separate predictions for backports and major upgrades.

If I went ahead, I’d publish the predictions and scoring rules before collecting the baseline data. The outline here leaves those choices open.

If I collected network information, I’d treat shared IP ranges and ASNs as clues. A CDN or shared host can put unrelated sites together. Sites changing within the same multi-hour scan interval do not prove a coordinated host deployment. I’d fix any cluster rule in advance and show results both including and excluding flagged groups.

I’d account for the sampling design and dependence between sites when estimating uncertainty. A simple interval around a percentage cannot resolve version-detection bias, caching, or uncertainty about who performed an update.

Privacy and retention

The October deletion is complete. Next time, I’d publish a retention rule before collecting anything: delete the private per-site records and their copies within 30 days of the final scan. With the last scan planned for two weeks after release, that would put deletion around day 44. The actual deadline would follow the final scan’s date, not an assumed release schedule. I’d record that deadline once the scan was complete and note when deletion was done.

That policy would cover working files, copies, logs, network identifiers and anything that could reconstruct the sample. I’d publish only aggregates. Before starting, I’d also settle storage and backup handling and provide a contact page for site owners who wanted to opt out.

Limits and a possible cross-check

This kind of study observes what sites expose publicly. Cached pages can lag behind an installation; hidden versions remain unknown. Tranco is a ranked-domain sampling frame, not a representative list of all WordPress installations. Rank alone does not establish that a site is better maintained, but we might be able to test and validate or falsify the assumption that high traffic, high-value sites are less likely to be left in a vulnerable state.

HTTP Archive documents WordPress version detections in its public dataset, including missing and unreliable values. A suitable crawl could provide a separate comparison, but not a precise check of this study’s release-relative timings. I’d check the query, coverage, and cost before including that comparison.

If I took this further, I’d first test the scanner, settle the sample and privacy arrangements, and publish the method before collecting the baseline. None of that work is scheduled here.

What the October study found

The October study completed checks on October 6, 8 and 15. Of 53 older-branch sites just one patch behind at the first check, 42 took their branch’s backport and one jumped to 7.1.3. Of 114 sites further behind, six moved within their branch and 20 jumped from 7.0 to 7.1.3. The surviving totals do not show whether those six reached their branch’s security backport.

Across those two older-branch groups, 93 of 167 sites (56%) still showed no version change nine days later; another five could no longer be read. Backports reached most sites already keeping up. Most movement among the further-behind sites came through major upgrades. The scans cannot tell us who installed an update or why a site stayed put.

The results and the work around them are here:

• Who actually gets WordPress security backports? has the final counts, the backport-versus-upgrade split, and the limits of the sample.
• You can’t observe a security backport by waiting for one examines the updater itself.
• Is your old WordPress site still getting security updates? explains how backports reach older branches and what can stop them arriving.

Those findings are why I’d separate backport uptake from major upgrades in any future study.

The other lesson: I’d decide which aggregate tables to keep before deleting the private records. They would need enough detail to support the published conclusions without identifying the sites.

Part of this series

• Your WordPress site’s age decides whether it installs major updates
• Is your old WordPress site still getting security updates?
• Keel Defaults
• Who actually gets WordPress security backports?
• You can’t observe a security backport by waiting for one
• WordPress security backports: quiet by design
• What I’d do differently next time: studying WordPress patch uptake (this post)