Published March 15, 2026 | Version 1.2.1

Masquerading bots

  • 1. Arising Technology Pty Limited

Description

Introduction

Large numbers of anonymous agents have been masquerading with browser strings [1] and walking the mediawiki farm metadata. On blocking the unwanted bot the number of IP addresses substantially increased. This lead to the postulate that there is probably an intersection between rejected bot IP addresses and these anonymous agent's IP addresses.

Table A contains that intersection across more than 79 days, where the IP, CC and Original Bot agent string is listed only when there is a correlation with an anonymous masquerading agent accessing the site metadata. The unwanted bots were blocked on purpose - yet evidence shows anonymous agents were launched from the same public IP address, which in many cases were /19 and /15 addresses from large address pools - and in some cases consecutive. Nginx logs have been archived since 2025-12-24 and the log format was changed to include CC some weeks after inception.

Note that these were contributing to the large anon/metadata counts split out in the Access Life Cycle Graphlets included in Publications Access Graphs [1]

The table is the total capture of blocked agents that have accessed the mediawiki farm, though this has background I must draw to the attention of the reader:

  • some agents were initially blocked and then permitted into the farm e.g. perplexity and bing
  • others were blocked after they were detected
  • the remaining were always blocked and yet accessed the farm at some point
  • the filtering on the farm has been modified during this period.

Rejected bots are signalled distinctly in the nginx logs, as are the unwanted metadata access, and these signals were used to obtain the IP address captured in the results.

Files

Masquerading bots - publications.pdf

Files (605.4 kB)

Name Size Download all
md5:b9e374b9fb4d7c7e43a4a99f6c38a967
605.4 kB Preview Download

Additional details

Related works

Is previous version of
Publication: https://publications.arising.com.au/pub/Masquerading_bots (URL)