We should investigate what is causing the high usage on multiple servers. CP2 has just been suspended.
Description
Description
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Resolved | None | T4039 Increased bandwidth usage | ||
| Declined | John | T4035 Write icinga check to alert if bandwidth usage exceeds certain thresholds |
Event Timeline
Comment Actions
cp2 has used 1TB since JANUARY 17th.
cp4 has used 500GB since JANUARY 4th.
If this has increased traffic, it’s coming from the Americas. If it’s abuse, it’s also coming from the Americas.
Comment Actions
Let's retrieve some Varnish Traffic graphs, jan. 20, 2019 23:24:34 to jan. 21, 2019 17:58:02.
cp2:
max avg total frontend 2.567 MiB 263 KiB 282.282 MiB backend 1.460 MiB 189 KiB 202.572 MiB
cp4:
max avg total frontend 1.55 MiB 163 KiB 174.79 MiB backend 6.30 MiB 122 KiB 130.60 MiB
Based on this graph, cp2's bandwidth usage is approximately 58.6% higher. The actual number in 'vnstat -d -i venet0' actually fluctuates very much (between 21 and 84% higher for a given day), but John's comment seems just about right. cp2 is serving way more traffic than any other cache proxy.
@Paladox pointed out that requests in the cp2 access log with User-Agent 'SemrushBot' popped up quite frequently. I did an investigation on this and could confirm SemrushBot has been responsible for 20% or more of all requests recorded in the access log between 18/Jan/2019:05:43:16 +0000 and 21/Jan/2019:05:43:16 +0000.
It is possible this bot contributed to the suspension of those servers. I have banned this bot for life now, so if traffic usage per day drops considerably now, we know who was responsible.