Site stability fixed (I think)

[Daeron]

Greetings everyone,

For the past weeks, both Ulthuan and Druchii sites have been suffering from downtime. I want to apologize for the inconvenience.
The source of the problem was that AI bots were scraping our forums, to the point that they completely shut down the normal working of the sites. I've blocked their access, and it seems the site is acting normally once more.

Unfortunately, our forum is longer being used to train AI LLM's. As a result, AI bots might become less versed in Warhammer and may start talking like normal people again.

If there are any other issues, please feel free to point them out and I'll have a look at it.
Cheers,
Daeron

--- For those interested in more technical details

Our site was struggling to communicate with the outside world (sftp and ssh access were still fine). This lead me to believe there was a congestion for HTTP. The Host support indicated the site was functioning as expected, and that there may have been "too much" access. The site had over 700K visits yesterday. It took me a while to figure it out, but the access logs showed a lot of bot activity. Some googling showed it's AI LLM training related.
Over 600K (88%) were from a single bot source.

I tried a different strategy for Ulthuan and Druchii, to see which one would be more effective. For Druchii I tried the nice way, by blocking the AI bot through the robots.txt file. Bots usually check this to see what they are and are not permitted to do. It seems this one, Claude, wasn't interested in playing nice. For Ulthuan I tried a Firewall rule to block Claude. Unfortunately, this isn't supported by our Host without paying a premium. It is possible to secure this at the DNS level though. So I set up the CloudFlare account to provide our DNS services, and set up the blocking rule there.
The traffic dropped almost immediately, and the firewall rule was triggered roughly 2000 times in just a few minutes.
Since the firewall is up, forum behaviour is back to normal.

The downside is that we're now dependent on an external party for the DNS service, but it gives us free (but limited) Web Application Firewall. This shouldn't be an issue at technical level. I just prefer to keeps things centered, but this was getting out of hand.

[Calisson]

Thanks for taking care of the sites.