Xenforo conversion thread ( ES 2.0 )

[neptronix]

Had another recurrence of alibaba bots - this time trying a new tactic. Same case as before - masquerading as a legitimate user and not following the robots.txt rules.
Server got so choked up that i couldn't even remote into it to see what was going on.

A few restarts, some analysis on their pattern, some new rules, i got em trapped...



Hopefully there's no round 2 left in them, haha..

 

[neptronix]

Quite the battle last night. After cutting the new hydra head off alibaba, a new challenger appeared.

These IP blocks in Vietnam had 1000's of IP addresses and a very good IP rotation schedule. It's a covert operation ( randomizes it's browser identifiers on each hit ) designed around my DDOS protection system. The problem was that they were hitting PHP, and PHPBB is freakin' slow. And also, robots.txt specifically forbids scraping it.



I'm glad they weren't able to take the site down, but wow, i need a bigger gun.

ES 1.0 is now offline to the general public, and only available by special request. It's way too much of a headache to support.

 

[riba2233]

Wow that's a pretty shitty situation.

 

[neptronix]

Yeah. What bugs me is how difficult it is to keep an independent website online in the current year, when things like this are lurking about.

I worry about the independent web at large when i see this kind of activity. The bar of systems administration + web development skill keeps going up as time goes on.


On the other end of the coin..


Just as i thought, Alibaba was training an open source AI model:

I tried this AI model last night and was very impressed so far.
Good performance from mid powered hardware ( 8 core cpu + nvidia 2070 )
I could get it to write some advanced code that worked.
We may use this at the dev shop. maybe one day we write server defenses with this AI..

At least we got something out of this server hassle, but it would be a lot cooler if these companies scraped more respectfully!