Nov 23, 2015 - 06:31 AM
One time several years ago our servers were attacked by DDOS attack. We went down for a week, and no one could have helped us. Luckily at that time we found Cloudflare.com that promised to mitigate the DDOS attack, and after working with them for several months we were able to develop a smart DDOS protection service to avoid future attacks.
After we developed our TEA Software predictive analytics that can detect human vs fake bot it got us thinking how much bots were visiting our websites compared to real humans. After excluding the friendly bots (ex: Googlebot) we found that 80% of our server traffic was fake bots. Ranging from SEO scrapers (ex: Spyfu.com), Web archives (Alexa Timemachine), dozens of Uptime monitors, CDN, Content scrapers, etc.
Usually bots will come in identifying themselves as bots, and if they are not white listed we started blocking them. We also started blocking malicious behavior such as entry without identify the browser (usually a web scraper). The block would be a captcha solve that once solved would white list the false positive visit.
We actively monitor all false positive traffic, and today there is virtually none. You can see for yourself if a false positive occurred by seeing how many referrers you got that came from TEA Software in your analytics.
The prove your human captcha will not affect your sales as we’ve measured it turned on and off for many of our clients and the sales are not affected.
We also block hacker injection. If you try to inject a hacking script into a form or URL it will block you. The false positives we’ve seen is when our customers use “0x1”in their URL (ex: domain.com/product/10x10inch). This will trigger our injection script to turn on and block the visitor trying to reach that page. We highly recommend that you do not use digits in your URL that may lead to such hacking injector patterns. Usually, anything starting with 0x# will cause a problem.
You may browse all your web pages to ensure you do not have such an issue, or export your product CSV sheet and sure for the patterns. The patterns can be shared with you by your account manager.
There is no way to turn off the scripts on your website as this would create a liability for us and all of our clients and open the gates to hackers to penetrate our system as well as bots to attack us via DDOS attack. Our rules are very flexible and will not trigger against real visitors. For it to trigger there has to be a very malicious behavior on the website. If you do experience a false positive please let our team know immediately so we can resolve it.