WhiteSites Blog

Block Foreign IPs to cut your server load.

Posted on Aug 3, 2009 by Paul White

Blocking IPs to keep the trash out is nothing new for Server Admins.  Before there were RBLs ( Real time Black Lists ), server admins would sometimes block traffic from an entire range of IPs.  However in my eyes, what is old, is new again.  Here are some of the situations where Blocking IPs will help

Submit Form Spam through your website


Recently many of the websites that I run and monitor have been getting an increased amount of spam.  This is not email spam, but form spam.  Its often pushing SEO services, claiming they can get you top rank on google, or some type of performance enhancing pill.  Like a good Webmaster I monitor all messages coming through the submit forms on my clients websites.  One of the things I include in these forms is the IP of the sender.  What I have found is that most of this form spam is coming form asian countries.  These clients don't have any customers in these markets, nor do they want to expand to these markets.  So to put an end to this, I have started to block IPs from these sites.

Stop Dictionary Attacks on your Mail Server


Unless you run the company mail server, and get access to the SMTP logs, you are probably unaware of the volume of Dictionary Attacks pinging your server every second.  This is when spam servers take a guess as to the emails that are on your mail server, using random usernames @ yourdomain.com.  Most of them won't be there, and your server will say User not found.  All your server is doing is playing a guessing game with the sending server.  It will keep trying every username in a dictionary until something works, then it will remember that this email works and continue to send you more spam.  This is very bad for users with simple emails like, mike@mydomain.com, or jill@mydomain.com.  The problem is even worse for catch all emails.  To stop this you should enable abuse detection on your mail server. This will detect when any IP is trying to send emails to user accounts that don't exists.  I would recommend setting up the detection for 2 instances over 6 hours.  Then if you catch an IP,  it will be blocked for a period of time from your mail server.  In my case the block period is 6 months ( I really hate spammers ).

Stop Spam on your Mail Server


Even though most spam comes from inside the USA.  using a combination of RBL lists, and greylisting can stop most of this spam.  Most ISPs inside the USA are very strict about spam, this makes getting IPs from inside the USA very expensive for spammers.  While IPs in asia where the oversight by government authorities is minimal, its relatively cheap to setup thousands of spam servers.  Either way spammers want to reach Americans, because we have more money than other countries, and we are dumb enough to acutally click on the links and checkout the spam sites.  Recent study found that 1 in 6 people actually open the same mail.  Wish I could get those kind of CTR rates on my clients newsletters.

What IPs should I block?


This is the easy part.  There are several regional registries, each with their own blocks of IPs they manage. 
For the most updated list see the IANA IPV4 address space registry  They also have the list available in an XML document.  Looking at the list is interesting to see which companies actually got their Class A IP block.  Here is how I used this list to build my own blacklist.  I wanted to block Europe, Africa, and Asian,  So I blocked all IP blocks that have the designation of RIPE, APNIC, AFRINIC.  I didn't do every single one, just the ones that were grouped together.  So for RIPE I am blocked all IPs from 77.0.0.0 - 95.255.255.255.  Next time you get some spam checkout the IP of the sending server, and see where it sits on this list.  Thus far it has cut down the spam traffic to my mail server.

UPDATE 8/4/2009


After doing this on 8/3 - 8/4  I checked my logs.  The way you can tell its working is because the number of SMTP requests you server has to handle will go down as all foreign IPs are blocked.  This will result in a much smaller SMTP log file Here are my results

7/29 SMTP Log  2,418 KB
7/30 SMTP Log  3,541 KB
7/31 SMTP Log  2,666 KB
8/01 SMTP Log  2,083 KB
8/02 - 8/03 Implimented IP block to Foriegn IPs
8/04 SMTP Log  633 KB

Pretty amazing huh?  over 75% of the requests my server was handling were spammers, and bots from Foreign IPs.  None of my clients have reports any problems getting emails from their legitimate customers.



Permalink
6599 Visitors
6599 Views

Categories associated with Block Foreign IPs to cut your server load.

Discussion

narutolost | May 6, 2010 2:20 AM
This is so stupid, even xenophobic.  There's no reason to block other countries from seeing your site.  How about you block Canada and then Colorado next while you're at it?
Paul | May 6, 2010 7:04 AM
Narutolost,
For websites that have a client base that is USA only.  Blocking foreign IPs makes sense.  If you saw the number of spam submits ( usually promising to increase a websites SEO ) that come through the comment forms on my websites you would not be so against the idea of blocking foreign IPs.  The IPs of these submits almost always go back to south east asia, and the person submitting always uses a gmail email address.  Of course complaining to google does nothing.

Narutolost | May 7, 2010 12:24 AM
Unless you're getting DOS attacks, why not just add a captcha and be done with bot spam?  As an web surfer who regularly explores foreign webpages, I'm opposed to blocking internet access to large numbers of innocent people because I think it goes against the principal of a free and interconnected, world wide web.  It's sort of like giving the finger to other countries, because how would you feel if a company you were interested in blocked your country's IP?  My feelings said, I acknowledge you're free to do with your site as you wish, including measures to save yourself money on bandwidth.

Paul | May 7, 2010 8:06 AM
Captchas are no good.  Captchas do nothing to prevent entries from being made in my error logs.  It basically comes down to the numbers.  99.9% of my foreign traffic is spambots, and spam servers and when my client does not care to have foreign customers, it makes sense to block foreign IPs. 
Bryan | Mar 11, 2013 7:03 AM
Thank you for this, previously I was just checking out my server log to find out the IP's of these spam bots. As Paul put it, blocking a single IP or relying solely on Captcha is ineffective as bots have gotten smarter and after a particular bot's IP has been banned, another makes an attempt on your site sometimes within minutes of the previous one. Unfortunately this detracts from the customer experience of targeted customers. So I'd prefer to just block the IP range and be done with it.
name
Email Needed to confirm comment, but not made public.
Website
 
 
When you Post your Comment, you'll be sent a confirmation link. Once you click this link your thoughts will be made public.. Posts that are considered spam will be deleted, Please keep your thoughts and links relavent to this Article