Geoblocking Wars
Command Palette
Search for a command to run...
Comments
Join the discussionNo comments yet. Be the first to comment.
Search for a command to run...
No comments yet. Be the first to comment.
Geoblocking, the practice of restricting access to content based on the user's geographical location, has become a hotly debated topic in the digital age. In this blog post, we'll explore the various facets of geoblocking, its implications for users, businesses, and content providers, and examine the evolving landscape of this controversial practice.
It all started with a firewall! Recently, I bought a Firewalla to protect my network from cyber threats. I decided to go with Firewalla for several reasons:
It's not very expensive but has a lot of features that I can use in my home office.
Low maintenance (someone else is maintaining the software).
Don't have the time to build my own firewall from scratch; I prefer to focus on my goals.
Received a recommendation from my colleague who has been using this device for years
Firewall
Intrusion Prevention
Ad Block
VPN Server + Client
Intrusion Detection
DNS over HTTPS
Vulnerability scan
Bandwith usage monitoring
Behaviour Analytics
Safe Search
Geo-IP Filtering
Site to Site VPN
Not to mention, it can use my cell phone's internet connection as a backup in case my main internet is down.
I've been using this device for three months, and I regularly receive attacks from one specific country: the Netherlands. So, I decided to share the basic mitigation for this problem, in case your Linux server is not expecting any traffic from specific countries, and you don't have a firewall to do it for you.
For decades, IPtables has been a familiar companion for every Linux user. It's very common in many distributions, and together with ipset it can be a great tool to block your attackers. ipset serves as a companion application to the iptables Linux firewall, enabling users to establish rules for swift and straightforward blocking of a set of IP addresses, among other functionalities. So we will utilize this app to block out countries or our own blocklist from recent attacks. In addition if you install fail2ban to your server. Fail2ban operates by monitoring log files (e.g., /var/log/auth.log, /var/log/apache/access.log, etc.) for specific entries and running scripts based on them. Most commonly, this is used to block selected IP addresses that may belong to hosts attempting to breach the system's security. It can ban any host IP address that makes excessive login attempts or engages in any other undesirable action within a time frame defined by the administrator.
This is not easy if you want to collect them by your own, fortunately there are some paid and free tools which you can use to kickstart your project. One of the options which you can use is https://www.ip2location.com/free/visitor-blocker By using this page you can select the country of your interest and export the lists into different formats, for sure you can directly export it to your web server you can choose from many options for example:
Apache .htaccess allow/deny
Nginx allow/deny
CIDR
Linux iptables accept/drop
Netmask / Inverse Netmask
Cisco ACL
MikroTik
etc.
But the most common which I like to use is CIDR, this format contains network addresses in CIDR - Classless Inter Domain Routing format like this:
# -------------------------------------------------------
# Free IP2Location Firewall List by Country
# Source: https://www.ip2location.com/free/visitor-blocker
# Last Generated: 26 Jun 2023 09:24:57 GMT
# [Important] Please update this list every month
# -------------------------------------------------------
2.58.168.0/22
2.59.196.0/22
4.68.14.62/31
4.68.15.178/31
4.68.62.150/32
4.68.74.67/32
4.68.74.68/32
4.68.111.70/32
...
This script will utilize this list to create ipset-s then configure the firewall to filter out all the traffic from the specific sets. By using this method, we can also create whitelists and ban everything else, but that's not the scenario I would like to discuss here.
In order to make the script work we should save the files in the following format:
<REGION>_firewall_ipv6.txt.gz
<REGION>_firewall.txt.gz
Then copy the downloaded lists to sources/blacklist directory the directory will be like this example:
$ ls -l sources/blacklist/
total 928
-rw-rw-r-- 1 user user 25249 jan 5 10:28 china_firewall.txt.gz
-rw-rw-r-- 1 user user 72576 jan 5 11:03 india_firewall_ipv6.txt.gz
-rw-rw-r-- 1 user user 48580 jan 5 10:28 india_firewall.txt.gz
-rw-rw-r-- 1 user user 7764 jan 5 10:27 iran_firewall.txt.gz
-rw-rw-r-- 1 user user 10980 jan 5 10:27 israel_firewall.txt.gz
-rw-rw-r-- 1 user user 577905 jan 5 11:04 netherlands_firewall_ipv6.txt.gz
-rw-rw-r-- 1 user user 124682 jan 5 11:04 netherlands_firewall.txt.gz
-rw-rw-r-- 1 user user 45331 jan 5 10:27 russia_firewall.txt.gz
-rw-rw-r-- 1 user user 18890 jan 5 10:27 ukraine_firewall.txt.gz
So the whole file hierarchy will be the following:
.
├── geoblock.sh
└── sources
├── blacklist
│ ├── china_firewall.txt.gz
│ ├── india_firewall_ipv6.txt.gz
│ ├── india_firewall.txt.gz
│ ├── iran_firewall.txt.gz
│ ├── israel_firewall.txt.gz
│ ├── netherlands_firewall_ipv6.txt.gz
│ ├── netherlands_firewall.txt.gz
│ ├── russia_firewall.txt.gz
│ └── ukraine_firewall.txt.gz
└── whitelist
Let's check what's inside the script, in order to make it easy to read I created multiple functions in the bash script. The variables are global variables, so please handle them accordingly.
At first there are some defined variables
_SOURCE_DIR=./sources ## This is the source directory where the blacklist and whitelist directories are located.
_BLACKLIST_DIR=blacklist ## This is the directory within the _SOURCE_DIR which will contain the GZIP files
_IPLIST_DIR=./lists ## This is the directory where the gzips will be extracted
_CLEANUP=true ## True if you want to empty your lists before you create them
_MAXIMUM_LANES=65536 ## Maximum number of rules in one set (ipset default)
If you are calling a function with arguments the arguments will be passed as positional parameters. They will be stored in the $1 $2 $3 variables.
Description: creates a directory in case it's absent. Variables:
function create_directory_if_not_exists {
mkdir -p $1
}
Description: deleting the set in case the _CLEANUP variable is true Variables:
function delete_ipset {
if [[ $_CLEANUP == true ]]; then
echo "[INFO] Cleaning up ipset"
ipset -X $1 2>/dev/null
fi
}
Description: is checking if the set already defined Variables:
function test_ipset {
echo "ipset list $1 >/dev/null" | bash
_return=$?
}
Description: Simply check if a file is available and getting the filename from the $1 variable Variables:
function test_file_exists {
_exist_return=0
if [ -f $1 ]; then
_exist_return=1
fi
}
Description: This function check if the iptables rule exists for the specific chain Variables:
$1 = iptables chain name
$2 = set name
function check_if_iptables_exists {
echo "[INFO] Checking if iptables $1 rule exists for $2."
_ipt_lines=$(iptables -L $1 | grep $2 | wc -l)
}
Description: extract the type from the archive name because the ipv6 list needs additional parameters upon creation. Variables:
function get_ipv4_or_ipv6 {
filename=$(basename -- "$1")
if [[ $filename =~ "v6" ]]; then
_TYPE=ipv6
else
_TYPE=ipv4
fi
}
Description: extract region name from the filename. Variables:
function get_REGION {
filename=$(basename -- "$1")
_REGION=$(echo $filename | sed 's/_.*//')
}
Description: extracts the Gzip files to target directory Variables:
$1 = input filename (Gzip)
$2 = output filename (txt)
function extract_file_to_iplist_dir {
gunzip -c $1 > $2
}
Description: clear the _REGION and _TYPE variables.
function clear_variables {
_REGION=""
_TYPE=""
}
Description: splits the files to multiple in case they have more lanes than the defined $_MAXIMUM_LANES. Variables:
function cut_file_to_multiple {
create_directory_if_not_exists ./tmp
filename=$(basename -- "$1" .txt)
split -l $_MAXIMUM_LANES $1 ./tmp/${filename}_ext
_counter=1
for split_files in ./tmp/* ;
do
_file_basename=$(echo $split_files | sed 's/_.*//')
_target_file_name=$(basename -- "$_file_basename")-${_counter}.txt
echo $_target_file_name
mv $split_files $_IPLIST_DIR/$_target_file_name
let "_counter+=1"
done
rm $1
}
Description: Check the length of the files and if they are longer than the defined $_MAXIMUM_LANES then calls the cut_file_to_multiple function. Variables:
function divide_file_by_lane_number {
_length=$(cat $1 | wc -l)
if [[ $_length -le $_MAXIMUM_LANES ]]; then
echo "[INFO] - File length of ($1) is less than $_MAXIMUM_LANES"
else
echo "[INFO] - File length of ($1) is larger than $_MAXIMUM_LANES, cutting to multiple files !"
cut_file_to_multiple $1
fi
}
Description: remove the comment lines from the lists as we don't need them. Variables:
function remove_comments_from_lists {
sed -i -e '/^[ \t]*#/d' $1
}
Description: This is the function that creates the IPsets and populates them with data. Variables:
function set_ipset {
_filename=$1
_IPSET_NAME=$(basename -- "$1" .txt)
which ipset
if [[ $? == 1 ]]; then
echo "[ERROR] Ipset is not installet or can't be found in \$PATH, please check ! "
exit 1
fi
echo "[INFO] Creating geoblocking for country: $_REGION"
delete_ipset $_IPSET_NAME
test_ipset $_IPSET_NAME
if [[ $_return == 1 ]]; then
echo "[INFO] Creating set $_IPSET_NAME"
if [[ ${_TYPE} == ipv6 ]]; then
ipset create $_IPSET_NAME hash:net family inet6 timeout 86400
else
ipset create $_IPSET_NAME hash:net timeout 86400
fi
else
echo "[WARNING] The set named ($_IPSET_NAME) already exists !"
fi
test_file_exists $_filename
if [[ $_exist_return == 0 ]]; then
echo "[ERROR] Can't find file $_filename"
else
cat $_filename | grep -v ^# | awk {'print "ipset -exist add '$_IPSET_NAME' " $1'} | bash
_CHAIN="INPUT"
_ipt_lines=0
check_if_iptables_exists $_CHAIN $_IPSET_NAME
if [ $_ipt_lines == 0 ]; then
echo "[INFO] Creating IPtables $_CHAIN rule of $_IPSET_NAME"
iptables -I INPUT -m set --match-set $_IPSET_NAME src -j DROP
else
echo "[INFO] IPtables INPUT rule already exists for set $_IPSET_NAME"
fi
_CHAIN="OUTPUT"
_ipt_lines=0
check_if_iptables_exists $_CHAIN $_IPSET_NAME
if [ $_ipt_lines == 0 ]; then
echo "[INFO] Creating IPtables $_CHAIN rule of $_IPSET_NAME"
iptables -I OUTPUT -m set --match-set $_IPSET_NAME dst -j DROP
else
echo "[INFO] IPtables OUTPUT rule already exists for set $_IPSET_NAME"
fi
fi
}
Description: This function iterates through the .txt.gz files in $_SOURCE_DIR/blacklist/, extracts the gzip files to the destination folder, and removes the comments from them
function prepare_lists {
for input_file in $_SOURCE_DIR/blacklist/*.txt.gz;
do
clear_variables
get_REGION $input_file
get_ipv4_or_ipv6 $input_file
_CURRENT_LIST_FILENAME=${_IPLIST_DIR}/${_REGION}-${_TYPE}.txt
extract_file_to_iplist_dir $input_file $_CURRENT_LIST_FILENAME
remove_comments_from_lists $_CURRENT_LIST_FILENAME
done
}
Description: In this function, the script cuts the files into smaller chunks if necessary, then populates the IPset sets and creates iptables rules to drop incoming and outgoing communication with the specific networks.
function create_sets {
for input_file in $_IPLIST_DIR/*.txt;
do
clear_variables
get_REGION $input_file
get_ipv4_or_ipv6 $input_file
divide_file_by_lane_number $input_file
set_ipset $input_file
done
}
create_directory_if_not_exists $_IPLIST_DIR
prepare_lists
create_sets
The source code of this post can be downloaded from my github repository here: