Geoblocking Wars

What is geoblocking anyway?

Geoblocking, the practice of restricting access to content based on the user's geographical location, has become a hotly debated topic in the digital age. In this blog post, we'll explore the various facets of geoblocking, its implications for users, businesses, and content providers, and examine the evolving landscape of this controversial practice.

WAIT !! We are under attack: TLS Hearthbleed Attack

It all started with a firewall! Recently, I bought a Firewalla to protect my network from cyber threats. I decided to go with Firewalla for several reasons:

• It's not very expensive but has a lot of features that I can use in my home office.

• Low maintenance (someone else is maintaining the software).

• Don't have the time to build my own firewall from scratch; I prefer to focus on my goals.

• Received a recommendation from my colleague who has been using this device for years

The feature list

• Firewall

• Intrusion Prevention

• Ad Block

• VPN Server + Client

• Intrusion Detection

• DNS over HTTPS

• Vulnerability scan

• Bandwith usage monitoring

• Behaviour Analytics

• Safe Search

• Geo-IP Filtering

• Site to Site VPN

Not to mention, it can use my cell phone's internet connection as a backup in case my main internet is down.

The Netherland pattern

I've been using this device for three months, and I regularly receive attacks from one specific country: the Netherlands. So, I decided to share the basic mitigation for this problem, in case your Linux server is not expecting any traffic from specific countries, and you don't have a firewall to do it for you.

GeoBlocking using ipset and iptables

For decades, IPtables has been a familiar companion for every Linux user. It's very common in many distributions, and together with ipset it can be a great tool to block your attackers. ipset serves as a companion application to the iptables Linux firewall, enabling users to establish rules for swift and straightforward blocking of a set of IP addresses, among other functionalities. So we will utilize this app to block out countries or our own blocklist from recent attacks. In addition if you install fail2ban to your server. Fail2ban operates by monitoring log files (e.g., /var/log/auth.log, /var/log/apache/access.log, etc.) for specific entries and running scripts based on them. Most commonly, this is used to block selected IP addresses that may belong to hosts attempting to breach the system's security. It can ban any host IP address that makes excessive login attempts or engages in any other undesirable action within a time frame defined by the administrator.

But how to create IP lists ?

This is not easy if you want to collect them by your own, fortunately there are some paid and free tools which you can use to kickstart your project. One of the options which you can use is https://www.ip2location.com/free/visitor-blocker By using this page you can select the country of your interest and export the lists into different formats, for sure you can directly export it to your web server you can choose from many options for example:

• Apache .htaccess allow/deny

• Nginx allow/deny

• CIDR

• Linux iptables accept/drop

• Netmask / Inverse Netmask

• Cisco ACL

• MikroTik

• etc.

But the most common which I like to use is CIDR, this format contains network addresses in CIDR - Classless Inter Domain Routing format like this:

# -------------------------------------------------------
# Free IP2Location Firewall List by Country
# Source: https://www.ip2location.com/free/visitor-blocker
# Last Generated: 26 Jun 2023 09:24:57 GMT
# [Important] Please update this list every month
# -------------------------------------------------------
2.58.168.0/22
2.59.196.0/22
4.68.14.62/31
4.68.15.178/31
4.68.62.150/32
4.68.74.67/32
4.68.74.68/32
4.68.111.70/32
...

This script will utilize this list to create ipset-s then configure the firewall to filter out all the traffic from the specific sets. By using this method, we can also create whitelists and ban everything else, but that's not the scenario I would like to discuss here.

The script

In order to make the script work we should save the files in the following format:

• IPV6 example:

<REGION>_firewall_ipv6.txt.gz

• IPV4 example:

<REGION>_firewall.txt.gz

Then copy the downloaded lists to sources/blacklist directory the directory will be like this example:

$ ls -l sources/blacklist/
total 928
-rw-rw-r-- 1 user user  25249 jan    5 10:28 china_firewall.txt.gz
-rw-rw-r-- 1 user user  72576 jan    5 11:03 india_firewall_ipv6.txt.gz
-rw-rw-r-- 1 user user  48580 jan    5 10:28 india_firewall.txt.gz
-rw-rw-r-- 1 user user   7764 jan    5 10:27 iran_firewall.txt.gz
-rw-rw-r-- 1 user user  10980 jan    5 10:27 israel_firewall.txt.gz
-rw-rw-r-- 1 user user 577905 jan    5 11:04 netherlands_firewall_ipv6.txt.gz
-rw-rw-r-- 1 user user 124682 jan    5 11:04 netherlands_firewall.txt.gz
-rw-rw-r-- 1 user user  45331 jan    5 10:27 russia_firewall.txt.gz
-rw-rw-r-- 1 user user  18890 jan    5 10:27 ukraine_firewall.txt.gz

So the whole file hierarchy will be the following:

.
├── geoblock.sh
└── sources
    ├── blacklist
    │   ├── china_firewall.txt.gz
    │   ├── india_firewall_ipv6.txt.gz
    │   ├── india_firewall.txt.gz
    │   ├── iran_firewall.txt.gz
    │   ├── israel_firewall.txt.gz
    │   ├── netherlands_firewall_ipv6.txt.gz
    │   ├── netherlands_firewall.txt.gz
    │   ├── russia_firewall.txt.gz
    │   └── ukraine_firewall.txt.gz
    └── whitelist

Script explanation

Let's check what's inside the script, in order to make it easy to read I created multiple functions in the bash script. The variables are global variables, so please handle them accordingly.

At first there are some defined variables

_SOURCE_DIR=./sources  ## This is the source directory where the blacklist and whitelist directories are located.
_BLACKLIST_DIR=blacklist ## This is the directory within the _SOURCE_DIR which will contain the GZIP files
_IPLIST_DIR=./lists    ## This is the directory where the gzips will be extracted
_CLEANUP=true          ## True if you want to empty your lists before you create them 
_MAXIMUM_LANES=65536   ## Maximum number of rules in one set (ipset default)

Functions

If you are calling a function with arguments the arguments will be passed as positional parameters. They will be stored in the $1 $2 $3 variables.

1. create_directory_if_not_exists

Description: creates a directory in case it's absent. Variables:

• $1 = directory name

function create_directory_if_not_exists {
  mkdir -p $1
}

2. delete_ipset

Description: deleting the set in case the _CLEANUP variable is true Variables:

• $1 = set name

function delete_ipset {
  if [[ $_CLEANUP == true ]]; then
    echo "[INFO] Cleaning up ipset"
    ipset -X $1 2>/dev/null
  fi
}

3. test_ipset

Description: is checking if the set already defined Variables:

• $1 = set name

function test_ipset {
  echo "ipset list $1 >/dev/null" | bash
  _return=$?
}

4. test_file_exists

Description: Simply check if a file is available and getting the filename from the $1 variable Variables:

• $1 = filename

function test_file_exists {
  _exist_return=0 
  if [ -f $1 ]; then
    _exist_return=1 
  fi
}

5. test_iptables_rule

Description: This function check if the iptables rule exists for the specific chain Variables:

• $1 = iptables chain name

• $2 = set name

function check_if_iptables_exists {
  echo "[INFO] Checking if iptables $1 rule exists for $2."
  _ipt_lines=$(iptables -L $1 | grep $2 | wc -l)
}

6. get_ipv4_or_ipv6

Description: extract the type from the archive name because the ipv6 list needs additional parameters upon creation. Variables:

• $1 = iptables chain name

function get_ipv4_or_ipv6 {
  filename=$(basename -- "$1")
  if [[ $filename =~ "v6" ]]; then
    _TYPE=ipv6
  else
    _TYPE=ipv4
  fi
}

7. get_REGION

Description: extract region name from the filename. Variables:

• $1 = filename

function get_REGION {
  filename=$(basename -- "$1")
  _REGION=$(echo $filename | sed 's/_.*//')
}

8. extract_file_to_iplist_dir

Description: extracts the Gzip files to target directory Variables:

• $1 = input filename (Gzip)

• $2 = output filename (txt)

function extract_file_to_iplist_dir {
  gunzip -c $1 > $2
}

9. clear_variables

Description: clear the _REGION and _TYPE variables.

function clear_variables {
  _REGION=""
  _TYPE=""
}

10. cut_file_to_multiple

Description: splits the files to multiple in case they have more lanes than the defined $_MAXIMUM_LANES. Variables:

• $1 = filename

function cut_file_to_multiple {
  create_directory_if_not_exists ./tmp
  filename=$(basename -- "$1" .txt)
  split -l $_MAXIMUM_LANES $1 ./tmp/${filename}_ext
  _counter=1
  for split_files in ./tmp/* ; 
  do 
    _file_basename=$(echo $split_files | sed 's/_.*//')
    _target_file_name=$(basename -- "$_file_basename")-${_counter}.txt
    echo $_target_file_name
    mv $split_files $_IPLIST_DIR/$_target_file_name
    let "_counter+=1"
  done
  rm $1
}

11. divide_file_by_lane_number

Description: Check the length of the files and if they are longer than the defined $_MAXIMUM_LANES then calls the cut_file_to_multiple function. Variables:

• $1 = filename

function divide_file_by_lane_number {
  _length=$(cat $1 | wc -l)
  if [[ $_length -le $_MAXIMUM_LANES ]]; then
    echo "[INFO] - File length of ($1) is less than $_MAXIMUM_LANES"
  else
    echo "[INFO] - File length of ($1) is larger than $_MAXIMUM_LANES, cutting to multiple files !"
    cut_file_to_multiple $1
  fi
}

12. remove_comments_from_lists

Description: remove the comment lines from the lists as we don't need them. Variables:

• $1 = filename

function remove_comments_from_lists {
  sed -i -e '/^[ \t]*#/d' $1
}

13. set_ipset

Description: This is the function that creates the IPsets and populates them with data. Variables:

• $1 = filename

function set_ipset {
  _filename=$1
  _IPSET_NAME=$(basename -- "$1" .txt)
  which ipset
  if [[ $? == 1 ]]; then
    echo "[ERROR] Ipset is not installet or can't be found in \$PATH, please check ! "
    exit 1
  fi
  echo "[INFO] Creating geoblocking for country: $_REGION"
  delete_ipset $_IPSET_NAME
  test_ipset $_IPSET_NAME
  if [[ $_return == 1 ]]; then
    echo "[INFO] Creating set $_IPSET_NAME"
    if [[ ${_TYPE} == ipv6 ]]; then 
      ipset create $_IPSET_NAME hash:net family inet6 timeout 86400
    else
      ipset create $_IPSET_NAME hash:net timeout 86400
    fi
  else
    echo "[WARNING] The set named ($_IPSET_NAME) already exists !"
  fi

  test_file_exists $_filename
  if [[ $_exist_return == 0 ]]; then 
    echo "[ERROR] Can't find file $_filename"
  else

    cat $_filename | grep -v ^# | awk {'print "ipset -exist add '$_IPSET_NAME' " $1'} | bash

    _CHAIN="INPUT"
    _ipt_lines=0
    check_if_iptables_exists $_CHAIN $_IPSET_NAME
    if [ $_ipt_lines == 0 ]; then 
      echo "[INFO] Creating IPtables $_CHAIN rule of $_IPSET_NAME"
      iptables -I INPUT -m set --match-set $_IPSET_NAME src -j DROP
    else
      echo "[INFO] IPtables INPUT rule already exists for set $_IPSET_NAME"
    fi

    _CHAIN="OUTPUT"
    _ipt_lines=0
    check_if_iptables_exists $_CHAIN $_IPSET_NAME
    if [ $_ipt_lines == 0 ]; then 
      echo "[INFO] Creating IPtables $_CHAIN rule of $_IPSET_NAME"
      iptables -I OUTPUT -m set --match-set $_IPSET_NAME dst -j DROP
    else 
      echo "[INFO] IPtables OUTPUT rule already exists for set $_IPSET_NAME"
    fi
  fi
}

14. prepare_lists

Description: This function iterates through the .txt.gz files in $_SOURCE_DIR/blacklist/, extracts the gzip files to the destination folder, and removes the comments from them

function prepare_lists {
  for input_file in $_SOURCE_DIR/blacklist/*.txt.gz; 
  do
    clear_variables
    get_REGION $input_file
    get_ipv4_or_ipv6 $input_file
    _CURRENT_LIST_FILENAME=${_IPLIST_DIR}/${_REGION}-${_TYPE}.txt
    extract_file_to_iplist_dir $input_file $_CURRENT_LIST_FILENAME
    remove_comments_from_lists $_CURRENT_LIST_FILENAME
  done
}

14. create_sets

Description: In this function, the script cuts the files into smaller chunks if necessary, then populates the IPset sets and creates iptables rules to drop incoming and outgoing communication with the specific networks.

function create_sets {
  for input_file in $_IPLIST_DIR/*.txt; 
  do
    clear_variables
    get_REGION $input_file
    get_ipv4_or_ipv6 $input_file
    divide_file_by_lane_number $input_file
    set_ipset $input_file
  done
}

This is the time to call the functions and let them do the heavy lifting.

create_directory_if_not_exists $_IPLIST_DIR
prepare_lists
create_sets

The source code of this post can be downloaded from my github repository here:

https://github.com/MaXIP21/GeoipBlocking