As mentioned in my previous blog Raspberry Pi with Squid Proxy Server for Testing Hand-Held Devices when Developing in a Sandbox it is possible (in fact quite usual) to use Squid to prevent user access to certain websites. This can include ad servers which serve up annoying ad content when trying to read articles. This tutorial shows how we can extend the functionality of the Squid server we have already set up to block ads.
The banning of ad content is achieved by downloading a blacklist of prohibited urls from a service provider, although it has to be mentioned that these lists depend upon the community to contribute any urls they discover and thus there can never be 100% confidence that such a list is ever complete.
Once the list is downloaded from the service provider, Squid needs to be informed of its presence in its configuration and the ban will take the form of an acl. Ok now we know what we need to achieve let's start the process.
Nigels-MacBook-Pro:Projects nigel$ ssh pi@192.168.0.201 pi@192.168.0.201s password: The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. Last login: Sun Sep 10 14:33:42 2017 from 192.168.0.7 pi@raspberrypi:~ $ sudo su root@raspberrypi:/home/pi# cd /etc/squid3/ root@raspberrypi:/etc/squid3# ls -las total 832 4 drwxr-xr-x 2 root root 4096 Sep 9 08:44 . 4 drwxr-xr-x 112 root root 4096 Sep 9 11:17 .. 4 -rw-r--r-- 1 root root 1547 Dec 24 2016 errorpage.css 4 -rw-r--r-- 1 root root 421 Dec 24 2016 msntauth.conf 272 -rw-r--r-- 1 root root 277585 Sep 9 08:44 squid.conf 272 -rw-r--r-- 1 root root 277565 Sep 3 11:06 squid.conf.local-vm 272 -rw-r--r-- 1 root root 277526 Sep 2 19:22 squid.conf.original root@raspberrypi:/etc/squid3# curl -sS -L --compressed "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=0&mimetype=plaintext" > ad_block.txt root@raspberrypi:/etc/squid3# ls -las total 872 4 drwxr-xr-x 2 root root 4096 Sep 17 15:20 . 4 drwxr-xr-x 112 root root 4096 Sep 9 11:17 .. 40 -rw-r--r-- 1 root root 40042 Sep 17 15:20 ad_block.txt 4 -rw-r--r-- 1 root root 1547 Dec 24 2016 errorpage.css 4 -rw-r--r-- 1 root root 421 Dec 24 2016 msntauth.conf 272 -rw-r--r-- 1 root root 277585 Sep 9 08:44 squid.conf 272 -rw-r--r-- 1 root root 277565 Sep 3 11:06 squid.conf.local-vm 272 -rw-r--r-- 1 root root 277526 Sep 2 19:22 squid.conf.original root@raspberrypi:/etc/squid3#
## disable ads ( http://pgl.yoyo.org/adservers/ ) acl ads dstdom_regex "/etc/squid3/ad_block.txt" http_access deny ads
root@raspberrypi:/etc/squid3# vi ad_block.txt root@raspberrypi:/etc/squid3# service squid3 restart
root@raspberrypi:/etc/squid3# tail -f /var/log/squid3/access.log #
root@raspberrypi:/etc/squid3# cat /var/log/squid3/access.log | grep TCP_DENIED | wc 44 440 5192 root@raspberrypi:/etc/squid3#
NME was totally successful - empty spaces in the screen where the blocked ads would've been seen. In this instance two ads have been blocked.
The native Android app Whisper was partially successful. The main ad shows a diagnostic that the ad failed to launch - I'm happy with that - but it failed to block the small ad at the bottom of the screen.
/etc/squid3/newads.sh
#!/bin/bash curl -sS -L --compressed "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=0&mimetype=plaintext" > /etc/squid3/ad_block.txt ## restart squid /usr/sbin/squid3 restart
crontab -e
# Squid3 fetch ad blocker blacklist 5 4 * * 1 /etc/squid3/newads.sh > /dev/null 2>&1