sysadmin

Self-Hosting Email is Possible

I self-hosted for well over 20 years, I did not throw the towel and I do not plan to. Self-hosting is a sign of pride. Neither my government nor my Prime Minister nor even my Ministry of Interior or Foreign Ministry can host their own email.

Last time I checked, only State Security self-hosted.

I was probably lucky, but I rarely had delivery problems. The last one was a couple years ago with Microsoft swallowing my emails and it was due to the combination of a fairly old exim and a TLS certificate verification quirk at *.protection.outlook.com. I found a fix in the form of a configuration option somewhere on SO.

In all fairness, there is very little maintenance involved, and whenever I have to do maintenance work, I take the opportunity to learn something new. Like this year, I decided to finally replace my aging Debian jessie setup by Arch Linux, and I rewrote all cron jobs as systemd timers.

I must admit that when I send a really important email, I check the mail server log if it went off without errors, but this does not bother me as checking logs manually once in a while is a good thing anyway.

Lastly, a piece of advice: treat self-hosting like a hobby and learn to enjoy it.

Oh and the very last thing: the person who designed Exim configuration for Debian deserves a special place in hell for all the hours wasted. If you set up Exim on Debian, just figure out how to use the upstream exim config and adapt it to your needs.

A small firewall of China

There's been quite a few spam{bots,turks} lately passing through Drupal Captcha , reCAPTHA, Honeypot module… After a bit of research, I decided to block China from accessing my site. It turned out to be easy:

# install geoip filter into iptables
apt-get install xtables-addons-common
# download the existing geoip database
/usr/lib/xtables-addons/xt_geoip_dl
# set up csv parser in perl
apt-get install libtext-csv-xs-perl
# create a directory for geoip filter's database
mkdir /usr/share/xt_geoip
# build the database
/usr/lib/xtables-addons/xt_geoip_build  -D /usr/share/xt_geoip  *.csv
# load the geoip filter module
modprobe xt_geoip
# block China
iptables -A INPUT -m geoip --src-cc CN -j DROP

PROFIT!!!

P.S. Next in sight is US. US an CH account for 97% of Drupal spam.

Hosting Drupal on bare metal vs. cloud (Acquia)

We hosted Drupal websites at Hetzner for a few years. While it's unbeatable on price, it requires a skilled Linux sysadmin, which weights on personnel costs. Our guesstimate was that we'll pay a third more for a Drupal hosting, but our sysadmin costs would go down by ⅔. Add in non-material considerations, such as lower personnel turnover risks and better DDoS protection, and alternatives to Hetzner start to look almost attractive.

So, we decided to check Drupal cloud hosting solutions and asked for quotes from Pantheon and Acquia. Pantheon was less expensive and had more features. It also edged out Acquia on technology by using Linux containers instead of Amazon Web Services as underlying infrastructure. Unfortunately, Pantheon servers were located near Chicago, while most of our readers are from Europe, so we had no choice but to go for Acquia.

Notes on setting up Debian with ChromeOS kernel on a SD card plugged into Samsung ARM Chromebook

Two main sources of my inspiration were this post by Daniel P. Berrangé and this very detailed HOWTO on Debian User Forums. There's also a great, albeit a bit stripped down rootfs from Vassilis Laganakos for those who don't want to bother using debootstrap. Here's my script to run in root shell on ChromeOS to put the signed kernel onto the SD card:
echo "console=tty1 debug verbose root=/dev/mmcblk1p3 rootwait rw lsm.module_locking=0" > /tmp/config
dd if=/dev/mmcblk0p2 of=/tmp/oldblob
vbutil_kernel --repack /tmp/newkern --keyblock /usr/share/vboot/devkeys/kernel.keyblock --version 1 \
  --signprivate /usr/share/vboot/devkeys/kernel_data_key.vbprivk --config=/tmp/config --oldblob /tmp/oldblob
dd if=/tmp/newkern of=/dev/mmcblk1p1
dd if=/tmp/newkern of=/dev/mmcblk1p2
cgpt add -i 1 -S 1 -T 5 -P 10 -l KERN-A /dev/mmcblk1
cgpt add -i 2 -S 1 -T 5 -P 5 -l KERN-B /dev/mmcblk1
Note the use of --oldblob option -- newer ChromeOS does now mount /boot anymore, but HOWTOs out there still suggest to point to the kernel using --vmlinuz . Also, lsm.module_locking=0 is important. Without it, you won't be able to load kernel modules in Debian.

How to backup to an external disk once it is plugged in

How it should work:

  • insert a disk with a partition labelled as backup
  • wait until the beep
  • remove the disk

To label an ext3/ext4 disk, use

e2label device [newlabel]

Then, create an udev rule that runs your script when you insert a disk with a partition named backup:

echo 'KERNEL=="sd*", ENV{ID_FS_LABEL}=="backup", RUN+="/usr/local/bin/backup.sh"' 
   >> /etc/udev/rules.d/99-backup.rules

your backup script can be as simple as this:

#!/bin/sh
/bin/mount /dev/disk/by-label/backup /media/backup && \
  /usr/bin/rsync -r /home/* /media/backup && \
  /bin/umount /media/backup && \
  /usr/bin/beep

Watch out, $PATH is not set, you shoud use absolute paths everywhere.

To make it all work, reload udev rules with

udevadm control --reload-rules

How to block Tor exit nodes from accessing your website

Internet trolls are using Tor nowadays to avoid bans by IP. However, banning Tor exit nodes is just slightly more complex. The Tor Project provides a regularly updated list of exit nodes that can access your IP here. As there may be many hundreds or even thousands of nodes, adding them to iptables can hurt your server's network performance. Enter ipset, a user-space hash table for iptables:

# create a new set for individual IP addresses
ipset -N tor iphash
# get a list of Tor exit nodes that can access $YOUR_IP, skip the comments and read line by line
wget -q https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=$YOUR_IP -O -|sed '/^#/d' |while read IP
do 
  # add each IP address to the new set, silencing the warnings for IPs that have already been added
  ipset -q -A tor $IP
done
# filter our new set in iptables
iptables -A INPUT -m set --match-set tor src -j DROP

Automatically switch between internal and external screens when using a laptop and a dock station

Assuming that the internal screen is LVDS and the external screen is DVI-0. inotifywait comes from the inotify-tools package.

# Wait for the changes detected by the kernel laptop mode.
while inotifywait -e modify /proc/sys/vm/laptop_mode
do
laptop_mode=`cat /proc/sys/vm/laptop_mode`
  # If on battery, switch on the internal screen.
  if [[ $laptop_mode -gt 0 ]]; then
    xrandr --output LVDS --mode 1280x800
    xrandr --output DVI-0 --off
  # Otherwise,
  elif [[ $laptop_mode = 0 ]]; then
     # if the external screen is connected, switch it on.
     if [[ `xrandr -q|grep DVI-0|cut -f2 -d\ ` = connected ]]; then
       xrandr --output DVI-0 --mode 1920x1200
       xrandr --output LVDS --off
     fi
  fi
done

A beautiful filter for SSH brute-force attacks for your admiration

A bad sysadmin changes the default SSH port.

A good sysadmin throttles the brute-force attackers by allowing one new connection in 20 seconds.

# requires xt_TARPIT
iptables -A INPUT -p tcp -m state --state NEW \
  --dport 22 -m recent --update --seconds 20 -j TARPIT
iptables -A INPUT -p tcp -m state --state NEW \
  --dport 22 -m recent --set -j ACCEPT

A brilliant sysadmin blocks intruders indefinitely

# requires hashlimit
iptables -A INPUT -p tcp -m tcp --dport 22 -m state \
  --state NEW -m hashlimit --hashlimit 1/hour \
  --hashlimit-burst 2 --hashlimit-mode srcip \
  --hashlimit-name SSH --hashlimit-htable-expire 60000 
  -j ACCEPT

iptables -A INPUT -p tcp -m tcp --dport 22 \
  --tcp-flags SYN,RST,ACK SYN -j DROP 

iptables -A INPUT -p tcp -m state \
  --state NEW -m tcp --dport 22 -j ACCEPT

In the last example, the 1st rule allows up to two connections per hour. After the limit of two connections per hour is reached, the second rule becomes active and the hashlimit module starts to countdown from 1 minute (60 000 milliseconds). If you connect within 1 minute, the hashlimit counter is reset to 60 000. If you connect after 1 minute, you drop to the 3rd rule and are allowed access.

Cherry-picked from Habr

Простейший сниффер TCP и UDP трафика в три строчки

mknod backpipe p 
mknod log p
while :; do cat backpipe |nc -l -p 3001 |tee log| nc localhost 3000 |tee log >backpipe; done
Зависимости: (ba,k)sh, netcat, mknod, tee. Работает следующим образом. Скажем, нам нужно просмотреть трафик на порту 3000. Устанавливаем netcat сервер на 3001 порту, форвардим соединение в оба конца на порт 3000. Копию всего, что передаётся в обоих направлениях, пишем в файл log. Если вдруг соединение закрывается, запускаем всё заново.

Pages