Test, test, test

It turns out that the best way to prepare for a test is by doing practice tests.
At some level, I have known this subconsciously. Want to have a successful Math exam? Do practice questions until you're bored to death. Having a 5-chapter History exam tomorrow? Do practice questions until dawn.

I think this also applies to things in general: expertise comes from repeated doings. In other words, expertise comes from experience.

Disabling LKM in Debian GNU/Linux

This instruction is applicable to Etch (4.x).
You may want to do this on your public servers to help keep off LKM trojan.

apt-get install lcap
lcap -c CAP_SYS_MODULE

There are more tips at Taking Advantage of Linux Capabilities.

What is in a machine name?

As I started to play with OpenVZ, it dawned on me that the cost of
machines are cheap. It is very cheap to have a specific (virtual)
machine for a specific purpose.

Already, I had several specific functionalities I wanted to have
implemented: intranet-wide (read: home) file server, intranet-wide
access to scanner, public-facing web server, public-facing mail
server, fresh OS base to test installation procedures on, etc.

Now they can each be implemented in its own machine. There will be
enough machines for me to justify entertaining some
system-administration practices, like impersonal machine naming.

I had always been naming my machines by the name of people I liked,
had crushes with, or dated. I cannot continue with that schema without
jeopardizing my marriage and lying to myself as I cannot start liking
someone just because I need to name a new machine.

A machine naming convention that I've always liked is one that encodes
the purpose and asset ID into the name. For example: WWW001 is a web
server machine, MAIL001 is a mail server machine, and WS001 is a
workstation machine. Some even go further by including physical
location information: DC06MAIL001 could be a mail server in data
center #6 in Washington DC.

My need for my simple system is simple. I choose the simple prefix "M"
which stands for machine. The names will be M01, M02, M03, and so on.
Reaching M99 is a strong indication that the system is no longer
simple and is a justification to use a more sophisticated naming
convention.

The machine name forms the basis for the canonical DNS name. Machines
with a single IP have similarly named DNS entries under the 'machine'
subdomain, e.g.: m01.machine.example.com.

Multi-Homed machines (having multiple IPs) also have similarly-named
DNS entries pointing to each of its interface. The entry names should
be descriptive enough to identify the interface, e.g.:
m01-internet.machine.example.com and m01-lan.machine.example.com if
m01 is an Internet-facing firewall machine.

Each interface may be assigned additional alias DNS name (CNAME
records). For example, if m01.machine.example.com is originally a
machine in LAN and is being upgraded to serve as an Internet-facing
firewall machine, then m01.machine.example.com then cease becoming
the canonical name. It may be dropped or added as an alias.

Publicly-Accessible servers also benefit from having an alias DNS
name, e.g.: www.example.com may be an alias for
m54.machine.example.com.

Publish Post

LKM in my server. Rootkitted.

 From: root@example.com (Cron Daemon)
 Subject: Cron test -x /usr/sbin/anacron || ( cd / && run-parts
 --report /etc/cron.daily )
 To: root@example.com
 Date: Sun, 27 Jan 2008 09:25:10 -0500

 You have 1 process hidden for readdir command
 You have 1 process hidden for ps command
 chkproc: Warning: Possible LKM Trojan installed

When I received that email from one of my public servers, two
questions popped up: did I just get rootkited again?

I had been rootkitted about three years ago. At that time the only
network-accessible services on the computer was a stock apache 1.3
serving static pages and an unpatched qmail. I didn't have tripwire
setup and ended up reinstalling the system from scratch because I
didn't know which files had been compromised.

This time, I had tripwire (AIDE, actually) setup and was able to
identify the infected files. Yet I still ended up reinstalling the
system from scratch partly because things were too easy for me and
mostly because I just found the excuse I needed.

Firstly, the compromiser didn't take out the regular chkrootkit
report. Secondly, subsequent chkrootkit report didn't report any
warning. Did the chkrootkit itself was compromised? Later verification
with an uncompromised AIDE data file showed that it was not, but
still, you wouldn't know that then. Thirdly, the same question for the
AIDE binary itself. Fourthly, it was really painful trying to do an
off-line verification on a remote system. Fifthly, was it apache 1.3
again?

One thing I discovered was that Linux allows you to disable loadable
kernel module (LKM) which supposedly makes it impervious to LKM
Trojan, like the one I had. No use crying over spilt milk.

The server was a Xen virtual machine hosted by quantact.com. It was
setup two years ago. This break-in gave me an excuse to reorganize my
hosting choices. Around last year, they started offering a cheaper
hosting option on OpenVZ. So I tried it and found that I liked it. I
liked it so much I installed one at home.

OpenVZ does not allow virtual machines to load kernel modules. I hope
that is enough to circumvent the same attack vector.

New site

My web server got rootkitted (see next post) and I got my excuse to
reorganise.

I had been thinking of using a public blog site, like blogger.com, for
my infrequent writings but had hesitated because of the perceived lack
of a decent editor to compose the entries with. I did not know about
the publish-via-email capability only until recently.

This capability allows me to keep using my favourite editor to create
a new email, do the compositions and send it to blogger.com for
publishing.

The ability to compose entries using the web interface is not
something to discount either. In my travelling, I have found hotels
with an over-eager filtering mechanism that restricted access to my
own web site because of its 'pornographic nature'. Yes, it is
pornographic only if you are turned on by boring technical writings.