2008-03-19

Accessing USB printer under OpenVZ VE

To access USB printer from within OpenVZ Virtual Environment (VE), you need to allow the device to be visible and accessible by doing the following from the Host Environment (HE):
vzctl set 999 --devices usb/lp0:rw

cat /proc/vz/devperms
#Version: 2.7
# 0 b 016 *:*
# 0 c 006 *:*
# 999 c 006 180:0


If there is any trouble with printer access from within VE, verify that there is none from HE .

In my case, CUPS within VE correctly detected my printer at ``usb://Samsung/ML-1740``. Yet it couldn't push any job to the printer. I spent hours trying to get it work to no avail. What I should have done was to verify if CUPS in HE could do it successfully. It couldn't. But CUPS-in-VE in another machine could push the job successfully. The problem, then, probably was with the hardware.

So, I settled with specifying the printer location (DeviceURI) at ``file:/dev/usb/lp0``. It works, except that it can't read printer status because it requires two-way communication which ``file`` protocol does not support.

2008-03-12

Shorewall and OpenVZ

A pretty basic Shorewall configuration can consume about 190 iptent (iptable entries) and the default limit is 128. So, if you are getting 'iptables: Cannot allocate memory' error message, check your /proc/user_beancounters because you might have hit the numiptent (number of iptable entries) limit.

OpenVZ VE's private and root directories must be on the same device

Files in a VE are stored under /var/lib/vz/private/${veid}. There is also /var/lib/vz/root/${veid} which is empty if the VZ is not alive and populated with what the VE sees when it is alive.

If you are moving the VE storage to another device, make sure to move both the private and root directories.

mount /mnt/newlocation
ln -s /mnt/newlocation/private/999 /var/lib/vz/private/999
ln -s /mnt/newlocation/root/999 /var/lib/vz/root/999


Otherwise you'll be getting this erronous error message:

> vzctl start 999
Starting VPS ...
vzquota : (error) Quota on syscall for 999: Device or resource busy
vzquota on failed [3]

2008-03-11

qmail in resource-constrained environment

An important thing to note when using virtualised environment
such as OpenVZ is that resources are artificially constrained. This can cause various funny and head-scratching behaviours.

My OpenVZ VE (plan VZ 128) from quantact.com, runs apache2 and qmail (as spooler).
I had to adjust tcpserver connection limit to 3 (tcpserver -c 3), and qmail-send's concurrencyremote to 1 (echo 1 > /var/qmail/control/concurrencyremote).

Each incoming SMTP request triggers an RBL lookup and may trigger a spamassassin check. The default tcpserver's incoming connection limit is 40 and there just isn't enough resources to support 40 spamassassin instances (1 spamd with 40 spamc processes). Instances will fail with various puzzling error messages. The only clear indication that they are failing because of resource constraint is by comparing the content of /proc/user_beancounters before and after failures.

Furthermore, if the remote host, where the real mail server is, is down for a period such that enough mails are queued up, when the remote host is up again, qmail-send will start 20 qmail-remote processes (the default) to send the mails all at the same time. Each process will send email to the real mail server through a stunnel connection. But there just aren't enough resources to support 20 qmail-remote processes along with 20 stunnel connections. Again, the only way to know this for certain is by comparing the content of /proc/user_beancounters before and after failures. Otherwise, all you are getting are error messages like below which could have been caused by various things like network or firewall problem:

2008-03-11T06:53:02.78164 2008.03.11 02:53:02 LOG3[11407:3058117552]: SSL_accept: Peer suddenly disconnected


Once the concurrency level is brought down, force qmail to resend the queue:

qmail-qstat; qmail-tcpok; pkill -ALRM qmail-send; qmail-qstat

OpenSSH: Hashed Known Hosts

Read more at https://itso.iu.edu/Hashing_the_OpenSSH_known__hosts_File.

To hash every user's known_hosts files and delete the known_hosts.old:

for username in `cut -f 1 -d : /etc/passwd`; do echo $username; sudo -u $username ssh-keygen -H; rm `sh -c "echo ~$username/.ssh/known_hosts.old"`; done

2008-03-08

Restoring Drive SnapShot from Linux

I use Drive Snapshot to archive our Windows machines. It is a Windows backup/disk imaging tool that does not require you to interrupt your other work in other applications. It is a good tool and has repaid its cost (US$ 57.56 in 2007 May) many times over.

Compared to other similar tools at the time, it was really the quickest and simplest one. However, its main downside was that it required DOS or Windows for restoring from scratch. DOS is troublesome to setup. There may be special drivers needed to be able to access the harddrive to be restored and also the backup files themselves. Restoring from Windows is slightly better, but only if you have another Windows machine and go through the hassle of hooking up the target drive to that machine.

Here is a method that I find easy enough for in-situ restoration. You need Live-CD Linux, QEMU and FreeDOS ISO image. I use Knoppix 5.11 since it includes QEMU.

  1. Boot Knoppix to GUI mode for QEMU UI.
  2. Make sure FreeDOS ISO image is accessible on a different partition/disk than the Snapshot image files (.SNA).
  3. The SNA and snapshot.exe filesmust be in a FAT32 volume
  4. qemu -cdrom freedos.iso -hda /dev/${fat32-partition-containing-sna-files} -hdb /dev/${target-harddrive} -boot d
  5. Follow the DOS restoration instruction. For example: snapshot.exe restore HD2 MBR image.sna which restores the MBR record from the image.sna (the CWD is assumed to be C:\ which corresponds to the fat32-partition-containing-sna-files device) to HD2 (the QEMU's hdb device which is the target-harddrive).

Hope this helps.


20080904 UPDATE: A step-by-step instruction
http://gnomicnotes.blogspot.com/2008/04/actual-restoration-from-drive-snapshot.html

2008-03-07

ntpd in openvz ve

ntpd needs to change the system time. In OpenVZ, VEs share the host's system time. The VE running ntpd needs to be given the CAP_SYS_TIME capability:
vzctl set 101 --capability sys_time:on --save



2008-03-01

Brake pedal needs to be adjusted after master cylinder replacement

After replacing the master cylinder in my sis-in-law car, I found that the braking performance didn't improve much. The pedal still went to the floor and the wheel could not be locked up.

I consulted with my brother who was an actual certified mechanic. He suggested that that the brake pedal free play needed to be adjusted because of depth differences of the piston cap in master cylinders. At first I was skeptical. True, the replacement part was a remanufactured one; but was it really that different than the old one?
I measured and found that the freeplay distance was out of spec: 12.7 mm instead of 5 mm. But what could a 7.7 mm difference do to the braking performance?

After adjusting the freeplay distance, I found that I couldn't floor the pedal anymore. Although the wheels still won't lock up, the brake is now good enough for everyday usage. I attribute the inability of the brake to lock up the wheel to older (weaker) brake hose and moderate glazing on the front rotors and pads.

Replacing brake master cylinder

My sis-in-law's car, a 1993 Nissan Sentra, had not been maintained by its previous owner. Since its purchase two years ago, I have been doing incremental maintenance on it. The last one I did was a brake fluid change.

Apparently, the brake fluid had not been changed in a long time. After changing it, the brake pedal became soft, mushy and the pedal could go to the floor. The car became dangerous to drive. Even with pedal on the floor, the wheel would not lock up. Repeated bleeding didn't help.

Then I stumbled on an explanation somewhere in the Internet (I forgot to note the URL). The old brake fluid could have been saturated with water. It caused the rubber seal within master cylinder to swell. Because brake fluid (DOT3&4) is hygroscopic, when the fluid was changed it pruned the seal, leaving it wrinkly like your fingers when you have been swimming for too long.
This causes the master cylinder to have an internal leak: instead of forcing the fluid along the brake lines to the slave cylinders, the seal allows the fluid to leak past them to the unpressurised region.

Symptoms of a master cylinder with an internal leak:
  • You can push the pedal to the floor with light continuous pressure even without the help of brake booster.
  • The pedal slowly sinks to the floor under its own weight (by itself).

Since diagnosing brake failure cannot be done with 100% certainty, if you have one of the above symptoms and you cannot attribute it to anything else (leaky slave cylinders, leaky brake lines, weak brake hoses, various other kinds of external leak), you need to replace the brake master cylinder.

Special Tools & Parts:

  • Replacement brake master cylinder. I got a rebuilt/remanufactured one for $60.
  • Flare-Nut wrench (see picture to see the difference).
  • Strong strings (optional. to help seat the reservoir in the new unit).
  • Aluminium foil / plastic food wrap / plastic bag / any barrier suitable for containing brake fluid.
  • Bench vise. Bench bleeding is much easier with one. Much much much easier.






Procedures:

  • Lift car on all wheels with wheels off.
  • You need some space around the brake master cylinder.



  • So, start by clearing up the area surrounding it.



  • Clean the area around master cylinder with brake part cleaner and wiping (physically). You want to get the area as clean as possible to prevent contaminant in your brake system. Use tooth brush (not the same one you used this morning) to clean tight spaces.

Do you know that brake part cleaner (Tetrachloroethylene) is a carcinogen? Do not let it drip to the floor. Do not let it free in the environment. I find an oil drain pan suitable to contain brake part cleaner liquid run-off. It's wide and short and can fit under various areas of the car. Don't use the same one as the one you use for engine oil. The chemical in the cleaner will render the oil unsuitable for recycling back as engine oil. It will force the recyclers to use it as fuel.

  • With the master cylinder is still fastened to the brake booster, loosen the brake line nuts using flare-nut wrench. The reason you need to use flare-nut wrench is because they are made of soft material and tighten pretty tight (13-20N.m or 10-15 ft/lb). Don't open them, don't let fluid drip. Just loosen them and tighten loosely back with finger.



  • Remove the master cylinder from the booster.




  • Now that you can manoeuvre the master cylinder slightly, put aluminium foil or fluid barrier under and to the sides of it because next you'll be removing the brake lines. Put some towels (that's the white thing on top of the aluminium foil in the picture below) too so the caught fluid is not sloshing around on the barrier. When the brake lines are separated, they won't drip fluid, but the master cylinder will. So, put it in a plastic bag as you carry it over the car.



  • Empty the master cylinder and reservoir of brake fluid.
  • Mount the master cylinder on the bench vise.
  • Remove the reservoir by moving it left and right while pulling it up. It's going be hard but it won't break as long as you are using your hand. The bench vise allows you to use both hands for pulling.
  • Pry off the filter screen in the reservoir
  • Clean the filter and the inside of the reservoir with brake part cleaner. Don't forget to do it above the drain pan to increase your karma. Afterwards, rinse with fresh brake fluid. Pour in some fluid into the reservoir, put on the cap and cover the ports with your fingers. Shake shake shake and let the fluid out. Repeat as necessary. Don't install the the filter yet, but clean it as well.

Here is a picture of the old and new master cylinders with the reservoir and metering valves removed.






  • Install the reservoir, metering valves and the bench bleed plugs (the green plug screwed to the metering valves and ports in the picture below). If the plug does not fit well, do not use it because it won't help in eliminating air during bench bleeding. You need to be prepared for getting messy during the bench bleeding, or do a long painful re-bleed session on all wheels (good thing you already have the car up, right?)

Putting back the reservoir was hard for me as I was not powerful enough to press it in. So, I used a long piece of string to help convert torque to pressure. Make sure that the string form a wide band so the pressure will not damage the nylon reservoir.






  • Install clear tubings from the plugs back to the reservoir. My green plugs takes 3/16" ID (internal diameter) tubings.
  • Fill reservoir with fluid.
  • Use a long screwdriver or a socket extension to push the piston repeatedly until you see no more air bubble. If there is a port without a plug (because it does not fit well), cover the port with your finger to prevent air being sucked in before releasing the pressure on the piston.
  • Installation is the reverse of removal. Do not forget to put the master cylinder in a plastic bag during transport as it will be dripping fluid. Take your time in screwing in the brake line nuts. Do not cross-thread the master cylinder (the nut is soft, but the master cylinder is softer). Do not get panicky because the fluid is dripping. That's why you put the barrier underneath, right?

If you did a good bench bleeding, you don't have to do anything else beside putting the car down. Otherwise, go bleed the brake. Yes, new master cylinder means you can push the pedal to the floor without damaging the seal.

Make sure to test drive it and check for external leak.

Keep checking for five days for any fluid level drop in the reservoir and for external leak. Some leaks are very small. I found a very small leak on the third day on one of the the brake line nuts that was not there previously.