DebianAMD64 and Samsung ML1740 (or othere printers)

Today, M. Aurélien Croc, the author of Splix, informed me that a new version of Splix, 0.0.2, is now usable in 64-bit.

Of course I am happy because now I am no longer limited to submitting 4294967296 documents to the spooler; now I can submit 18446744073709551616 documents at a time. I mean why else would you even bother running a spooler in a 64-bit environment? (I kid).

Packages needed:

libcupsys2-dev
libcupsimage2-dev



/tmp/splix-0.0.2 $ make
make[1]: Entering directory `/tmp/splix-0.0.2/src'
g++ -O2 `cups-config --cflags` -I../include -c -o rastertospl2.o rastertospl2.cpp
g++ -O2 `cups-config --cflags` -I../include -c -o raster.o raster.cpp
g++ `cups-config --ldflags` -lcups -lcupsimage -o rastertospl2 spl2.o printer.o band.o compress.o rastertospl2.o raster.o
make[1]: Leaving directory `/tmp/splix-0.0.2/src'
make[1]: Entering directory `/tmp/splix-0.0.2/ppd'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/tmp/splix-0.0.2/ppd'


/tmp/splix-0.0.2 $ sudo make install
make[1]: Entering directory `/tmp/splix-0.0.2/src'
install -m 755 -s rastertospl2 `cups-config --serverbin`/filter
make[1]: Leaving directory `/tmp/splix-0.0.2/src'
make[1]: Entering directory `/tmp/splix-0.0.2/ppd'
install -d -m 755 `cups-config --datadir`/model/samsung
for filename in ml1510 ml1520 ml1610 ml1710 ml1740 ml1750 ml2010 ml2150 ml2250 ml2550; do \
install -m 644 $filename.ppd `cups-config --datadir`/model/samsung;\
for lang in fr it de; do \
install -m 644 $filename$lang.ppd `cups-config --datadir`/model/samsung;\
done; \
done \

make[1]: Leaving directory `/tmp/splix-0.0.2/ppd'

--- Everything is done! Have fun ---





The files installed are:


/usr/lib/cups/filter/rastertospl2
/usr/share/cups/model/samsung/*.ppd (among them is ml1740.ppd)


We can add the printer from cups's web interface (typically found at http://localhost:631). When prompted for printer model, just provide the ppd (in my case it is /usr/share/cups/model/samsung/ml1740.ppd).


(originally from http://microjet.ath.cx/WebWiki/2006.11.17_DebianAMD64_and_SamsungML1740_Part2.html)

Continuation support in ruby

I heard today that Ruby 2.0 is going to abandon support for continuation, at least initially until things have stabilised a bit. It is no longer a news, it is just that I have not been active in Ruby community for a while.

The Problem with Continuation Support in Ruby 1.x

My initial reaction upon hearing that was "What a pity". On second thought, I realised that dropping continuation initially makes sense. The support for continuation in Ruby 1.x is missing an ensure procedure that is continuation-friendly; meaning one that is called only once one a given path is no longer accessible.

$create_cont_k=nil

def do_something(k)
  puts "Do something called"
  $create_cont_k = k
end

def do_something_else
  return unless $create_cont_k
  puts "Do something else called"
  k = $create_cont_k
  $create_cont_k = nil
  k.call
end


def create_cont
  callcc{|create_cont_k| do_something(create_cont_k)}
ensure
  puts "Ensure called"
end

create_cont
do_something_else

# /tmp $ ruby1.8  /tmp/ensure-run-multiply.rb 
# Do something called
# Ensure called
# Do something else called
# Ensure called

A proper ensure mechanism that supports explicit continuation formation system, like Ruby 1.8, would have produced:

# /tmp $ ruby1.8  /tmp/ensure-run-multiply.rb 
# Do something called
# Do something else called
# Ensure called

Notice how "ensure" is supposed to be called only once. I have a sample implementation of such continuation-observant ensure mechanism in system-managed-unwind-protect-in-sisc for SISC, a scheme implementation.

"But, but, what if I really want something like ensure that is called each time a code section is called?", you asked. In other words, you want something like Ruby 1.8's multiple-shot ensure in this new world of single-shot ensure.

That is not a problem and don't think that you are being unreasonable for asking this feature. It is useful to be able to guarantee execution of some code upon exiting a dynamic environment.

A dynamic environment is the environment that your program can access at any given time. The environment can change with each instruction executed by the machine.

But such one-sided guarantee does not really do anything interesting. How about a guarantee that some code is also executed upon entering a dynamic environment, thus having a symmetric aspect.

Scheme has dynamic-wind that does the above:

(define *CREATE-CONT-K* #f)

(define (do-something k)
  (display "Do something called\n")
  (set! *CREATE-CONT-K* k))

(define (do-something-else)
  (when *CREATE-CONT-K*
    (display "Do something else called\n")
    (let ((k *CREATE-CONT-K*))
      (set! *CREATE-CONT-K* #f)
      (k))))

(define (create-cont)
  (dynamic-wind
      (lambda () (display "Entering...\n"))
      (lambda ()
        (call/cc
         (lambda (create-cont-k)
           (do-something create-cont-k))))
      (lambda () (display "Exiting...\n"))))

(create-cont)
(do-something-else)

;; /tmp $ sisc -x /tmp/dynwind.scm 
;; Entering...
;; Do something called
;; Exiting...
;; Do something else called
;; Entering...
;; Exiting...

The utility in having a in/excursion guard can't be understated. In SRFI-34, dynamic-wind is used to install a custom exception handler. Yes, a custom exception handler; this is probably a strange concept for some people. Why would one have different exception handlers on the same piece of code? The first reason is "why not?". A dynamic language restricted to having a static exception handler feels incomplete. The second reason is more practical. Since you can in/ex-curse from/to any context (via calling a provided continuation object), how an exception is handled in a given context is not necessarily the same as in other different contexts.

Thus, a complete example of how multiple-shot and single-shot ensure example co-existing (sorry, in scheme as there is no Ruby implementation with single-shot ensure yet):

(class-path-extension-append! (list "file:/home/ysantoso/share/project/scheme/unwind-protect/"))

(require-library 'unwind-protect)
(import unwind-protect)

(import s2j)
(define (do-gc) ((generic-java-method '|gc|) (java-null (java-class '|java.lang.System|))))

;; do-something and do-something-else functions are elided for brevity

(define (create-cont)
  (dynamic-wind
      (lambda () (display "Entering...\n"))
      (lambda ()
        (unwind-protect
          (call/cc
            (lambda (create-cont-k)
              (do-something create-cont-k)))
         (display "Ensure\n")))
      (lambda () (display "Exiting...\n"))))


(create-cont)
(do-gc)
(do-something-else)
(do-gc)

;; /tmp $ sisc -x /tmp/dynwind.scm 
;; Entering...
;; Do something called
;; Exiting...
;; Do something else called
;; Entering...
;; Exiting...
;; Ensure

Notice how "ensure" is called just once even though there are two incursion into the protected code block.

The last deficiency in Ruby 1.x is the restriction that a continuation has to be resumed from the same thread that reifies it.

def do_something_else
  return unless $create_cont_k
  puts "Do something else called"
  k = $create_cont_k
  $create_cont_k = nil
  t=Thread.new { k.call}
  t.join
end

# /tmp $ ruby1.8  ensure-run-multiply.rb 
# Do something called
# Ensure called
# Do something else called
# ensure-run-multiply.rb:13:in `call': continuation called across threads (RuntimeError)
#       from ensure-run-multiply.rb:14:in `join'
#       from ensure-run-multiply.rb:14:in `do_something_else'
#       from ensure-run-multiply.rb:25

(originally from http://microjet.ath.cx/WebWiki/2006.11.05_Continuation_Support_in_Ruby.html)

System-managed UNWIND-PROTECT in SISC

Authentication

Login with:New JS-Kit AccountMy Google ProfileMy Twitter AccountMy FriendFeed AccountMy Yahoo! AccountMy Blogger AccountMy JS-Kit AccountMy Haloscan AccountMy OpenID

FORM: (unwind-protect BODYFORM UNWINDFORMS...)
Scheme's call/cc provides a flexible way to control execution flow. Its flexibility allows for different approaches to implementing unwind-protect.

Dorai Sitaram's "Unwind-Protect in Portable Scheme" shows a portable implementation of unwind-protect. It does so by constricting the flexibility of call/cc. User would have to specify when to call the unwindforms part of unwind-protect. In certain situations, it is desirable to have this level of control. In others, it would be akin to having to manually manage memory allocation.

The other approach is having an automatically executed unwindforms such as the one described by Taylor R. Campbell, "2006-05-01 On control brackets and resource release". Having the system manages the execution of unwindforms is more in line to traditional implementations of unwind-protect in other languages. It is this approach that interests me.

By associating a protector cell to the bodyform, we can rely on the GC mechanism cleaning up the protector cell once =bodyform='s continuation is no longer reachable. If it is no longer reachable, then it is safe to execute the unwindforms.

He also describes an optimisation of a common case where =bodyform='s continuation is never reified. Without it, there is only one =bodyform='s continuation and once the execution passes through the end of bodyform, it is safe to continue to unwindforms.

In his 'blog', he provides a sample implementation for MIT-Scheme. The implementation would have to be platform-specific since it hooks into the GC. However, it is easy to port that to other platform.

 

An Implementation in SISC

As a user of SISC, and as someone who has seen many applications (including SISCWeb) using dynamic-wind as a poor-man's unwind-protect, having a system-managed unwind-protect in SISC is a good idea. As I was not aware of any existing implementation, I proceeded to create one using Taylor Campbell's example as the starting point.

To hook into Java's GC finalisation process, I used GCHook.java. Then unwind-protect.scm implements the rest.

unwind-protect-in-sisc


(originally from http://microjet.ath.cx/WebWiki/2006.10.24_UnwindProtectInSISC.html and http://microjet.ath.cx/WebWiki/2006.10.27_UnwindProtectInSISCPart2.html)

Debian Sarge (3.1), LVM2, and Linux >= 2.6.16

I upgraded Linux kernel from 2.6.15 to 2.6.17 and found out that the weekly backup process appeared to hung the whole machine. I narrowed the problem to when the snapshot volume was being removed:

lvremove /path/to/snapshot/volume

I found this email that describes a workaround, but the more proper workaround is to upgrade sarge's lvm2 package made available by Juergen Kreileder.

(originally from http://microjet.ath.cx/WebWiki/2006.09.04_DebianSarge_LVM2_Linux2.6.16.html)

perl-like lock for ruby

Authentication

Login with:New JS-Kit AccountMy Google ProfileMy Twitter AccountMy FriendFeed AccountMy Yahoo! AccountMy Blogger AccountMy JS-Kit AccountMy Haloscan AccountMy OpenID

cdfh was asking in #ruby-lang about a locking library that allows one to do:

lock(an_obj) { ..... }

similar to the lock library in perl.

"That won't not that hard to build", I thought. I did the first version in about 5 minutes. Then thought that it would be nice to make it robust to object_id wrap-around and also to discard unnecessary records associate to an_obj when an_obj has been garbage collected.

require 'thread'
require 'weakref'


class Lock
  Record = Struct.new("Record", :weakref, :mutex)

  def initialize(auto_cleanup_per_record = 1000)
    @mutex = Mutex.new
    @obj_records = {}
    @create_count = 0
    @auto_cleanup_per_record = auto_cleanup_per_record
  end

  def lock(what, &block)
    record = acquire_obj_record(what)
    record.mutex.synchronize {
      if block_given?
        block.call
      end
    }
  end
  

  private

  def cleanup_proc
    lambda {|id|
      @mutex.synchronize {
        to_delete = []
        @obj_records.each_pair{|k,v|
          if not v.weakref.weakref_alive?
            to_delete << k
          end
        }
        to_delete.each{|id|
          @obj_records.delete(id)
        }
        #puts "Cleanup id: #{to_delete.join(",")}. Size: #{@obj_records.size}: #{@obj_records.keys.join(",")}"
      }
    }
  end

  def acquire_obj_record(what)
    @mutex.synchronize {
      if record = @obj_records[what.object_id] and record.weakref.weakref_alive? 
        record
      else
        ObjectSpace.define_finalizer(what, cleanup_proc)
        @obj_records[what.object_id] = Record.new(WeakRef.new(what), Mutex.new)
      end
    }
  end

  public
  
  @instance = Lock.new
  def self.lock(what, &block)
    @instance.lock(what, &block)
  end
  def self.cleanup(verbose=false)
    @instance.cleanup(verbose)
  end
end



if $0 == __FILE__
  module Test
    def self.test1(finalizer_block)
      puts "Test1"
      s="str"*1024*1024
      ObjectSpace.define_finalizer(s, finalizer_block)
      puts s.object_id
      
      t1 = Thread.new {
        Lock.lock(s) {
          4.times {
            puts "Thread1"
            sleep(0.25)
          }
        }
      }
      
      t2 = Thread.new {
        puts "Thread2 started"
        Lock.lock(s) {
          4.times { 

            puts "Thread2"
            sleep(0.25)
          }
        }
      }
      
      t1.join
      t2.join
    end

    def self.test2(finalizer_block)
      puts "Test2"
      s="noo"*1024*1024
      puts s.object_id
      ObjectSpace.define_finalizer(s, finalizer_block)
      Lock.lock(s)
    end

    def self.test3(finalizer_block)
      puts "Test3"
      10.times {
        s="noo"*1024*1024
        puts s.object_id
        ObjectSpace.define_finalizer(s, finalizer_block)
        Lock.lock(s)
      }
    end
  end



  Test.test1(lambda {|id| puts "Finalized: #{id}"})
  GC.start
  puts "GC invoked"

  Test.test2(lambda {|id| puts "Finalized: #{id}"})
  GC.start
  puts "GC invoked"

  Test.test3(lambda {|id| puts "Finalized: #{id}"})
  GC.start
  puts "GC invoked"
end


=begin
/mnt/vg0.home/ysantoso/tmp/ruby-postgres/tests $ ruby /tmp/lock.rb 
Test1
-604990708
Thread1
Thread2 started
Thread1
Thread1
Thread1
Thread2
Thread2
Thread2
Thread2
GC invoked
Test2
-610787788
GC invoked
Test3
-610788148
-610953348
-615730956
-615809476
-615887996
Finalized: -615730956
Finalized: -610953348
-615902566
-615981086
-615697446
Finalized: -615809476
Finalized: -615887996
Finalized: -615902566
-614121002
-614199522
Finalized: -614121002
Cleanup id: -615809476,-615730956,-610953348,-615902566,-615887996. Size: 7: -614199522,-615697446,-615981086,-610787788,-614121002,-610788148,-604990708
Finalized: -614199522
Cleanup id: -614121002. Size: 6: -614199522,-615697446,-615981086,-610787788,-610788148,-604990708
Finalized: -615697446
Cleanup id: -614199522. Size: 5: -615697446,-615981086,-610787788,-610788148,-604990708
Finalized: -615981086
Cleanup id: -615697446. Size: 4: -615981086,-610787788,-610788148,-604990708
Finalized: -610787788
Cleanup id: -615981086. Size: 3: -610787788,-610788148,-604990708
Finalized: -610788148
Cleanup id: -610787788. Size: 2: -610788148,-604990708
GC invoked
Finalized: -604990708

=end 

(originally from http://microjet.ath.cx/WebWiki/2006.08.07_LockLibraryForRuby.html)

Producing divx-device compatible files with xvid

Found this little gem:
http://forum.doom9.org/showthread.php?p=855604

Safe params:

• Home theatre profile (max bitrate 4854 instead of 8000 for AS@L5 )
• No custom matrix (use MPEG or H263) 
• NO QP, AQ, Trellis, GMC NO PACKED bitstream

Params that helps:

• MAX consecutive B-VOPS set to 1.00 
• CBR audio 
• No openDML 
• Recommended max resolution 640*xx

http://forums.afterdawn.com/thread_view.cfm/165637


(originally from http://microjet.ath.cx/WebWiki/2006.07.30_ConstrainingXVIDToPlayInDIVXDevices.html)

Christmas in May, Muvo FM TX, PATA to USB adapter

I went nuts. I decided to have an early Christmas.
I have been wanting a portable MP3 player for a long time to replace my broken Rio (first-gen MP3 player). Nothing caught my eyes in terms of price-feature ratio until I happened upon an ad for Creative MuVo FM TX 1GB for $57.50. That's a great buy as other stores were still selling it for about $100+. I bought it.
I also bought a USB-PATA adapter. It is this small adapter you attached to the back of an IDE drive. On the other side of the plug is a wire with a USB connector.
I wanted to move the DVDRW drive out from secondary IDE which was shared with another hard drive. The hard drive contained my /usr volume. Burning a DVD would cause frequent jerkiness in interactive session. The cheap solution to that is to either buy a PATA to SATA adapter or PATA to USB adapter.
I tried the PATA to SATA adapter first primarily because my motherboard had two SATA ports and they had never been used. I bought the adapter from EBay. It arrived a month later from China. Opening the package, I found that adapter wrapped in a non-anti-static plastic bag. Uh oh.
I detached all my disks and use an old 3GB hard drive for experimentation with this PATA-SATA adapter. Based on my reading the hard drive must be set to master. When I turned on the computer, the light on the adapter shone green. Since this was going to be my first SATA device, I tried booting with Knoppix 4.2 which boasted a large number of drivers. It was not recognised. The SATA controller, SiS964, was recognised, but not the attached device. Not being familiar with SATA handling in Linux, and not having a real SATA device to verify against, I chose not to pursue this path any further, at least until I have a real SATA device.
So, I bought a non-brand-name USB-PATA adapter. Supposedly it presents the hard disk as a USB Mass Storage device, which Linux has been supporting since last century.
There was no problem. I burned a DVD without having The adapter worked and it could pump data from and to the DVDRW at 20MBps without me noticing any jerkiness in interactive session.
I'm enjoying this early Christmas.

(originally from http://microjet.ath.cx/WebWiki/2006.05.20_ChristmasInMay.html)

Linksys WRT54GX2

After I returned WRE54G, I bought WRT54GX2.
I don't know if it really increases my range, but I noticed that all the deadspots in the house that I know of have disappeared. In the past, interactive session was not very pleasant as the connection frequently became laggy. It is not occurring that frequently anymore; there are still some occurrences, typically in the morning (06:00 or so).
The bad thing about WRT54GX2 was the wired ethernet port could not reliably auto-negotiate 100baseT protocol. Googling for it lead me to http://www.dslreports.com/forum/remark,14714795. The firmware on my device was 1.01.06. So, I downloaded the new firmware (1.01.14 as of now) and tried to upgrade the device.
But I was unable to update the firmware. None of the suggestions in that URL did the trick. Experimenting for one hour with various settings didn't yield anything either.
Just when I was close to giving up, I had this crazy notion of trying to do the upgrade using Internet Explorer. Behold! It worked.
I find it hard to believe that a device manufactured in 2005 by a company using GPL code would still require a particular browser. But believe I had to.
Anyway, don't bother to backup your configuration by Administration -> Config Management -> Backup. It's worthless as the configuration file is version-specific. Pen and paper would work better.

(originally from http://microjet.ath.cx/WebWiki/2006.05.03_Linksys_WRT54GX2.html)

Ruby's hash implementation

Authentication

Login with:New JS-Kit AccountMy Google ProfileMy Twitter AccountMy FriendFeed AccountMy Yahoo! AccountMy Blogger AccountMy JS-Kit AccountMy Haloscan AccountMy OpenID

Ruby's Hash class is implemented using the same engine that it uses for symbol table.

It is not meant for high-volume usage.

/tmp $ ruby testspeed.rb 
Rehearsal ----------------------------------------------------------------------------------------------
reading with while, name=1million                            1.240000   0.030000   1.270000 (  1.272282)
reading with readline, name=1million                         0.980000   0.210000   1.190000 (  1.215839)
reading with while and inserting into hash, name=1million    5.330000   0.200000   5.530000 (  5.715192)
reading with while and inserting into array, name=1million   5.640000   0.210000   5.850000 (  5.975710)
------------------------------------------------------------------------------------ total: 13.840000sec

                                                                 user     system      total        real
reading with while, name=1million                            1.750000   0.020000   1.770000 (  1.785138)
reading with readline, name=1million                         1.440000   0.010000   1.450000 (  1.454656)
reading with while and inserting into hash, name=1million    4.050000   0.020000   4.070000 (  4.102691)
reading with while and inserting into array, name=1million   2.290000   0.020000   2.310000 (  2.320377)

def create_file(name, size)
  File.open("/tmp/largefile_#{name}", "w") {|f| size.times {|i|f.puts "foo#{i}"; } }
end

# do these once
# create_file("1million", 1*1000*1000)
# create_file("5million", 5*1000*1000)

def read_with_while(name)
  File.open("/tmp/largefile_#{name}") {|fh|
    while line = fh.gets
      line.chomp!
    end
  }
end

def read_with_readlines(name)
  File.readlines("/tmp/largefile_#{name}")
end

def read_into_hash(name)
  hash={}; array=[]; File.open("/tmp/largefile_#{name}"){ |fh| while line = fh.gets; line.chomp!; 
                                                                 hash[line] = 1; 
                                                               end} 
end
def read_into_array(name)
  array=[]; File.open("/tmp/largefile_#{name}"){ |fh| while line = fh.gets; line.chomp!; 
                                                                 array << line
                                                               end} 
end

require 'benchmark'
Benchmark.bmbm {|r|
  ["1million"].each{|name|
    GC.start
    r.report("reading with while, name=#{name}") {read_with_while(name)}
    GC.start
    r.report("reading with readline, name=#{name}") {read_with_readlines(name)}
    GC.start
    r.report("reading with while and inserting into hash, name=#{name}") { read_into_hash(name)}
    GC.start
    r.report("reading with while and inserting into array, name=#{name}") { read_into_array(name)}
  }
} 

(originally from http://microjet.ath.cx/WebWiki/2006.04.15_Ruby%27sHash.html)

Linksys WRE54G version 2.0 (repeater/extender)

Authentication

Login with:New JS-Kit AccountMy Google ProfileMy Twitter AccountMy FriendFeed AccountMy Yahoo! AccountMy Blogger AccountMy JS-Kit AccountMy Haloscan AccountMy OpenID

I finally did something to address weak signals from my wireless router, a 2-year old Netgear device. I went to CompUSA thinking of buying Linksys WRT54G router, but instead ended up buying the Linksys WRE54G version 2.0.

Toms Hardware reviewed this device sometimes ago. They were not really impressed by the product and the setup was painful. They were reviewing version 1.0.

True enough, things have changed in version 2.0. The device now includes a RJ-45 port for wired setup. This makes the setup easier somehow.

Setup was not difficult, except for the following:

1. The SSID must be the same as the router's SSID. This is an annoyance since that means I can't choose which device to connect to for comparison purposes.
2. The router has been setup to use 64bit WEP. But the expander's WEP implementation seems to not be compatible as it couldn't connect to the router. Fortunately, its WPA Personal implementation is compatible with the router's.

I put the expander in the middle of the signal edge and the router in contrary to putting it at the edge as written in the manual. Putting it at the edge does not really make much sense since at the expander would have difficulty communicating with the router. I bought the device to expand range but I do not want to sacrifice signal quality.

With the expander activated, I now can go beyond the previous range. However, interactive session sucked real bad. It seems the device is buffering data, which is bad for interactive session.
I returned it after several days because I work with interactive session most of the time.

(originally from http://microjet.ath.cx/WebWiki/2006.03.25_Linksys_WRE54G_v2.html)

Hexifier and XML entities escaper

Happy belated Valentine's day.



I have a present for Emacser out there: a hexifier and XML Entities
escapers.

ys-hex306.el

(require 'hexl)

(defun ys/escape (string to-escape-char &optional escape-char)
  ;; Escapes all instances of to-escape-char in the given string by prefixing each instance with escape-char (default to backslash ``\``).
  ;;
  ;; (let ((str "/ \\/ \\\\/ \\n"))
  ;;   (insert (concat "\n" str "\n" (ys/escape str ?/))))
  ;; / \/ \\/ \n
  ;; \/ \\\/ \\\\\/ \\n



"  Escapes all instances of to-escape-char in the given string by prefixing each instance with escape-char (default to backslash ``\\``).
  
  (let ((str \"/ \\\\/ \\\\\\\\/ \\\\n\"))
    (insert (concat \"\\n\" str \"\\n\" (ys/escape str ?/))))
  / \\/ \\\\/ \\n
  \\/ \\\\\\/ \\\\\\\\\\/ \\\\n
"
  (if (eq escape-char nil)
      (setq escape-char ?\\))
  (with-temp-buffer
    (insert string)
    (goto-char (point-min))
    (save-match-data
      (let ((regexp (concat (regexp-quote (char-to-string to-escape-char))
                                   "\\|"
                                   (regexp-quote (char-to-string escape-char)))))
        (while (re-search-forward regexp (point-max) t)
          (replace-match (concat (char-to-string escape-char)
                                 (match-string 0)) t t))))
    (buffer-substring (point-min) (point-max))))











(defun ys/unescape (string &optional escape-char) 
  ;; Converts ``\/ \n \\\/`` to ``/ n \/``. Basically converts ``\x`` to ``x`` where x is any single character and \ is the escape-char."
  ;; (let ((str "/ \\/ \\\\/ \\n"))
  ;;   (insert (concat "\n" str
  ;;              "\n" (ys/escape str ?/)
  ;;              "\n" (ys/unescape (ys/escape str ?/)))))
  ;; / \/ \\/ \n
  ;; \/ \\\/ \\\\\/ \\n
  ;; / \/ \\/ \n


"  Converts ``\\/ \\n \\\\\\/`` to ``/ n \\/``. Basically converts ``\\x`` to ``x`` where x is any single character and \\ is the escape-char.\"
  (let ((str \"/ \\\\/ \\\\\\\\/ \\\\n\"))
    (insert (concat \"\\n\" str
                  \"\\n\" (ys/escape str ?/)
                  \"\\n\" (ys/unescape (ys/escape str ?/)))))
  / \\/ \\\\/ \\n
  \\/ \\\\\\/ \\\\\\\\\\/ \\\\n
  / \\/ \\\\/ \\n

"
  (if (eq escape-char nil)
      (setq escape-char ?\\))
  (with-temp-buffer
    (insert string)
    (goto-char (point-min))
    (save-match-data
      (let ((regexp (concat (regexp-quote (char-to-string escape-char))
                            "\\(.\\)")))
        (while (re-search-forward regexp (point-max) t)
          (replace-match (match-string 1) t t))))
    (buffer-substring (point-min) (point-max))))
  





(defun ys/hexstring-to-charstring (hexstring)
;(insert (concat "\n" (ys/hexstring-to-charstring "039900000000000000000000000046")))
; ™������������F
"(insert (concat \"\\n\" (ys/hexstring-to-charstring \"039900000000000000000000000046\")))
 \231������������F
"
  (let ((expected-length (/ (length hexstring) 2)))
    (save-excursion
        (let ((result 
               (with-temp-buffer
                 (insert hexstring)
                 (goto-char (point-min))
                 (save-match-data
                   (while (and (not (eobp))
                               (looking-at "\\([0-9A-Fa-f]\\)\\([0-9A-Fa-f]\\)"))
                     (replace-match (char-to-string (hexl-htoi (string-to-char (match-string 1))
                                                               (string-to-char (match-string 2)))) 
                                    t 
                                    t)))
                 (buffer-substring (point-min) (point-max)))))
          ;; check if the conversion is valid
          (let ((result-length (length result)))
            (if (/= result-length expected-length)
                (error "Expected a charstring of length %d, but instead the length is %d" expected-length result-length)))
          result))))



(defun ys/charstring-to-hexstring (charstring)
                                        ;(insert (concat "\n" (ys/charstring-to-hexstring " ™������������F")))
                                        ;039900000000000000000000000046
  "(insert (concat \"\\n\" (ys/charstring-to-hexstring \" \231������������F\")))
039900000000000000000000000046

"
  (mapconcat #'(lambda (char) 
                 (format "%02x" char))
             charstring
             ""))





(defun ys/hex306string-to-char306string (hex306string)
                                        ;(insert (concat "\n" (ys/hex306string-to-char306string "303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f")))
                                        ;0123\/convert this\/ but /don't convert this/ finally \/convert this\/
  "(insert (concat \"\\n\" (ys/hex306string-to-char306string \"303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f\")))
0123\\/convert this\\/ but /don't convert this/ finally \\/convert this\\/
"
  (save-excursion
    (with-temp-buffer
        (insert hex306string)
        (goto-char (point-min))
        (save-match-data
          (while (and (not (eobp)) ;; needed because the regexp below won't fail due to *
                      (re-search-forward "\\([0-9A-Fa-f]*\\)" (point-max) t))
                                        ;(message (match-string 0))
            (let* ((hexstring (match-string 0))
                   (charstring (ys/hexstring-to-charstring (match-string 0)))
                   (escapedcharstring (ys/escape charstring ?/)))
                                        ;(message (format "hexstring:%s\ncharstring:%s\nescaped:%s" hexstring charstring escapedcharstring))
              (replace-match escapedcharstring t t))
                                        ;(message (format "reminder:%s" (buffer-substring (point) (point-max))))
            (if (looking-at "/.*?/") 
                (goto-char (match-end 0))
              (if (< (point) (point-max))
                  (error "Invalid hex306 string. Encountered a non-hexchar char but also not surrounded with //")))))
        (buffer-substring (point-min) (point-max)))))





(defun ys/char306string-to-hex306string (char306string)
;(insert (concat "\n" (ys/char306string-to-hex306string "\n0123\\/convert this\\/ but /don't convert this/ finally \\/convert this\\/")))
;0a303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f

  "(insert (concat \"\\n\" (ys/char306string-to-hex306string \"\\n0123\\\\/convert this\\\\/ but /don't convert this/ finally \\\\/convert this\\\\/\")))
0a303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f

"
  (save-excursion
    (with-temp-buffer 
      (insert char306string)
      (goto-char (point-min))
      (save-match-data
        (let ((charstring-begin (point-min)))
          (while (not (eobp)) 
            (cond ((looking-at "\\\\.") ; skip over escaped char. we must consume escaped char first so when we scan for a don't-convert-region, we won't get a false positive
                   )
                  ((looking-at "/\\(.*?\\)/") ; leave the don't-convert-this-region alone, but convert previous characters to hex
                   (let ((charstring (buffer-substring charstring-begin (match-beginning 0))))
                     (re-search-backward (regexp-quote charstring))
                     (replace-match (ys/charstring-to-hexstring (ys/unescape charstring)))
                                        ;(message (format "remaining:%s" (buffer-substring (point) (point-max))))
                                        ; the match-data info would have changed (because the length of charstring changed) , so we re-run the search
                     (if (looking-at "/\\(.*?\\)/")
                         (setq charstring-begin (match-end 0))
                       (error "Lost pointer to the don't-convert-region"))))
                  ((looking-at ".\\|\n") ; skip over other char
                   )
                  (t (error "Shouldn't have happened")))
            ;(message (format "going to %d/%d, remaining:%s" (match-end 0) (point-max) (buffer-substring (match-end 0) (point-max))))
            (goto-char (match-end 0)))
          (let ((charstring (buffer-substring charstring-begin (match-end 0))))
            (re-search-backward (regexp-quote charstring))
            (replace-match (ys/charstring-to-hexstring (ys/unescape charstring))))))
      ;(message (format "End-state:%s" (buffer-substring (point-min) (point-max))))
      (buffer-substring (point-min) (point-max)))))








;; ys/charstring-to-hexstring-region and ys/hexstring-to-charstring-region converts between these two forms:
;; 0123\/convert this\/ but /don't convert this/ finally \/convert this\/ 
;; 303132335c2f636f6e7665727420746869735c2f20627574202f646f6e277420636f6e7665727420746869732f2066696e616c6c79205c2f636f6e7665727420746869735c2f


(defun ys/hexstring-to-charstring-region (region-begin region-end)
  "Converts ``039900000000000000000000000046`` to `` ™������������F``"
  (interactive "r")
  (save-excursion
    (let ((charstring (ys/hexstring-to-charstring (buffer-substring region-begin region-end))))
      (delete-region region-begin region-end)
      (goto-char region-begin)
      (insert charstring))))


(defun ys/charstring-to-hexstring-region (region-begin region-end)
  "Converts `` ™������������F`` to ``039900000000000000000000000046``"
  (interactive "r")
  (save-excursion
    (let* ((charstring (buffer-substring region-begin region-end))
           (hexstring (ys/charstring-to-hexstring charstring)))
      (delete-region region-begin region-end)
      (goto-char region-begin)
      (insert hexstring))))




;; ys/char306string-to-hex306string-region and ys/hex306string-to-char306string-region converts between these two forms:
;; 0123\/convert this\/ but /don't convert this/ finally \/convert this\/
;; 303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f

(defun ys/char306string-to-hex306string-region (region-begin region-end)
  "Converts ``0123\/convert this\/ but /don't convert this/ finally \/convert this\/`` to 
``303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f``"
  (interactive "r")
  (save-excursion (let* ((char306string (buffer-substring region-begin region-end))
                         (hex306string (ys/char306string-to-hex306string char306string)))
                    (delete-region region-begin region-end)
                    (goto-char region-begin)
                    (insert hex306string))))

(defun ys/hex306string-to-char306string-region (region-begin region-end)
  "Converts ``303132332f636f6e7665727420746869732f2062757420/don't convert this/2066696e616c6c79202f636f6e7665727420746869732f`` to
``0123\/convert this\/ but /don't convert this/ finally \/convert this\/``"
  (interactive "r")
  (save-excursion
    (let ((char306string (ys/hex306string-to-char306string (buffer-substring region-begin region-end))))
      (delete-region region-begin region-end)
      (goto-char region-begin)
      (insert char306string))))



(provide 'ys-hex306)

ys-xml-escape.el

(require 'cl)
(defun ys/xml-escape-entities-and-non-printable-ascii (string)
  "Escapes XML entities and any non-ascii characters and also ascii characters that are not ``printable`` ( 32 < x < 126 )."
  (mapconcat 
   #'(lambda (char)
       (case char
         (?< "&lt;")
         (?> "&gt;")
         (?& "&amp;")
         (?' "&apos;")
         (?\" "&quot;")
         (t  (if (and (<= 32 char)
                      (<= char 126))
                 (char-to-string char)
               (format "&#%02d;" char)))))
   string
   ""))
;(insert (concat "\n" (ys/xml-escape-entities-and-non-printable-ascii "<goo&ten, \"'night\", he said>�")))
;&lt;goo&amp;ten, &quot;&apos;night&quot;, he said&gt;&#00;

(defun ys/xml-unescape-entities (string)
  "Unescapes XML entities"

  (save-excursion
    (with-temp-buffer
      (insert string)
      (goto-char (point-min))
      (while (not (eobp))
        (cond ((looking-at "&#\\([[:digit:]]+?\\);")
               (let ((char (string-to-number (match-string 1) 16)))
                 (replace-match (char-to-string char))
                 (goto-char (match-end 0))))
              ((looking-at "&\\(.+?\\);") 
               (let ((entity (match-string 1)))
                 (replace-match (case (intern entity)
                                  ('lt "<")
                                  ('gt ">")
                                  ('amp "&")
                                  ('apos "'")
                                  ('quot "\"")
                                  (t (error "Unknown XML entity: %s" entity))))))
              (t (goto-char (+ 1 (point))))))
      (buffer-substring (point-min) (point-max)))))
;(insert (concat "\n" (ys/xml-unescape-entities "&lt;goo&amp;ten, &quot;&apos;night&quot;, he said&gt;&#00;")))
;<goo&ten, "'night", he said>�

(eql (intern "lt") 'lt)


(defun ys/xml-escape-entities-and-non-printable-ascii-region (region-begin region-end)
  "Escapes XML entities and any non-ascii characters and also ascii characters that are not ``printable``."
  (interactive "r")
  (save-excursion
    (let ((escaped (ys/xml-escape-entities-and-non-printable-ascii (buffer-substring region-begin region-end))))
      (delete-region region-begin region-end)
      (goto-char region-begin)
      (insert escaped))))

(defun ys/xml-unescape-entities-region (region-begin region-end)
  "Unescapes XML entities"
  (interactive "r")
  (save-excursion
    (let ((unescaped (ys/xml-unescape-entities (buffer-substring region-begin region-end))))
      (delete-region region-begin region-end)
      (goto-char region-begin)
      (insert unescaped))))

(provide 'ys-xml-escape)

What I put into my .emacs for these two utils:

(require 'ys-hex306)
(global-set-key "\C-chc" 'ys/hex306string-to-char306string-region)
(global-set-key "\C-chh" 'ys/char306string-to-hex306string-region)
(require 'ys-xml-escape)
(global-set-key "\C-cxe"
'ys/xml-escape-entities-and-non-printable-ascii-region)
(global-set-key "\C-cxu" 'ys/xml-unescape-entities-region)

(defun ys/quote-for-docstring (region-begin region-end)
  (interactive "r")
  (save-excursion
    (let* ((original-mode-name mode-name)
           (commented (buffer-substring-no-properties region-begin
  region-end))
           (uncommented (with-temp-buffer
                          (funcall (symbol-function (intern (format
  "%s-mode" (downcase original-mode-name)))))
                          (insert commented)
                          (uncomment-region (point-min) (point-max))
                          (buffer-substring-no-properties (point-min)
  (point-max))))
           (quoted (with-output-to-string (print uncommented))))
      (insert quoted))))

The function ys/quote-for-docstring= is especially handy for
constructing a docstring with a lot of backslashes.


(originally from http://microjet.ath.cx/WebWiki/2006.02.15_Hexifier_and_XML_Entities_Escaper.html)

RSS and VSZ are not an accurate measure of memory usage

Yesterday I had the honour to chat with Bosko Milekic, a FreeBSD developer, regarding his concern of the outrageous VSS and RSS (virtual memory size and resident size, respectively) of a ruby app. I remembered that I had some code showing that VSS and RSS do not accurately measure an app's memory usage.
Since this was Bosko Milekic I was talking with, he was able to use the code to further investigate the problem matter. The best part is, he wrote a write-up (Ruby on Rails and Application Memory Consumption Patterns) of what he has learnt including what his further investigation revealed. As a result, now I know more than yesterday.
Isn't collaboration simply wonderful?

Updated 2007-04-20: updated linked URI.

(originally from http://microjet.ath.cx/WebWiki/2006.01.12_RSS_and_VSZ_AreNotAccurateMeasureOfMemoryUse.html)