^(almost) murphy.de

TL;DR If you want to truncate a SQLite table, use DELETE FROM table; VACUUM.

While cleaning up my blog (backend, frontend and everything) for the new year, I noticed that the SQLite database I use to store its data had become quite huge — 28 MB huge to be exact. Since I could not dream of having written this much text, I had no idea what happened.

Since I despise writing SQL to inspect tables, I first searched for a simple, free graphical SQLite client (for OS X). No success in the Mac App Store. What I found then was a Firefox plugin called “SQLite Manager”. It’s horrible (Windows 3.1 style icons and poor interface design), but it did its job: After it hung up when I tried to look at the “sessions” table (connecting over SFTP wasn’t such a good idea), the culprit was found: ActiveRecord was storing sessions for each visitor without ever cleaning them up.

*sigh*

So, back to SQL:

I totally missed this new Rouge gem which came out in September. It’s a Pygments port in pure Ruby!

Jay Adkisson did a great job in my opinion, and the project is destined to become popular — I guess, in the long run, even more popular than CodeRay. Like Pygments, it has a long list of supported languages, a plethora of compatible stylesheets (even with 256 color terminal output), and the ability to guess the language of a piece of code.

If you’re using CodeRay or one of the Pygments wrappers, you should definitely check out this project.

Rouge also inherits from Pygments the DSL-based lexer definitions, which means it’s much cleaner to maintain, and also slower to execute.

Here’s a very quick benchmark that helped me to get a first impression:

At least when working with graphics and hexagons.

Well, CodeRay has had support for Themes for a long time. Seems I need to push that more.

When you include this in your ~/.bash_login file:

# Show current git branch or SVN subfolder in prompt.
      GREEN="\[\033[0;32m\]"
LIGHT_GREEN="\[\033[1;32m\]"
       GRAY="\[\033[1;30m\]"
 LIGHT_BLUE="\[\033[1;34m\]"
 LIGHT_GRAY="\[\033[0;37m\]"
  COLOR_OFF="\[\e[0m\]"
function prompt_func() {
  branch_pattern="\* ([^$IFS]*)"
  PS1="$LIGHT_GRAY`date +%H:%M:%S`$COLOR_OFF $LIGHT_GREEN\w$COLOR_OFF"
  if [[ -d "./.git" && "$(git branch --no-color 2> /dev/null)" =~ $branch_pattern ]]; then
    branch="${BASH_REMATCH[1]}"
    PS1+=":$LIGHT_BLUE$branch$COLOR_OFF"
  elif [[ -d "./.svn" ]]; then
    path_pattern="URL: ([^$IFS]*).*Repository Root: ([^$IFS]*).*Revision: ([0-9]*)"
    if [[ "$(svn info 2> /dev/null)" =~ ${path_pattern} ]]; then
      branch=`/usr/bin/ruby -e 'print ARGV[0][ARGV[1].size + 1..-1] || "/"' ${BASH_REMATCH[1]} ${BASH_REMATCH[2]}`
      revision="${BASH_REMATCH[3]}"
      PS1+=":$LIGHT_BLUE$branch$GRAY@$revision$COLOR_OFF"
    fi
  fi
  PS1+=" "
}
PROMPT_COMMAND=prompt_func

…it should show you, depending on what kind of folder you’re in, the current git or svn branch:

~ cd ruby/coderay
~/ruby/coderay:trunk@644 cd ../coderay-0.9
~/ruby/coderay-0.9:branches/0.9@641 cd ~/studies/ikiwiki/
~/studies/ikiwiki:master 

It is quite fast, but needs Ruby (any idea for a bash-only version?)

Update: Fixed a few things.
Update 2: Cleanup, SVN revison number.

Yes. I’m sorry. I’m busy with:

• starting a new job at sofatutor
• friends
• Shadowrun
• reading/writing on Twitter
• preparing for my New Zealand trip from late October till New Year
• watching JAG, season 5 (in English)

Because of this, I could not find the time yet to write about

• rbJL’s Zucker gem
• the latest updates to the Schatten project
• CodeRay 1.0
• my new iMac
• my love for m&m‘s
• what I like/hate about Twitter

You can follow me on @murphy_karasu (manchmal schreibe ich dort auch Deutsch), and/or you can have a completely unrelated “motivational” poster:

I’m sure I’m not the first one to find this, but here’s a working example for a i18n version of Integer#ordinalize.

Setup

First, you need to define <locale>.rb files in you lib/locales folder for each language. Here are examples for English, German, and French:

# lib/locales/en.rb
{
  :en => {
    :ordinal => proc { |number| number.ordinalize }
  }
}

This is using the built-in ordinalize extension, which does the English job.

# lib/locales/de.rb
{
  :de => {
    :ordinal => proc { |number| "#{number}." }
  }
}

German ordinalization is really easy.

# lib/locales/fr.rb
{
  :fr => {
    :ordinal => proc { |number| "#{number}#{number == 1 ? 'er/re' : 'e'}" }
  }
}

You’ll have to find out how to apply the correct genus all by yourself if you don’t like the self-service-slash :P

More languages can be added, but it gets really complicated for Spanish or Finnish.

Usage

After you restart your server (also in development mode!), I18n.t('ordinal') will return a Proc object. You can use it in your view like this:

<%= I18n.t('ordinal')[count] + ' ' + I18n.t("article") %>

Notice the use of [] for the object returned by the I18n.t call: It’s calling the Proc, passing the number argument, which is then ordinalized according to the current locale. You could also use .call(count), or even .(count) in Ruby 1.9, but I prefer the brackets.

Licenser:

(defn <3 [love & loves]
  (loop [l (str "I love " love) loves loves]
    (let [[love & loves] loves]
      (if (nil? love)
        (str l ".")
        (if (empty? loves)
          (str l " and " love ".")
          (recur (str l ", " love) loves))))))
(<3 "cookies" "cake" "chocolat") ;; =>  "I love cookies, cake and chocolat." 

murphy:

I = Object.new
def I.<(three, *args)
  puts "I love " + args[0..-2].join(', ') + ' and ' + args.last + '.'
end
I.<3, 'test', 'foo'
#~> I love test and foo.

I like the Clojure one better, even without syntax highlighting :)

Update: Licenser complained, so here’s an improved version:

Here’s the ultimate reason why Ruby is better than Python:

Jus ried o clean my keyboard, and now he key is suck. I can no longer run my Pyhon programs, bu I can sill run Ruby scrips. A definiive improvemen.

A gem from Fredrik Lundh, 2003.

Revision 534 of my syntax highlighting library updates the HTML output to HTML5 and CSS 3, which means IE users will need to install a browser to see CodeRay highlighted code.

Yes, I’m radical. Feels so good.

(By the way: Microsoft can choose to support CodeRay again if they wish :)

Man sollte allerdings bedenken, dass manche Männer sich als Frauen ausgeben und umgekehrt. Aber im Allgemeinen beantwortet das die Frage, wo die Frauen abgeblieben sind – zumindest frage ich mich das oft, wenn ich mich durch die Tech-Blogosphäre bewege.

Soziale Netzwerke sind offenbar nichts anderes als die weibliche Seite des Internets. Männer haben es lange Zeit nur dafür benutzt, Informationen auszutauschen (oder Pornos). Kommunikation über das Web (Twitter, Facebook, Gästebücher auf MySpace, das Gebrabbel auf StudiVZ usw.) ist aber heute mindestens so grundlegend wie Information (Wikipedia, Blogs, News, Google).

Es war auch überfällig :) Mails sind nämlich Mist, da sie aus der Gedankenwelt technischer Beschränkungen (nur Text, international eindeutige E-Mail-Adressen, die viel zu lang sind) und Bürometaphern (interne Post, Empfänger/Absender, Blindkopie, Briefform) stammen.

Die Idee mit dem IE8-Frauenbrowser hingegen ist peinlich. Frauen brauchen keine voreingestellten Favoriten, sondern einen schnellen, zuverlässigen Browser und moderne Webstandards – genau wie alle anderen auch.

I have written a little TextMate command (download) that allows you to count Lines of Code of the current file:

Here, this command is being used to verify that blokk stays at 318 LoC.

It needs CodeRay 0.9.1 or later.

In January, I wrote about my thoughts on the iPad:

Somehow, I’d like to code on that thing! TextMate 2 for iPad?

Some weeks later, I realized that this might have been a stupid idea. Isn’t the iPad a media viewer, with editing capabilities for visual things like presentations at most?

On third thought…

Well, now Matt Gemmell writes about iPad Application Design. The first line that got me was:

Look like a viewer, and behave like an editor.

Wow, what a statement :) Isn’t that what TextMate is all about?

If something has more than, say, 5 parameters, it should be divided into components that have less.

• iMac – 4 variants
• Keyboard options in German Apple store: 12
• Windows 7 – 10+ versions (32/64 bit)
• CreateWindowEx – 12 parameters
• script.aculo.us’ Ajax.InPlaceEditor – 24 options
• RDoc – 40+ parameters

Less is more. Hierarchies, if sensibly applied, can manage complexity.

Think LEGO.

1 puts, p, print

2 require

3 raise, throw, warn

4 new

5 to_s, to_i, to_f, to_sym

6 private, protected

7 class

They do. As do Germans, Spaniards, Californians, Czechs, Polish, Indians, and propably some Japanese. I think that’s cool :)

Ruby 1.8.6

…defines the following constants:

Two weeks ago, I demonstrated how to draw text on a WebGL texture. Eric Shepherd is even rendering video [warning: with sound]. Today, I’m rendering web content – that’s to say, a HTML page.

Again, I’m using the canvas 2d context to draw the content before using its data for the texture. The example is also based on Giles’ WebGL Lesson 7.

Mozilla’s Firefox has a unique feature called drawWindow. It’s not secure and thus you have to ask the user for explicit permission. So, it’s not ready for general usage, but it’s a nice demonstration.

Note: Before the example works, you have to enable the signed.applets.codebase_principal_support option in about:config. Disable it afterwards!

I also use an IFrame. So, this is about as evil as it gets.

You need a WebGL-enabled Firefox to see this.

Here’s the code:

Kirk Haines just stated:

If superior execution time is only achieved by offloading extra work
to an idle core, then that really isn’t a gain.

Agreed. Just because we have multiple cores now, that doesn’t mean we have to spawn threads for everything. In my opinion, achieving great single thread performance with good algorithms and clever optimization is still the best way of programming fast applications.

I compiled a pie graph of times needed for CodeRay scanner tests. More test data for a language means more time to run the tests.

As you can see, I have a lot of tests for Ruby (60,000 lines of code) and C (70,000), and less for Delphi (11,000). The combined “other” languages are C++, the CodeRay debug format, diff, ERB and Nitro HTML templates, Scheme, and XML.

I used the excellent Gruff graph library.

Here’s the code:

WebGL has no text rendering functions. But it’s pretty easy to create a texture with text using canvas 2D context:

You need a WebGL-enabled browser to see this.

The interesting code:

When cp -r takes a lot of time, I want to have an estimate how long it will take. My first approach was based on checking the size of the copied data repeatedly with du -sh.

This is obviously now very clean. Marco (a fellow student) told me that cp reacts to the SIGINFO signal by printing the progress to stdout. The manpage says:

If cp receives a SIGINFO (see the status argument for stty(1)) signal, the current input and output file and the percentage complete will be written to the standard output.

This is what the output looks like:

$ cp3 large.file large.file.copy 
large.file -> large.file.copy  42%
large.file -> large.file.copy  88%

Here’s the code:

I’m trying out a new scanner/encoder concept for CodeRay that would (hopefully) make highlighting even faster while improving the code at the same time.

Basically, it’s about bypassing the Tokens representation altogether.

The tests can be seen at Odd-eyed code.

Here are my results so far:

(1..100).select { |n| p n if ('1' * n) !~ /^1?$|^(11+?)\1+$/ }

Basically, the /^(11+?)\1+$/ part checks if the 1-string can be factorized. See Avinash Meetoo.

I use TextMate for everything that is text. Programming, organizing, shell-foo, writing for studies, protocolling, writing songs or guitar tabs, looking at patches and code, organizing, learning. And I have given up some great editors for it – Delphi, RDE, VIM, Weaverslave…actually, I switched to Mac in 2006 because of this software.

I just found out that I am [one of] the 129th most busy Rails Contributor[s], for my work on 7 patches back in 2006 and 2007.

Actually, I’m even #97 because 3 tickets for murphy where counted separately. Yay! I’m one of the top 100 Rails contributers. Now that’s something to brag about :D [Update: Xavier Noria added the alias. Great!]

Funny: Ezra Zygmuntowicz, Ola Bini, Sam Ruby, _why (the lucky stiff) and Zed Shaw come in last with only 1 commit. Even JEG2 has only worked on 3 tickets.

Also funny: There are currently exactly 1337 contributors listed!

devtail.com, a new site about developers’ lifes and works by Ian Stewart, is using my CodeRay syntax highlighter :) Nice.

Quote from his mail to me:

*Such* a great gem!

But it’s more due to Perl being so unpopular lately. Ruby stagnates; maybe Ruby 1.9.2, Rails 3, JRuby, MacRuby and Gemcutter can make Ruby more interesting in the future.

Update: It seems my prediction was wrong. But I wait another month before I take it back :)

Update: Epic fail. I was totally wrong; instead, Ruby is out of the Top 10 (Delphi took its place again). Every Top 10 language except C and Perl seems to weaken because of the advent of Objective-C and Google Go (the so-called company languages).

A cool feature of JavaScript 1.6 is called E4X:

var xhtml = <html xmlns="http://www.w3.org/1999/xhtml">
        <head>
                <title>Embedded SVG demo</title>
        </head>
        <body>
                <h1>Embedded SVG demo</h1>
                <svg xmlns="http://www.w3.org/2000/svg" 
                        viewBox="0 0 100 100">
                        <circle cx="50"
                                cy="50"
                                r="20"
                                stroke="orange"
                                stroke-width="2px"
                                fill="yellow" />
                </svg>
        </body>
</html>;

alert(xhtml.name().localName); // Alerts “html”
alert(xhtml.name().uri); // Alerts “http://www.w3.org/1999/xhtml”

I’m trying to highlight this syntax with CodeRay, but it’s a bit more complicated than I thought.

The last weeks I have been experimenting with WebGL, the upcoming 3D extension to the HTML 5 Canvas element. It is supported by the latest WebKit and Firefox nightly builds (read about getting WebGL to run on your system).

Now, here’s a rotating 500 4000 about 4900 polygon semi-transparent 3D sphere complete with a texture and lightning, animated via OpenGL by your graphics card – no Flash plugin needed (click the image):

If you want to stay updated about this developing technology, you can subscribe to the Planet WebGL meta-blog feed.

I can also recommend Giles’ step-by-step tutorials on learningwebgl.com, which I based my sphere example on.

This is just the beginning :) Some people are already creating games, demoscenes, and Viewers for 3D models with WebGL.

Update: Much faster mesh generation, more polygons. Always use Array.push instead of Array.concat.

Update 2: The Mesh library is now a separate file. Fixed some bugs with mesh generation.

In my tests, Ruby 1.8.7 runs simple benchmarks about 3-8% faster than 1.8.6 – that’s nice!

But Ruby 1.9.1 is far more fast, with speedups up to 2x for some real-life cases, and a reliable speedup around 10-20%.

JRuby performs better than 1.8, sometimes even better than 1.9, but only for longer tasks. For short benchmarks and day-to-day tasks, it’s often 2 or 3 times slower than any C-Ruby. The startup time is around 1 second, and thus JRuby is not the right tool for scripting tasks. It runs best on Java 6.

I updated my Ruby 1.8 vs. 1.9 benchmark.

It seems they continue to speed up Ruby in the latest 1.9 trunk, eliminating some slowness and stabilizing the new VM.

For the projects I am programming lately, namely Fukubukuro (a JavaScript interpreter in Ruby) and EPIC/EVIL (a secret project ;), Ruby 1.9 turns out to be about twice as fast as Ruby 1.8.6. I also profit from faster testing, spec-checking and RDoc generation.

Licenser and the folks of the Divided Space team want to use a hexagonal grid for their free massive multiplayer online role-play science-fiction fantasy browser game…thingy. Hex fields are so much cooler than squares or diamonds…remember Siedler 2? We want that. But it’s not easy…even the most basic thing, recognizing which hex field is currently under the cursor, proved to be hard to solve. So I worked hard to get it done, fast and reliable, because we can’t even begin to make an interface without such a function.

Licenser’s approach was absolutely funny, he actually checked the distance to the center coordinates of all hex fields to select the one nearest to the cursor – perfectly simple, but with quadratic complexity. Since the function has to be evaluated onmousemove, it needs to be FAST.

This is the function that I came up with, using school geometry knowledge and a great deal of trial-and-error:

A Chess simulation with JavaScript and HTML 5 Canvas.

It is to be played by two human players. I’m planning a computer opponent.

• shows board and pieces (as letters)
• shows possible moves
• move pieces: click on piece, then click on target square
  • all move rules except check, castling, en passant, draw game rules
• tested with Safari 4.0.2, Firefox 3.5.1, Opera 9.64, and iPhone OS 3.0
  • No shadows on Opera :-(
• graphical pieces, thanks to Cburnett from Wikipedia
  • I actually extended the Shapes library to interpret SVG paths!

Ever wanted to know how much longer your cp -r needs? This little Ruby script might help you.

$ cp2 coderay coderay-0.8
From: 307M	coderay
To:   154M	coderay-0.8

The second line updates every second while copying. It’s fast enough on OS X, but I’m sure it can be optimized.

Here’s the code:

Here’s another experiment with canvas: a simple map editor (click on the image).

You need Firefox, Opera, or Safari to see this; preferably one of the current beta versions (Safari 4 runs fastest, Firefox 3.5 is okay).

TeX is much to complicated for me, especially when it comes to simple text – I prefer the simplicity of Textile. But when it comes to typesetting formulas, there’s no real alternative.

So I’m trying to combine them, with this little Ruby script which runs on Leopard and uses latex2png:

Another great article about programming is the quite short piece Teach Yourself Programming in Ten Years from Peter Norvig, written 2001. The title should be enough to stir your interest :)

According to this article, I’m some years and a few programming languages away from becoming a “master”. Yay me!

One of the best articles I’ve read about general programming language topics is Jonathan Amsterdam’s Java’s new Considered Harmful (click on “Print” for a less ad-cluttered version.) He criticizes the use of the new Constructor(...) statement in Java on a functional level, concluding that “…for statically typed languages, it may not be possible to do better…”.

If you are interested in the never-ending discussion about static versus dynamic typing, I encourage you to read it. If you’re not, well, do something else.

Amsterdam also mentions Ruby’s approach to object creation, stating that it fixes a problematic behaviour of Smalltalk. Still, according to him, one problem remains:

A second drawback with […] Ruby is that initialize, being an ordinary method, does not chain: You must remember to begin your initialize methods with a call to the superclass’s initialize method.

1 Ruby

2 Python

3 JavaScript

4 C

5 QBasic

6 LOLCODE

7 Java

-7 TeX – hate it, can’t find a replacement

-6 MSN Messenger, ICQ Client – I hate ads

-5 TWiki – the ulcer of Web 2.0

-4 coderay executable – ugly, cryptic, incomplete – needs a rewrite

-3 RDoc – hopelessly bad

-2 chatcity.de – so buggy that it hurts

-1 Internet Explorer 6 – it’s the devil

It’s final. I fell in love with JavaScript.

I still love Ruby, but she can’t do this:

(Essentially the clock from the Mozilla Canvas tutorial, a little tweaked.)

No GIF, no Flash, no Silverlight, no SVG, no ugly hacks – just HTML and JavaScript. As an added bonus, it doesn’t work in Internet Exploder.

Another variation of Ruby-chan for the CodeRay Website.

Nobody can solve a big problem. But some people are good at dividing big problems into smaller problems that everybody can solve.

Those people are called programmers; their skill is division.

Ever wondered where a deep symlink points to? Try tracelink.

13:59:48 ~ $ tl foo2
foo2 --> foo (relative)
foo --> bla (relative)
bla: empty
14:00:27 ~ $ tl `which tl`
/usr/local/bin/tl --> /usr/local/bin/tracelink (absolute)
/usr/local/bin/tracelink: a ruby script text executable
14:00:34 ~ $ cat `which tl`

Did you know?

Every time you run the Ruby 1.9 test suite, a child dies :(

I wrote a simple version of the UNIX directory disk usage tool du (disk usage). Its output should be exactly the same as du -sh. Call it with the name of a file or folder.

Update: This is the second version, using a better method for the bytesize output and guessing the system’s blocksize.

Teaching Rails more languages is now as simple as:

• downloading example locales,
• putting them in your config/locales folder, and
• adding a method to ApplicationController that detects the user’s language:

class ApplicationController < ActionController::Base
  before_filter :set_locale
  def set_locale
    I18n.locale = request.env.fetch('HTTP_ACCEPT_LANGUAGE', '')[/(?:,|^)(de|en)\b/, 1]
  end
end

You have to adjust the regexp to allow automatic recognition of more languages; I haven’t found out yet how to get a list of all available locales without using I18n.load_path, but maybe the community helps me out.

It’s cool that changes to the locale files are applied instantly in development mode! A great improvement :D

(German translation on Ruby-Mine.)

Now, I’m about to write the Java Scanner for CodeRay (ticket #42 btw), and I thought it would be nice to have a list of built-in types highlighted – like String or IllegalStateException. I knew that TextMate had quite good highlighting for Java, so I checked the bundle for something to use.

Indeed, some smart guy included a very long regular expression into the token definitions, it looks like this:

support-type-built-ins-java = {
  name = 'support.type.built-ins.java';
  match = '\b(R(GBImageFilter|MI(S(ocketFactory|e(curity(Manager|Exception)|
    rver(SocketFactory|Impl(_Stub)?)?))|C(onnect(ion(Impl(_Stub)?)?|
    or(Server)?)|l(ientSocketFactory|assLoader(Spi)?))|IIOPServerImpl|
    JRMPServerImpl|FailureHandler)|SA(MultiPrimePrivateCrtKey(Spec)?|
    ...this goes on for several screens...

Apparently, they converted a long list of types into a minimal regexp, surely using a script for this. But that wasn’t exactly what I had searched for. How could I convert this back into the original plain list?¹

Camping sessions not working lately? Well, it’s a problem with Rails 2.1, to be more precise: ActiveRecord 2.1.0.

This took me at least 5 hours, but I finally got it: the latest version of AR does Dirty tracking and partial SQL updates. This means it only saves a new value when it thinks the attribute has changed. Nice, but it bugs Camping. So, here’s the patch:

Beim Recherchieren für unseren Vortrag über CUDA bin ich auf ein schönes mathematisches Bild des Mathematikers Claudio Rocchini aus Florenz gestoßen.

Der zugehörige C++-Code wollte bei mir nicht funktionieren; offenbar habe ich eine andere complex-Bibliothek. Nach ein bisschen Herumprobieren ging es dann (make colorful); allerdings bekomme ich ein anderes Bild! Woran das wohl liegt?

History

I began writing the application that runs this blog on 2008-01-13, shortly after exploring Camping, _why’s extremely small web framework (which has only 4KB of code). I was amazed how easy it is to create a web application with it.