Ruby Can’t open(‘http://digg.com/’)

This is a bit of a Ruby bug, but it’s so much more of a Digg bug that it’s worth telling the public about:

This code, as of Ruby 1.8.5 and March 29th, 2007, will not work. In fact, it will crash Ruby (which I should report, yes).

require 'open-uri'
open('http://digg.com/robots.txt')

I suspect the crash happens when the timeout thread stops the TCPSocket‘s sysread. But the question that brings up is: why would sysread timeout?

Turns out that Digg expects the User-Agent header every time and will not respond properly unless it receives it. (Thanks to Matt Horan for solving this one for me.) So this code works:

require 'open-uri'
open('http://digg.com/robots.txt',
     'User-Agent' => 'yourmom')
Advertisements

3 Comments

  1. Posted March 29, 2007 at 5:50 pm | Permalink

    ‘Accepts’ => ‘text/ymml’ # yourmom markup language

  2. Posted March 29, 2007 at 6:00 pm | Permalink

    <snaps>
    <snap>When she sits around the house, she sits <em>around</em> the house!</snap>
    <snap>She got hit by a parked car!</snap>
    </snaps>

  3. Vamsee
    Posted January 2, 2009 at 1:55 pm | Permalink

    Thank You. This bug, who ever responsible has been driving me crazy.


%d bloggers like this: