This is a bit of a Ruby bug, but it’s so much more of a Digg bug that it’s worth telling the public about:
This code, as of Ruby 1.8.5 and March 29th, 2007, will not work. In fact, it will crash Ruby (which I should report, yes).
require 'open-uri' open('http://digg.com/robots.txt')
I suspect the crash happens when the timeout thread stops the
sysread. But the question that brings up is: why would
Turns out that Digg expects the
User-Agent header every time and will not respond properly unless it receives it. (Thanks to Matt Horan for solving this one for me.) So this code works:
require 'open-uri' open('http://digg.com/robots.txt', 'User-Agent' => 'yourmom')