Oh my, a post from squish. What is going on in the world? (Apart from riots in London, falling stock markets, general despair, doom and gloom…)

I’ve revamped my DNS traverser service. All rejoice at the rails-ness and AJAX of it all.

Thanks lots to the ISC for providing me with a server to host it.

I’m sure you can’t help but have noticed that fish sustainability has been in the news a lot recently. My sister decided for her 30th Birthday to take a whole bunch of us to see the film that’s created the media storm: End Of The Line. It’s a really good film (lots of graphs to back up the facts and agreement that there’s a big problem from real fishermen) and I highly recommend it. Ted Danson narrated! (I couldn’t help humming the Cheers theme tune…)

The main message that I’ve taken away from the film is that you can do your bit.

1. Don’t just order fish at a restaurant without consideration for where your fish came from. Ask them whether the fish was trawled, farmed or line caught. It was shocking to see what trawling involves and the damage it does to the sea bed, not to mention the fact that all the unwanted fish is thrown back in the sea dead.

2. When buying from a supermarket, check for items that are labelled sustainable, or better yet with the MSC logo:

MSC Logo

Having been around my supermarket the logo is very visible once you know what to look for.

3. Check out the Marine Conservation Society’s pocket Guide. This is an excellent resource. I hope they turn it in to an iPhone app someday. Not that I have an iPhone. Yet.

The topic is really fascinating just because how complicated it is. To pick two well known favourites from the pocket guide:

  • Cod from the Atlantic is AVOID, but Cod from the Pacific if MSC certified is EAT - not even CAUTION!
  • Salmon that’s of the 5 MSC certified types from the Pacific are EAT, and it’s less good to eat farmed Salmon. (Unbelievably farmed Salmon is fed on wild fish apparently 7 tonnes to make 1 tonne of Salmon.)


I’m pleased to announce the first beta of dnstraverse, a Ruby Gem API and associated command line program that performs a similar job to my (Perl based) dnscheck service.

Many people have asked for the source to dnscheck over the years but it was written as a prototype and regrettably I’d be very embarrassed if I released it. So I’ve re-written it from the ground up in OO Ruby and the gem is the result.

Conceptually dnstraverse does the same thing as a DNS resolver, with the major difference that it doesn’t just pick the fastest answer it picks all the answers. It can then calculate the %age probabilities of which answer will be returned. It’s a great way of debugging DNS problems such as lame server delegation and intermediate server connectivity issues. dnstraverse is more standards compliant than dnscheck and returns errors in a more meaningful way.

Please give dnstraverse a go and let me know what you think. It’s beta, so there may be issues. If so, drop me a note and I’ll be more than happy to help you out.

On an aside note, I love Ruby as a language. I’m not sure I can ever go back to Perl again. Unfortunately, Perl still wins hands down for support modules, documentation and system administration.

del.icio.us links:

rails background jobs: bj vs spawn vs backgroundrb (tags: ruby rails distributed)

Cor, Andrew Bednarz has written an iPhone client for mserv. Thanks Andrew!

mserv controller for iPhone

If only I had an iPhone :-(

I’ve recently done some work to talk to Sharepoint with Perl and thought I would share my experiences. I couldn’t find any example code out there in the wild for doing this, so I had to figure a lot of this out by trial and error. It’s actually quite simple once you’ve got it set up. I hope this helps someone.

This code shows you how to connect via the Web Services interface with NTLM authentication (i.e. standard Windows authentication) to manipulate Lists, but you could do almost anything.

You will need:

  • SOAP::Lite for talking to Sharepoint Web Services interface
  • LWP::Authen::Ntlm to enable LWP to talk NTLM
  • Authen::NTLM which LWP::Authen::Ntlm uses for the NTLM authentication

Some information sources that you’ll find useful:

  • MSDN Sharepoint Web Services documentation
  • Go to /_vti_bin/lists.asmx on your Sharepoint server for a lot of useful information
  • Go to /_vti_bin/lists.asmx?WSDL for the WSDL definitions (if all else fails)

So here’s how to get started:

use LWP::UserAgent;
use LWP::Debug;
use SOAP::Lite on_action => sub { "$_[0]$_[1]"; };
import SOAP::Data 'name', 'value';
our $sp_endpoint = 'http://sp.example.com/sites/mysite/_vti_bin/lists.asmx';
our $sp_domain = 'sp.example.com:80';
our $sp_username = 'DOMAIN\username';
our $sp_password = 'xyz';

The SOAP::Lite module needs to be told how to construct the SOAPAction header properly for Sharepoint. The on_action does just this, and means you’ll end up with a SOAPAction appending the URL and the method name together without anything in between (stops the default # that Sharepoint doesn’t want).

if ($debug) {
    SOAP::Lite->import(+trace => 'all');

Use the above code to turn on debugging if you get errors.

my @ua_args = (keep_alive => 1);
my @credentials = ($sp_domain, "", $sp_username, $sp_password);
my $schema_ua = LWP::UserAgent->new(@ua_args);
$soap = SOAP::Lite->proxy($sp_endpoint, @ua_args, credentials => \@credentials);

This complete mess is the necessary steps to get SOAP::Lite to use a properly configured LWP UserAgent to do NTLM authentication. SOAP::Lite uses two UserAgents, one for the main SOAP calls and one for the Schema fetching. Although you don’t need to fetch a schema, I’ve included the proper set up above in case you want to call $soap->service(”$sp_endpoint?WSDL”); for some reason.

$lists = $soap->GetListCollection();
quit(1, $lists->faultstring()) if defined $lists->fault();

That’s all you need to do to get a list of all the lists on your Sharepoint site. And we can print them out:

my @result = $lists->dataof('//GetListCollectionResult/Lists/List');
foreach my $data (@result) {
    my $attr = $data->attr;
    foreach my $a qw/Title Description DefaultViewUrl Name ID WebId ItemCount/ {
        printf "%-16s %s\n", $a, $attr->{$a};
    print "\n";

Or if you need to find a particular list to do operations on it, search for it in the results by looking up the Title with something like this:

sub lists_getid
    my $title = shift;
    my @result = $lists->dataof('//GetListCollectionResult/Lists/List');
    foreach my $data (@result) {
        my $attr = $data->attr;
        return $attr->{ID} if ($attr->{Title} eq $title);
    return undef;

And here’s another useful subroutine to get all the items in a list:

sub lists_getitems
    my $listid = shift;
    my $in_listName = name('listName' => $listid);
    my $in_viewName = name('viewName' => '');
    my $in_rowLimit = name('rowLimit' => 99999);
    my $call = $soap->GetListItems($in_listName, $in_viewName, $in_rowLimit);
    quit(1, $call->faultstring()) if defined $call->fault();
    return $call->dataof('//GetListItemsResult/listitems/data/row');

That will use the default view. The 99999 is a hack to get all the items and stop the server “paging” the results. Putting this together you’d do something like this:

my $list_id = lists_getid('MyList');
print "List ID is: $list_id\n";
my @items = lists_getitems($list_id);
foreach my $data (@items) {
    my $attr = $data->attr;
    # print Dumper($attr);

Here’s some code to add a new list item:

my $field_id = name('Field', 'New')->attr({ Name => 'ID'});
my $field_linktitle = name('Field', $title)->attr({ Name => 'Title'});
my $field_something = name('Field', $something_else)->attr({ Name => 'Something_x0020_Else'});
my $method = name('Method', [$field_id, $field_linktitle, $field_something])->attr({ ID => "anything", Cmd => 'New'});
my $batch = name('Batch', \$method);
my $in_listName = name('listName' => $list_id);
my $in_updates = name('updates' => \$batch);
my $call = $soap->UpdateListItems($in_listName, $in_updates);
quit(1, $call->faultstring()) if defined $call->fault();

The content for Name=”ID” must be “New”. Where it says “anything” it really can be anything, it’s just an identifier for responses. You can also see that spaces are encoded as _x0020_.

my $field_id = name('Field', $sp_id)->attr({ Name => 'ID'});
my $field_something = name('Field', $something_else)->attr({ Name => 'Something_x0020_Else'});
my $method = name('Method', [$field_id, $field_appname])->attr({ ID => $jira_name, Cmd => 'Update'});

The above is for modifying an item. In this case the $sp_id must be set appropriately from the “id” attribute of a list item you previously fetched.

I hope that helps someone. Perhaps one day someone can put the effort in to writing a module to do all this.


del.icio.us links:

The sky at night (and during the day) | Gulliver | Economist.com (tags: airline travel)

del.icio.us links:

How Wall Street Lied to Its Computers - Bits Blog - NYTimes.com (tags: news)

del.icio.us links:

Shiny Shiny: Friday Video Fun: Alice in Wonderland does electronica (tags: disney music video)

del.icio.us links:

Seven Misunderstandings About Classical Architecture (tags: architecture)

Next Page »