In Deep Waters with Perl API's

clar·i·ty /ˈklærɪti/ Pronunciation Key - Show Spelled Pronunciation[klar-i-tee] Pronunciation Key - Show IPA Pronunciation
–noun

1.	clearness or lucidity as to perception or understanding; freedom from indistinctness or ambiguity.

2.	the state or quality of being clear or transparent to the eye; pellucidity: the clarity of pure water.

For those of you that don't know what an API is. Then please move onto a post about photography, because this will mean nothing to you. But if you do lets jump in with both feet.

Last week I was asked to write an API with another software Client. I don't like to name names so we will keep it on the low down. But there instructions where this:

The general process for making a request of an XYZ system is:
1) Gather information from user
2) Construct an XML data structure for request with REQUEST root element
3) Open HTTP 1.1 connection to XYZ ‘service’ URL
4) Send XML request to XYZ with content type “text/xml”
5) Get XML response from XYZ
6) Close connection

I figured that with perl this was going to be easy. So I create a perl script to do this. I was going to use my good old tools LWP and just submit a STRING over as the content of the lwp::useragent. Here is my code:

my $xml = '
SOME XML HERE
SOME XML HERE
SOME XML HERE
';
my $cgi_url= "http://THE_URL/DIR_OF_SERVICE";

my $ua = LWP::UserAgent->new(keep_alive =>1);
$ua->timeout(10);
$ua->env_proxy;

my $response = $ua->post($cgi_url, Content_Type => 'text/xml', Content=>$xml);
if ($response->is_success) {
$c->log->dumper($response->as_string);
} else {
$c->log->dumper($response->status_line);
}

Every time that I ran this code I would get a 200 code response and NO body back from the other service. So onto trouble shooting the problem. Here are a few things to keep in mind when working on an issue with a third party vendor api:

1. Always Assume that the problem is in your code and fix that first.
2. Start with the simplest of things.
3. Warn, Warn, Warn the crap out of your code.
4. Don't go changing the core of your code until you know it isn't a problem with the vendor.
5. Contact the vendor to have them check there end of the code.
6. When all else fails goto lunch.
7. After lunch if you still don't know what is wrong then change how your doing it.

I didn't follow any of these rules on this issue. Had I have I would have saved my self two days of programming. So when it was all said and done this is what I ended up doing. I contacted a friend that has also made an api into this same vendor but in PHP. Once I saw his code I noticed that he was opening a socket and submitting the info over. Had he been using Curl to do this I would have know my code was fine. Because useing LWP is pretty much the same as using Curl. So off to using IO::Socket. Here is the new code:

my $xml = '
SOME XML HERE
SOME XML HERE
SOME XML HERE
';
my $len = length($xml);
my $sock = IO::Socket::INET->new(PeerAddr =>'ROOT_DOMAIN_OR_IP',
PeerPort => '80',
Proto => 'tcp'
);
die "Could not create socket: $!\n" unless $sock;

if ($sock){
print $sock "POST PATH_FROM_ROOT HTTP/1.1\r\n ";
print $sock "HOST: ROOT_DOMAIN_OR_IP \r\n";
print $sock "Content-Type: text/xml \r\n";
print $sock "Content-Length: $len \r\n";
print $sock "Connection: close \r\n";
print $sock "\r\n";
print $sock $xml;
my $stuff ;
while (<$sock>){
$stuff .= $_;
}
close $sock;

$c->log->dumper($stuff);

When that was completed two days later, I got a 200 success and no body of the message. WTF. At this point I decided to follow my steps for trouble shooting the issue. So onto the simplest answers first. Maybe they can only get XML that doesn't have new lines. So I added
$xml =~ s/\n//ig;

Submitted the code and low and behold it worked. Got a response from there server.

THE PROBLEM:
There are two sides of this issue, mine and theres. Any good perl guy will have already caught my problem so lets start with theres.

There code will only respond when it received completed post, with a proper close. In other words if I say I am going to be sending a specific content_length then it will only respond if it gets it.

My problem is that when I was sending my $xml it would only send the first line tell it reached the first \n or new line. Had I submitted my xml one line at a time it would have been fine.

So todays lesson is, KISS. Keep it simple stupid. It could save your life.