HTTP socket, without using bridge

[Hemmendorff]

Hey, i've tried searching for this but didn't come up with much.

What i want to do is to open a connection with socket and GET the response of a simple http request.
I also want to download a image from the web directly with photoshop.

EDIT: I've succeeded with both these things(see next posts) but being pretty new to this, i would appreciate any advice on how to improve my code!

I'm making simple requests to a server to retrieve the http response.
I've cooked it down to a function:

Code: Select allfunction httpGet(url, port){
   if(url.substr(0,7)=="http://") url=url.substr(7); // removing http:// from domain
   if(!port) port=80; // set to default port if missing

   var domain=url.split("/")[0]+":"+port;
   var call="GET ";
   if(url.indexOf("/")<0){
      call+="/";
   }else{
      call+=url.substr(url.indexOf("/"));
   }
   call+=" HTTP/1.0\n\n";
   
   var reply = new String();
   var conn = new Socket();
   if (conn.open (domain)) {
      conn.write (call);
      reply = String(conn.read(999999));
      reply = reply.substr(reply.indexOf("\n\n")+2); // Removing header
      conn.close();
   }else{
      alert("CONNECTION TO DATABASE FAILED");
      reply = "";
   }
   return reply;
}

// Example:
alert(httpGet('http://www.ps-scripts.com'));
// alerts: the entire html code for the start page of ps-scripts


This actually works pretty neatly... But for some reason, some domains only return 404 "page not found" pages.

Does anyone have any experiences with using socket?

The reason i don't want to use the Bridge http connection is i only need to fetch simple json strings, and using bridge slows down my process so much.[/b]

[Hemmendorff]

It seems it's having some problems redirecting to the right IP depending on how the DNS stuff is set up... Unfortunately this goes way over my head.

I enhanced it by using HTTP 1.1 protocol instead of 1.0. This made the redirecting better, but still buggy on some hosts.

New function:
Code: Select allfunction httpGet(url, port){
   if(url.substr(0,7)=="http://") url=url.substr(7);
   if(!port) port=80;
   var domain=url.split("/")[0]+":"+port;
   var call="GET ";
   if(url.indexOf("/")<0){
      call+="/";
   }else{
      call+=url.substr(url.indexOf("/"));
   }
   call+=" HTTP/1.1\n";
   call+="Host: "+domain+"\n\n";
   
   var reply = new String();
   var conn = new Socket();
   if (conn.open (domain)) {
      conn.write (call);
      reply = String(conn.read(999999));
      reply = reply.substr(reply.indexOf("\n\n")+2);
      conn.close();
   }else{
      alert("CONNECTION TO DATABASE FAILED");
      reply = "";
   }
   return reply;
}

[Hemmendorff]

And now i finally succeeded to download a file directly from the web to the javascript, so i can copy it to the harddrive or view it as a iconbutton. Very useful! At least to me...

Code: Select allfunction httpGetFile(url, port){
   if(url.substr(0,7)=="http://") url=url.substr(7);
   if(!port) port=80;
   var domain=url.split("/")[0]+":"+port;
   var fileName=url.substr(url.lastIndexOf("/")+1);
   var call="GET ";
   if(url.indexOf("/")<0){
      call+="/";
   }else{
      call+=url.substr(url.indexOf("/"));
   }
   call+=" HTTP/1.1\n";
   call+="Host: "+domain+"\n\n";
   
   var reply = new String();
   var file = new File();
   file.encoding = "binary";
   file.open("w");
   var conn = new Socket();
   conn.encoding = "binary";
   if (conn.open (domain, "binary")) {
      conn.write (call);
      reply = conn.read(99999999999);
      reply = reply.split("\r\n").join("\n");
      reply = reply.split("\r").join("\n");
      file.write(reply.substr(reply.indexOf("\n\n")+2));
      file.close();
      file.rename(fileName);
      conn.close();
   }else{
      alert("CONNECTION TO DATABASE FAILED");
      reply = "";
   }
   return file;
}

// EXAMPLE:
var gotFile=httpGetFile("bb/templates/subSilver/images/logo1.gif");

alert("GOT FILE "+gotFile.length);

app.load(gotFile);


The only problem now(except the bugs with some hosts), is that the socket is slowww, even when downloading a very short text response it takes a couple of seconds. Downloading a jpg doesn't seem to slow down the process much so the slowness should be in opening or connecting the socket?

Any ideas on this?

[xbytor]

For a possible performance boost, try replacing this:
Code: Select all      reply = reply.split("\r\n").join("\n");
      reply = reply.split("\r").join("\n");


with this:
Code: Select all      reply = reply.replace(/\r\n?/, '\n');


Also, run the script in ESTK with per-line profiling enabled to get a better idea of where the time is actually being spent.

-X

[Hemmendorff]

Thanks!

That solution with replacing all \r is pretty bad anyway, i need to find a better way to split the http header from the data... Fetching png's fail because it seems i get a different header separation there. The use of \r \n or \r\n seems kinda wiggly... Strange.

Hmm... I'll have to look into this tomorrow i guess.

[xbytor]

That solution with replacing all \r is pretty bad anyway, i need to find a better way to split the http header from the data.

Start looping through the contents of the result a byte at a time and keep look for \n\n, \r\n\n, or \r\n\r\n.

Then write the rest of the result to disk. Try different techniques for this as well.

-X

[Hemmendorff]

Stupid question, when you say loop a byte at a time, would that mean looping through the string like

for(i=0; i<string.length; i++){
string
}

? Working with bytes is new to me. I was suprised that i could output the string i got when downloading a jpg into a file and just save it as the proper format and it worked

[Mike Hale]

X may have a better way but it's more like;

Code: Select allfor(i=0; i<string.length; i++){
string.charCodeAt(i);
}


\n = char code 10
\r = char code 13

[Hemmendorff]

Nice, thank you!

[Ryan0723]

I found a few more things that will speed up the connection for my situation (Indesign CS3 reading from Apache 2.2).

Turns out HTTP 1.1 automacically keeps the connection alive after the transfer unless you specifically turn it off. To do that add "Connection: close" to the request headers.

Code: Select allcall+=" HTTP/1.1\n";
call+="Host: "+domain+"\n";
call+="Connection: close\n\n";

Also instead of reading a way more than we need to from the socket we can calculate how much to read from the "Content-Length: #" header.

Code: Select all//Read enough to get the headers
reply = conn.read(300);

//parse content length out of the header
var contentLengthHeader = String(reply.match(/Content-Length: [0-9]*/));
var contentLength = contentLengthHeader.substr (16);
//parse for header length
var headerLength = reply.indexOf("\n\n")+2;

//read the rest of the message (content + header - what we have already read)
reply += conn.read(contentLength + headerLength - 300);