Prev/Next Blog
by date

TorqueScript Replacement to HTTPObject

Name:	David Higgins
Date Posted:	Feb 07, 2007
Rating:	4.5 out of 5
Public:	YES
Comments:	YES
RSS Feed:	or Subscribe with .
Profile Page:	View profile page for David Higgins

Blog post

UPDATE: httpPage now supports basic "POST" using either a URL-Encoded Query String style or a SimSet that contains SimObjects with "key" and "value" properties.

-- ORIGINAL POST (with updated code for POST support) --

This is pretty simple, you just instantiate a copy of it like so:

new TCPObject(httpPage) { };
httpPage.get("http://gearworxprod.com/index.php");
echo(httpPage.getResult());

Now, of course, the 'getResult' has to be called after the page has actually been retrieved -- I've left this up to the developer using the code, I put a skeleton 'onDisconnect' method for TCPObject in the code, just fill in the blanks :)

Here's the code:

$MAX_HTTP_QUERY_STRING = 255;


function httpPage::init(%this, %url) {
   %host = "";
   %page = "";

   if(strpos(%url, "http://") == 0)
   {
      %host = getSubStr(%url, 7, strpos(%url, "/", 8) - 7);
      %page = getSubStr(%url, strpos(%url, "/", 8), $MAX_HTTP_QUERY_STRING);
   }
   else
   {
      %host = getSubStr(%url, 0, strpos(%url, "/", 8));
      %page = getSubStr(%url, strpos(%url, "/"));
   }

   if(strpos(%host, ":") < 0) %host = %host @ ":" @ "80";
   %this.Address = %host;
   %this.Page = %page;
}


function httpPage::get(%this, %url)
{
   %this.Buffer = "";
   %this.doBuffer = false;


   %this.init(%url);
   warn("Connecting to: " @ %this.Address @ %this.Page);
   %this.Method = "GET";
   %this.connect(%this.Address);
}


function httpPage::post(%this, %url, %data)
{
  %this.Data = "";
  if(isObject(%data)) {
    warn("Data is Object: true");
    for(%x = 0; %x < %data.getCount(); %x++) {
      %datum = %data.getObject(%x);
      if(strlen(%postData) > 0) %postData = %postData @ "&";
      %this.Data = %datum.key @ "=" @ %datum.value;
    }
  } else {
    warn("Data is Object: false");
    %this.Data = %data;
  }


  error("Data: " @ %this.Data);
  error("%data: " @ %data);

  %this.init(%url);
  warn("Connecting to: " @ %this.Address @ %this.Page);
  %this.Method = "POST";
  %this.connect(%this.Address);
}


function httpPage::onConnected(%this)
{
   warn("Connected ...");

   %query = %this.Method @ " " @ %this.page @ " HTTP/1.0\nHost: " @ %this.Address;
   if(%this.Method $= "POST") {
     %query = %query @ "\n" @ "Content-Type: application/x-www-form-urlencoded\n";
     %query = %query @ "Content-Length: " @ strlen(%this.Data) @ "\n\n";
     %query = %query @ %this.Data @ "\n";
   } else {
     %query = %query @ "\n\n";
   }
   warn("QUERY: " @ %query);

   %this.send(%query);
}

function httpPage::onLine(%this, %line)
{
   warn("LINE: " @ %line);
   if(!%this.doBuffer && %line $= "") { %this.doBuffer = true; return; }
   if(%this.doBuffer)
   {
      error("BUFFER: " @ %line);
      if(%this.Buffer !$= "") %this.Buffer = %this.Buffer @ "\n";
      %this.Buffer = %this.Buffer @ %line;
   }
}

function httpPage::getResult(%this)
{
   return %this.Buffer;
}

function httpPage::onDisconnect(%this)
{
  warn("Disconnected: " @ %this.Address);
}

function httpPage::onConnectFailed(%this)
{
   error("Connection Failed: " @ %this.Address);
}

UPDATE: To pass parameters to your page, you can just build out a 'Query String' like you normally would for a regular browser call ("http://site.com/page.php?var1=val1&var2=val2"). One thing to note, is that 'httpPage' uses a TCP Socket to communicate with the server, and is not intelligent enough to make 'improper' url's 'proper' -- what does this mean? It means ... make sure "spaces" are "%20" and other things ... you can refer to the HTTP RFC for what is a valid request and what is not, 'httpPage' conforms to 'HTTP/1.0' requests (you can make it 1.1 if you want, by changing 'HTTP/1.0' to 'HTTP/1.1' in the 'onConnected' function, if you for some reason need extra features ... You can also, if you want ... extend it to support more then just 'get' by just, again, reading the HTTP RFC and implementing the POST or PUT, etc actions on your own. I may flesh this out to support POST eventually (maybe soon, ;p).

I put this together, using some of the code from my Forum/Blog GUI Kit I started working on a few months back, for a poster in the TGB Private Forums -- as this is TGE, TGB and TGEA friendly, I figured I'd post a .plan and share it for anyone else who may be struggling with HTTP Connectivity -- I leave the idea of "what to do with the http data" up to you ... :)

Recent Blog Posts

List:	11/19/07 - Cacheable Web Resources... Oh My! 09/29/07 - The Adventures of Coco the Gorrila in: CocoNuts 09/23/07 - How's it all Add Up? 07/11/07 - Ever wondered how to get your game project update to the rest of the team? 06/30/07 - The dog ate my homework, I swear! 06/20/07 - My latest news, and the new site I just launched ... 05/09/07 - $5,000 sound interesting? 05/01/07 - What Time is It?

Submit your own resources!

Tom Bentz (Feb 07, 2007 at 20:56 GMT)

Cool stuff David! I have TGB working as a simple web server using TCPObject. It's amazing what these engines can do.

Stefan Lundmark (Feb 08, 2007 at 00:27 GMT) Resource Rating: 4

Cool! :)

David Higgins (Feb 08, 2007 at 02:45 GMT) Resource Rating: 5

@Tom, do I even want to ask? :)

Tom Bentz (Feb 11, 2007 at 07:49 GMT)

I've been learning Ajax and was toying around with the idea of communicating to TGB with the web browser. Wacky computer tricks...

Prodigy Andy (Feb 16, 2008 at 12:58 GMT) Resource Rating: 5

David, could you tell me how to send url with query string just like http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=12262.

David Higgins (Feb 16, 2008 at 17:01 GMT) Resource Rating: 5

It's really just as simple as you want it to be,

new TCPObject(httpPage) { };
httpPage.get("http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=12262");
echo(httpPage.getResult());

You can build the query string however you wish, using whatever string concatenation methods and variable data you wish ...

Prodigy Andy (Feb 17, 2008 at 02:14 GMT) Resource Rating: 5

thank you , david.

:)

B. Burton (Mar 03, 2008 at 03:04 GMT)

Great resource! This is SO much easier than trying to use HttpObject!

David Higgins (Mar 03, 2008 at 03:21 GMT) Resource Rating: 5

thanks :)

MMPlus (Greg) (Mar 06, 2008 at 05:15 GMT)

Is there more to using this code than simply plugging it in? I tried testing it using my MainMenuGui with a button that runs a function containing those first 3 lines of code (using a custom URL), with the function declarations added to the end of the gui file. I get a script compilation error (syntax error in line 1 -- I've gotten this before, there's never anything wrong with line 1...), but when i remove the function declarations i don't get the error and it still doesn't work. Is there anything else I need to know about implementing this code? Does "httpPage" need to be delcared as a datablock or className or something somewhere? Do the function declarations need to be in a separate .cs file? I'm still learning here..

David Higgins (Mar 06, 2008 at 05:19 GMT) Resource Rating: 5

MMPlus, just toss the code into something like 'httpPage.cs' and 'exec' it from somewhere ('game.cs or main.cs') ... then use the 'first three lines' ...

but keep in mind ... waiting for the 'disconnect' message is necessary ... you can't just call httpPage.get() and then immediately call 'getResult' and expect to get something ... it's a non-blocking operating ...

if your getting compilation errors, then you've either copied and pasted the code incorrectly, or in an inappropriate place .... without seeing your code, I can't say for sure either is true, however.

-- David

MMPlus (Greg) (Mar 06, 2008 at 23:05 GMT)

Thanks, David. I got it working just as I needed it to. BTW, the compilation error was from a missing semicolon in the code, line 7 (%host = ""). Incidentally, is there any way to modify this to support HTTPS connections? I tried a few things, but to no avail.

David Higgins (Mar 07, 2008 at 01:47 GMT) Resource Rating: 5

MMPlus, thanks ... I updated the post to reflect the missing semi-colon ... obviously a bad copy/paste job on my part ... since the code was working when I posted it ;p

As for HTTPS ... that's not possible with 'httpPage', and would require a C++ library to be added to Torque to support the handshake and encryption required for SSL ... you'd then also have to add some sort of support for wrapping the TCPObject requests inside of the SSL commands ...

It is possible, just not feasable ... ;)

/Opinion

-- David

B. Burton (Mar 19, 2008 at 17:36 GMT)

David,
First, this is a great resouce a works wonderfully. My problem is when I try to do a subsequent connect to the tcpobject. I always get a bad request response. Should I create a new object everytime I do a get?
Here is my code:

         %query = $AuthServer @ "/VLE/chat.php?nick=" @ $pref::chat::NickName ;
         %chat_query = %query @ "&message=" @ %text;
         
         if($chatCounter !$= "1")   // only create the httpChat object once!
            {
            $ChatConnect = new TCPObject(httpChat) {};
            error("TCPObject intialized!!!");
            $chatCounter = "1";         
            }
         $ChatConnect.get(%chat_query);
         echo(%chat_query);

I set it to only create the TCPObject once. I was hoping that once would be enough, and I had problems with creating it multiple times: it also wouldn't consistently write to the database.

and here is the applicable part of the log file:

*** Initial Control Object
Activating DirectInput...
keyboard0 input device acquired.
keyboard0 input device unacquired.
TCPObject intialized!!!
Connecting to: www.gardenschools.com:80
http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=test message
keyboard0 input device acquired.
Mapping string: messageSent to index: 3
Mapping string: ChatMessage to index: 13
Mapping string: %1: %2 to index: 14
Connected ...
LINE: HTTP/1.1 400 Bad Request
LINE: Content-Type: text/html
LINE: Date: Wed, 19 Mar 2008 17:33:27 GMT
LINE: Connection: close
LINE: Content-Length: 20
LINE: 
LINE: <h1>Bad Request</h1>
BUFFER: <h1>Bad Request</h1>
keyboard0 input device unacquired.
Connecting to: www.gardenschools.com:80
http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=second message
keyboard0 input device acquired.
Connected ...
LINE: HTTP/1.1 400 Bad Request
LINE: Content-Type: text/html
LINE: Date: Wed, 19 Mar 2008 17:33:33 GMT
LINE: Connection: close
LINE: Content-Length: 20
LINE: 
LINE: <h1>Bad Request</h1>
BUFFER: <h1>Bad Request</h1>
keyboard0 input device unacquired.
Connecting to: www.gardenschools.com:80
http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=third message
keyboard0 input device acquired.
Connected ...
LINE: HTTP/1.1 400 Bad Request
LINE: Content-Type: text/html
LINE: Date: Wed, 19 Mar 2008 17:33:39 GMT
LINE: Connection: close
LINE: Content-Length: 20
LINE: 
LINE: <h1>Bad Request</h1>
BUFFER: <h1>Bad Request</h1>

The only thing I'm trying to do is save all chat messages. It is part of my dissertation research and I have to have the chat messages to analyze as part of the research. So if you have a better way to store the chats on a server, I'm open to suggestions!

Thank you for your help!
Brian

David Higgins (Mar 20, 2008 at 00:47 GMT) Resource Rating: 5

Brian, can we try to toss one more piece of information into the mix ...

Let's add another warn() statement into the onConnected, under the %query = ....

warn(%query);

I'd like to know what it is this.send(%query)'ing ...

and, let's also toss something into the 'onDisconnect', warn('Disconnected: ' @ %this.Address)

This will help me understand what exactly is going on, unfortunately, I don't have a copy of TGB sitting in front of me ... so I can't just whip up a quick test example ... so, please excuse the 'online remote debugging' process and bare with me ;)

-- David

B. Burton (Mar 20, 2008 at 01:16 GMT)

Okay David, I have added the lines. Here are the applicable results:

Activating DirectInput...
keyboard0 input device acquired.
keyboard0 input device unacquired.
TCPObject intialized!!!
Connecting to: www.gardenschools.com:80
http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=message 1
keyboard0 input device acquired.
Mapping string: messageSent to index: 3
Mapping string: ChatMessage to index: 13
Mapping string: %1: %2 to index: 14
Connected ...
GET /VLE/chat.php?nick=ProfB&message=message 1 HTTP/1.0
Host: www.gardenschools.com:80


LINE: HTTP/1.1 400 Bad Request
LINE: Content-Type: text/html
LINE: Date: Thu, 20 Mar 2008 01:13:16 GMT
LINE: Connection: close
LINE: Content-Length: 20
LINE: 
LINE: <h1>Bad Request</h1>
BUFFER: <h1>Bad Request</h1>
25www.gardenschools.com:80
keyboard0 input device unacquired.
Connecting to: www.gardenschools.com:80
http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=message 2
keyboard0 input device acquired.
Connected ...
GET /VLE/chat.php?nick=ProfB&message=message 2 HTTP/1.0
Host: www.gardenschools.com:80


LINE: HTTP/1.1 400 Bad Request
LINE: Content-Type: text/html
LINE: Date: Thu, 20 Mar 2008 01:13:19 GMT
LINE: Connection: close
LINE: Content-Length: 20
LINE: 
LINE: <h1>Bad Request</h1>
BUFFER: <h1>Bad Request</h1>
25www.gardenschools.com:80

I appreciate your help!

B. Burton (Mar 20, 2008 at 02:25 GMT)

David,
I'm using TGE 1.5.2

David Higgins (Mar 20, 2008 at 02:57 GMT) Resource Rating: 5

B, ok -- this is most likely the issue -- you'll want to most likely replace all instances of ' ' (that's a space, heh) with '%20' in your query -- more important, in your "message=message 2".

So, before setting what message is, wrap it with a simple string replace for ' ' (thats a space, again) with '%20' to conform to standards ...

Basically, the server's throwing an exception and calling it a bad request, because it doesn't know what to do with "1" or "2", because it was expecting "HTTP/1.0" or something ... using the space as the delimiter ...

Try that, and let me know ... or, optionally, just try making multiple calls with single words ... your choice :)

-- David

B. Burton (Mar 20, 2008 at 03:17 GMT)

David,
Wow, can't believe I missed that!
Thank you!
Brian

UPDATE: I added a

%chat_query = strreplace(%chat_query, " ", "%20");

to the messaging, and now everything writes to the database!
Edited on Mar 20, 2008 23:09 GMT

David Higgins (Mar 20, 2008 at 03:33 GMT) Resource Rating: 5

B, your email and comments on this thread ... along with some other feedback I've received recently on this resource ... has brought it back into light for me ... I'm working on implementing an httpPage.post() ... that will accept a URL and a SimSet with "name/value" for the post data ... or, optionally, a string that is in the form of the query-string ...

Should be posted soon :)

-- David

David Higgins (Mar 20, 2008 at 04:07 GMT) Resource Rating: 5

Ok -- I just posted an update to the main post, which updates the httpPage "class".

I used the following code to test this, so please, let me know if you run into any problems:

test.cs

cls();

exec("./httpPage.cs");
new TCPObject(httpPage);


$url[0] = "http://projects.zoulcreations.com/httpPage/get.php";
$url[1] = "http://projects.zoulcreations.com/httpPage/get.php?foo=bar";
$url[2] = "http://projects.zoulcreations.com/httpPage/get.php?foo=message%20text";


function httpPage::onDisconnect(%this)
{
  warn("Disconnected: " @ %this.Address);
  next();
}

$urlNbr = 0;

function next() {
  warn("next(): " @ $urlNbr);
  httpPage.get($url[$urlNbr]);
  $urlNbr = $urlNbr + 1;
}

httpPage.post("http://projects.zoulcreations.com/httpPage/post.php?method=post", "foo=bar&msg=message%20text");

I simply exec("./test.cs") in my game.cs file and checked the console output. As you can tell, I did -NOT- test the SimSet functionality, I'm just "assuming" it works -- :)

The URL's in this are valid, and your welcome to use them for yourself to test this functionality out -- get.php simply spits back the query-string to you, and post.php spits out both the query-string AND the post data.

post.php

<b>POST DATA</b><br />
<?
foreach($_GET as $k=>$v) {
	echo "G: <b>$k</b>: $v<br />";	
}
foreach($_POST as $k=>$v) { 
	echo "P: <b>$k</b>: $v<br />";
}
?>

get.php

<?
foreach($_GET as $k=>$v) {
	echo "<b>$k</b>: $v<br />";
}
?>

Marijn (May 08, 2008 at 08:19 GMT)

I've copied the code in News.cs and exec it in client/init.cs but it doesn't work for me.
I'm using TGEA 1.7

console:
==>new TCPObject(httpPage) { };

httpPage.get("http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=12262");

echo(httpPage.getResult());
Connecting to: www.garagegames.com:80/index.php?sec=mg&mod=resource&page=view&qid=12262

Error connecting to www.garagegames.com: No error
Connection Failed: www.garagegames.com:80

Robert "Robc" Charney (May 31, 2008 at 21:05 GMT)

I am also getting errors that look like:

Error connecting to www.garagegames.com: No error
Connection Failed: www.garagegames.com:80

and I also using TGEA 1.7. Has anyone used this resource under 1.7 with success?

David Higgins (May 31, 2008 at 21:36 GMT) Resource Rating: 5

Can anyone confirm that 1.7's TcpObject is the same as previous versions? Perhaps there were some bug fixes, or alterations to the way TcpObject works in 1.7 that are breaking this?

Also, can anyone experiencing issues with 1.7 confirm that it does in fact work with previous versions and 1.7 is the issue?

Edited on May 31, 2008 21:36 GMT

Robert "Robc" Charney (Jun 01, 2008 at 00:43 GMT)

I went back to a very old build on a game I left stranded back in early 2007, think it was TGEA 1.01? I am not even sure, lol. Anyway, it was the only older version of the engine I apparently still have laying around that was available to test immediately. I installed httpPage and succesfully connected to my webserver and executed a get() during startup before the client promptly crashed... hehe, don't know or care why it crashed. Anyway, the test you requested was whether this is a 1.7 issue, I can't confiirm it definitively, but this test does eliminate a lot of operator error possibilities on my part and suggests that 1.7 might be the culprit.

David Higgins (Jun 01, 2008 at 07:37 GMT) Resource Rating: 5

thanks for the quick and prompt test -- seems they probably fixed a bug, or introduced one ... in the 1.7 (or slightly previous?) code ... I'll just have to simply say "This is apparently not supported in TGE/A 1.7" :)

Matt Cuda (Sep 14, 2008 at 22:00 GMT)

Here is something weird that I hope someone can shed some light on. I am using TGE 1.5.2 and had this resource working great on my Vista box (if you can imagine). Well I got sick of Vista and went back to XP and suddenly the resource no longer works. Basically what I am doing is "GET" to a web page which emitts xml. Works great when I navigate to it manually but when I use this resource the connection terminates before the xml is sent. Here is what I am getting...
LINE: HTTP/1.1 200 OK
LINE: Server: Microsoft-IIS/5.1
LINE: Date: Sun, 14 Sep 2008 21:54:56 GMT
LINE: X-Powered-By: ASP.NET
LINE: X-AspNet-Version: 2.0.50727
LINE: Cache-Control: private
LINE: Content-Type: text/xml
LINE:

Disconnected: mattxp:80

The xml should be coming in after the blank line but instead the server just disconnects. Anyone else seen this behavior?

David Higgins (Sep 29, 2008 at 14:15 GMT) Resource Rating: 5

from the looks of it ... your HTTP response headers are missing the Content-Length ... which, if I remember correctly, means that your response is ended ... so, seems like it's a server-side issue, not related to httpPage ...

The server is just sending over a response header, and then disconnecting ... most likely, because there is no content to send over. Validate this with your page making the same request with HttpWebRequest in .NET or something ...

You must be a member and be logged in to either append comments or rate this resource.