Game Development Community

Easy libCurl integration for T3D

by Frank Bignone · in Torque 3D Professional · 07/07/2009 (8:47 pm) · 37 replies

First of all, this is based on the works done by community that are available in this thread libCurl implementation. The main changes are that the proposed package is an easy-to-install package (just unzip it), then everything should work correctly for all new project you will create using the project generator tool of T3D.

First of all download the package here libCurl zip package. It includes also the redistributable of libCurl. Then unzip this package inside your T3D installation folder. It will update the tools, engine and template folder. So you can now create a new project with T3D tool; and this project will automatically have the libCurl available.

The proposed library is supporting also SSL protocol. You just need to install one SSL library for your computer, like Win32OpenSSL.

As I do not have a Mac platform, I was not able to make a package for Mac. However, it is very simple to edit the curllib.inc file inside tools/projectgenerator/modules, to add such support for Mac if required.

You can look on our new website for additional information.

About the author

Real programmers don't waste time recompiling; they patch the binary files... ... Real programmers don't waste time patching binary files; they patch memory.

Page «Previous 1 2
#1
07/08/2009 (9:08 am)
Thanks Frank, curl is an exceptional library.
Also thanks to bank of course. I wish curl would make it into Torque3D.
#2
07/08/2009 (11:06 am)
Yeah, the ability of downloading files off HTTP/FTP is very important for anything running in a browser and account-based online games.
#3
07/12/2009 (3:19 pm)
Filedownload is possible out of the box since TGE 1.5 at very least through the HTTPObject actually (our patcher was basing on it)

None the less, libcurl is nice for a few other things like HTTPS Post
#4
07/12/2009 (5:28 pm)
@marc: with some modifications, it is possible to use the HTTPObject right (there is a resource to use it for binary file download). But as you mentioned libCurl gives a lot of interesting features like FTP, HTTPS, Upload...
#5
07/13/2009 (2:16 pm)
Definitely :)

Forgot to thank above, I'm sorry about that so: thank you very much for the resource, it will surely find its use :)
#7
08/22/2009 (3:46 pm)
How would you go about setting an HTTP header for libcurl? I am trying to set a user-agent because it is currently blank. Any ideas?
#8
08/23/2009 (1:50 am)
Just call curl_easy_setop(curl, CURLOPT_HTTPHEADER, headerlist) before sending the information to your server (see this link).

You can construct the headerlist with curl_slist_append as shown in this example.

CURL handle;  
 struct curl_slist *slist=NULL;

 slist = curl_slist_append(slist, "pragma:");  
 curl_easy_setopt(handle, CURLOPT_HTTPHEADER, slist);

 curl_easy_perform(handle);
 curl_slist_free_all(slist); /* free the list again */


#9
08/23/2009 (4:13 pm)
I get the following errors when I add the above code:
1>simCurl.cpp
1>..........EnginesourcesimsimCurl.cpp(330) : error C2182: 'handle' : illegal use of type 'void'
1>..........EnginesourcesimsimCurl.cpp(333) : error C2664: 'curl_easy_setopt' : cannot convert parameter 1 from 'int' to 'CURL *'
1>        Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
1>..........EnginesourcesimsimCurl.cpp(334) : error C2664: 'curl_easy_perform' : cannot convert parameter 1 from 'int' to 'CURL *'

This is what it looks like in simCurl.cpp:

if(stream || storagetype == SIMCURL_STORAGE_BUFFER)
	{
		// set all curl options
		if(upload)
		{
			curl_easy_setopt(curl, CURLOPT_READFUNCTION, &read_data);
			curl_easy_setopt(curl, CURLOPT_INFILE, this);
			curl_easy_setopt(curl, CURLOPT_INFILESIZE_LARGE, stream->getStreamSize());
			curl_easy_setopt(curl, CURLOPT_UPLOAD, 1);
		} 
		else
		{
			curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &write_data);
			curl_easy_setopt(curl, CURLOPT_WRITEDATA, this);
			
			CURL handle;  //ADDED
            struct curl_slist *slist=NULL;  //ADDED
            slist = curl_slist_append(slist, "User-Agent:Axis2/C");   //ADDED 
            curl_easy_setopt(handle, CURLOPT_HTTPHEADER, slist);  //ADDED
            curl_easy_perform(handle);  //ADDED
            curl_slist_free_all(slist); /* free the list again ADDED*/  //ADDE
		}
#10
08/23/2009 (10:18 pm)
My example was a generic example of the use of libCurl and not linked to the integration inside T3D. It should be adapted in order to work correctly.
#11
09/30/2009 (11:16 am)
Added an update to it for T3D v1.0 here.
#12
09/30/2009 (3:49 pm)
thanks Frank.
#13
09/30/2009 (3:59 pm)
so i assume one would use this from script something like this ?:
function sendRequest()
{
   %sc = new SimCurl();
   %sc.setURL("http://mysite.com/foo?a=1&b=2");
   %sc.setFinishCallback("finishCallback(" @ %sc @ ");");
   %sc.perform();
}

function finishCallback(%simCurlObject)
{
   %results = "";
   %delim   = "";

   while (!%simCurlObject.isEOF())
   {
      %results = %delim @ %simCurlObject.readLine();
      %delim   = "\n";
   }

   %simCurlObject.delete();

   // do stuff w/ %results
}

#14
10/03/2009 (11:44 pm)
you can use either %sc.perform or %sc.start. The first one is blocking and the second one is multi-threaded so it will not block your current script execution.
#15
10/05/2009 (1:50 am)
ah, thanks. start() would be the one the above was designed for.
#16
10/06/2009 (2:07 am)
Some info regarding multi-threading inside libCurl support implementation:
1. the current implementation makes use of a single thread for all %sc.start() action with a FIFO implementation. This means that you cannot have concurrent downloads.
2. if during the execution of some callbacks (which will be fired inside the "curl" thread), you have others scripts callbacks using execute, exec... you will completely mess the engine.


#17
10/06/2009 (1:57 pm)
> you cannot have concurrent downloads

ai yai yai, that's a pretty severe limitation. thanks for the heads-up.
it seems it should use multiple threads, and the callbacks shouldn't execute directly but rather go into the message queue.

this would make a great bounty-style project for GG.
having a robust HTTP library seems crucial for developing apps in this web 2.x world.
#18
10/06/2009 (10:38 pm)
Regarding the callbacks, you can:
- either have a queue for all Namespace::Entry::execute , ... which is mainly where you will have issue in multi-threaded application
- or protect the code with critical section / mutex

I have already solved both issues (concurrent downloads, multi-thread) in some new code I'm doing as support to all my projects. Indeed, I'm working on a package library (see my blog) where it will be possible to manage download of files / packages with a main aim targeted to update of game assets, or download of game assets (in case of webplugin for example). It also allow dynamic download of mission objects while player is playing the mission (like static object...).

I'm now testing the library and I did not yet decide to release it.
#19
11/22/2009 (4:20 am)
Here is a draft document of the extension I'm currently coding for T3D which will allow download of object resources on demand, file package management and so on.

Just look at the draft documentation here
#20
11/27/2009 (10:56 am)
First off, thank you all for sharing this resource! It is exactly what I was looking for. However, I can't get it to work (probably because of my inexperience with Torque). Whatever I try, I get a "Unable to instantiate non-conobject class simCurl." What am I doing wrong? I followed the instructions, created a new project and checked that curl is included in the game folder.
Also, I noticed the zip has a typo in the buildFiles folder name (it is currently without the 'l': libCurl\Templates\Full\buidFiles).
Page «Previous 1 2