Fun With FFA pages...

[uniprosper]

Hi,

I'm trying to figure out a way to automatically post to FFA (Free-4-all) link pages.

I made a script several weeks ago that can post to search engines using the GET method, and that was no problem. But when it comes to using the POST method to post to FFA pages, YIKES! Where do I start?

For the search engine script, I used the GET method through a SOCKET. Here is part of the script I used:

openconnection("www.server.com", "80");

$add="/addurl.cgi?newurl=$url&email=$email";
$submit = "GET $add HTTP/1.0\n\n";
send(SOCK,$submit,0);
close(SOCK);

The above works great for search engines that use the GET method. But how do I convert the above into POST method?

I've tried this code below, but that doesn't work either:

openconnection("www.server.com", "80");

$add="/free4all/links.cgi";
$submit = "POST $add HTTP/1.0\n\n";
$submit.= "Content-length: 150\n";
$submit.= "Content-type: text/plain\n\n";
$submit.= "title=Test";
$submit.= "url=http://www.uniprosper.com/?unipro";
$submit.= "section=Business";
$submit.= "email=lee\@uniprosper.com";

send(SOCK,$submit,0);
close(SOCK);

I have a feeling it may be something to do with the content length line, but how do I know what number to set the length to? I have no idea how to do this.

Then someone else told me I could use the LWP cgi modules, and I could forget about all this socket module stuff. But then I don't know how to specifically use the post method with that either.

Has anyone got any ideas of how to do this? I have an FFA site up that I use to test the script on. Its at http://www.uniprosper.com/free4all/links.html

If you have a script and you want to test it on that site, you can do.

Any help will be greatly appreciated.

And Merry Christmas!
Lee

[Terra]

$submit.= "title=Test";
$submit.= "url=http://www.uniprosper.com/?unipro";
$submit.= "section=Business";
$submit.= "email=lee\@uniprosper.com";

The best way to do this is reverse engineer it... http://www.aota.net/ubb/wink.gif

Setup an HTML page and fire off a POST method to a test script, like - oh, ummm...

TTdebug.pl
#!/usr/local/bin/perl
$|++;

my $cl = $ENV{'CONTENT_LENGTH'};
my $sanity = -1;
open(TTdebug, ">/big/dom/xuniprosper/cgi-bin/debug/post_slurp.txt") or die "A horrible death: $!\n";
if ($sanity = read(STDIN, $buffer, $cl)) {
print TTdebug join "\n", ("CL:", $cl, "Post:", $buffer);
}
else {
print TTdebug "Dave, I think my mind is fading: $sanity\n";
print TTdebug "But Dave, I was expecting: $cl\n";
}
close TTdebug;
__END__

I think you'll find your first problem is that what you are submitting is *not* URI %-encoded...

After you do the encoding, then it's as simple as:
$cl = "Content-length: " . length($encoded_post) . "\n";

$submit = "POST $add HTTP/1.0\n\n";


I believe that should only have "\n" x 1, not x2...

--
Terra
sysAdmin
FutureQuest

[This message has been edited by ccTech (edited 12-20-98).]
--minor mods for sanity nullification handling--

[This message has been edited by ccTech (edited 12-20-98).]
--did I ever mention I like 2010--

[This message has been edited by ccTech (edited 12-20-98).]

[uniprosper]

Thanks Terra,

I'll give that a shot.

Where did you guys learn to do all this stuff? I can't find any good Perl/cgi tutorials anywhere.

Lee.

[Terra]

Where did you guys learn to do all this stuff?

Heheheh... Lot's and Lot's and Lot's of headaches/nightmares/patience/flashbacks/knawed_fingernails/patience/pacing/moon_walking/patience/trial-n-error/coffee/etc...

Oh yeah, did I happen to mention 'patience'? http://www.aota.net/ubb/wink.gif

I can't find any good Perl/cgi tutorials anywhere.


Ummm, how much are you willing to spend??? http://www.aota.net/ubb/wink.gif

My Perl library is pretty extensive - and have the *perfect* blend of books... Though not cheap...

--
Terra

[This message has been edited by ccTech (edited 12-20-98).]

[uniprosper]

uhh, let me see....

Ok, how much would it cost for me to learn how to set-up this FFA submitter without all the headaches/nightmares/patience/flashbacks/knawed_fingernails/patience/pacing/moon_walking/patience/trial-n-error/coffee/etc...

hehe.

You know, a month ago I didn't know a single thing about this CGI/Perl stuff. I'm gradually learning. But I've been at this FFA submission project for about 3 weeks, and I seem to be going through the symptoms that you described above, LOL.

I've seen loads of free FFA submission sites out there that submit upto 1900 sites. But whenever I email them to ask if they could give me any tips on how to do it, they never reply! I don't think they want to share their secrets.

At the minute I submit all my customers pages with some FFA software I have on my PC. However, Internet access is really expensive over here in the UK, so during the daytime I'm actually spending more on the phone bill than I by submitting the pages using the FFA submitter!

Is there any way I could bribe you guys into helping me? Let me see....
I've already offered people a FREE subscription to my services worth $85, but I don't think this will be any good to you as you seem to already have a submission service!

I don't know, how much do you charge for this kind of info?

Maybe I'll be able to get some sleep before Christmas, hehe.

Thanks *very* much,
Lee.

[Terra]

Wish I could help you out more, but my plate is **very** full at the moment...

For reference, please visit:
http://www.aota.net/cgi-bin/forumdisplay.cgi?action=topics&forum=Server+-Alpha-/-Beta-/-Gamma-&number=12&DaysPrune=60&LastLogin=

As you can see - I'm swamped for awhile... http://www.aota.net/ubb/biggrin.gif

Maybe later on I'll kick out all the ISBN's to the books I have crawling around me...

--
Terra

[This message has been edited by ccTech (edited 12-20-98).]

[Deb]

Hey There...

I am *not* the coder around here... but..

I am good at 'learning by example', some call it cheating.. but I'd never do that http://www.aota.net/ubb/wink.gif

Maybe by viewing some of the scripts available that do stuff like this you could find the magic key needed for you to accomplish it also...

There's a short list at cgi-resources found at http://www.cgi-resources.com/Programs_and_Scripts/Perl/Website_Promotion/Submitting_URLs/

It might help a bit to peak at those scripts...

Give it a shot -- Other then that you may need to pay a programmer for to build it for you.. I believe cgi-resources also has a list of people for hire in this area too... If there isn't anyone around here willing?

Deb

------------------
Unlocking the Power of the Internet!
www.FutureQuest.net (http://www.FutureQuest.net)

[uniprosper]

Thanks very much you guys,

I will check out the cgi-resources section.

Good luck with all those server projects. Does anyone get any sleep around here? hehe.

That Ez-mlm project looks promising. I'll take a look at that too.

:-) :-) :-) :-)
Lee

[Terra]

I did some digging on your POST question...

In the LWP module suite - the one you need to read and study is:
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;

The HTTP one contains the POST routines you require, as it will automatically encode all the information for you...

==Code Snippet==
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;

$ua = LWP::UserAgent->new();
my $req = POST 'http://www.perl.com/cgi-bin/cpan_mod', [module => 'DB_File', readme => 1];
$content = $ua->request($req)->as_string;
==Code Snippet==

The snippet was derived from the 'Perl Cookbook'... AWESOME book!!! If you don't have it, seriously consider purchasing it... http://www.aota.net/ubb/smile.gif
ISBN: 1-56592-243-3
$39.95

--
Terra

[meikel]

Hi uniprosper,

sorry that my first tip did not get you round.
I strongly suggest, that you get an offline webreader (e.g. http://www.metaproducts.com
and then go to http://www.itlibrary.com/reference/dir.programminglanguages.perlandcgi1.html

You find complete books about perl there freely available to read.

I already downloaded much stuff there and I'm getting more and more accustomed to reading on screen.

For your current problem:
1)
The Content-length: must match *exactly* the number of bytes you want to submit. The HTTP spec states that any server must ignore any POST request not having a 100% correct Content-length. So better calc carefully.

2)
The post is devided in two parts
The first part is the header that should look like

POST /cgi-bin/script.pl
Content-lenght:10

Each line has a \n at the end.

Then follows an empty line and the content of *exactly* 10 bytes.

1234567890

So your complete submission must look like

POST /cgi-bin/whatever.pl HTTP/1.0
Content-length: 10

1234567890


3)
You must encode the data to post. The only way to learn what to post is to see what gets posted to your scripts.
Everything has a funny %HEX encoding in it for special characters. Usually a form is posted like
field=value&field=value&field=value
You must have this "&" between the fields.

To get what you need, I suggest you first restrict your tests to pure ascii characters and then set up a test-script and see which character gets encoded to which %HEX value using your browser.

It should be no problem for scripts if you encode all characters to a %HEX value, but this will triple the amount of data you must send.

So your code could look like:

$submit = 'field1=value1';
$submit .= '&field2=value2';

$contentlength = length($submit);

$submit = 'POST ...[etc]...\n' .
"Content-length: $contentlength\n\n" . $submit;

And I'm not sure if your approach works for all websites. E.g. our sites here at FQ all share the same IP. So if you open the connection to any FQ hosted site, you get the same connection. Now the HTTP POST request must include some information for the Apache webserver which site you really mean. I don't know how to put that into the headers.
So it seems a good idea to use the LWP library. The small code snippet above looks pretty easy (even if I have no idea what it does.... I have no idea on how OO works in perl).

Greetings from Bonn, Germany
Meikel Weber
http://www.tickerland.com
http://www.meikel.com

[uniprosper]

Hi,

Thanks for your replies!

I will try out code in both instances.
This will help me out very much, thanks.

Once I get the submitter up and running, I'll let you know how it goes.

Thanks Again! :-)
Lee.