Comments on: Parasite Scraping http://seocracy.com/2007/09/parasite-scraping/ A blog about technical SEO, Ruby, Web Apps, and more Mon, 05 Jul 2010 17:10:23 -0700 http://wordpress.org/?v=2.8.2 hourly 1 By: Nis http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-48 Nis Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-48 <p>Isn't it kind of risky to have that kind of site? --> http://opensource.votio.com/php/forum/poker-gambling </p> Isn't it kind of risky to have that kind of site? –> http://opensource.votio.com/php/forum/poker-gambling

]]> By: Seocracy http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-49 Seocracy Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-49 <p><font size="2">Yeah, as we're talking about here, it is definately easy to exploit scraper sites....not even just ones that have the scraped query in the URL through mod_rewrite.....there are other techniques to exploit other kinds of scrapers....but thats fodder for a later post.</font></p><p> </p><p><font size="2">There are tons of sites like this this you know how to find them</font> </p> Yeah, as we're talking about here, it is definately easy to exploit scraper sites….not even just ones that have the scraped query in the URL through mod_rewrite…..there are other techniques to exploit other kinds of scrapers….but thats fodder for a later post.

 

There are tons of sites like this this you know how to find them 

]]> By: Money Maker Blogs http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-50 Money Maker Blogs Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-50 <p>Thanks for this post. I have been reading about scraping recently and I'm trying to learn how to do it. This is interesting - scraping the scrapers... I like it. :) </p><p>This is a newbie question, but how do you implement the code examples above on a webpage to show the scraped content?</p> Thanks for this post. I have been reading about scraping recently and I'm trying to learn how to do it. This is interesting – scraping the scrapers… I like it. :)

This is a newbie question, but how do you implement the code examples above on a webpage to show the scraped content?

]]> By: Seocracy http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-51 Seocracy Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-51 That is Ruby. It's not as simple to run as just copy and pasting.....It requires its own interpreter program to execute the code, and it is not standard on many hosts. Do some research. That is Ruby. It's not as simple to run as just copy and pasting…..It requires its own interpreter program to execute the code, and it is not standard on many hosts. Do some research.

]]> By: Money Maker Blogs http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-52 Money Maker Blogs Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-52 Thanks for the response, off to my friend google I go :) Thanks for the response, off to my friend google I go :)

]]> By: taky http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-53 taky Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-53 <p>Good post, hopefully all the newbies will be able to benefit off of everyone elses Wordze accounts.</p><p>Hehe. </p><p>I just got finished writing my own content generation program that is similar, the first example scraper site is really good. That is a good blackhatter, and the sites are similar to mine (very clean and neat looking template).</p><p>I have been pretty sick of the spammiest looking spam sites you can ever imagine coming up in a G search. I'd also be suprised if those sites converted at all, for clicks or for affiliate programs. They just look so shitty.</p><p>The secret is to look neat, and have your PPC ads above the fold. Easy. </p> Good post, hopefully all the newbies will be able to benefit off of everyone elses Wordze accounts.

Hehe.

I just got finished writing my own content generation program that is similar, the first example scraper site is really good. That is a good blackhatter, and the sites are similar to mine (very clean and neat looking template).

I have been pretty sick of the spammiest looking spam sites you can ever imagine coming up in a G search. I'd also be suprised if those sites converted at all, for clicks or for affiliate programs. They just look so shitty.

The secret is to look neat, and have your PPC ads above the fold. Easy.

]]> By: Seocracy http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-54 Seocracy Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-54 <p><font size="2">Whenever I write a post in the wee horus of the morning (like this one), I invariably forget to mention some key things....so here's a very quick run-down of the key things you should also thing about...</font></p><p><font size="2">Be anonymous. Proxy. Rotate IPs if possible. Install tor on your server if you have to.....</font></p><p><font size="2">You shouldn't do anything half ass, especially this....don't forget to consider all aspects of your code that might leak your sites IP or other identify features. </font></p><p><font size="2">So be slick by remembering to pay attention to how you display the CSS for a page....I usually have my scraper scrape all pages ending in .css and then copy them into <style> tags in the header of the page I am scraping. </font></p><p><font size="2">Always filter your html to avoid opening yourself to XSS</font></p><p><font size="2">Keep in mind that the people you are going to be targeting will generally have no qualms about fucking with you if they catch you. It wouldnt be hard for them to feed your scraper malicious code. </font></p><p><font size="2">Oh...that reminds me....I didnt proxy those examples.....oh shit..uhhhhhhhhhhhhhh......</font></p> # bus error Whenever I write a post in the wee horus of the morning (like this one), I invariably forget to mention some key things….so here's a very quick run-down of the key things you should also thing about…

Be anonymous. Proxy. Rotate IPs if possible. Install tor on your server if you have to…..

You shouldn't do anything half ass, especially this….don't forget to consider all aspects of your code that might leak your sites IP or other identify features.

So be slick by remembering to pay attention to how you display the CSS for a page….I usually have my scraper scrape all pages ending in .css and then copy them into <style> tags in the header of the page I am scraping.

Always filter your html to avoid opening yourself to XSS

Keep in mind that the people you are going to be targeting will generally have no qualms about fucking with you if they catch you. It wouldnt be hard for them to feed your scraper malicious code. 

Oh…that reminds me….I didnt proxy those examples…..oh shit..uhhhhhhhhhhhhhh……

# bus error

]]> By: Bruce http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-55 Bruce Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-55 <p>You mentioned installing Tor..</p><p>Do you know of any sites that give instructions for installing Tor on a linux server, so that I can run my scripts a "bit" more secretly..</p><p> </p><p>Hopeing you can help</p><p> </p><p>Bruce </p><p> </p> You mentioned installing Tor..

Do you know of any sites that give instructions for installing Tor on a linux server, so that I can run my scripts a "bit" more secretly..

 

Hopeing you can help

 

Bruce 

 

]]> By: Rob http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-56 Rob Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-56 <p>Bruce, my suggestion is that you read all the help files on the tor website.</p><p><br />There are so many ways to set up an install on linux dependent on the environment, so I suggest you read the doc files.</p><p> </p><p>Make sure you have libevent installed on your machine (that was a stumbling block for me at first) </p> Bruce, my suggestion is that you read all the help files on the tor website.

There are so many ways to set up an install on linux dependent on the environment, so I suggest you read the doc files.

 

Make sure you have libevent installed on your machine (that was a stumbling block for me at first) 

]]> By: pcallisto http://seocracy.com/2007/09/parasite-scraping/comment-page-1/#comment-57 pcallisto Wed, 31 Dec 1969 16:00:00 +0000 http://127.0.0.1/seocracy/?p=12#comment-57 Another way to avoid detection would be to do your scraping off-site on a test server, injecting content into your own db, then uploading to your own sites.  It's also a good work around if you don't have VPSes or your own dedicated servers yet.  Can you say WP splogs? Another way to avoid detection would be to do your scraping off-site on a test server, injecting content into your own db, then uploading to your own sites.  It's also a good work around if you don't have VPSes or your own dedicated servers yet.  Can you say WP splogs?

]]>