Google Webmaster Tools has released some new data for all you data-junkies to get your fix on. Specifically, they’re now showing impression numbers and click-through percentages for organic search queries. Rad right?  It looks that way on the surface…but the good folks on Dave Naylor’s team have done a bit of research into the numbers and what they’re calling the ‘Google Gap’.

Read on http://www.davidnaylor.co.uk/google-webmaster-tools-2.html, and remember to always question the numbers.

Big thanks Dave’s team for writing this post…if you’re looking for a top notch UK SEO, look no further.

The code was poorly presented, and poorly formatted. It wasn’t at ALL my normal caliber of work, so I went ahead and rewrote the concept into a tighter little wrapper.

I am using the Gattica gem to interact with the Google Analytics API.

This little script provides a wrapper around the gem that easily allows you to extract your sites organic keywords and sort them based on the number of unique entrances to your site via each keyword.

I thought I’d show you all a quick crash introduction on how you can access the API using Ruby.

The following code uses Open-Uri to access the API and Hpricot to parse the response. In this example, we’re going to authenticate with the API, and then we’re going to use it to extract all the keywords (organic and paid) which have brought visitors to our site within the past year (limited to 100 keywords max)

NB: sorry for the shitty formatting. I tried like hell to get this code to display well, but I gave up. Just copy and paste..you’ll be able to sort it out.

require ‘rubygems’
require ‘open-uri’
require ‘hpricot’

USER_EMAIL=”XXXXXX” #Insert your Google Account email address here
USER_PASS=”XXXXXX” #Insert your password here
PROFILE_ID=”XXXXXX” #Insert your profile ID here (Find this in your analytics profile under profile settings)
results = {}
keywords = []
auth_url = “https://www.google.com/accounts/ClientLogin?Email=#{USER_EMAIL}&Passwd=#{USER_PASS}&accountType=GOOGLE&source=open-uri-limbo-v01&service=analytics”

auth_key = (open(auth_url).read).split(”\n”).last

#now query for keywords, you can change the date values as you see fit, as well as the max-results value
response = Hpricot.XML(open(”https://www.google.com/analytics/feeds/data?start-date=2008-10-01&end-date=2009-04-21&dimensions=ga:keyword&max-results=100&ids=ga:#{PROFILE_ID}&prettyprint=true”,
“Authorization” => “GoogleLogin #{auth_key}”).read)
## storing some of the extra information in a “results” hash in case we need it later
results[:total_results] = (response/’openSearch:totalResults’).inner_text
results[:start_index] = (response/’openSearch:startIndex’).inner_text
results[:items_per_page] = (response/’openSearch:itemsPerPage’).inner_text
results[:start_date] = (response/’dxp:startDate’).inner_text
results[:end_date] = (response/’dxp:endDate’).inner_text
results[:aggregates] = (response/’dxp:aggregates’).inner_text
results[:data_source] = (response/’dxp:dataSource’)
results[:entries] = (response/’entry’)
## the entries contain each individual keyword

## for my purposes, I’m not interested in visitors coming from search operators like “site:”
## so we’re scrubbing those out.
for entry in results[:entries]
keyword = (entry/’dxp:dimension’).first[:value]
keywords << keyword unless keyword.include? “site:” or keyword.match(/\(.*\)/)
end

## Now print out our keywords
keywords.each {|kw|
puts kw
}

UPDATE just seconds ago, a new gem was released called Gattica which you can use to access the Google Analytics API. How’s that for up to the minute blogging? Thanks Twitter!

I got the idea when I found some domains that ranked for various viagra search terms back in 2001 that are now unregistered. They are old, plus have backlinks, which is a double bonus. We all know how much Google loves old domains!

Anyways, I created a small script that goes through the 2001 SERPs and pulls each domain into a list that is run through a whois search to see if it’s available. In under and hour I found almost 15 great domains. The thing that I like about this, compared to domain auctions is that you know you’re buying a domain that is relevant to your niche! And chances are, whatever backlinks those domains still have will also be relevant to your niche.

Give it a whirl! Some of you might be pleasantly surprised with what you find.

Notice how every single entry says “…Password Obscured…”

Interesting eh? Google has finally caught onto the fact that people have been leveraging their index to find exploitable systems.

As most of you know, the practice of using Google to find exploitable systems has been going on for a long long LONG time, but this is the first time I have seen Google react to the problem.

I wonder, does this means we are soon going to see google restricting certain wildcard searches? I can’t imagine they’d be foolish enough to restrict/disallow certain queries, as Im sure the fallout from angry bloggers and searchers would be something to reckon with. You know what it would be like: Freedom of information, slipperly slope, etc etc

However, I like the steps they have taken in the above example by simply “obscuring” sensitive information like a password; this might be a viable solution for Google to combat the ever-increasing amounts of server pwnage that they seem to be facilitating.

This weird little application, hosted on code.google.com is described as the following:

Google Hacks is a compilation of carefully crafted Google searches that expose novel functionality from Google’s search and map services. For example, you can use it to view a timeline of your search results, view a map, search for music, search for books, and perform many other specific kinds of searches. You can also use this program to use google as a proxy.

It’s a neat (if somewhat halfbaked) little tool that lets you try out a bunch of the more popular google hacks out there. The proxy option doesn’t work at all….trust me. But otherwise, it’s worth a spin!

Check it out:

http://code.google.com/p/googlehacks/

*burp*

There, I did it.  I can’t believe I just did that….At least I’ve got a good reason for it:

Cutts just wrote about a very useful Google search feature that lets you return results for a given term sorted by recency. I won’t go into it too much because I’m in the middle of a hundred other things, but click the link to Matt’s blog to read more about it.

Here is an example that returns results for Seocracy added to Google’s index within the last 24 hour: Last 24 hours of Seocracy

Why is this so useful? Well it gives me a much easier way to chart the success of my link building and indexing strategies. I can now tell exactly when the links are coming in and sort them by the date they were added to the index…..

‘nuf said.

Didn’t think so.

I just wanted to make a quick post today to share this tip with you all on how to find some decent proxy lists.

This query will return a huge amount of proxy lists. You’ll want to either write your own shell script to filter out the dead ones, or plug them into your program of choice.

To find vbulletin databases, the modification to the google query is quite simple:Try this

I myself wasted almost an entire evening sifting through data to bring you guys a couple really GREAT finds.

Some of this data will make its way into the free section of the website for posterity (it is just TOO good to pass up!), but in the meantime, you are welcome to peruse the links below and grab whatever tickles your fancy:

Info for some kind of Chat-Bot plus a bunch of Vbulletin data:

http://eroticasp.com/gg.sql

More Vbulletin databases (amazing, eh?)

http://www.nsmarabia.com/vbb.sql
http://forumsokak.net/dump/vbulletin.sql
http://www.alskor.com/forumbackup-05-Mar-2007-8aded.sql
http://www.thehulkster.org/goldint_rth2.sql
http://www.hgvpdl.com/vb2.sql
http://stardvb.com/tester/forumbacku…2006-850e0.sql

258 megabytes of articles
http://www.centralwebtesting.com/sg/articles.sql

Here are some more interesting ways to modify this google hack:

“phpMyAdmin” “running on” inurl:”main.php”

“# INSERT INTO”  filetype:sql phpbb2

“# INSERT INTO”  filetype:sql  inurl:private

“# INSERT INTO” emails filetype:sql

“#mysql dump” filetype:sql 21232f297a57a5a743894a0e4a801fc3
(md5 for ADMIN)

“vbulletin” filetype:sql seo
(This one was particularly juicy)

“# Dumping data for table ” -site:mysql.com -cvs

UPDATE (Oct 17,2007): Since this post came out, many MANY people started downloading alot of the databases that were discovered via the above search queries. As a result, quite a few of them are no longer online…I suppose their respective owners noticed when their bandwidth ballooned by several gigs. ANyways, some of the above google queries may no longer be super productive, but there are HUNDREDS of other google queries you can use to turn up more great stuff. Put your thinking caps on!

I LOVE google hacking.

Google hacking is the art of using google to turn up information that you really should not be able to access in a normal environment.

here is a great example of how to use google to search for peoples database dumps. This is a great way for you to source content for your website. Or if you like things a bit more black, a great way to find email address, passwords, usernames, and tons of other VERY personal information.

Check this out

Now try modifying the search query to search for `users table or `contacts`  table.

Cool eh? Get digging! And if you find anything really cool, feel free to post a link to it in the comments.

-Rob