Google Search Scraper written in Google Scripts

Google search will block your IP address temporarily if you use a download utility like wget to download search pages for scraping Google. However, if you run the scraper on Google servers itself through Apps Script, the request won’t be blocked as the IP address of the scraper would be one of Google’s own IP address.

The other alternative is Import HTML, a Google Docs function that you can used to scrape Google search results inside a Google Sheet.

function scrapeGoogle() {
  var response = UrlFetchApp.fetch('http://www.google.com/search?q=labnol');

  var myRegexp = /<h3 class=\"r\">([\s\S]*?)<\/h3>/gi;

  var elems = response.getContentText().match(myRegexp);

  for (var i in elems) {
    var title = elems[i].replace(/(^\s+)|(\s+$)/g, '').replace(/<\/?[^>]+>/gi, '');
    Logger.log(title);
  }
}
Amit Agarwal

Amit Agarwal

Google Developer Expert, Google Cloud Champion

Amit Agarwal is a Google Developer Expert in Google Workspace and Google Apps Script. He holds an engineering degree in Computer Science (I.I.T.) and is the first professional blogger in India.

Amit has developed several popular Google add-ons including Mail Merge for Gmail and Document Studio. Read more on Lifehacker and YourStory

0

Awards & Titles

Digital Inspiration has won several awards since it's launch in 2004.

Google Developer Expert

Google Developer Expert

Google awarded us the Google Developer Expert award recogizing our work in Google Workspace.

ProductHunt Golden Kitty

ProductHunt Golden Kitty

Our Gmail tool won the Lifehack of the Year award at ProductHunt Golden Kitty Awards in 2017.

Microsoft MVP Alumni

Microsoft MVP Alumni

Microsoft awarded us the Most Valuable Professional (MVP) title for 5 years in a row.

Google Cloud Champion

Google Cloud Champion

Google awarded us the Champion Innovator title recognizing our technical skill and expertise.

Email Newsletter

Sign up for our email newsletter to stay up to date.

We will never send any spam emails. Promise.