[FIXED] How to scrape data using google api

Issue

import requests

def search(query, pages=4, rsz=8):
    url = 'https://ajax.googleapis.com/ajax/services/search/web'
    params = {
        'v': 1.0,     # Version
        'q': query,   # Query string
        'rsz': rsz,   # Result set size - max 8
    }

    for s in range(0, pages*rsz+1, rsz):
        params['start'] = s
        r = requests.get(url, params=params)
        for result in r.json()['responseData']['results']:
            yield result

At first 2,3 attempts it is retrieving all required pages, but after 2,3 attempts it is not getting any result. It is returning "None" or []. Is google blocking my IP after few attempts? Any solution?

Solution

Well the problem was sorted out using requests and BeautifulSoup.

import requests, import BeautifulSoup
url = 'http://www.google.com/search'
payload = { 'q' : strToSearch, 'start' : str(start), 'num' : str(num) }
r = requests.get( url,params = payload, auth=('user', 'pass')) 
subSoup = BeautifulSoup( subR.text, 'html.parser' )
text = soup.get_text(separator=' ')       

Answered By - Muhammad Zeeshan