Import requests import pandas as pd Step

[mahmud220]

Send our request through ScraperAPI Traditionally, to get the data we're looking for, we'd have to build a scraper that would navigate to Google, fetch the information using CSS selectors, collect the information, format it... you know what I mean? It's usually a lot of logic. Open new terminal in VScode Instead, we can use a web scraping tool to reduce costs, implementation time and maintenance. By sending our request through ScraperAPI structured data endpoints, we will be able to retrieve PAA questions for any query without worrying about parsing HTML, crashes, or any other issues we may face. To get started, create a free ScraperAPI account and go to your dashboard to copy your API key. Next, we will create a payload like this: payload = { 'api_key': 'YOUR_API_KEY', 'country': 'us', 'query': 'keyword+research', } The country parameter will tell ScraperAPI where to send your requests from - remember that Google shows different results depending on your location - while the query parameter contains your keyword. With these parameters ready.

RequestsStep 3: Printing the PAA questions If we print(this is the information we get: Printing PAA questions As you can see, the tool returns the entire SERP as JSON data, and the top questions related to the query are inside the "related_questions" key. Since we provide thailand telegram database are getting structured data rather than raw HTML data, we can select specific elements using their key name: serp = response.json() all_questions = serp['preguntas_relacionadas'] for paa in all_questions: print(paa['pregunta']) We store the entire JSON response in a serp variable We get the "related_questions" and create a list of items, where each item is a PAA question. To get the questions, we loop through the list and print only the "question" key.

The result is a list of PAA questions printed to the console: print PAA questions This is important because it is the basis of the CSV file and will help us identify where the query is coming from when we expand the scraper to thousands of keywords. For the last step, we are going to create the CSV file using Pandas for easy export: Run code Step 5: Compiling PAA scaled questions Of course, getting the questions for a single keyword can be done by hand, so how do we scale this project? Well, here's the beauty of web scraping. It's all about the loop. First, create a list of your desired keywords: keywords = { 'keyword+research', 'keyword+tracker' } We will then put all of our previous code inside a new loop, which will take each term within the keyword list and run through the entire process. Here is the final, complete code snippet: