Web Scraping Use Cases For Small Businesses (With Code Snippets!)

The world is littered with small businesses trying to survive.

400 million, to be exact (2023).?

These businesses are the backbone of the world’s economy, generating the largest share of jobs. However, despite their importance, only 78.5% of small businesses survive their first fiscal year ... and half of them reach their fifth.

It is said that the most common reasons for failure are low sales, lack of market research, and unsuccessful marketing—which, in hindsight, can be saved through web scraping and prediction analysis.?(Learn the basics of scraping.)

Here's couple of use cases for web scraping (with code snippets!) to get you on your competition's level in minutes. See the full five use cases for SMBs here.

1. Price Intelligence and Competitor Gap Analysis?

A great pricing strategy ensures you’re optimizing your products’ value and bringing your profit margins up without hurting your sales or alienating your customers.?If you're in the ecommerce industry, for instance, the competition is high. You need to be on top of market shifts and trends, your competitors’ pricing models, as well as consumer behavior. You need insights into all these areas.

By scraping marketplaces like Amazon and Google Shopping, or competitors’ pricing pages, to name a few, you can automatically retrieve this information.

Here’s a ready-to-use script you can use to gather pricing information from Google Shopping within a specific search query. You can see the outcome here, as well as considerations for using this script to get the best success.

import request

import json

products = []

payload = {

??‘api_key’: ‘YOUR_API_KEY’,

??‘query’: ‘gps+tracker’

??}

response = requests.get(‘https://api.scraperapi.com/structured/google/shopping’, params=payload)

product_data = response.json()

for product in product_data[“shopping_results”]:

???products.append({

???????‘Product Name’: product[“title”],

???????‘Price’: product[“price”],

???????‘URL’: product[“link”],

???})

with open(‘prices.json’, ‘w’, encoding=‘utf-8’) as file:

???json.dump(products, file, ensure_ascii=False, indent=4)
        

2. Stock Market and Financial News Monitoring

Small investment firms know how important data is to making high-performing, low-risk investments. With accurate data, you might avoid missing out on big opportunities or fail to predict and prevent losses that could set you back.

Using web scraping, you can build?alternative data pipelines?to gather as much data as possible about the companies you’re looking to invest in. Some of the key data points you can collect include product data, stock prices, and news.

Here's a ready-to-go script to collect stock data from Investing.com. (We used this example to collect data for Nike, Coca-Cola, and Microsoft—see more!)

import?request

from?bs4?import?BeautifulSoup

import?csv

from?urllib.parse?import?urlencode

urls?=?[

???‘https://www.investing.com/equities/nike’,

???‘https://www.investing.com/equities/coca-cola-co’,

???‘https://www.investing.com/equities/microsoft-corp’,

???]

file?=?open(‘stockprices.csv’,?‘w’)

writer?=?csv.writer(file)

writer.writerow([‘Company’,?‘Price’,?‘Change’])

for?url?in?urls:

???params?=?{‘api_key’:?‘YOUR_API_KEY’,?‘render’:?‘true’,?‘url’:?url}

???page?=?requests.get(‘https://api.scraperapi.com/’,?params=urlencode(params))

???soup?=?BeautifulSoup(page.text,?‘html.parser’)

???company?=?soup.find(‘h1’, {‘class’:‘text-2xl font-semibold instrument-header_title__gCaMF mobile:mb-2’}).text

???price?=?soup.find(‘div’, {‘class’:?‘instrument-price_instrument-price__xfgbB flex items-end flex-wrap font-bold’}).select(‘span’)[0].text

???change?=?soup.find(‘div’, {‘class’:?‘instrument-price_instrument-price__xfgbB flex items-end flex-wrap font-bold’}).select(‘span’)[2].text

???print(‘Loading :’,?url)

???print(company,?price,?change)

???writer.writerow([company.encode(‘utf-8’),?price.encode(‘utf-8’),?change.encode(‘utf-8’)])

file.close()
        

If you want to learn how we built this script ?? read our?stock price scraping guide. Follow our thought process step-by-step to create your own scraper!

With web scraping, your data collection opportunities are endless. It’s a useful, scalable solution for any small business trying to improve marketing, sales, and research output—without burning through budgets. It can significantly help you lower costs by optimizing decision-making and automating repetitive tasks.

Ready for more scripts? This article is filled with projects for small businesses. If you sign up for ScraperAPI, you’ll get 5,000 free credits to try each one out!

___________

Like what you see?

Keep subscribing for the latest insights and tips. Until next time, happy scraping!

Your ScraperAPI Team! ??