Scrape competitor prices from eBay

As a seller on eBay it’s crucial to understand how your competitors are pricing their products. Consistently collecting all pricing data from eBay can help you adjust your prices and learn which products are in demand.

Web scraping is one technique that will help you with business and pricing intelligence. If you are manually copying and pasting products from eCommerce websites – scraping will save you a lot of time. You can scrape a list of products from multiple webpages into a CSV/Excel file in 10 minutes with a web scraping tool.

A web scraping tool can help you:

  1. Get prices for competing products on eBay, so you can price your products accordingly
  2. Find out which products are in demand, have the most bids or are out of stock
  3. Monitor the best and worst reviews for products in your niche
  4. Stay on top of new sellers and products
  5. Save time copying and pasting pricing data from the web

ParseHub is a browser extension for scraping dynamic, Javascript and AJAX websites. Using ParseHub, instead of a price comparison or competitor analysis software, gives you the flexibility to choose which data to extract from eBay or similar eCommerce websites. You can also use ParseHub to automate tasks, such as searching for competing products and calculating price differences. You can also get products by searching through a list keywords.

In this tutorial I will show you how to:

  1. Install a browser extension plug-in for web scraping
  2. Scrape a list of products from eBay
  3. Get the name of each product and enter it into the search box, to search for competing products
  4. Scrape the list of competing products for each product
  5. Scrape the products into a CSV/Excel file

Step 1: Get started with web scraping software

Download the ParseHub web scraping tool and open products in the triathlon category of eBay. You can also open any eCommerce website to scrape.

  1. Install the ParseHub browser extension by going here
  2. Open http://www.ebay.ca/sch/Triathlon-/2918/i.html in the ParseHub desktop application
  3. Click on "New Project" and "Start project on this URL".

Step 2: Scrape a list of products from eBay

Select and extract the names of all of the products on the page. This process is the same for any eCommerce website that shows products in a grid or list layout.

search through keywords on eBay with web scraping

  1. Select the first product on the webpage by clicking on it. Click on the second product to select all 50 products on the page.
  2. Rename your selection products. You will now have the name and URL of all 50 products extracted.

You can do more on this page with web scraping:

  • Use the relative select tool to get the image url, price and any other details for each product.
  • Use the extract tool again and select “href” from the dropdown instead of the text. This will get you the link for each product on the page.
  • Select all of the links at the bottom and navigate through the pagination.

Step 3: Search for products on eBay to find competitors

Set up ParseHub to search for each product that we just scraped. ParseHub will get all of the competing products listed on eBay.

search through keywords on eBay with web scraping

  1. Click on the "plus" button next to the command "Begin new entry in products to open the command menu.
  2. Add a Select command and click on the search bar at the top of the page to select it. An Input command will also be added.

    enter a list of product names with web scraping on eBay

  3. In the input value text box type in name. From the Input type dropdown, select “expression”. This will tell ParseHub to input the name of each product into the search bar separately. Rename your selection search_bar.

    navigate through products on eBay with a web scraping tool

  4. Switch from Select Mode to Browse Mode, by flipping the switch at the top of the toolbar.
  5. Copy and paste the first product name that you see into the search bar. This will give ParseHub an example of what to search for.

    navigate through products on eBay with a web scraping tool

  6. Switch back to Select Mode.
  7. Click on the plus button next to the "Begin new entry in products command and add another Select command.
  8. Click on the "Search" button next to the search bar to select it. Rename the sealection search_button.
  9. Click on the plus button next to the Select and Extract command to add a Click command.
  10. In the Click configuration pop-up window, choose to Create New Template and call the template competitor_products.

Step 4: Scrape the prices of competing products

On the new template that we just created we will select all of the competing products listed. We will also get the price for each product.

  1. Click on the first two products to select all of the products on the page.
    Note: if there is only one product on the page then you will have to search for a different product with more competitor products.
  2. Rename your selection competitors.
  3. You will automatically have the name and URL of each competitor product extracted.

    scrape a list of competitor prices from eBay

  4. Add a Relative Select command by clicking on the plus button next to "Begin new entry in competitors.
  5. Click on the first product name. Now, click on the price. Notice how all of the prices on the page are now selected and extracted for each product. Rename your selection price.

You can do more with web scraping!

  • Navigate to the details page of each product.
  • Use ParseHub to extract more information about the seller, items in stock and manufacturing details.
  • Do simple calculations with ParseHub. Find the difference between the original product prices and their competing products.

Step 5: Download competitor data in a CSV/Excel

scrape a list of products into CSV/Excel

  1. Click the green “Get Data” button. Click “Run Once” on the next page.
  2. Save the project and click “Run on Servers”.
  3. Use the blue button at the top to refresh the page. You should also get an email when the run is finished.
  4. Click the CSV icon to download your results in CSV. If you have Excel, the product
    data will open in Excel automatically.

You can do more!

  • Get a new product and pricing information consistently on a schedule. Just click the “Run of Schedule” button and select how frequently you would like ParseHub to scrape eBay.
  • Connect to the ParseHub API and feed pricing and product data into your mobile or web app.

A web scraping tool can scrape a list of products from eBay and track availability, sellers, prices and shipping costs and timelines. Now you can monitor price fluctuations and find profitable items to sell on eBay.

Do you want this project right in your ParseHub account? Email me at angelina[at]parsehub [dot]com and I will send you the project file! Any questions, ping me!