Skip to main content

Projects

Introduction​

Projects in ScrapeHero Cloud allow you to customize and manage your data scraping tasks. This guide will help you understand how to create, configure, and manage projects effectively.

Creating a New Project​

  1. Go to the App Store in ScrapeHero Cloud.
  2. Find the scraper app you want to use.
  3. Click "Create new project".

Project Configuration​

When setting up a project, you can configure the following:

  • Input URLs: Provide the URLs of the product/listing pages from which you want to collect data.
  • Website-specific filters: Set any additional filters required for your scraping task.
  • Data Record Limit: Optionally limit the number of records to scrape per run.

Tip: Name your project based on the input URLs or the specific data you're gathering. This makes it easier to identify projects in your dashboard.

Running a Project​

After configuration, you can:

  • Run the project immediately, or
  • Schedule the project to run at a later time.

Project Details Page​

The Project Details page contains several sections:

1. Overview​

  • View the status of different runs for your project.
  • See the number of data records collected and credits used per run.
  • Access or download the gathered data in Excel, CSV, or JSON format.

2. Settings​

  • Modify project name, input URLs, and other configurations.
  • Adjust the limit on data records to be gathered per run.

3. Schedules​

  • Set up recurring project runs.
  • Specify run times and optional end dates for schedules.

4. Integrations​

  • Connect to various integrations available on the ScrapeHero Cloud platform.
  • Note: Some integrations require a "Standard" plan or higher.

5. Credit Usage​

  • Track data credit usage for the project over time.
  • View total used credits, total runs, and data records collected for specified dates.

6. Instructions​

  • Access documentation and instructions related to the scraper app.
  • View the dataset schema for the project.

Depending on your needs, you can either:​

  1. Create multiple projects: Useful when scraping different types of data or from different sources regularly. Example: Create separate projects for scraping laptop data and smartphone data from Amazon.

  2. Use a single project with multiple runs: Suitable when scraping similar data with changing inputs. Example: Use one Amazon Product Details Scraper project, changing the input URLs between runs for laptops and smartphones.