Octoparse vs Import.io: which is the best for web scraping and extracting data?

Web scraping software, also known as a data extraction tool, is the software used to collect the data from the website. It’s usually not easy for us to pick up a web scraping tool as there are so many web scraping tools available now (refer to Top 28 Free Web Scraping Software to learn more). That’s why I decided to put Octoparse head to head with import.io to see how the two tools compare. Here is everything you need to know when deciding which web scraping tool best suits you.

Feature Comparison

Here is a general comparison between Octoparse and Import.io features:

What could both web scrapers do for you?

Both interfaces are built according to the point-and-click principle, which is easy for you to extract data without coding. Both of the scrapers could deal with Javascript and AJAX pages and are able to scrape behind a login. Like a bot, they could follow the links to go into the deeper web pages by clicking the items and extracting the data on the other pages. Also, they are able to get data in CSV format and transform data by manually modifying Regular expression or XPath.

They all provide cloud services, which are able to offer API options, IP rotation and services to schedule extractors running in real time. With that, it is easy to get up-to-date data regularly without having to keep your computer on.

What could Octoparse do for you?

The biggest difference between Octoparse and its web scraping alternatives is that Octoparse can get data from interactive websites. It totally mimics human behaviour when browsing a website.

You can instruct Octoparse to scrape data from very complex and dynamic sites, because it can:

• Sign in to accounts to scrape behind a login
• Select choices from dropdown menus (single and multiple), tabs, pop-up windows
• Enter keywords and search with a search bar
• Go to a new page simply by clicking on the “next” button
• Get data from infinitely scrolling pages
• Able to input Captcha on local machine
• Visual workflow to understand the logic of the scraper (Variables, loopsand conditionals) and could be changed easily with point-and-click interface
• Smart mode to deal with a simple website just by entering the target URL
• Extract inter and outer HTML and attributes and customize the values for further extraction
• Advanced RegEx tool and XPath tool to modify the regular expression or XPath, which means you don’t need to know how regular expression and XPath are written(see the screenshots below)

And more! Except for the first one, these are all things that import.io cannot handle.

Here is the full list of Octoparse’s scraping features:

The downside of using Octoparse as an alternative to import.io is that you need to install the application on your own computer. And because the software is written in .Net, it only supports Windows system. A Mac visual machine is needed if you want to run Octoparse on Mac. You would also be annoyed if the Internet is unstable and the scraper stopped unexpectedly, you need to rerun the crawler from scratch. The other one is that it may take longer to learn Octoparse for you to easily make mistakes if you don’t understand the logic of the workflow. Luckily, there are plenty of tutorials and great support if you get stuck!

Besides, Octoparse is not able to extract the images and files directly; you need to extract their URLs and download them with other applications. And the function of API is quite limited.

What import.io could do for you?

First of all, import.io is a cloud-based platform, which means you don’t need to run the scraper on your machine and the data could be kept in the cloud. Therefore, you can access your data from any computer connected to the Internet. Also, you don’t need to worry about scraping process maintenance and scalability.

Unlike Octoparse advanced mode, import.io tries to guess what you want from the page, and would build an extractor for you in just a few seconds. Other features include:

• Connect one data source with another and thus produce new, valuable, real-time data sets
• Integrate with Google Sheets and Tableau
• Able to extract images and files
• API integration

Here is the full list of Import’s scraping features:

The downside of using import.io is that it’s not as widely used as Octoparse to deal with websites. As mentioned above, it couldn’t deal with websites with dropdown menus, pop-up windows and captcha. It’s also not possible to scrape with infinite scrolling pages, which are quite common for most web pages. There’s also no scraper logic like conditions for further extraction to specifically locate the web page or items.

And for pagination, it’s not easy as you need to enter a list of pages. As for transforming the data into regular expression and XPath, there are no built-in tools for you and you need to enter the expression yourself, which means you need to master XPath and regular expression if you want to explore more on import.io.

Cost Comparison

There’s no doubt that Octoparse has overwhelming advantages. It provides a free version with powerful functions! To summarize, that is:

Import.io Pricing 

Octoparse’s plans are limited by:

• the number of crawlers
• the number of crawlers you can concurrently run on your machine
• the speed at which you can collect data (different cloud servers)

There are unlimited pages for each crawler and unlimited computer license for each version, including the free one.

(Note: When you enter URLs in URL list, it would suggest LESS THAN 20,000 URLs. All versions are limited to numbers such as Octoparse and need to ensure that the CPU runs the crawler at one time. But you can copy the crawler to extract the rest URLs.)

Import.io’s plans are limited by: 

• the number of queries per month or year
• the expiry date of the queries
• limited functionality like image and file download, API, and up-to-date reporting
• support

It’s sad to find out that import.io doesn’t provide free version anymore.

Most people build one or two crawlers per website on Octoparse. One is to extract the separate web pages URLs and the other one is to use URL lists to bulk extract the data with the extracted URLs. It’s highly recommended when using the cloud service.  

On the other hand, Import.io counts an extractor as one query and it doesn’t provide URL lists to bulk extract the web pages. Therefore, you either need to spin over these separate web pages in one extractor (which usually means missing data) in import.io, or you need to upgrade your version for more queries.

For both Octoparse and Import.io, you have to subscribe to a premium plan for scheduling feature —— the ability to collect data from a website continuously on a schedule (real-time, daily, weekly, monthly).

If you don’t want to learn how to use a tool and just want your data on demand, both Octoparse and Import.io provide data service extracting data for you. Just contact the sales of both companies and they will scrape the data from the website you want —— delivering them in CSV/Excel or API format.

Conclusion

It is not difficult to start a project either with Octoparse or import.io. They all deal well with both static and dynamic websites. XPath and regular expression are needed if you want to explore more, though they claim that no programming knowledge is needed. Also, both have their limits.