Web Crawler & Web Scraping

Change big data to asset

What is Keywalker Web Crawler?

Based on the automatic operation technology of the browser developed by Key Walker independently, various document files on the Internet It is a service to collect according to rules.
The page information collected by the crawler visiting the target site is extracted by Web scraping service for each item.
For example, in the information collection of the EC site, the category and name of the product, the model number, the price, the number of inventory etc. In collecting sales information, company name, address, telephone number, type of business etc.
Disassemble and organize the information of each item and store it in the database.

Sites to be crawled

1620k site

Development experience 13 years, 620,000 site above, there is a 10 billion or more pages of crawling experience.

Introduced breakthrough


From large to small-scale, it has been introduced as a Web marketing tool of the various companies.

Service continuity rate


In many companies and organizations as a new infrastructure of the big data era it has been continuously utilized.


Features of the Web crawler

Cloud-ReadyCloud-Ready Because the cloud-enabled that can be set from anywhere at anytime
Parallel crawlingParallel crawling Also supports crawling tens of millions, several hundreds of million page scale by the parallel crawling
Crawl corresponding to any siteCrawl corresponding to any site The browser auto-manipulation language of its own development that Keywalker.js, can also extract information from a dynamic site
Flexible crawling setting functionFlexible crawling setting function
  • IP distributed setting, the Tour Schedule setting Allowed
  • Level document and a string with possible
  • Crawl route adjustment function
You can acquire a variety of document file formatYou can acquire a variety of document file format HTML、RSS、SITEMAP、PDF、Office、and so on…
Tableau cooperation (optional) The acquired data, in conjunction with Tableau (BI tool) analysis and visualization by

It offers a series of solutions for big data

In the key Walker, as well as Web crawling, related to big data,
collection, sorting, analysis, visualization, additional functions, offers a series of solutions to operation.


Flow of up to quote

  • STEP 1

    Hearing investigation of workflow in the field (free of charge)

  • STEP 2

    Rough estimate of when it is automated (free of charge)

  • STEP 3

    Based on the workflow definition, building automation system

  • STEP 4

    The installation process (user PC environment or our cloud environment,)

  • STEP 5

    Get started

Click here for information