“WP Crawler” Documentation by “WP Crawler Team” v1.0


“WP Crawler”

Created: 10/26/2015
By: WP Crawler Team
http://codecanyon.net/user/wpcrawler

Thank you for purchasing my plugin. If you have any questions that are beyond the scope of this help file, please feel free to email via my user page contact form here. Thanks so much!


Table of Contents

  1. Installing Plugin
  2. How to Add a Source
  3. Adding Example Source
  4. Import / Export Sources
  5. Running Crawler
  6. Sources and Credits

A) Installing Plugin - top

  1. Log into your WordPress blog admin area:

  2. Once logged in, click "Plugins" on the left hand menu (If you don't have a "Plugins" option you will have to ask a blog administrator to do this and the following steps!):

  3. Once, on the "Plugins" page click the "Add New" button in the top middle of the page:

  4. On the next page "Install Plugins", click the "Upload Plugin" button in the top middle of the page:

  5. On the next page you will be asked to "Choose File" - Click this button and here you must select the .zip you have downloaded from CodeCanyon, once you have selected that .zip file Click "Install Now":

  6. The next page will be entitled "Installing Plugin from uploaded file (e.g. wp-crawler.zip) on this page click "Activate Plugin":


B) How to adding a Source - top

  1. Click "WP Crawler" on the left hand menu:

  2. Once on the "All Sources" page click the "Add New" button in the top middle of the page:

  3. Type source's title into the Title field (We will using this later when running crawl)

  4. Now you need to analyze the webpage that you need to crawl. Then insert value in the fields below to convert into WordPress content:

    Each field you must input 2 values: "Selector" and "Attribute":

    • Selector: Unique pattern used to select an element you want to get content (Reference)
    • Attribute: Additional information about an element you want to get from selector. If you want to get content between opening and closing tag, you can input: innertext on this field.
      Accepted attributes: outertext, plaintext, href, src

C) Adding Example Source - top

To help you understand the analyzing process and adding new source, in this part I will use the example by trying analyze & add "Stackoverflow" source.

  1. Firstly, I will analyze listing page: http://stackoverflow.com/questions. I need to select elements that include link to the posts:

  2. I'm using Google Chrome Browser, and right-click on the link and choose "Inspect Element" to find out unique selector of this element

  3. In the image below, I realize that selector #questions .question-summary h3 a will be unique.

  4. You can re-check this selector is unique or not by right-clicking on browser, choose "View Page Source" and search id="questions". If the result appears once, so, that element is unique.

  5. After analyzing, I come back to WordPress Dashboard, in "Add New Source" and insert this selector in Listing Page > Selector:

  6. Next, I will analyzing in the same way with other parts in child page. And insert into Single Item’s fields like the image below (If you can’t find the field’s selector, you can let it be blank):

  7. Now click Publish this source and it will be ready to crawl on next step!


D) Import / Export Sources - top

Sources can be Import/Export like Posts or Pages. You can using WordPress Import/Export under Tools to Import/Export Sources. We already provide a sample-sources.xml file (located on plugin folder) to help you understand how it works.


E) Running Crawler - top

  1. Click "Run" inside the "WP Crawler" menu:

  2. Choose "Source" you have added on previous step
  3. Choose "Type" to select insert Single or Multiple items
  4. If you choose Single, you must input single item URL to "URL" field below.

  5. The left-over fields help you setting some additional values after crawl this post into WordPress.
  6. Below is another example I need to crawl multiple item from stackoverflow:

  7. After finishing setting, click “Run Crawler” to start crawling.
  8. Then when it’s “Well Done”, that mean your work is finished, click "Posts" on the left hand menu to see the result:


F) Sources and Credits - top

I've used the following resources, libraries as listed.


Once again, thank you so much for purchasing this plugin. As I said at the beginning, I'd be glad to help you if you have any questions relating to this plugin. No guarantees, but I'll do my best to assist. If you have a more general question relating to the plugin on CodeCanyon, you might consider visiting the forums and asking your question in the "Item Discussion" section.

WP Crawler Team

Go To Table of Contents