# Parsing scheme settings

Each scheme has some not required values to set up

# Name

This human-readable name used to distinguish one scheme from another in the schemes list.

# Webhook url

The setting to send parsing result of each page to some URL address. Payload will have the next format

  "parsingSchemeId": 0,
  "parsingResultId": 0,
  "code": 200,
  "error": "",
  "pageUrl": "https://test.com/page1",
  "result": {}

result property will contain parsing result of this page. When field is filled the button "Send test data through a webhook became available".

# Threads count

Max amount of parallel requests to a resource. An integer value between 1 and 10. Too big value can create a big load to the site, so a crawling can be unsuccessful.

# Http timeout

Max timeout in milliseconds to load a response from the page. An integer value between 5000 and 20000.

# Scheduling

This setting allows to make the crawling repeatable. When the scheduling is set, the scheme has option "Enable scheduling" in the parsing schemes list. When scheduling is enabled, the scheme will run by the system automatically according to scheduling.

# Proxy type

All requests from the service are use proxies. As default the setting set in transparent mode. It is enough for the most existing sites. But sometimes it is necessary to use residential mode when the resource is very popular and have a protection from a big amount of requests

# Parsing mode

As default html code of each page loads in the system and then a data is extracting from it. But some sites uses javascript to render content of the page. Html code of such pages doesn't contain needed sections until it will not be rendered by browser. In such cases a parsing mode can be set in the "browser" mode and on the page can be chosen a element to wait. The browser will load this page, then wait while the chosen block appears on the page and then take rendered html which will be used for a crawling.

# Note: When the "browser" mode chosen, "Result JSON" block still show results for page without "browser" mode. To reload it with current configuration, close settings window and press reload page button on the scheme. The cost of reloading will be equal to the one page crawling.