Scheduled import

Prev Next

It is possible to schedule files extracting from specific remote folder on FTP server.

To make scheduled FTP import works it is important to configure:

  1. FTP connected service;

  2. Import settings(import items via xml/csv or import file structure)

  3. Action with new type "Scheduled import";

Action with new type "Scheduled import" have link to connected service (FTP), import setting and remote folder.

Settings for scheduled FTP import action with various configuration options displayed.

"Scheduled import action" form

Important:

  • Remote folder mentioned in config should be already created and user need to have Read+Edit access to it.

  • Path to the remote folder must be specified as an absolute path (with preceding slash /) from the server root.

  • “Remote Files Regex” defines which files would be picked up by scheduler during import.

  • Once action is triggered it removes the file from SFTP once imported (Without removing file will be imported every time import is triggered. That's why we do that in both import from file system and from ftp).

  • “Skip Processing If Modified in Last” defines period in seconds, if wile has modified during specified period  it will be skipped during import. This option was added to avoid importing files that weren’t fully uploaded to remote ftp server.Default value for newly created actions - 60 seconds.For existing actions - 0 seconds.

  • “FTP Server Timezone Shift” defines period in minutes.This option was added for cases when timezone settings in webapp server and remote server for extracting file are different.As an example sftp server has timezone in UTC, but ftp server can have timezone settings + 01:00.To adjust this gap between webapp and remote server timezones this option was added to shift timezone by specifying period in minutes.

    Pay attention: this shift should be only set after analysis actual comparisons in debug mode. Without debug it can be not clear what shift should be added. Another suggestion is to use sftp server where this shift is not needed, also more secure, reliable. If this case with shift is present then probably after changing summer/winter time this shift will need to be changed as well correspondingly.

Known limitations:

Remote Folder need to be specified in action settings, it cannot be inherited from Import or scheduler settings;

Scheduled Import ignores following options in Import settings:

  • Folder for lookup files by references

  • Ignore Files/Folders starting with

  • Ignore Mime Types

Note! Unfortunately, we could not test cases with FTPS and FTPES connections. There are problems with configuration of our server.

What we reccomend:

  • if security is important to the client - use sftp,

  • if not - use ftp.

The ftps / ftpes option has a serious compatibility issue with Java security.
Problem - Java uses a separate security channel for each new port. And in ftps / ftpes - 2 ports are used. And in the default settings, the server wants the session to be one, and Java does two.