File Validation and Extraction

  • Published on Feb 10, 2025
  • 4 minute(s) read
  • Migration Team
 Prev Next

File Name Mapping and Text Extraction

Configuration

When uploading a file it is possible to analyze its name to validate if it conforms to the specific rules and then set the corresponding values to the mapped fields.

Currently, file name validation and extraction supports the following checks for the options:

  • option is generally available for selection in the field

  • option is available for a current user by access restriction (e.g. groups, status in target module in case of module link fields)

  • option is available for selection by the field dependencies

  • option is available for selection by the filter configured in module link field definition

The feature can be configured in the field definition settings of the File field.

Screenshot_2019-11-05_at_16.16.34.png

Parameter

Description

Enable File Validation and Extraction

Enables the feature

Delimiting regular expression

Regular expression to split the file name into several logical parts

  • _ (underscore)

  • - (dash)

  • \s (any space character) 

  • \W (any non-word character, 

    including non-alphanumeric characters and underscore)

File name mask

A mask which represents a typical file being uploaded:

var1var2var3

Ex.: region_company_unit

where

  • region, company, unit - custom variables

  • _ - delimiter

So, the file name  Europe_CNN_Design.jpg will be split by _ delimiter and associated with variables:

  • region = Europe

  • company = CNN

  • unit = Design

Add mapping

Allows mapping the variable extracted by splitting the file name to some existing field in the module.

  • Variable - variable name used in the File name mask

  • Field - module field the variable to be saved to 

  • Linked field Id - when mapping to the module link, it is possible to additionally specify the field from the linked module the variable to be compared to. If Linked field Id is left empty, by default the pointed field will be used. Linked field Id is not applicable to other field types except module links

Supported field types for Linked field Id are: text, date, searchable concatenation

  • Required - by default the mapped variables are mandatory though it is possible to make some variables optional

Ex.

Mask: region_company_unit
unit - optional variable.

Result:

File region1_company1.jpg will pass the validation
File region1_company1_unit1.jpg will pass the validation
File region1.jpg will not pass the validation

Only the last variable can be optional, otherwise, the system will not be able to determine which variable is absent

  • Apply order - the order in which fields will be evaluated for validation. This setting is important when validation of options respecting field dependencies or module link filters is required. For example, if Company field has Region field as a filtering parent, the Region field should preceed the Company field by the Apply order settings in order to validate the company values that do not belong to the selected region.

 

Where and How Validation is Applied

Currently, validation is supported in the following places in the system:

- Bulk File Upload window

- Item info sheet (for both new items and already existing ones)

- Copy item window

Where and How Extraction is Applied

Currently, extraction is supported in the following places in the system:

- Item info sheet (for both new items and already existing ones)

- Copy item window

Please note: File Name extraction can be applied only for file fields with limit=1

File(s) with invalid name(s) will be shown as the error messages that explain which part of the name is wrong or whether the file matches the mask at all.

It is also possible to download the list of failed file names as CSV file (functionality is only applicable to the Bulk File Upload window).

Example of the file name validation in the Bulk File Upload window:

Screenshot_16.png

Example of the file name validation in the Bulk File Upload window:

Screen_Shot_2019-10-29_at_16.20.55.png

File Field Validation

Forbidden Characters in File Names

In addition to validation of the file names against the mapped fields and their values, it is also possible to specify the list of illegal characters in the file names. 

Screen_Shot_2019-10-29_at_16.27.15.png

To enable the feature, perform the following steps:

  1. Select the "Enable File Validation and Extraction" check box in the File field definition window.

  2. List the characters that are not allowed in the uploaded file names in the "Illegal characters in file name" field in the File field field definition.

Characters list should not be separated by comma, but listed as is. In case you want to specify whitespace as a forbidden character, it is recommended to just type the space in the middle of the other characters for better readability of the settings.

For the newly-created File field definitions, the following default list of forbidden characters will be pre-filled

!@#$%^&*()+= 

The above described functionality is applicable to both file upload in item info sheet and bulk file upload.

Limitations

  • For Linked Field Id only the following field types are supported: text, date, searchable concatenation. Module link is not supported.

  • Only the last variable can be mapped as optional ("required" unchecked).

  • Validation is triggered only on file upload (in all places), previously uploaded and saved files are not validated.

  • File name validation and extraction is allowed for multiple file filed with limit=1

Allowed/Not Allowed Extensions Validation

It is possible to specify the list of file extensions that are allowed/forbidden for upload. Configuration of this validation is available for both single and multiple file fields.

To enable validation specify the comma-separated list of allowed/forbidden formats for upload, and enter text that will be displayed in validation message.

Screenshot_2020-07-03_at_13.06.08.png

Please, note that double extensions are currently not supported. It means that if you need to validate for e.g. .tar.gz extension, .gz should be specified in the allowed/forbidden list.

Max Number of Files for Upload for Multiple File Fields

As per release 9.97, this functionality is controlled by the feature flag in webapp.yaml and is disabled by defualt.

To enable this validation feature, add the following to the webapp.yaml file.

encode:                            
  webapp:                                
    field-definition:              
      multiple-files-limit: true

Default limit is 1000.

Screenshot_2020-07-03_at_14.30.42.png

Please, note: Currently, in case limit is exceeded, "Item was not saved" standard growl error will be displayed, with descriptive exception in server log.

Related articles