How to prepare for migration

Overview

Migrating to utilise the new international standards of locale formats for dates, times and numeric values is a major upgrade that requires careful evaluation and planning. Please take note of the following breaking changes, known issues and guidance on how to understand the configuration impact

The schema available by this link can be used for getting an overview of the rules of applying timezone and locale settings across the system:

https://docs.google.com/presentation/d/1t92r2PCV4VBjQ6qie9uU18BEdY2Y22TMPNS9pHwK0E8/edit?usp=sharing

Note! Even though "System" - level settings are specified to be applied for the scheduler jobs, the site/user locale/timezone settings will be applied in case those are specified in scheduler job configuration parameters.

Even though it is generraly not recommended to change System locale settings, it can be done via JAVA_OPTS as a temprorary workaround if needed locale is not applied:

After the upgrade, the customer database will no longer be compatible with previous versions. In case a rollback is required during the upgrade please ensure a copy of the database is taken prior to the upgrade to ensure it can be restored.

How to evaluate configuration impact

Taking into account that Timezones functionality bears relatively high risk on existing systems and integrations, it is recommended to follow these steps after upgrade to evaluate the impact of the changes on the system in question:

Before switching to version 9.101 or higher we recommend evaluating your configuration to understand how the system will be adjusted to utilise locale formats for dates, times and numeric values.

Site Configuration > Health > Fields Overview will provide an overview of the formatting settings before and provide a report of how the dates, times, and numeric values will be adjusted after the migration script. Any configuration changes can be applied prior to the upgrade and migration.

In addition, we recommend following these steps after upgrade to evaluate the impact of the changes on the system in question:

• Check date field data representation on UI (both classic and new UI)

• Check validation and formatting rules based on date fields

• Check calculation fields using specific functions modifying original date field values.

• Use the new functionality added in 9.100.1 to list all fields of a certain type across all sites- this to gain an overview of what and where such checks should be performed.

• Check import of date field data is working correctly

• Check integration via API is working correctly

• Due to data migration being performed during the upgrade, it will not be possible to downgrade a system, once it has been upgraded.

Breaking Changes and Known Issues

Time Zone

• During the upgrade, the time zone for each site will be set following the time zone set for the server. The system administrator can adjust the Locale and Time Zone by editing the general site settings.

• Encodify supports three levels of locale and time zone settings; Server, Site, User. All dates and numeric values within a site will follow the user's local and time zone settings. If a user has not chosen a locale and a time zone, the settings from the current site will be applied.

• If the time zone of the user is different from the browser, a pop-up will prompt you to either update user settings to the time zone of the browser or keep the current time zone. Read more about how to change your time zone in the following helpdesk article

• Locale and time zone settings set for a user are applicable to all sites. This means that changing the settings for a user on 1 site will change the settings for all sites for this particular user.

• The localisation page located in Site Configuration > Health > Localization can be used to check actual locale and timezone settings for System, Site and currently logged in user. This can be use

Date Fields

• Historically, Encodify supported four different date field types: date, date-time, week-year, month-year. When upgrading these will be migrated into a single date/time field type, which then via formatting can represent itself as both date or DateTime.

• After upgrade, locale setting for site and user will be set from the date format settings used before upgrade: If the date format on a site before the upgrade is 'US' - locale 'en-US' will be set, if the format was 'international' - the system will pick the locale that corresponds to the language of the site. In case matching locale could not be found, default locale 'EN-DE' (English/German) will be set.

Before Upgrade:

After Upgrade:

• A skeleton can be selected to define the required format to extract each element of a date/time, and the locale will then determine how those elements are displayed and which separators are being applied.

• If the list of supported skeleton formats does not provide the format you require then a custom format can be specified within the configuration of a date fields.

• Not all custom skeleton formats will be supported by the date-picker and therefore it's important to verify that entered skeleton is displayed correctly.

• It is important to note that custom skeletons are case-sensitive. e.g. dd-mm-yyyy will not work, dd-MM-yyyy should be used instead.

• The new locale formats for week/year fields might be different after the upgrade where they are now correctly formatted based on the international standards.  

• First day of week can be displayed incorrectly for some locales in date picker and calendar widget. For example, 1st week day for Australia (en-AU locale) will be Monday, instead of Sunday. This is a current limitation coming from the 3d party library used.

• It is not recommended to change a field type from date to DateTime after the upgrade as this can cause data inconsistencies. For example, Harry Donovan from Copenhagen enters a deadline date of 01/03/2021, if this is changed to a DateTime field then the date will be recorded as 01/03/2021 00:00, and when Sophia Perez from New York views the same deadline date she will see the deadline as 28/02/2021 18:00.

• For configured "Item Expiration" and "Release date" functionality: Please note that in case exact specific moment should be considered as item expiration or release, date-time field format should be used. For the field of the type date, specific date will be used across all timezones.

• Classic UI, Filters panel: Focus is lost after selecting a value from datepicker.

• Can’t translate “today” button in datepicker.

• Before using custom formats it is important to ensure that the format is supported in date picker (for instance, formats with Z/z/time, like "yyyyMMMMddHHmmssZZZZ" are not supported in date picker).

• Entered date changes after changing field skeleton

• Inconsistent values of numeric fields in Change log

• Classic UI, Filters panel: Page refreshes each time after inserting a value in datepicker.

• New UI, Filters panel: Datepicker closes sporadically during editing.

• Datepicker FireFox and Safari): Cannot update a time in datepicker (Workaround: delete previous time, then insert a new values).

Numerical Fields

• Numeric fields format will follow the user's locale settings. It will no longer be possible to specify locale(format) on field definition level.

Before Upgrade:

After Upgrade:

• Validation of entered decimal separator matching locale is no longer present. This change is applicable to both classic UI and import. Example: Field number format == International (1.234.567,89) > User enters "1.25" > Behavior before 9.101: Validation about incorrect number format is displayed.  Behavior starting from 9.101: 125,00

AdBuilder and AdBriefing

• AdBuilder text elements do not currently support applying locale or time zone formatting for date and numeric data entered in AdBuilder. Values will be displayed as they have been entered.

Calculations

• It is important to keep consistency for the format of the date fields when used in calculations to avoid inconsistencies in calculation results after upgrade. Source date field used in calculation should have the same skeleton(format) as calculation field itself and vice versa. Namely, if source field is of the type date, calculation field should also have date as format (not dateTime).

• Calculation fields using functions that modify/transform date field values might be broken in case the format of the date has changed. It is recommended to review the configuration to ensure it works.

Rules

• Starting from release 9.101 use of milliseconds and timestamps without zones in rules is deprecated. It is recommended to use literals or ISO timestamps with time zone instead in rules and DSL query configuration. E.g.  Instead of CURRENT_TIMESTAMP() + 432000000 or CURRENT_TIMESTAMP() - 240*60000,  startOfPeriod(addDate(CURRENT_TIMESTAMP(), n, ‘days’) should be used.  Please, also note that after upgrade to 9.101, for existing rules/DSL queries using milliseconds in date queries site time zone will be applied, which might result in date value shift. Therefore it is recommended to evaluate if such rules/DSL queries still work as expected and change to the recommended format to avoid inconsistencies.  Using timestamps without time zones will also be deprecated in rules and in query DSL, e.g  2021-02-03T15:27:20 will be deprecated as it contains no time zone in the timestamp. 2021-02-03T15:27:20Z or 2021-02-03T15:27:20+01:00 should be used instead. Read more about rules in the following helpdesk article

• Site Configuration > Health > Rules Overview page can be used to get the list of rules that need special attention. Page is available in version 9.101.1 and later.

• Site timezone is used for rules/DSL queries evaluation if timestamps without timezone or milliseconds are specified is used as criteria.

Importing

• When importing both date and numerical field values its important to ensure that the value being imported has the same format as the target field definition, including separators. E.g. 03/02/2021, 08:00:00 will not be imported into the field with format 03-02-2021 08:00. If the values cannot be matched then the import settings can be adjusted specify the local and format to be provided. Read more about how to import numeric and date values in the following helpdesk article.

• BulkUserImport: as previously, dates should be in ISO 8601  format. e.g. 2021-01-01. Read more about rules in the following helpdesk article

Actions, Exports and Reports

• Scheduled reports and actions do not support time zone. The server's time zone is always used for triggering events. User-generated reports will adjust dates based on their locale and time zone preferences.

API

• For use of the External API, the Locale of the user used for authentication will be applied.

System Logs

• Numeric/Date field values in the item changelog will continue to be displayed in the format they were saved in as they are stored as text. User time zones are not applied either.

• User’s TZ&Locale is respected in System Logs (for example: Login Log, Site List etc.) if locale for user defined, if it is not defined - system's TZ&Locale will be used.

Master Modules

When upgrading, we automatically create the below master modules:

1. Locales

2. Timezones

They will be created with a locked metadata schema preventing the ability to change or rename field values. Additional custom fields can be added to these modules where required.

If the User Management Module is not enabled and configuted for the site, it will be created and mapped automatically during upgrade.