Zenario 8.2

This release of Zenario builds on 8.1 with a further development of its data protection and GDPR-related features. There are also improvements to the documents system, and more.

GDPR and Data Protection features

• There is now a Consents Log, which records every time a new user or content gives consent to having their details stored
• Before taking a backup of a site that may contain personal data, administrators must now re-enter their password; this should make it harder for an unauthorised person to take a backup which may contain personal data
• We now have a feature which allows an administrator to completely delete all data relating to a user; not just data in the users tables, but also references in the email log, form response log, sign-in and content item access logs
• The Data Protection panel (in Organizer->Site Settings) now shows how many records of each type exist, and the date of the oldest record
• The DP panel now shows summary information about backups (both Zenario backups, and automated mysqldump backups) which may exist
• The Diagnostics panel now shows a warning if your Zenario site has administrators have not logged in for several weeks (which could indicate a person who's left the organisation and whose account should be trashed).

Improvements to the Documents system

We've made the Documents system clearer to use, particularly in its handling of privacy. 

When a document is first uploaded, the administrator can now explicitly say whether the document is intended to be Public, Private, or Offline. (Offline is the new name for what used to be called "auto").

Public documents are intended for free download, and have a fixed, short path to the file, which search engines are permitted to store. Private documents are only available after a user goes through a password-protected content item, whose privacy (extranet user, a specific group or groups, a smart group or groups, or a role or roles) determines the precise policy.

If the user can't decide immediately, the document is made "Offline". This means it is not available for download by any site visitor or extranet user (though administrators may access it).

When making a managed link to a document (e.g. from a Document Container, or from a Banner), it is possible to pick any kind of document, but documents will only appear when they are linked-to from a content item with the appropriate privacy level (or if not, the plugin will show a warning). For example, a document that is Public can be freely put on a content item that's Public, but if a document that's Private is placed on a Public content item then no link will appear (because a user must authenticate and be on a Private content item before seeing the link).

Document Container plugin

We've improved the Document Container plugin. It now allows you to specify filters and the sort order in the plugin settings, thereby controlling which documents are shown to visitors and in what order. 

End of Life for Contact Form plugin

The Contact Form plugin is now removed from Zenario, please migrate to the new Zenario Forms system. You should be able to start this module, create a Form, and put a Form Container plugin on pages where you need it. The Forms system has many more features, including support for data protection features.

Improvements for managing Plugins/Nests/Slideshows

We've made improvements for organising plugins, nests and slideshows.

There is now a Nest Library and a Slideshow Library, separate from the Plugin Library.

Note that the deprecated Slideshow 2 module is not included in this change, and is currently still inside the Plugin Library. A future release will remove this module and move any slideshows created using it into the Slideshow Library.

Improvements for viewing where things are used in Organizer

When you view a form, image, plugin, nest or slideshow in Organizer, you now see a list of everywhere that form/image/plugin/nest/slideshow is used.

If the form/image/plugin/nest/slideshow is just used in one place, you'll see a name and a link to where it's used.

If the form/image/plugin/nest/slideshow is used in multiple places, you'll see one name as an example, and a link to a full list of everywhere else it is in use.

Improvements to Conductor

The navigation in conductor has been rewritten to eliminate many of the problems in the previous version, and to reduce the number of things that an administrator needs to set up.

• Any variables on a slide (such as the current page number or search term) will now be remembered when you return to that slide.
• The system now tracks which plugins use with variables on each slide. You no longer need to enter this in manually when setting up the conductor. (Unless you make a custom path, in which case you can still specify the variables you want.)
• To reduce the amount of clutter in the URL, the system no longer adds any variables on links to slides if those variables are not used on that slide.
• Added a "Show variables" option to the Organizer Panel where you define the conductor navigation. This allows you to see which slides and which paths use which variables.
• The system now makes it clear which plugins can generate smart breadcrumbs, and will no longer allow you to nominate a plugin to generate smart breadcrumbs that does not have this ability.

Minor changes

• You can now view which administrators are currently logged in to your site from Organizer.
• Added a warnings to the diagnostics screen for administrators who have not logged in to your site for a long time, but who have not been deleted.
• If a menu node has an image/rollover image/feature image, you'll now see an icon for these on the admin toolbar.
• Most of the icons from the bottom left of Organizer have now been moved to the top right, to reduce clutter.
• The Content Type Audio module no longer uses Flash, and has been rewritten to use HTML 5 instead.
• The Storefront Documents and Storefront Physical Products modules have been merged into one module, the Storefront Products module.
• The Contact Form has been removed.

MailChimp integration in Forms

The CRM Integration module (which accompanies Forms) now has MailChimp integration. If your form is a sign-up system (e.g. to subscribe to a newsletter) you can now integration with your MailChimp account by adding the appropriate settings to your Zenario Form's settings.

Please see the MailChimp API which defines the integration, but you should just need to enter 2-3 settings in Zenario to make this work.

Notes for hosting providers

Changes to Administrator Permissions

All administrators without the "every possible permission" option checked will be removed of the ability to create, restore and download database backups.

This is a one-time-only change that's made upon updating your site to version 8.2. You can restore their permissions again later after updating, should you wish.

Changes to Conductor

Any modules you have written that used conductor will need their description.yaml files to be updated with the new information needed by conductor - e.g. you need to specify which variables each different mode of your plugin uses. Their revision numbers will then need to be incremented so that these updates are read into the system.

You should also re-test the functionality of the conductors on your site, to ensure that they are still working.

Changes to Modules

The Contact Form module has been removed. You will need to replace any Contact Form plugins that you were previously using with User Forms.

If you have used one of the Storefront Products modules, you will need to update your CSS as the class names have changed.

Changes to site-config Files

The DB_NAME_PREFIX constant has been renamed to just DB_PREFIX, and the DB_NAME_PREFIX_GLOBAL has been renamed to just DB_PREFIX_GLOBAL.

Zenario will still check for the old names for backwards compatibility, so updating to the new names is optional.

Notes for developers

Better support for multiple database connections

The ze\row and ze\sql libraries now support connecting to multiple databases at the same time, without needing to "switch" the database that they're currently pointed to.

In previous versions, whenever you wanted to access a different database, you always had to switch which database was flagged as the active one, e.g.:

ze\db::connectGlobal();
$globalAdmins = ze\row::getArray('admins', $adminColumns, []);
ze\db::connectLocal();

This behaviour has completely changed in 8.2. There are now multiple copies of the database API functions, one for each database.

If you need to use the global database, you can now access it using the ze\rowGlobal and ze\sqlGlobal libraries. This simplifies the code as you don't need to worry about switching databases, e.g.:

$globalAdmins = ze\rowGlobal::getArray('admins', $adminColumns, []);

Note that if you want to write an optional dependancy you can still do so, e.g.:

if (ze\db::hasGlobal()) {
    $globalAdmins = ze\rowGlobal::getArray('admins', $adminColumns, []);
}

Improvements for accessing encrypted fields in the database

Some limited support has been added to the ze\sql library of functions to automatically apply decryption when SELECTing the values of two-way encrypted columns.

You can now do simple queries such as:

$sql = "
    SELECT id, salutation, first_name, last_name, email
    FROM ". DB_NAME_PREFIX. "users
    WHERE u.id = ". (int) $id;
$result = ze\sql::select($sql);
$row = ze\sql::fetchAssoc($sql);

...and the ze\sql library will automatically detect that the columns you are selecting from are encrypted, and get the decrypted values for you.

This works when:

• The table has a unique primary key column, and you SELECT the primary key value in the query.
• You SELECT the values directly - i.e. you can't use any MySQL functions like MIN(), MAX() or CONCAT().
• If using table joins, you don't join to the same table more than once.

Other minor changes to the database API functions

If you have an int or a float in the database, the ze\row and ze\sql libraries will now always return its value as a int or a float, and not a string.

The ze\sql::fetchArray() function has been removed. If you were using it in your code, you should replace it with either the ze\sql::fetchAssoc() or the ze\sql::fetchRow() functions.

You can no longer use the ze\sql::select() function to updating data in the database. You must now always use the ze\sql::update() function for this.

(If you were ze\sql::select() function as a work-around to update to the database without clearing the page-cache, you can now replace it with the new ze\sql::cacheFriendlyUpdate() instead, which does just that.)

You can now use foreach on the queries returned by the ze\row and ze\sql libraries, e.g.:

foreach (ze\row::query('admins', 'last_name', [], 'last_name', 'id') as $adminId => $lastName) {
    var_dump($adminId, $lastName);
}

foreach (
   ze\row::query('admins', ['id', 'first_name', 'last_name'], [], 'last_name') as $adminDetails
) {
    var_dump($adminDetails);
}

foreach (ze\sql::select('
    SELECT id, first_name, last_name
    FROM '. DB_PREFIX. 'admins
    ORDER BY last_name
') as $adminDetails) {
    var_dump($adminDetails);
}

Assetwolf support

Support for a separate MySQL database for asset data has been added

Zenario underpins the Assetwolf IOT platform. Zenario now supports a third database, intended to be a data archive that holds large amounts of data separate to a site's content.

This database is not included when taking a CMS backup in Organizer.

To enable the data-archive, enter the following into your zenario_siteconfig.php file:

///////////////////////////////////////////////
 // MySQL database settings for data archives //
///////////////////////////////////////////////

// These parameters allow you to specify a different database for asset data and
// other large storage.
define('DBHOST_DATA_ARCHIVE', 'localhost');
define('DBNAME_DATA_ARCHIVE', 'dbname');
define('DBUSER_DATA_ARCHIVE', 'username');
define('DBPASS_DATA_ARCHIVE', 'password');
define('DBPORT_DATA_ARCHIVE', 'port');
define('DB_NAME_PREFIX_DATA_ARCHIVE', '');

You'll then be able to access this this database using the ze\rowDA and ze\sqlDA libraries.

If you need to issue database updates for this new database, you can do so by naming your patch-files data_archive.inc.php. When the database updater sees a patch-file with this name, it will assume it is for the data-archive database.

Support for MySQL JSON has been added

If you're running MySQL 5.7, you can now use MySQL's JSON capabilities in Zenario.

To get started, create a patch-file called data_archive.inc.php with a table-definition as follows:

ze\dbAdm::revision( 1
, <<<_sql
    DROP TABLE IF EXISTS `[[DB_NAME_PREFIX_DATA_ARCHIVE]]test`
_sql
, <<<_sql
    CREATE TABLE `[[DB_NAME_PREFIX_DATA_ARCHIVE]]test` (
        `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
        `data` json DEFAULT NULL,
        PRIMARY KEY (`id`)
    )
_sql
);

You can then start using the ze\rowDA::insert() and ze\rowDA::update() functions to insert data, e.g.:

$data = ['name' => 'Bob', 'age' => 21, 'list' => [1, 2, 3]];
ze\rowDA::insert('test', ['data' => $data]);

If you want the data back you can use the ze\rowDA library to load the whole document:

$data = ze\rowDA::get('test', 'data', ['id' => 1]);

or alternately you can get individual elements from the document by specifying a path:

$name = ze\rowDA::get('test', 'data.name', ['id' => 1]);

$li = ze\rowDA::get('test', 'data.list[0]', ['id' => 1]);

Finally, you can also update or remove individual paths rather than whole documents

ze\rowDA::update('test', ['data.name' => 'Fred', 'age' => null], ['id' => 1]);

If you want to use the ze\sqlDA library for this instead of the ze\rowDA library then you can, but you need to use MySQL's syntax for this, which is a little less friendly. You can read up about this at dev.mysql.com/doc/refman/5.7/en/json.html#json-paths.

Support for MongoDB is being dropped

We're in the process of dropping support for MongoDB.

In Zenario 8.2, you can no longer use the database updater to create collections or add indexes in MongoDB.

The ze\mongo library of functions is deprecated in this version, and will be removed in Zenario 8.3.

If you've been using MongoDB for your modules, the intention is that you should use this version of Zenario to migrate your data to MySQL JSON, as this version is a brief window where both the MongoDB and MySQL JSON libraries are available.