Scripting with Groovy 3.0
Breaking Change
Sophora 5 uses Groovy 3.0 as scripting engine. As there are some major changes, e.g. to the Groovy parser, we advise to check your scripts on a non-production system before upgrading your production system. Note that scripts are used by several Sophora components (e.g., Server, DeskClient, Importer, and customer-specific tools). See the Release Notes for Groovy 3 for further details on potential incompatibilities and workarounds. We recommend the Upgrading Guide by Collibra.
Sophora 4 initially used Groovy 2.5, but recent versions also use Groovy 3.0. We highly recommend that, before upgrading to Sophora 5, you first update to recent Sophora 4 Server, DeskClient and Importer versions and thorougly check the scripting with that versions.
XPath is now deprecated
The underlying storage technology of Sophora, Jackrabbit/JCR, will be replaced in Sophora 6. Therefore, the XPath query language will no longer be available in Sophora 6. Prior to that release, we will provide alternatives to XPath-based features, and there will be detailed guides how to migrate to those alternatives. We strongly advise not to use XPath any more where alternatives already exists. For instance, when searching for documents, use IQuerys or Solr instead.
Minimum Sophora Version prior to Upgrade
For compatibility reasons all of your Sophora Servers must be running with at least version 4.26.0 prior to the upgrade to 5.
Major Changes in Version 5
Sophora Server version 5 comes with the following changes:
- SolrCloud and the Sophora Indexing Service replace the internal Solr engine and indexing
- Introduction of the Version Store
- The module ACS has been removed
- The plugin Linkchecker has been sourced out and is now a separate module
Removal of the embedded Solr
Until server version 4, the Sophora Server came with an integrated Solr engine. This has been removed completely in Sophora 5. The Sophora Server itself does not require Solr. Queries from Sophora Clients are mapped to Solr however and require the external SolrCloud. Support for SolrCloud was introduced in Sophora 4 alongside the Sophora Indexing Service. You must have migrated to SolrCloud and the Sophora Indexing Service prior to the upgrade. See indexing service migration guide for more details.
We encourage you to not see this as part of the update to Sophora 5. It is rather a prerequisit for this update.
Introduction of the Version Store
Sophora Server 5 does no longer contain the Archive Repository. It was used to store access versions of documents. All those versions plus the versions from the main repository are now placed in the new Version Store which is based on PostgreSQL. To see what the Version Store does, refer to the Version Store Guide.
The Sophora Server does not automatically migrate all the versions. For migrating all your document versions into the new Version Store, refer to its migration guide.
PostgreSQL does not support the unicode character U+0000 <Null>. As a consequence, we decided to drop support for this character as well. The Sophora Server will replace all occurrences of U+0000 automatically with the empty string. Based on our observations of larger Sophora repositories, this character realistically only occurs in a few (partly invalid) image metadata properties parsed from IPTC data.
List of breaking changes
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | Before the update, preferrably months before | The Server does no longer come with an internal Solr. You must have migrated to Solr Cloud and the Sophora Indexing Service prior to the upgrade. See indexing service migration guide for more details. |
| 5.0.0 | Before the update, preferrably months before | Support for LevelDB as Jackrabbit repository has been discontinued. LevelDB used to be an alternative repository technology for Staging Servers but it cannot be used Sophora 5 any more. Sophora 4 repositories stored in a LevelDB cannot be started using a Sophora 5 Server. If you were using LevelDB with any of your Sophora Servers then you have to migrate those before the update. In this case we recommend to setup an empty staging server and synchronize it from scratch. On larger repositories this might take a few days. But afterwards this newly synchronized server can be used as copy-paste prototype for all the other Sophora Staging Server. |
| 5.0.0 | Before the update, preferrably months before | The SolrCloud connection now requires a Zookeeper url. A direct connection to one of the SolrCloud nodes using sophora.solr.cloud.url has been discontinued. For more information about the connection configuration see the Sophora Server installation guide. |
| 5.0.0 | Before the update | Support for database binary store has been discontinued. A separate binary store is mandatory in Sophora Server 5. The option to use the database binary store no longer is available. So you either have to use the default file system binary store or the cloud binary store. If you have been using the database binary store or no binary store whatsoever before then you have to migrate your binary data into one of the other options. The easiest way to do this is through a full sync into a newly set up server configured to use a file system binary store (which is the default). The update to 5 requires a full-sync anyway to migrate document versions to the new version store, so you can do this in one go. |
| 5.0.0 | Before the update | Versions have to be migrated into the new VersionStore, see the migration guide for more details. These configuration options are no longer needed after the update:
|
| 5.0.0 | Before the update | Support for legacy categories has been discontinued. Use enhanced categories (modelled as select values) instead prior to updating to Sophora 5. |
| 5.0.0 | Before the update | The Content Manager method getConfigurationDocumentProperties no longer includes values from the Sophora Server's internal configuration file (sophora.properties).This especially covers the properties:
|
| 4.20.3 | Before the update | The following configuration options are no longer needed since 4.20.3 and 5.0.0 and can be removed from the Sophora Server's sophora.properties
|
| 4.15.2 | Before the update | Server side event scripts had access to the IScriptDocumentManager via the method getDocumentManager(). The IScriptDocumentManager provides access to the saveDocument method which underwent some changes:
|
| 5.0.0 | Before the update to 5.0.0, after using at least 4.21.0 | The script sophoraServerControl.sh now has to be used to start a Sophora Server; it replaces the old start script sophora.sh. The new script is already available since 4.21.0. See Using the Sophora Server for further information on how to use the new start script. This does not apply to you, if you use our Docker image or Helm chart. |
| 5.0.0 | Before or during the update | Additional configurations for the Server plugin content-api have to be part of the Servers sophora.properties Older versions of the Sophora Server allowed adding configurations for this Server plugin via the content-api.properties file. This is no longer possible. From version 5.0.0 onwards, these properties have to be part of the regular sophora.properties file. If you have been using the specific configuration files, you can just copy their content into the regular sophora.properties file before the update and then remove the content-api.properties afterwards. Alternatively you can do all this in one go before launching the Sophora Server in version 5.0.0 (or greater) for the first time. |
| 5.0.0 | After the update | The IDs of document versions have changed. They are now generated with a determistic pattern to be equal on all cluster servers. This is unlikely to be relevant. However, as the version IDs have been different between different servers prior to version 5.0.0 and therefore were unreliable. |
| 5.0.0 | After the update | The format for describing document thumbnails has changed. Every nodetype configuration comes with the property "Thumbnail" on the "attributes" tab. This used to contain some pseudo XPath-Pattern which is no longer supported. Instead, it now uses an item path syntax: The new syntax supports accessing child nodes (with or without an index) and properties. The nodes and properties are separated by forward slashes, similar to URL paths. The item path follows this structure: [<child-node>/.../]<property> with zero or more child nodes at the begining and ending in a property. No prefix is required. Here are a few examples of the new item path syntax for thumbnails:
order by jcr:score at the end should be removed. Invalid item paths will cause empty thumbnails.The thumbnail path is resolved recursively. A chain like "Document A -> Document B -> Image Document" is possible. The last element in the chain should always point to an actual image document. Most real world examples of Thumbnail configurations are valid in the former XPath syntax and the new item path syntax and therefore don't have to be adjusted, but you should check this for every node type with thumbnails. |
| 5.0.0 | After the update | Starting with Sophora 5, the Server IDs are generated using a combination of a Server's hostname, http and https port, making it more robust against failure. The previously used server.properties file, which persisted the randomly generated Server ID, is automatically deleted during the first start of Sophora 5. This means that the IDs of every server will change during the update to Sophora 5. This should not have critical consequences, but will likely result in new time series data in the Sophora Dashboard. All this happens automatically. |
Sophora Client API Update Notes
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.2.0 | After the update to Sophora Client 5.2.0 | In com.subshell.sophora.api.structure.StructureNode the methods getDefaultDocumentUUID(), getHierarchyDocumentUUID() and getStructureNodeDocumentUUID() are changed so that they will return null instead of throwing a RuntimeException. That happens if the Reference of the concerning document or structureNode is null or an external Reference is set. |
| 5.0.0 | After the update to Sophora Client 5.0.0. | The class com.subshell.sophora.commons.locking.StringKeyLockManager has been removed. Please use com.subshell.sophora.commons.locking.ReentrantLockProvider instead. |
| 5.0.0 | After the update to Sophora Client 5.0.0. | Improbable breaking change: The (previously deprecated) methods getNodeTypeConfigurationHashEntries(), getDefaultPropertyConfigurationHashEntries(), getDefaultChildNodeConfigurationHashEntries(), and getFormFieldGroupHashEntries() have been removed from com.subshell.sophora.client.impl.SophoraClient. Those methods are typically not used by regular tools. |
Info
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | After the update to Sophora Client 5.0.0 | As of version 5.0.0 the archive repository is no longer part of Sophora. Accordingly, it is not possible to query completely deleted documents, since they don't exist anymore. The property com.subshell.sophora.api.search.SearchParameters#completelyDeletedDocumentsOnly as well as the respective setter and getter methods are deprecated. Calls to these methods can be removed safely. |
Automatic Update via Update Site
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | Before updating to Sophora 5.0.0 | An automatic update of the DeskClient via update site from version < 5.0.0 to version >= 5.0.0 is not supported. The DeskClient 5.0.0 must be rolled out separately (e.g. next to existing installations). |
Document Type Configuration
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | Before updating to Sophora 5.0.0 | The "Categories" feature ("Schlagworte") has been removed from Sophora. Make sure to replace the input fields of type "Category (multi)" in your property configurations by using the "Tags", "Enhanced Categories" or "Taxonomies" feature. |
Deprecation
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | Before updating to Sophora 5.0.0 | Nodetype icons and icons in selectvalue documents or script documents with the legacy size of 16x16 pixels may not be displayed correctly, e.g. without transparency. These icons must be replaced by 24x24 pixel icons (required since Sophora 2.3.0). |
Custom Form Fields/Plugins
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | Plugins providing or accessing form fields must be adapted prior to using them in a DeskClient >= 5.0.0. | Since DeskClient versions 2.5.58, 3.1.0 and 4.0.0 the method AbstractFormField.labelLinkActivated() is deprecated. Replace all calls with ((IFormField2) formField).getLabelLinkAction().isPresent().When providing form fields via the extension point "com.subshell.sophora.eclipse.formInputFields" the attribute labelAsLink must be removed from the tag formInputField. |
| 5.0.0 | Plugins providing or accessing form fields must be adapted prior to using them in a DeskClient >= 5.0.0. | The code structure of the document editor has changed. It was converted to an e4 part. The "globalActionBars" were replaced by handlers for (global) commands. The constants IEditorPart.PROP_INPUT or IWorkbenchConstants.PROP_INPUT should be replaced with DocumentEditorPart.PROP_DOCUMENT. The method IPropertyFieldConfiguration.getEditorOpener() is now available as IFormFieldSite.getEditorOpener().Some icons were moved to the new project 'com.subshell.sophora.eclipse.icons'. These icons must now be accessed by IconsPlugin.getIcon() instead of SophoraPlugin.getDefault().getIcon().If you have other problems let us know so we can help you. |
| 5.0.0 | Plugins providing EditorComponent must be adapted prior to using them in a DeskClient >= 5.0.0. | Change access of getSite().getWorkbenchPart() to getSite().getEditor() for accessing methods of the document editor.When using an IEditorComponentTabProvider change the parameter of method createEditorPart() to DocumentEditorPart. |
Open document by URL in clipboard
Removed
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | any time | The configuration key "openDocumentFromUrl.usePreviews" no longer has any effect. This option to get the UUID of a document by querying the IdForUrlServlet of the previews directly by iterating over the preview URLs of the published preview documents has been removed.The function to open a document via its URL now uses always the client/server method to get the UUID of a document via its URL. |
Native support of Apple ARM architecture (aarch64)
Info
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | After updating to Sophora 5.0.0 | The DeskClient is now also released for the Apple ARM architecture. The corresponding download file is named according to the following scheme: sophora-deskclient-<version>-<product>-macosx.cocoa.aarch64.dmgNote that the sophora-deskclient-<version>-<product>-macosx.cocoa.x86_64.dmg builds will also work on Apple ARM in compatibility mode. |
Sophora Importer Update Notes
Breaking change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | The possibility to import "categories" has been removed from Importer 5 | |
Configuration Changes
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | This must be done before the Importer is started for the first time in version 5. | Since Importer 5 the configuration property sophora.client.server-connection.url is removed and replaced by sophora.client.server-connection.urls, where multiple connection urls may be used. |
Deprecation
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | After the update to Importer 4.4.0 and before updating to 6.0.0 | Since importer version 4.4.0 there are two new modules to build and represent Sophora-XML. One of them is the model, which now contains some classes that resided in the Importer core before. To use the new artifact use the artifactId com.subshell.sophora.importer.model in the groupId com.subshell.sophoraThese Classes will be removed in 6.0.0!
|
API Changes
Breaking Changes
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | com.subshell.sophora.importer.core.creators.CategoryCreator andcom.subshell.sophora.importer.core.creators.CategoryPath are removed from Importer 5 | |
| 5.0.0 | The Method IErrorTracker#writeToFile(File file) is removed from Importer 5 |
Sophora URL Library: Update Notes
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | After the update to 5.0.0 | The configuration option sophora.url.domain-name was renamed to sophora.url.default-domain.The configuration option sophora.url.prefer-site-domain was removed.The precedence for generating an absolute URL for a document is now as follows:
|
| 5.0.0 | After the update to 5.0.0 | The configuration options for image URL generation have changed, there is a new property imageUrl in structure node documents and a new property sophora.image-service.url.structure-path-to-domain in the configuration properties. The existing property sophora.image-service.url.domain-name was renamed to sophora.image-service.url.default-domain.The precedence for generating an URL for image documents is now as follows:
|
Sophora Scripting Update Notes
The dependencies commons-lang:commons-lang and commons-collections:commons-collections have been removed from the Sophora Core. They are superseded by org.apache.commons:commons-lang3 and org.apache.commons:commons-collections4, which already have been part of the Sophora core libraries for a long time.
This affects all types of scripts, as those libraries cannot be used any more in scripts. Before update to Sophora 5, make sure that none of your scripts reference those old libraries. Also note that the default preamble for scripts was changed to reflect the new package names.
Migration to the new libraries is mostly straightforward, as in most cases the classes and methods kept their names within the new packages org.apache.commons.lang3 and org.apache.commons.collections4. For details on the migration see the release notes for Apache Commons Lang and Apache Commons Collections.
Configuration
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | This must be done before the indexing service is started for the first time in version 5. | The configuration property sophora.client.server-connection.url is replaced by sophora.client.server-connection.urls. Multiple connection urls may be used. |
| 5.0.0 | This must be done before the indexing service is started for the first time in version 5. | The json codec configured under spring.redis.redisson.config has changed. Use the following configuration: codec: !<com.subshell.sophora.indexingservice.core.codec.SISIJsonJacksonCodec> {} |
Solr Index Mapping
- A new boolean solr field
sophora_isEnabled_bis added to all working collections that indicates if the document is enabled within the structure.
TableStar Controller
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | This must be done before the TableStar Controller is started for the first time in version 5.0.0. |
|
Sophora Teletext: Update Notes
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | After the update to Teletext 5.0.0 | The util class for Teletext scripts com.subshell.sophora.teletext.script.common.IndexReferenceAnalyzer changed:
|
Info
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | After the update to Teletext 5.0.0 | The index document type sophora-teletext-nt:index must have a new mixin sophora-teletext-mix:index.Please import the mixin sophora-teletext-mix:index from the maven artifact com.subshell.sophora.teletext:teletext-repository-base-config to your Sophora repository and activate it in the index document type sophora-teletext-nt:index. Since Teletext 5 new updates (e.g. scripts) will identify index documents based on the mixin. |
UGC Service
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | This must be done before UGC is started for the first time in version 5. | Several deprecated API endpoints that used the externalId or the username as parameters have been removed.The new API endpoints require the sophoraId instead of the externalId and are marked by a trailing /bySophoraId. These endpoints have been existing prior to Sophora 5 so that adjustments can be done before the update. Instead of username, it is now required to use the externalUserId.Please note: If the ugc submitter and the ugc taglib are used, it is absolutely necessary to use these modules in version 5 due to this adjustment. |
| 5.0.0 | After the update to UGC 5.0.0 | There are slight changes in the URL paths of the UGC Webapp due to an update of libraries used by the UGC Webapp frontend. The fragment separator #! is not needed anymore. Therefore paths like https://<hostname>:<port>/#!/comments change to https://<hostname>:<port>/comments. |
| 5.0.0 | After the update to UGC 5.0.0 | You need to change the URL field in the UGC related tab documents. Otherwise some tabs of UGC specific documents won't be displayed correctly. In the URL field, replace every occurence of ${sophora:externalId} with ${sophora:id}. Additionally, you need to remove the #! from the URL. For example: localhost:9080/#!/comments/${sophora:externalId}?nav=false should be changed to localhost:9080/comments/${sophora:id}?nav=falseIf you don't want to change every tab document manually, you can reimport the tab documents that are included in the ugc-5.*.*-sophora.zip as part of the release and are located in the folder 06_tabdocuments. |
| 5.0.0 | After the update to UGC 5.0.0 | The prior deprecated configuration property group double-opt-in.transport has been removed. Please use email.transport instead. Consult the UGC Webapp documentation page for details on the configuration. |
| Removed API endpoints | New API endpoints |
|---|---|
| /comments/{externalId}/simple | /comments/{sophoraId}/simple/bySophoraId |
| /comments/{externalId}/pinned | /comments/{sophoraId}/pinned/bySophoraId |
| /comments/{externalId}/all | /comments/{sophoraId}/all/bySophoraId |
| /commentTemplates/delete/{externalId} | /commentTemplates/deleteBySophoraId/{sophoraId} |
| /quizzes/{externalId}/updatecalculation | /quizzes/{sophoraId}/updatecalculation/bySophoraId |
| /votings/{externalId}/votingitems | /votings/{sophoraId}/votingitems/bySophoraId |
| /votings/{externalId}/report | /votings/{sophoraId}/report/bySophoraId |
| /votings/{externalId}/report/xls | /votings/{sophoraId}/report/bySophoraId/xls |
| /votings/{externalId}/resetreport | /votings/{sophoraId}/resetreport/bySophoraId |
| /legacyratings/{externalId} | /legacyratings/bySophoraId/{sophoraId} |
| /ratings/findDocument/{id} | /ratings/findDocument/bySophoraId/{sophoraId} |
for /imageUploads and /comments: | |
| /byUsername/{username} | /byExternalUserId/{externalUserId} |
| /{externalId}/reclassify | /{sophoraId}/reclassify/bySophoraId |
| /approveByFilter | /approveByFilter/bySophoraId |
| /denyByFilter | /denyByFilter/bySophoraId |
| /deleteByFilter | /deleteByFilter/bySophoraId |
| /deletePermanentlyByUsers/{username} | /deletePermanentlyByUserId |
| /deletePermanentlyByUsers | /deletePermanentlyByUserIds |
| /checkCategory/{externalId} | /checkCategory/bySophoraId{sophoraId} |
for /quizzes and /votings: | |
| /{externalId} | /bySophoraId/{sophoraId} |
| /{externalId}/participants | /{sophoraId}/participants/bySophoraId |
| /{externalId}/participants/paged | /{sophoraId}/participants/bySophoraId/paged |
| /{externalId}/participants/isAvailable | /{sophoraId}/participants/bySophoraId/isAvailable |
| /{externalId}/participants/xls | /{sophoraId}/participants/bySophoraId/xls |
| /{externalId}/columnheaders | /{sophoraId}/columnheaders/bySophoraId |
| /{externalId}/random/{number} | /{sophoraId}/random/bySophoraId/{number} |
| /{externalId}/reset | /{sophoraId}/reset/bySophoraId |
for /comments, /imageUploads and /ratings: | |
| /{externalId}/random/{number} | /{sophoraId}/random/bySophoraId/{number} |
Configuration Changes
Deprecation
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | This must be done before the AV-Tool/YouTube-Connector is started for the first time in version 5. | The configuration property akamaiUrlPrefixes has been deprecated with version 4.1.0 of the AV-Tool/YouTube-Connector. It has been removed with version 5.0.0. Please use akamaiUrlPatterns instead. |
Categories
Breaking Change
| Version | Timing | Changes |
|---|---|---|
| 5.0.0 | Before updating to Sophora Delivery 5.0.0. | The following classes have been completely removed:
The following methods have been completely removed:
|
