Staged Database Reorganizations

There is a limit to the number of database structures that can be reorganized in a single DMSII Reorganization. This limit is close to 300 structures, which includes any actual user structure changes, as well as any fixups that are implicitly included, such as the reorganization of all the associated sets/subsets for a changed dataset.

In an AB Suite application with a large database, the limits to a single DMSII reorganization might be an issue when a change is made to an attribute, such as a Dictionary item, and there are many persistent attributes that inherit the changed attribute. Potentially, a very large number of structures might need reorganizing (more than 300) as a consequence of the change.

To overcome this issue, the MCP Runtime deployment software has the capability of supporting automatic staged database reorganizations, which allows almost unlimited numbers of user and fixup structure changes in a single deployment of the application. The need for a staged reorganization is determined during the deployment process. If a staged reorganization is needed, the changes are assessed, the number of stages required to apply all changes is determined and the changes are allocated to the stages. This is done automatically, ensuring that no more than 290 structure changes are applied in any one of the stages. The deployment process then manages the multiple reorganizations with the last reorganization concluding with the final installation of the application.

Note: The deployment is still treated as “all complete” or “nothing completed”. Although multiple reorganizations might have been performed during the deployment process, recovery from a failure always restores to the start of the first reorganization. Hence the deployment process will ask for confirmation that the database has been backed up, before commencing the first staged reorganization. Prior to first reorganization stage, some database related files are backed up with the ../SAVED suffix.

Original File	Backup File
<system>/SOURCE/DMS	<system>/SOURCE/DMS/SAVED
DESCRIPTION/<database>	DESCRIPTION/<database>/SAVED
<database>/CONTROL	<database>/CONTROL/SAVED
DMSUPPORT/<database>	DMSUPPORT/<database>/SAVED
RMSUPPORT/<database>	RMSUPPORT/<database>/SAVED
<system>/DBINTERFACE	<system>/DBINTERFACE/SAVED
<system>/SOURCE_FILE/DBRECORD/ LAYOUT	<system>/SOURCE_FILE/DBRECORD/ LAYOUT/SAVED

After a failed deployment, in which there was a staged database reorganization, these files, along with the database backup, must be reinstated prior to restarting the deployment. On a successful completion of the staged database reorganization, the backup files is removed.

If the staged database reorganization feature is not wanted for any particular application, it might be disabled from Application Builder. Use the WAITIFSTAGE command to enable or disable the Staged Database Reorganization feature for a specific system. Refer to WAITIFSTAGE in ClearPath MCP Application Builder Commands.

If the staged database reorganization is required and the Reorganization Type selected for the database reorganization is neither ‘Offline’ nor ‘Offline No Post Dump’ for the current generate or WAITIFSTAGE is enabled, then the user is prompted to confirm the staged database reorganization.

Generate Suspended during Dasdl syntax-only compile//<mix#>AX DS TO ABORT, OK

FOR STAGED OR IGNORE FOR NON-STAGED.

Respond with <mix#>AX OK to continue with a staged database reorganization.

Respond with <mix#>AX IGNORE to continue with a regular database reorganization.

Respond with <mix#>AX DS to terminate the deployment.

The Application Builder command SYSTEM n for the appropriate system shows if WAITIFREORG is set for this system.

Building an application with the Preview option provides a database summary that indicates whether a staged database reorganization is necessary, detailing the scope and number of stages required.

Additionally, when deployment, which includes a staged database reorganization, is in progress, the following client log status messages are returned:

User changed structures to be reorganized: N (excludes FIXUPS)
REORGANIZATION MUST BE STAGED AND OFFLINE STYLE, ENTER xxxxx AX DS TO ABORT, OK FOR STAGED OR IGNORE TO CONTINUE.
>>> This waiting entry appears if a staged deployment is in progress but the database reorganization type is not OFFLINE. A staged database reorganization must be done with OFFLINE or OFFLINE NOPOST DUMP reorganization.
OFFLINE NOPOSTDUMP is recommended to avoid being prompted for a database dump at each stage.
Respond with xxxxAX OK to continue with the staged database reorganization.
Respond with xxxxAX IGNORE to continue with a regular database reorganization.
Respond with xxxxAX DS to abort the deployment.
Note: xxxx is the mix number that appears in the message.
Changed structures to be reorganized: <nnn> (excludes FIXUPS )
Reorganization is staged, impacts <xxxx> database structures
Requires <x> Reorganization Stages (<xxxx> structures includes FIXUPS)
Structure changes/fixups per Reorganization: 290
Starting Reorganization 1 of <x> [J <jjjj>]
Staged Reorganizations commencing, AX OK to confirm DB is backed up
Waiting for Accept BDPSINCO/GEN [J <jjjj>]
AX OK TO CONFIRM OR DS TO TERMINATE GENERATE.
>>> This waiting entry is presented at the start of the first Staged reorganization to verify the database has been backed up.
Respond with <jjjj>AX OK to confirm that a backup of the database has been taken.
Respond with <jjjj>AX DS to abort the deployment.
Starting Reorganization <s> of <x> [J <jjjj>]
A message that is similar to this appears at the beginning of each interim stage.
Starting Reorganization<x> of <x> [J <jjjj>]
This message indicates the start of the final stage of the reorganizations.

The Staged Database Reorganization process cannot handle database changes that are applied to all, or most of the, structures in one go. For example, setting the segment configuration property Extended Edition to True causes the DMSII EXTENDED option to be applied to all structures. In this case, DMSII implicitly reorganizes every Data Set and Set/Subset. The nature of this change is such that the staged reorganizations process cannot implement the changes in stages. If the user has enable EXTENDED globally and the number of structures impacted by the change exceeds the staged reorganization threshold, the summary from a Build Preview includes an XE warning. If the user continue with an actual deployment, then the same XE warning is shown in the client log.

The default trigger for staged reorganization is 290 structure changes. This might be altered to any value between 290 and 5, inclusive, using the SETTRIGGER Application Builder command. This is really only for testing purposes. For example, you might set the trigger for a staged reorganization to a low number of structure changes for the purpose of validating, or familiarization with, the staged reorganization process by using a small system. However, the user value supplied with SETTRIGGER might be overridden if either any changed data set plus its associated sets/subsets numbers greater than the threshold (a dataset and all its sets/subsets must be handled within the one reorganization), or the number of reorganizations required for all the structure changes exceeds 40. The Application Builder command SYSTEM n might be used for the appropriate system to show if SETTRIGGER is set (similarly the SETTRIGGER command with no arguments shows the current state). Refer to SETTRIGGER in ClearPath MCP Application Builder Commands.