In Option 42 of version 7.1 of the IBM System i operating system, an important new feature was added to remote journaling: the ability to filter out specific journal entries so that they are not sent to the remote system. Using this feature can significantly reduce the amount of data that needs to be sent over your communications line. However, there are a few restrictions to be aware of before implementing remote journal filtering.

Written by:

Don Zimmerman

Journaling Team

IBM Rochester,MN

What Is remote journal filtering?

Remote journaling is a feature of IBM System i that allows you to automatically make copies of local journal receivers on one or more backup systems. The journal receivers could be broadcast to multiple systems from one system, or cascaded from one system to the next in a chain. These remote journal receivers can be used for a number of operations, such as saving them to tape on the backup system instead of tying up resources on the local system, or replaying changes to journaled objects on a backup system so as to have a remote copy of your data.

Traditionally, remote journal receivers have always been exact duplicates of the original local journal receivers. Remote journal filtering breaks this tradition and allows remote journal receivers to only have a subset of the journal entries existing in the local journal receivers. By setting filtering criteria, journal entries that match that criteria are filtered out on the source system and are never sent across the communication line. The remaining journal entries are completely unchanged and are sent to the remote system as normal. The resulting remote journal receiver will appear to have "holes" where journal entries are missing. Since the remote receiver may have less journal entries than the original local receiver, it may also consume less space.

There are three flavors of remote journal filtering: before images filtering, filtering by object, and filtering by program. These three flavors are not mutually exclusive and may be used in conjunction with each other. A journal entry is filtered if it matches any of the filtering criteria for a remote journal connection. Each remote journal connection may have a different set of criteria, even if it is broadcast from the same local journal. In a cascade environment, only the original source system may specify filtering criteria; remote journal connections farther down the chain may not specify filtering criteria. Filtering criteria is specified when activating a remote journal environment, whether by the Change Remote Journal (CHGRMTJRN) command or the QjoChangeJournalState API.

Before images filtering

Database files and data areas have the option of journaling before images (as opposed to only after images). These are images of the objects written as journal entries before each update occurs. Before images are necessary in some environments, particularly for commitment control. They are, however, rarely needed on a remote system. All before images can be filtered from a remote journal by using the FTRIMAGES parameter on the Change Remote Journal (CHGRMTJRN) command or the Filter images field on format CJST0500 of the QjoChangeJournalState API.

While remote journal receivers that may have had journal entries filtered cannot normally be used for replaying journal entries with the Apply Journaled Changes (APYJRNCHG) command, journal receivers that have only had before images filtered may be used with APYJRNCHG. They cannot be used with the Remove Journaled Changes (RMVJRNCHG) command.

Filtering by object

Not all objects that are journaled need to be replicated to a backup system. Temporary files, for example, may come and go on the source system and are never needed on a target system. For such objects, remote journal filtering by object could be very useful.

Using remote journal filtering by object is a two-step process. First, you need to indicate which objects are to be filtered. This indication is a new journaling attribute that can be set for database files, data areas, and data queues. To set this attribute for existing journaled objects, use the RMTJRNFTR parameter of the Change Journaled Object (CHGJRNOBJ) command. Objects that automatically start journaling when they are created, moved, or restored into a journaled library can have this attribute set automatically. Check the inheritance rules for the journaled library using the Display Library Description (DSPLIBD) command, and change them using the CHGJRNOBJ command. The inheritance rules can also be set when the library is initially journaled using the Start Journal Library (STRJRNLIB) command.

The second step of the process is turning on remote journal filtering for the remote journal connection. This is done with the FTROBJ parameter on the Change Remote Journal (CHGRMTJRN) command or the "Filter by object field" on format CJST0500 of the QjoChangeJournalState API. There are only two values, *YES and *NO. What this means is that either all journal entries for all objects with RMTJRNFTR(*YES) are filtered from the remote journal or none of them.

Only journal entries deposited while an object is set to RMTJRNFTR(*YES) are filtered. Journal entries deposited before the attribute is changed for the object are not filtered. The converse is true as well: journal entries deposited while RMTJRNFTR is set to *YES will continue to be filtered even after the object is changed to RMTJRNFTR(*NO). That is, the attribute that the object had at the time a journal entry was deposited determines whether or not that journal entry is eligible to be filtered.

It is worth noting that multiple remote journal connections broadcast from the same source journal cannot filter different sets of objects. Each connection must decide to filter all journal entries by objects marked as such, or filter none of them.

Filtering by program

In some cases it is useful to filter out journal entries sent on behalf of changes made by a specific program. Format CJST0500 of the QjoChangeJournalState API allows you to specify up to 20 different programs for filtering. All journal entries sent on behalf of any of these programs will not be sent to the remote journal. The programs are specified as 10-character name and library name fields and are case sensitive.

Each remote journal connection broadcast from the same source journal may have a different set of programs to filter.

Remote Journal Filtering Restrictions

There are a few restrictions associated with remote journal filtering that you need to be aware of before using this support.

If you end your remote journal environment and then simply try to restart it with different filtering criteria, the remote journal connection will likely fail to restart. This is because a journal receiver can only have one set of remote journal filtering criteria associated with it. The receiver that is still attached to the remote journal has the old filtering criteria associated with it and cannot have its criteria changed. To restart this remote connection with different filtering criteria, you need to first delete the journal receiver attached to the remote journal.

As mentioned before, only the original local journal in a cascaded remote journal environment can set filtering criteria. Remote journal connections farther down the chain cannot modify or add to this criteria.

Remote journal filtering is not allowed to releases prior to 7.1. Attempts to activate remote journal environments to previous releases will fail. Remote journal receivers that may have had journal entries filtered will not be allowed to be saved to prior releases.

In most cases, you may not use the Apply Journaled Changes (APYJRNCHG), Apply Journaled Changes Extend (APYJRNCHGX), or the Remove Journaled Changes (RMVJRNCHG) commands with a remote journal receiver that has filtering criteria associated with it. This is true even if no journal entries were actually filtered. So do not use remote journal filtering if you plan to use any of these commands on the remote journal receivers. The one exception to this is that you may still use journal receivers that filtered before images on the APYJRNCHG and APYJRNCHGX commands. You may not use them with RMVJRNCHG.

Finally, journal entries may be missing from the remote journal receiver, obviously. Be careful not to filter out any journal entries needed for replication, backup, or auditing on your remote system.

Summary

Remote journal filtering is a potentially powerful tool for limiting the amount of bandwidth used by a remote journal connection. By using a combination of before images filtering, filtering by object, and filtering by program you can ensure that only those journal entries that are needed on a target system are sent over the communication line.