Integration Broker

• 1: IB Operation Errors
• 2: IB Operations Stalled
• 3: IB Publication Contract Errors
• 4: IB Publication Contracts Stalled
• 5: IB Subscription Contract Errors
• 6: IB Subscription Contracts Stalled
• 7: Abnormal IB Operation Volume
• 8: Abnormal IB Publication Contract Volume
• 9: Integration Broker Down
• 10: Abnormal IB Subscription Contract Volume
• 11: IB No Active Domain
• 12: IB Dispatcher Down
• 13: IB Sync Operation Exceptions
• 14: IB Nodes Down

1 - IB Operation Errors

IB Operation Errors Alert

Alert ID: ib_operation_errors Category: Integration Broker Default lookback: 24 hours

What This Alert Detects

This alert finds asynchronous Integration Broker operation instances that are in Error or Timeout status within a configurable lookback window. These are messages that attempted to process but did not complete successfully.

An IB operation instance represents a single execution of a Service Operation through the Integration Broker. When an instance errors, the message did not reach its destination. The original publish/subscribe data is preserved in the IB tables and can usually be resubmitted from the Service Operations Monitor.

Severity Logic

Error status means the processing actively failed. Timeout means it ran out of time, which may be a transient issue but still warrants investigation.

Alert Details

Each alert item includes:

• Operation instance ID
• Service operation name
• Status (Error or Timeout)
• The originating node
• When the instance was created
• A link to the IB Monitor detail page

Configuration

alerts:
  checks:
    ib_operation_errors:
      enabled: true
      lookbackHours: 24          # How far back to look for errors
      excludeOperations:         # Operation names to skip
        - SOME_NOISY_OPERATION

How to Respond

1. Click the alert link to go to the IB Monitor entry for the failed operation
2. Review the error details. The IB Monitor shows the error message or exception
3. Check whether this is a configuration issue (wrong endpoint, auth failure) or a data issue
4. If the message needs to be reprocessed, you can do so from PeopleSoft’s Service Operations Monitor
5. If this is a recurring operation, investigate the root cause before errors pile up

Relationship to Other IB Alerts

This alert finds operations that have already ended in error. For operations that are stuck in progress, see IB Operations Stalled.

For similar alerts on publication and subscription contracts, see:

• IB Publication Contract Errors
• IB Subscription Contract Errors

2 - IB Operations Stalled

IB Operations Stalled Alert

Alert ID: ib_operation_stalled Category: Integration Broker Default threshold: 30 minutes

What This Alert Detects

This alert finds asynchronous Integration Broker operation instances that are stuck in New or Working status and have been in that state longer than the configured threshold. These are messages that started processing (or are waiting to be processed) but have not completed in a reasonable amount of time.

Cross-reference with IB Dispatcher Down first. A stalled queue plus a down dispatcher is almost always the dispatcher.

Severity Logic

For example, with the default threshold of 30 minutes:

• An operation stuck for 35 minutes → Warning
• An operation stuck for 65 minutes or more → Critical

Alert Details

Each alert item includes:

• Operation instance ID
• Service operation name
• Current status (New or Working)
• How long it has been stuck (in minutes)
• The originating node
• A link to the IB Monitor detail page

Configuration

alerts:
  checks:
    ib_operation_stalled:
      enabled: true
      thresholdMinutes: 30       # Minutes before flagging as Warning
      excludeOperations:         # Operation names to skip
        - BULK_SYNC_OPERATION

How to Respond

1. Click the alert link to go to the IB Monitor entry for the stalled operation
2. Check whether the IB dispatcher/handlers are running on the PeopleSoft application server
3. Look for signs of a larger IB backlog (many operations in New status)
4. Check the gateway and connector configuration if the operation can’t reach a node
5. If the operation is safe to reprocess, you can cancel and resubmit from PeopleSoft’s Service Operations Monitor

Relationship to Other IB Alerts

This alert finds operations that are stuck in progress. For operations that have already ended in error, see IB Operation Errors.

For similar alerts on publication and subscription contracts, see:

• IB Publication Contracts Stalled
• IB Subscription Contracts Stalled

3 - IB Publication Contract Errors

IB Publication Contract Errors Alert

Alert ID: ib_pub_contract_errors Category: Integration Broker Default lookback: 24 hours

What This Alert Detects

This alert finds Integration Broker publication contracts that are in Error or Timeout status within a configurable lookback window.

In the PeopleSoft Integration Broker architecture, a publication contract tracks the delivery of a published message to a specific subscribing node. When a publication contract fails, it means a message that PeopleSoft published was not successfully delivered to one or more subscribers.

When This Matters

Publication contract errors typically mean:

• An outbound message from PeopleSoft was not delivered to a downstream system
• An integration partner did not receive data it was expecting
• A workflow or data sync that depends on this message may be incomplete

Severity Logic

Alert Details

Each alert item includes:

• Publication contract ID
• Service operation name
• Status (Error or Timeout)
• The target subscribing node
• When the contract was created
• A link to the IB Monitor detail page

Configuration

alerts:
  checks:
    ib_pub_contract_errors:
      enabled: true
      lookbackHours: 24          # How far back to look for errors
      excludeOperations:         # Operation names to skip
        - HIGH_VOLUME_SYNC

How to Respond

1. Click the alert link to go to the IB Monitor entry for the failed contract
2. Review the error details. The IB Monitor shows the error message
3. Check whether the target subscribing node is reachable and its connector is configured correctly
4. Review the node’s authentication settings (see the Nodes with No Password report if auth may be the issue)
5. If the message can be safely reprocessed, resubmit from PeopleSoft’s Publication Contracts Monitor

Relationship to Other IB Alerts

For pub contracts stuck in progress, see IB Publication Contracts Stalled.

For similar alerts on async operations and subscription contracts:

• IB Operation Errors
• IB Subscription Contract Errors

4 - IB Publication Contracts Stalled

IB Publication Contracts Stalled Alert

Alert ID: ib_pub_contract_stalled Category: Integration Broker Default threshold: 30 minutes

What This Alert Detects

This alert finds Integration Broker publication contracts that are stuck in New or Working status and have not progressed beyond that state within the configured threshold.

A stalled publication contract means PeopleSoft has published a message to a subscriber, but the delivery has not completed. The message is either waiting to be picked up (New) or is in the process of being delivered but taking too long (Working).

When This Matters

Check IB Nodes Down and IB Dispatcher Down first. A stalled pub contract is usually one of those two conditions.

Severity Logic

Alert Details

Each alert item includes:

• Publication contract ID
• Service operation name
• Current status (New or Working)
• How long it has been stuck (in minutes)
• The target subscribing node
• A link to the IB Monitor detail page

Configuration

alerts:
  checks:
    ib_pub_contract_stalled:
      enabled: true
      thresholdMinutes: 30       # Minutes before flagging as Warning
      excludeOperations:         # Operation names to skip
        - LARGE_BATCH_SYNC

How to Respond

1. Click the alert link to go to the IB Monitor entry for the stalled contract
2. Check whether the IB dispatchers are running on the PeopleSoft application server
3. Look for a larger backlog (many contracts in New status may mean the dispatcher is down)
4. Check whether the target node’s endpoint is reachable
5. Review connector configuration for the target node

Relationship to Other IB Alerts

For pub contracts that have already ended in error, see IB Publication Contract Errors.

For similar stalled alerts on operations and subscription contracts:

• IB Operations Stalled
• IB Subscription Contracts Stalled

5 - IB Subscription Contract Errors

IB Subscription Contract Errors Alert

Alert ID: ib_sub_contract_errors Category: Integration Broker Default lookback: 24 hours

What This Alert Detects

This alert finds Integration Broker subscription contracts that are in Error or Timeout status within a configurable lookback window.

In the PeopleSoft Integration Broker architecture, a subscription contract tracks the processing of an inbound message by a subscribing handler. When a subscription contract fails, it means PeopleSoft received a message from an external system but was unable to process it completely.

When This Matters

Subscription contract errors typically mean:

• An inbound message from an integration partner was not fully processed
• PeopleSoft was unable to apply the data changes the message carried
• A business process that depends on this message may be incomplete or in an error state

Severity Logic

Alert Details

Each alert item includes:

• Subscription contract ID
• Service operation name
• Status (Error or Timeout)
• The originating (publishing) node
• When the contract was created
• A link to the IB Monitor detail page

Configuration

alerts:
  checks:
    ib_sub_contract_errors:
      enabled: true
      lookbackHours: 24          # How far back to look for errors
      excludeOperations:         # Operation names to skip
        - KNOWN_RETRY_OPERATION

How to Respond

1. Click the alert link to go to the IB Monitor entry for the failed contract
2. Review the error details. The IB Monitor typically shows the exception or error message from the handler PeopleCode
3. Check the subscription handler code for the operation (viewable in the Service Operation detail page in psLens)
4. Investigate whether the data in the message is valid. Handler errors often come from unexpected data
5. If the subscription can be safely reprocessed, resubmit from PeopleSoft’s Subscription Contracts Monitor

Relationship to Other IB Alerts

For sub contracts stuck in progress, see IB Subscription Contracts Stalled.

For similar alerts on operations and publication contracts:

• IB Operation Errors
• IB Publication Contract Errors

6 - IB Subscription Contracts Stalled

IB Subscription Contracts Stalled Alert

Alert ID: ib_sub_contract_stalled Category: Integration Broker Default threshold: 30 minutes

What This Alert Detects

This alert finds Integration Broker subscription contracts that are stuck in New or Working status and have not progressed beyond that state within the configured threshold.

A stalled subscription contract means PeopleSoft received an inbound message from an external system, but has not yet finished processing it. The message is either waiting to be handled (New) or is in the process of being handled but taking too long (Working).

When This Matters

For sub contracts, the most common cause specific to this alert is subscription handler PeopleCode that runs unusually long or waits on an external resource. For broader IB-side causes (dispatcher down, backlog), cross-reference IB Dispatcher Down and IB Operations Stalled.

Severity Logic

Alert Details

Each alert item includes:

• Subscription contract ID
• Service operation name
• Current status (New or Working)
• How long it has been stuck (in minutes)
• The originating (publishing) node
• A link to the IB Monitor detail page

Configuration

alerts:
  checks:
    ib_sub_contract_stalled:
      enabled: true
      thresholdMinutes: 30       # Minutes before flagging as Warning
      excludeOperations:         # Operation names to skip
        - BULK_INBOUND_SYNC

How to Respond

1. Click the alert link to go to the IB Monitor entry for the stalled contract
2. Check whether the IB dispatchers are running on the PeopleSoft application server
3. Look for a broader backlog. Many New contracts may mean no dispatcher is running
4. Check application server logs for errors in the subscription handler
5. For Working contracts, check whether the handler PeopleCode is looping or waiting on an external resource

Relationship to Other IB Alerts

For sub contracts that have already ended in error, see IB Subscription Contract Errors.

For similar stalled alerts on operations and publication contracts:

• IB Operations Stalled
• IB Publication Contracts Stalled

7 - Abnormal IB Operation Volume

Abnormal IB Operation Volume Alert

Alert ID: ib_operation_volume Category: Integration Broker

What This Alert Detects

This alert detects when the volume of IB async operation instances (PSAPMSGPUBHDR) is significantly higher than the rolling historical average.

A sudden volume spike may indicate a runaway integration sending messages in a loop, an upstream system retrying failed messages, or an unusually large but legitimate batch publish event.

How Baselining Works

psLens maintains a rolling history of up to 288 volume snapshots (approximately 24 hours at a 5-minute check interval). Each check cycle records the current message count for the lookback window.

Once at least 6 baseline snapshots have accumulated, the alert begins comparing the current count against the historical average. This prevents false alerts during the first few minutes after psLens starts.

Severity Logic

For example, with the default threshold of 50%:

• Current count is 75% above average → Warning
• Current count is 100%+ above average → Critical

Configuration

alerts:
  checks:
    ib_operation_volume:
      enabled: true
      lookbackHours: 1         # Window for counting current messages
      thresholdCount: 50       # Percentage increase to trigger Warning

Alert Details

Each alert item includes:

• Current message count for the lookback window
• Historical average count
• Percentage increase above average
• Number of baseline samples used
• Link to the IB Monitor page

How to Respond

1. Navigate to the IB Monitor in psLens to see which operations are generating the volume
2. Check if any operations are in Error or Stalled status (see related IB alerts)
3. Review the specific operations with high message counts to determine if the volume is expected
4. If a runaway integration is identified, investigate the upstream system sending the messages
5. Consider using the Daily IB Volume report to review historical patterns

Tables Queried

8 - Abnormal IB Publication Contract Volume

Abnormal IB Publication Contract Volume Alert

Alert ID: ib_pub_contract_volume Category: Integration Broker

What This Alert Detects

This alert detects when the volume of IB publication contracts (PSAPMSGPUBCON) is significantly higher than the rolling historical average. Publication contracts represent messages being delivered to subscribing nodes, so a spike here can indicate unexpected fan-out, retries, or a high-volume event.

A sudden volume spike may indicate repeated retries of failed contracts inflating the count, an upstream system publishing in a loop, or a legitimate but unusually large batch publication event.

How Baselining Works

psLens maintains a rolling history of up to 288 volume snapshots (approximately 24 hours at a 5-minute check interval). Each check cycle records the current publication contract count for the lookback window.

Once at least 6 baseline snapshots have accumulated, the alert begins comparing the current count against the historical average.

Severity Logic

For example, with the default threshold of 50%:

• Current count is 75% above average → Warning
• Current count is 100%+ above average → Critical

Configuration

alerts:
  checks:
    ib_pub_contract_volume:
      enabled: true
      lookbackHours: 1         # Window for counting current contracts
      thresholdCount: 50       # Percentage increase to trigger Warning

Alert Details

Each alert item includes:

• Current publication contract count for the lookback window
• Historical average count
• Percentage increase above average
• Number of baseline samples used
• Link to the IB Monitor page

How to Respond

1. Navigate to the IB Monitor in psLens to see which publication contracts are generating the volume
2. Check for contracts in Error status (see IB Publication Contract Errors alert)
3. Review whether retries are inflating the count. Repeated errors create new contract instances.
4. Consider using the Daily IB Volume report to compare against historical patterns

Tables Queried

9 - Integration Broker Down

Integration Broker Down Alert

Alert ID: ib_down Category: Integration Broker Default threshold: Immediate

What This Alert Detects

This alert fires when the SWS endpoint returns a connection error, timeout, 404, or 5xx. psLens cannot fetch IB data without SWS, so this also blocks every other psLens IB check. A connection failure indicates that the Integration Broker gateway, local node, or application server is down.

Severity Logic

What Gets Checked

The alert invokes the standard SWS connection test method. If it receives any network connection error (e.g., HTTP gateway timeout, connection refused, dns resolving failure, or HTTP 404/500 errors), a Critical alert is raised immediately.

Alert Details

Each alert item includes:

• The base REST URL of the SWS endpoint (baseURL)
• The raw network or connection error message
• Troubleshooting links to verify connection settings

Configuration

alerts:
  checks:
    ib_down:
      enabled: true

How to Respond

1. Verify that the PeopleSoft Web Server and PIA are up and running.
2. Check the Integration Gateway web application status (typically /PSIGW/PeopleSoftListeningConnector).
3. Ensure that the application server domain is booted and the Integration Broker handlers/dispatchers are active.
4. Verify there are no firewalls, proxies, or security policies blocking outbound HTTPS traffic from the psLens server to the Integration Broker gateway port.
5. Inspect the basic auth credentials in the config.yaml to ensure the API service user has not expired or been locked.

10 - Abnormal IB Subscription Contract Volume

Abnormal IB Subscription Contract Volume Alert

Alert ID: ib_sub_contract_volume Category: Integration Broker

What This Alert Detects

This alert detects when the volume of IB subscription contracts (PSAPMSGSUBCON) is significantly higher than the rolling historical average. Subscription contracts represent inbound messages being processed by local subscribers, so a spike here can indicate a flood of inbound messages, excessive retries, or an integration firing more frequently than expected.

How Baselining Works

psLens maintains a rolling history of up to 288 volume snapshots (approximately 24 hours at a 5-minute check interval). Each check cycle records the current subscription contract count for the lookback window.

Once at least 6 baseline snapshots have accumulated, the alert begins comparing the current count against the historical average.

Severity Logic

For example, with the default threshold of 50%:

• Current count is 75% above average → Warning
• Current count is 100%+ above average → Critical

Configuration

alerts:
  checks:
    ib_sub_contract_volume:
      enabled: true
      lookbackHours: 1         # Window for counting current contracts
      thresholdCount: 50       # Percentage increase to trigger Warning

Alert Details

Each alert item includes:

• Current subscription contract count for the lookback window
• Historical average count
• Percentage increase above average
• Number of baseline samples used
• Link to the IB Monitor page

How to Respond

1. Navigate to the IB Monitor in psLens to see which subscription contracts are generating the volume
2. Check for contracts in Error status (see IB Subscription Contract Errors alert)
3. Identify which operations are receiving the high volume of inbound messages
4. Contact the sending system’s administrators if the volume is unexpected
5. Consider using the Daily IB Volume report to compare against historical patterns

Tables Queried

11 - IB No Active Domain

IB No Active Domain Alert

Alert ID: ib_no_active_domain Category: Integration Broker Default threshold: Immediate

What This Alert Detects

This alert triggers when no active message domains are found in the Integration Broker. In PeopleSoft, message domains correspond to individual application servers running the Integration Broker background processes. If there are no active domains, Integration Broker cannot route, dispatch, or process any asynchronous publication or subscription messages.

Severity Logic

What Gets Checked

The alert queries the PSAPMSGDOMSTAT table to retrieve all registered domains. It counts the domains where the status is "A" (Active). If the count of active domains is zero, it triggers a Critical alert.

Alert Details

Each alert item includes:

• The total count of registered domains
• A list of inactive domains along with their machine names and app server paths
• A link to the Integration Broker Monitor on the dashboard to inspect the domain statuses

Configuration

alerts:
  checks:
    ib_no_active_domain:
      enabled: true

How to Respond

1. Log into the PeopleSoft server or use PSAdmin to check the status of the application server domains.
2. Verify if the Quick-Start or normal app server configuration has Integration Broker (Pub/Sub) processes enabled.
3. If the application server was recently restarted, check if the domains were configured to boot automatically or if they need to be booted manually.
4. Check system logs for application server boot crashes, memory errors, or database connection failures.

12 - IB Dispatcher Down

IB Dispatcher Down Alert

Alert ID: ib_dispatcher_down Category: Integration Broker Default threshold: 10 minutes

What This Alert Detects

This alert monitors Integration Broker dispatcher processes (such as the publication dispatcher, subscription dispatcher, or handler dispatchers) on active domains and triggers when any dispatcher is inactive or has stopped reporting health updates.

When a dispatcher process is down, messages assigned to that dispatcher fail to process and queue up indefinitely, leading to a backlog.

Severity Logic

For example, with the default threshold of 10 minutes:

• No health update for 11 minutes → Warning
• No health update for 20+ minutes → Critical

What Gets Checked

The alert queries PSAPMSGDSPSTAT for dispatcher process statuses. For each dispatcher associated with an Active domain (from PSAPMSGDOMSTAT), it verifies:

1. The status string is "ACT" (Active).
2. The health timestamp (DspHealthDttm) has been updated within the threshold window.

Note: Dispatchers on inactive domains are ignored by this check (they are covered by the IB No Active Domain check).

Alert Details

Each alert item includes:

• Dispatcher process name
• Physical machine/host name
• App server path
• The last updated health timestamp
• Reason for the down status (e.g. status not active, or elapsed minutes since last update)
• A link to the Integration Broker Monitor page

Configuration

alerts:
  checks:
    ib_dispatcher_down:
      enabled: true
      thresholdMinutes: 10

How to Respond

1. Go to the Integration Broker Monitor on the psLens dashboard to identify which specific dispatcher on which host is failing.
2. Log into the affected PeopleSoft application server and check the status of the dispatcher processes via PSAdmin.
3. Review the application server and Pub/Sub subdirectories log files (such as APPSRV.log, TUXLOG, or stderr / stdout logs) for crashes, Tuxedo errors, or database lockups.
4. Restart the Pub/Sub processes on the application server if the dispatcher has locked up or crashed.

13 - IB Sync Operation Exceptions

IB Sync Operation Exceptions Alert

Alert ID: ib_sync_exceptions Category: Integration Broker

What This Alert Detects

This alert detects synchronous IB service operations that have logged errors in the sync transaction log (PSIBLOGHDR) within the configured lookback window. Results are aggregated by operation name to avoid noise from high-volume sync operations that may have individual errors but function normally overall.

Unlike async IB errors (which create persistent queue entries), sync operation errors are logged transiently in PSIBLOGHDR. Without active monitoring, these errors are often missed until a consuming system reports a problem.

Severity Logic

All findings are reported at Critical severity. Sync errors are always surfaced immediately because they affect real-time integrations.

What Gets Checked

The alert queries PSIBLOGHDR for records with error status within the lookback window, aggregated by IB operation name. Only operations with at least one error are reported. Operations in the exclude list are skipped.

Alert Details

Each alert item includes:

• Service operation name (with link to Service Operation detail page)
• Number of errors in the lookback window
• Lookback window in hours

Configuration

alerts:
  checks:
    ib_sync_exceptions:
      enabled: true
      lookbackHours: 24          # How far back to check for sync errors
      excludeOperations: []      # Operation names to ignore

How to Respond

1. Click the alert link to open the Service Operation detail page
2. Review the operation’s handler configuration and routing
3. Check PSIBLOGHDR directly for the specific error messages (psLens links to the operation, not individual log entries)
4. Verify that the operation’s service handler is still functional and the underlying code has not changed
5. Contact the calling system to understand whether they are seeing failures on their end
6. Use the Sync Operations Without Logging report to identify any sync operations that are not logging and may have undetected errors

Prerequisites

The PSIBLOGHDR table must be whitelisted in the PeopleSoft SWS framework on each target environment. Sync logging must be enabled on the relevant routings. If logging is disabled, errors will not appear in PSIBLOGHDR. See the Sync Operations Without Logging report to find operations where logging may be disabled.

Tables Queried

14 - IB Nodes Down

IB Nodes Down Alert

Alert ID: ib_nodes_down Category: Integration Broker Default threshold: Immediate

What This Alert Detects

This alert triggers when there are entries in the PeopleSoft table PSNODESDOWN. An entry in this table indicates that a message node is blocked or offline.

When the Integration Broker attempts to publish a message to an external or remote node and the connection fails, the system registers the node as “down” in PSNODESDOWN. While the node is down, all subsequent publication contracts to that node are automatically paused/blocked and remain queued in the database until the node status is resolved.

Severity Logic

What Gets Checked

The alert queries PSNODESDOWN for any active rows. For each row found, it groups the results by message node and counts the number of blocked transactions.

Alert Details

Each alert item includes:

• The name of the offline message node
• The count of blocked transactions queueing for the node
• A link to the Node detail page in the browser showing connection diagnostics

Configuration

alerts:
  checks:
    ib_nodes_down:
      enabled: true

How to Respond

1. Click the node link in the alert to inspect the node configuration and test connectivity.
2. Check if the external target service (e.g. an external API gateway, third-party system, or another PeopleSoft application node) is offline or undergoing maintenance.
3. Verify network routing and gateway connector configurations.
4. Once the external endpoint is confirmed to be healthy, delete the down-node status entry in PeopleSoft (typically via the Service Operations Monitor > Administration > Nodes Down page) and resubmit or force retry the stalled publication contracts.