Process Scheduler

• 1: Long-Running Processes
• 2: Process Errors
• 3: Backlogged Processes
• 4: Locked OPRID Scheduled Processes
• 5: Queue Latency
• 6: Process Run Check
• 7: Process Scheduler Down
• 8: No Process Completed
• 9: Stalled Recurrences

1 - Long-Running Processes

Long-Running Processes Alert

Alert ID: long_running_processes Category: Process Scheduler Default threshold: Dynamic (4x rolling median, min 10 min baseline) or static fallback (20 minutes)

What This Alert Detects

This alert finds Process Scheduler requests that are currently in Initiated or Processing status and have been running longer than their expected duration.

Instead of relying solely on a static global threshold, psLens dynamically calculates an expected duration for each process using its own history:

• It fetches the last 10 successful run durations for the specific process name (PRCSNAME).
• It calculates the rolling median runtime of those successful runs.
• It raises an alert if the current execution duration exceeds a multiple of the median (default: $4\times$ the median, with a minimum baseline of 10 minutes).
• If no successful runs are found (e.g. a brand new process or a process that has never completed successfully), it falls back to the static global thresholdMinutes (default: 20 minutes).

A process that has been running for a long time may be stuck, consuming excessive server resources, or waiting on a lock or resource that will never become available.

Severity Logic

For example:

• If a process usually runs in 3 minutes: median is 3m. $4\times 3 = 12$ minutes. The alert triggers if it runs for 12 minutes or more.
• If a process usually runs in 1 minute: median is 1m. $4\times 1 = 4$ minutes. Since this is below the 10-minute minimum baseline, the baseline is used: the alert triggers if it runs for 10 minutes or more.
• If a process has no history: fallback is used. The alert triggers if it runs for 20 minutes or more.

What Gets Checked

The alert queries the Process Scheduler request table for processes in run status 6 (Initiated) or 7 (Processing). For each result, it calculates how long the process has been running based on its BeginDttm (begin datetime) and the current server time.

Processes with no BeginDttm value are skipped (the process hasn’t truly started yet).

Alert Details

Each alert item includes:

• Process name (PRCSNAME)
• Process instance number
• How long the process has been running (in minutes)
• The rolling median runtime (if available, in minutes)
• The operator who submitted the request
• A link to the Process Monitor detail page for that instance

Configuration

alerts:
  checks:
    long_running_processes:
      enabled: true
      thresholdMinutes: 20            # Fallback static minutes when history is empty
      anomalyMultiplier: 4.0          # Multiplier applied to rolling median
      anomalyMinBaselineMinutes: 10   # Minimum baseline runtime before alerting
      excludeProcesses:               # Process names to skip
        - SOME_LONG_BATCH_JOB

How to Respond

1. Click the alert link to go directly to the Process Monitor entry for the flagged process
2. Review the process details: what it is, who submitted it, when it started
3. Check whether the process appears to be making progress or is stuck
4. If the process is genuinely stuck, you may need to cancel it from PeopleSoft’s Process Monitor
5. Investigate why it got stuck: look for locks, resource contention, or data issues

Tuning the Threshold

The right threshold depends on your environment. You can adjust anomalyMultiplier or anomalyMinBaselineMinutes globally or per-database to reduce noise, or use excludeProcesses to ignore specific jobs entirely.

2 - Process Errors

Process Errors Alert

Alert ID: process_errors Category: Process Scheduler Default lookback: 24 hours

What This Alert Detects

This alert finds Process Scheduler requests that have failed within a configurable lookback window. It catches processes that ended in one of three error statuses:

Severity Logic

• Recurring Processes: Any failure fires Critical immediately.
• Non-Recurring Processes: Fire Warning after the thresholdMinutes grace period.

Alert Details

Each alert item includes:

• Process name and instance number
• Run status label (Error, Not Successful, Unable to Post)
• The operator who submitted the request
• When the process ran
• A link to the Process Monitor detail page for that instance

Configuration

alerts:
  checks:
    process_errors:
      enabled: true
      lookbackHours: 24        # How far back to look for failures
      thresholdMinutes: 15     # Grace period buffer in minutes for non-recurring errors
      excludeProcesses:        # Process names to skip
        - KNOWN_FLAKY_PROCESS

How to Respond

1. Click the alert link to go directly to the Process Monitor entry for the failed process
2. Review the process details: run status, begin and end times, server
3. Look for output files or log information that might explain the failure
4. Check whether this is a one-time failure or a repeating issue
5. If the process needs to be rerun, submit a new request from PeopleSoft

Common Causes of Process Failures

• Data errors: The process encountered unexpected data (null values, bad formats, constraint violations)
• Resource issues: The server ran out of memory or disk space
• Timeout: The process exceeded its allowed run time
• Configuration problems: A required configuration parameter is missing or incorrect
• Dependency failures: A process that runs after another failed because the first one didn’t complete correctly

Reducing Alert Noise

If certain processes fail regularly and you’re already tracking them separately, add them to excludeProcesses to keep the alert list focused on unexpected failures.

3 - Backlogged Processes

Backlogged Processes Alert

Alert ID: backlogged_processes Category: Process Scheduler Default threshold: 30 minutes

What This Alert Detects

This alert finds Process Scheduler requests that are in Queued or Blocked status and whose scheduled run time (RUNDTTM) has already passed by more than the configured threshold. This alert focuses exclusively on processes currently waiting to start. Once a process begins running, it is cleared from this alert and is instead tracked by the Queue Latency alert.

Severity Logic

For example, with the default threshold of 30 minutes:

• A process scheduled 40 minutes ago that is still queued → Warning
• A process scheduled 65 minutes ago that is still queued → Critical

What Gets Checked

The alert queries the Process Scheduler request table for processes in run status 5 (Queued) or 18 (Blocked) whose RUNDTTM (scheduled run datetime) is in the past. For each result, it calculates how far past the scheduled time the process is based on RUNDTTM and the current server time.

Processes with no RUNDTTM value are skipped. Completed or active runs are skipped.

Alert Details

Each alert item includes:

• Process name (PRCSNAME)
• Process instance number
• How long the process is overdue (in minutes)
• Current run status (Queued or Blocked)
• The operator who submitted the request
• A link to the Process Monitor detail page for that instance

Configuration

alerts:
  checks:
    backlogged_processes:
      enabled: true
      thresholdMinutes: 30         # Minutes overdue before flagging as Warning
      excludeProcesses:            # Process names to skip
        - SOME_LOW_PRIORITY_JOB

How to Respond

1. Click the alert link to go directly to the Process Monitor entry for the flagged process
2. Check whether the Process Scheduler server is running and accepting work
3. Look at how many processes are currently running on the server. It may have hit its concurrency limit
4. Check if the process type or class has reached its maximum allowed concurrent instances
5. For blocked processes, investigate what is blocking them (dependencies, server restrictions, etc.)
6. If the Process Scheduler server is down, restart it from PeopleSoft’s Process Scheduler administration

Tuning the Threshold

The right threshold depends on how busy your Process Scheduler is. In environments where many jobs are submitted at once, some queuing is normal. Set thresholdMinutes high enough to avoid false positives during peak batch windows but low enough to catch genuine problems. You can also use excludeProcesses to exclude specific low-priority processes that are known to queue for long periods.

4 - Locked OPRID Scheduled Processes

Locked OPRID Scheduled Processes Alert

Alert ID: locked_oprid_processes Category: Process Scheduler

What This Alert Detects

This alert finds queued or scheduled Process Scheduler requests where the submitting operator’s account (OPRID) is currently locked in PSOPRDEFN (ACCTLOCK = 1).

When an operator account is locked after a process has been queued, PeopleSoft will refuse to run the process, or run it under the locked account and immediately fail. PeopleSoft does not surface this condition anywhere obvious: Process Monitor shows the job queued, the operator’s user page shows them locked, but nothing connects the two. This alert does.

Common scenarios:

• A service or batch account had its password expire and was locked
• An employee left and their account was locked, but scheduled jobs were not transferred
• A security lockout from failed login attempts affected a batch account

Severity Logic

All findings are reported at Warning severity. Every queued or scheduled process with a locked submitting account is flagged.

What Gets Checked

The alert queries PSPRCSRQST joined to PSOPRDEFN for process requests in Queued or Scheduled run status where the submitting OPRID has ACCTLOCK = 1.

Alert Details

Each alert item includes:

• Process name and instance number
• Submitting OPRID (with link to User detail page)
• Current run status (Queued, Scheduled, etc.)
• Scheduled run date/time
• Recurrence name (if applicable)

Configuration

alerts:
  checks:
    locked_oprid_processes:
      enabled: true
      excludeProcesses: []   # Process names to ignore

How to Respond

1. Click the alert link to open the Process Monitor detail page for the affected instance
2. Identify the locked OPRID shown in the alert
3. Navigate to the User detail page to review the account lock status
4. Either unlock the account (if appropriate) or re-queue the process under an active operator account
5. For recurring processes, update the recurrence definition to use a non-locked operator
6. Investigate why the account was locked. If it was a failed login lockout, check the Failed Logins alert for additional context

Tables Queried

5 - Queue Latency

Queue Latency Alert

Alert ID: queue_latency Category: Process Scheduler Default threshold: 15 minutes

What This Alert Detects

This page documents the queue latency alert, which monitors the delay between a process’s scheduled run time and its actual start time. It queries the PSPRCSRQST table for processes that have started running or completed within the lookback window and calculates the delay as BEGINDTTM - RUNDTTM.

Severity Logic

For example, with the default threshold of 15 minutes:

• A process scheduled for 10:00 that started running at 10:20 → Warning

What Gets Checked

The alert queries the Process Scheduler request table (PSPRCSRQST) for processes whose BEGINDTTM (begin datetime) is in the past lookback window (default 24 hours). For each process, it calculates the difference between BEGINDTTM and RUNDTTM (scheduled run datetime).

Processes that have not started running yet (empty BEGINDTTM value) are skipped. Active queuing checks are handled by the backlogged_processes alert.

Alert Details

Each alert item includes:

• Process name (PRCSNAME)
• Process instance number
• Start delay duration (in minutes)
• Current run status (Initiated, Processing, Success, Error, etc.)
• The operator who submitted the request
• A link to the Process Monitor detail page for that instance

Configuration

alerts:
  checks:
    queue_latency:
      enabled: true
      thresholdMinutes: 15         # Minutes delay before flagging as Warning
      lookbackHours: 24            # Hours to look back for completed/running processes
      excludeProcesses:            # Process names to skip
        - LOW_PRIORITY_AE

How to Respond

1. Click the alert link to go to the Process Monitor entry for the flagged process.
2. Review the process server definition to identify if it is running and accepting work.
3. Check the max concurrent limits configured on the Process Scheduler server or category definitions.
4. Verify if other higher-priority processes occupied all available channels.
5. If the delay is caused by category stalls, adjust the process class concurrency settings.

Tuning the Threshold

Environments with heavy batch schedules may experience normal queue delays during peak hours. Set thresholdMinutes high enough to prevent alerts on minor delays but low enough to flag scheduler capacity bottlenecks or server category stalls.

6 - Process Run Check

Process Run Check Alert

Alert ID: process_run_check Category: Process Scheduler

What This Alert Detects

This alert monitors configured critical processes and fires when one has not completed successfully within its expected time window. It is the alert equivalent of the Process Run Check report. The difference is that this runs automatically on every check cycle and surfaces failures on the dashboard without any manual action.

Use this alert for processes that must run on a regular cadence, such as:

• Nightly batch jobs that must complete before business hours
• Data synchronization processes that run every few hours
• Critical integrations that should run multiple times per day
• Post-maintenance verification of essential processes

Severity Logic

Configuration

Process checks are configured per process name in config.yaml. Each entry specifies the process name and the number of hours within which a successful run is expected.

alerts:
  checks:
    process_run_check:
      enabled: true
      processChecks:
        SOMEJOBNAME: 24      # Must run successfully within 24 hours
        ANOTHERJOB: 8        # Must run successfully within 8 hours
        NIGHTLY_ETL: 12      # Must run successfully within 12 hours

If a process name is listed with 0 or a negative value, the check defaults to a 24-hour window.

What Gets Checked

For each configured process, psLens queries PSPRCSRQST for successful runs (RunStatus = 9 / Success) within the configured time window. If none are found, it then checks for any run history to determine severity:

• No successful run in window + recent run history found: Warning
• No run history at all: Critical

Alert Details

Each alert item includes:

• Process name
• Configured threshold (hours)
• Last known run status (if any history exists)
• Last known run time (if any history exists)
• Link to the Process Definition detail page

How to Respond

1. Click the alert link to open the Process Definition detail page for the affected process
2. Review recent run history to understand what happened. Did the process run but fail, or did it not run at all?
3. Check the Process Scheduler server configuration if the process never ran
4. Investigate error logs if the process ran but ended in a failed state
5. If the process ran and succeeded but outside the expected window, consider adjusting the threshold in config.yaml

Tables Queried

7 - Process Scheduler Down

Process Scheduler Down Alert

Alert ID: process_scheduler_down Category: Process Scheduler Default threshold: 10 minutes

What This Alert Detects

This alert triggers when any active Process Scheduler server registered in PSSERVERSTAT has not reported a status update (heartbeat) within the configured amount of time.

Severity Logic

For example, with the default threshold of 10 minutes:

• A scheduler that hasn’t heartbeat’ed for 12 minutes → Warning
• A scheduler that hasn’t heartbeat’ed for 22 minutes → Critical

What Gets Checked

The alert queries the PSSERVERSTAT table to retrieve all server status definitions. For each active scheduler (status not Down/Offline), it calculates the elapsed time since its LASTUPDDTTM timestamp. If that time exceeds the configured threshold, the alert fires.

Alert Details

Each alert item includes:

• Server name (SERVERNAME)
• Current status code and friendly string status (e.g., Running, Error, Suspended)
• Last heartbeat timestamp (LASTUPDDTTM)
• Host name (SRVRHOSTNAME)
• A detailed explanation of how long the heartbeat has been stale
• A link to the Server Definition detail page for that server

Configuration

alerts:
  checks:
    process_scheduler_down:
      enabled: true
      thresholdMinutes: 10          # Minutes stale before flagging as Warning
      excludeProcesses:             # Server names (e.g., PSUNX, PSNT) to skip
        - PSUNX_OLD

How to Respond

1. Click the alert link to go directly to the Server Definition detail page for the affected scheduler.
2. Check the Host Name where the Process Scheduler daemon runs.
3. Access the server host and verify whether the Process Scheduler processes (e.g., psadmin, PSAESRV, etc.) are running.
4. Review the Process Scheduler logs (e.g., TUXLOG, SCHED_*.LOG) on the host machine to diagnose why the process has hung or crashed.
5. If the scheduler has hung, stop the process scheduler daemon and restart it using psadmin.
6. If the server definition is obsolete or decommissioned, consider deleting it in PeopleSoft Server Definitions configuration to clean up the PSSERVERSTAT row.

8 - No Process Completed

No Process Completed Alert

Alert ID: no_process_completed Category: Process Scheduler Default lookback: 1 hour

What This Alert Detects

This alert fires when no process has successfully completed within the configured lookback window. It is a broad scheduler health check. If nothing has finished successfully in the past hour, the Process Scheduler may be down, stalled, or not dispatching jobs.

This is distinct from the Process Run Check, which monitors specific named processes. This alert monitors overall scheduler activity.

Severity Logic

What Gets Checked

The alert queries PSPRCSRQST for any process with RunStatus = 9 (Success) and an end datetime within the lookback window. If no rows are returned, the alert fires.

Only one result is needed to resolve the alert. The check uses a limit of 1 for efficiency.

Alert Details

When firing, the alert produces a single item:

• Summary: No process completed successfully in the last N hour(s)
• Lookback hours used for the check

Configuration

alerts:
  checks:
    no_process_completed:
      enabled: true
      lookbackHours: 1    # How far back to look for completed processes

How to Respond

1. Check PeopleSoft’s Process Monitor to see if any processes are running, queued, or have recently completed
2. Verify the Process Scheduler server is running (PeopleSoft > PeopleTools > Process Scheduler > Servers)
3. If processes are queued but not running, the scheduler daemon may need to be restarted
4. If this fires regularly during off-hours when no jobs run, increase lookbackHours or disable the alert for those periods

Tuning

If your environment has periods where no batch jobs are expected to run (e.g., overnight maintenance windows), consider increasing lookbackHours to cover those gaps, or disable the alert entirely during those windows.

9 - Stalled Recurrences

Stalled Recurrences Alert

Alert ID: stalled_recurrences Category: Process Scheduler Default lookback: 336 hours (14 days)

What This Alert Detects

This alert fires when a scheduled recurring process has finished a run recently (within the lookback window) but does not have a subsequent scheduled instance.

In PeopleSoft, the Process Scheduler is responsible for scheduling the next run of a process based on its recurrence definition when the current one runs. If the scheduler is interrupted, a database is refreshed, or an error occurs during the scheduling process, the recurring job can fall off the schedule. This check identifies those occurrences so they do not go unnoticed.

Severity Logic

What Gets Checked

The alert queries PSPRCSRQST for all process requests that have a recurrence name assigned. It then groups these requests by:

• Process Name (PRCSNAME)
• Run Control ID (RUNCNTLID)
• User ID (OPRID)
• Recurrence Name (RECURNAME)

For each unique combination, if the latest instance has a completed status and there are no active/pending requests (such as Queued, Blocked, Hold, Pending, Initiated, or Processing) to follow, the recurrence is flagged as stalled.

Alert Details

When firing, the alert produces an item for each stalled recurrence:

• Summary: Recurrence RECURNAME for PRCSNAME — Stalled (Last status: STATUS)
• Details:
  • prcsName: The process name
  • runCntlId: The run control ID
  • oprid: The user ID that submitted the process
  • recurName: The recurrence definition name
  • lastInstance: The process instance number of the last run
  • lastRunStatus: The status of the last run (e.g. Success, Error)
  • lastRunDttm: The date and time the last run was scheduled

Configuration

alerts:
  checks:
    stalled_recurrences:
      enabled: true
      lookbackHours: 336   # How far back to look for completed runs (14 days)

How to Respond

1. Click the link in the alert detail to view the last process instance in the Process Monitor.
2. Check the logs for that instance if the status was an error.
3. If the recurrence should continue running, go to PeopleSoft and submit the process again on the same recurrence name using the matching User ID and Run Control ID.
4. If the recurrence has intentionally finished its life cycle (e.g., reached its end date), you can ignore the warning or configure the process in the exclusions list.