Long-Running Processes

Long-Running Processes Alert

Alert ID: long_running_processes Category: Process Scheduler Default threshold: Dynamic (4x rolling median, min 10 min baseline) or static fallback (20 minutes)

What This Alert Detects

This alert finds Process Scheduler requests that are currently in Initiated or Processing status and have been running longer than their expected duration.

Instead of relying solely on a static global threshold, psLens dynamically calculates an expected duration for each process using its own history:

• It fetches the last 10 successful run durations for the specific process name (PRCSNAME).
• It calculates the rolling median runtime of those successful runs.
• It raises an alert if the current execution duration exceeds a multiple of the median (default: $4\times$ the median, with a minimum baseline of 10 minutes).
• If no successful runs are found (e.g. a brand new process or a process that has never completed successfully), it falls back to the static global thresholdMinutes (default: 20 minutes).

A process that has been running for a long time may be stuck, consuming excessive server resources, or waiting on a lock or resource that will never become available.

Severity Logic

For example:

• If a process usually runs in 3 minutes: median is 3m. $4\times 3 = 12$ minutes. The alert triggers if it runs for 12 minutes or more.
• If a process usually runs in 1 minute: median is 1m. $4\times 1 = 4$ minutes. Since this is below the 10-minute minimum baseline, the baseline is used: the alert triggers if it runs for 10 minutes or more.
• If a process has no history: fallback is used. The alert triggers if it runs for 20 minutes or more.

What Gets Checked

The alert queries the Process Scheduler request table for processes in run status 6 (Initiated) or 7 (Processing). For each result, it calculates how long the process has been running based on its BeginDttm (begin datetime) and the current server time.

Processes with no BeginDttm value are skipped (the process hasn’t truly started yet).

Alert Details

Each alert item includes:

• Process name (PRCSNAME)
• Process instance number
• How long the process has been running (in minutes)
• The rolling median runtime (if available, in minutes)
• The operator who submitted the request
• A link to the Process Monitor detail page for that instance

Configuration

alerts:
  checks:
    long_running_processes:
      enabled: true
      thresholdMinutes: 20            # Fallback static minutes when history is empty
      anomalyMultiplier: 4.0          # Multiplier applied to rolling median
      anomalyMinBaselineMinutes: 10   # Minimum baseline runtime before alerting
      excludeProcesses:               # Process names to skip
        - SOME_LONG_BATCH_JOB

How to Respond

1. Click the alert link to go directly to the Process Monitor entry for the flagged process
2. Review the process details: what it is, who submitted it, when it started
3. Check whether the process appears to be making progress or is stuck
4. If the process is genuinely stuck, you may need to cancel it from PeopleSoft’s Process Monitor
5. Investigate why it got stuck: look for locks, resource contention, or data issues

Tuning the Threshold

The right threshold depends on your environment. You can adjust anomalyMultiplier or anomalyMinBaselineMinutes globally or per-database to reduce noise, or use excludeProcesses to ignore specific jobs entirely.