Alert fatigue has a well-defined failure signature: the system generates dozens of alerts per shift, operators acknowledge them without acting, the alert log fills with entries where "acknowledged" and "resolved" timestamps are identical — meaning the operator dismissed the notification without a response. After a few weeks, something genuinely important fires, and nobody notices because every notification looks like every other notification.
This pattern is documented in process industries under the framework of IEC 62682 (Management of Alarm Systems for the Process Industries), which was developed specifically in response to industrial accidents where alarm floods — hundreds of alerts per hour during upset conditions — contributed to operator failure to respond to critical signals. The manufacturing version of alarm flood is less acute but equally damaging to the system's usefulness over time.
Threshold tuning is the technical mechanism for addressing it. But threshold tuning without a systematic approach tends to produce oscillation: tune thresholds up to reduce noise, observe that something important is now missed, tune them back down, restore the noise. The solution requires a framework, not just a parameter adjustment.
Why Factory Floor Alerting Is Different from Process Plant Alarms
The IEC 62682 framework is a useful reference for principles but was designed for continuous process environments — petroleum refining, chemical plants, power generation — where equipment runs continuously, process variables change continuously, and alarms protect against dangerous deviations. Discrete manufacturing has different dynamics that affect how alert logic should be designed.
First, discrete manufacturing has defined production cycles. A PLC-driven assembly line has a known cycle time. An OEE deviation alert that fires when cycle time exceeds 110% of ideal for a single cycle is noisy; the same alert firing when cycle time exceeds 110% for 5 consecutive cycles is more meaningful because it indicates a sustained deviation rather than a transient. Cycle-count-based confirmation windows are a natural fit for discrete manufacturing that have no equivalent in continuous process alerting.
Second, discrete manufacturing has scheduled downtime that looks like unplanned downtime to a simple threshold rule. A packaging line that stops for a changeover will trigger a "line stopped" alert if the alert logic does not distinguish between planned and unplanned stops. The alert system needs awareness of the production schedule — or at minimum, operator-initiated production mode flags that suppress downtime alerts during planned changeovers and breaks.
Third, discrete manufacturing operators are frequently responsible for multiple machines or a full line section simultaneously. A mobile alert system that fires a push notification for each of 8 monitored signals independently generates multiplicative noise. Alert grouping — aggregating related signals into a single notification describing the production state — is more appropriate for a multi-machine operator than individual per-signal alerts.
The Four-Step Tuning Process
Effective threshold tuning for factory floor alert systems follows a systematic sequence that starts with data before touching configuration.
Step 1: Audit current alert volume. Pull the alert log for the past 4 weeks. Count total alerts fired per shift per line, and break down by alert type. Identify the 3 alert types with the highest volume. For most over-configured systems, 60-80% of alert volume comes from 2-3 signal types. These are the starting points for tuning, not the full alert library.
Step 2: Calculate the response rate for each alert type. The response rate is the percentage of alerts where there is a confirmed operator response within the expected response window — an acknowledgment followed by a resolution note or a maintenance action within 30 minutes. Alert types with response rates below 40% are likely generating excessive noise for the value they provide. Alert types with response rates above 90% are well-calibrated.
Step 3: For low-response-rate alerts, identify the cause. There are three common patterns. Threshold too sensitive: the alert fires on normal process variation that does not require action. Missing context: the alert fires on a real deviation but operators cannot determine what action to take from the alert information alone. Wrong recipient: the alert fires correctly but is routed to operators who cannot address the underlying cause without maintenance or engineering involvement.
Step 4: Adjust one alert type at a time, wait 2 weeks, re-measure. Adjusting multiple alerts simultaneously makes it impossible to attribute changes in response rate or missed-event frequency to specific tuning actions. The discipline of one-at-a-time adjustment is slow but produces a tuning history that is interpretable — which matters when the next new engineer joins and wants to understand why the current thresholds are what they are.
A Worked Example: Cycle Time Deviation on a Welding Cell
A mid-size welding and fabrication facility in Chiba Prefecture was running a Yaskawa MA1440 arc welding robot on a parts-feeding cell, monitored via Modbus TCP polling from a process intelligence layer. Initial alert configuration: cycle time deviation alert at 15% above ideal cycle time (8.2 seconds per part), firing after any single cycle exceeding the threshold.
After 3 weeks, the alert log showed 62 alerts per shift on this cell, with an operator response rate of 11%. The alert was firing constantly on the first 2-3 cycles after a pallet changeout — a normal operational sequence where the parts loading creates brief cycle time variation. It was also firing on legitimate cycle extensions caused by the robot's seam-tracking algorithm engaging for welds with higher geometric variation.
Tuning sequence: First, the threshold was raised from 15% to 25% to eliminate false positives from normal pallet changeout variation. Alert volume dropped to 31 per shift; response rate climbed to 34%. Still too noisy. Second, a confirmation window was added: alert only after 4 consecutive cycles at or above the threshold. This eliminated the seam-tracking variation spikes, which typically lasted 1-2 cycles. Alert volume dropped to 8 per shift; response rate climbed to 79%. Third, alert routing was changed to also include the welding process engineer, not just the line operator — because the remaining alerts were primarily indicating consumable wear (nozzle or wire feed issues) that required maintenance attention, not just operator observation.
After tuning, 8 alerts per shift with 79% response rate. The previous 62 alerts per shift with 11% response rate was not "more monitoring" — it was effectively no monitoring, because operators had learned to dismiss the alerts reflexively.
The Role of Sustained-Deviation vs. Instantaneous Alerts
One of the most powerful threshold design choices for factory floor alerting is the distinction between instantaneous alerts (fire when a single sample exceeds threshold) and sustained-deviation alerts (fire when the moving average or a sliding window of samples exceeds threshold for a defined duration).
Instantaneous alerts are appropriate for hard limits: equipment fault codes, safety interlock triggers, machine E-stop events. These require immediate response regardless of duration. Sustained-deviation alerts are appropriate for performance and quality monitoring: OEE dropping below target, cycle time running slow, rejection rate climbing above baseline. Performance deviations are more meaningful when sustained than when momentary.
For Omron Sysmac NX-series controllers with structured data logging, sustained-deviation alerts can be computed on the edge, using the controller's built-in data analytics functions introduced in firmware 1.40. This keeps the alert computation close to the data source and reduces the latency and network dependency of routing raw time-series data to a cloud layer for threshold evaluation. For older controllers without this capability, the edge agent handling data collection can perform the windowed evaluation locally.
Maintaining Alert Hygiene Over Time
Alert configuration has a natural tendency to accumulate: new signals are added when new problems are observed, but old alerts are rarely removed when they become obsolete. A line configuration change, a new product family with different cycle characteristics, or a maintenance improvement that eliminates a recurring fault source — all of these can make previously calibrated alerts newly miscalibrated, without anyone noticing because the original justification for the alert is no longer remembered.
A practical maintenance protocol: quarterly review of alert response rates. Any alert type with response rate below 50% for two consecutive quarters gets reviewed for reconfiguration or removal. Any alert type that has not fired in 6 months gets reviewed for relevance — either it is well-calibrated to detect rare but important events (keep it), or the condition it was monitoring has been resolved (archive it). This review takes about an hour per line per quarter. It is the operational equivalent of the 5S methodology applied to the alert configuration itself: keep what is useful, remove what creates clutter.
An alert system that operators trust is more valuable than one that covers every conceivable signal. That trust is built incrementally, by demonstrating that the alerts that fire are worth responding to — and that when an alert fires and the operator responds, they can actually do something useful with the information. Threshold tuning is the technical work; the measure of success is behavior on the production floor.