0035329: scheduled processes are not executed in some cases (ie. after some OOM)

Openbravo Issue Tracking System - Openbravo ERP

View Issue Details

Project

Category

View Status

Date Submitted

Last Update

0035329

Openbravo ERP

A. Platform

public

2017-02-21 16:30

2022-02-01 08:09

Reporter

alostale

Assigned To

Triage Platform Base

Priority

normal

Severity

major

Reproducibility

random

Status

acknowledged

Resolution

open

Platform

OS Version

Product Version

Target Version

Fixed in Version

Merge Request Status

Review Assigned To

OBNetwork customer

Web browser

Modules

Core

Support ticket

Regression level

Regression date

Regression introduced in release

Regression introduced by commit

Triggers an Emergency Pack

Summary

0035329: scheduled processes are not executed in some cases (ie. after some OOM)

Description

If QuartzSchedulerThread throws an unhandled exception (it handles RuntimeException [1]), this thread gets terminated not being possible to schedule any new process till Tomcat is restarted.

This exceptional situation is difficult to reproduce but it has been observed in some real customers in OOM.

Exception in thread "DefaultQuartzScheduler_QuartzSchedulerThread" java.lang.OutOfMemoryError: Java heap space
        at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:62)
        at java.lang.StringBuffer.<init>(StringBuffer.java:96)
        at org.quartz.Trigger.getFullName(Trigger.java:454)
        at org.quartz.simpl.TriggerComparator.compare(RAMJobStore.java:1504)
        at java.util.TreeMap.getEntryUsingComparator(TreeMap.java:368)
        at java.util.TreeMap.getEntry(TreeMap.java:339)
        at java.util.TreeMap.remove(TreeMap.java:597)
        at java.util.TreeSet.remove(TreeSet.java:276)
        at org.quartz.simpl.RAMJobStore.acquireNextTrigger(RAMJobStore.java:1224)
        at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:266)

Note if the exception is thrown by a scheduled process, it is properly handled without further consequences.
---
[1] https://github.com/alostale/quartz/blob/quartz-1.6.2/src/java/org/quartz/core/QuartzSchedulerThread.java#L438 [^]

Steps To Reproduce

It's hard to reproduce because it is necessary to cause an unhandled exception in this concrete piece of code.

1. Replace quartz-1.6.2.jar with attached jar. This jar throws an OutOfMemoryError whenever Alert Process is executed.
2. Start tomcat
   -> OK: check with jstack there is a running thread named "DefaultQuartzSecheduler_QuartzSchedulerThread"
3. Schedule Alert Process
4. After it tries to run:
   -> FAIL: check with jstack there IS NOT a running thread named "DefaultQuartzSecheduler_QuartzSchedulerThread"
5. Schedule any other process
   -> FAIL: it does not get executed

Proposed Solution

Additional Information

Tags

No tags attached.

Relationships

related to

defect

0029490

closed

egoitz

The quartz scheduler is interrupted and the background process stop working

Attached Files