0045246: OOM exception if misconfigured or files fail to upload consistently

Openbravo Issue Tracking System - Modules

View Issue Details

Project

Category

View Status

Date Submitted

Last Update

0045246

Modules

Integration with Amazon S3

public

2020-10-19 12:41

2022-12-23 12:18

Reporter

cberner

Assigned To

Triage Platform Conn

Priority

normal

Severity

major

Reproducibility

have not tried

Status

new

Resolution

open

Platform

OS Version

Product Version

Target Version

Fixed in Version

Merge Request Status

Regression date

Regression introduced by commit

Regression level

Review Assigned To

Support ticket

OBNetwork customer

Regression introduced in release

Summary

0045246: OOM exception if misconfigured or files fail to upload consistently

Description

If Amazon S3 is misconfigured or connection is missing while executing "Amazon S3 Migration Process" process, the application will eventually run out of memory because of retrying to upload the same files indefinitely.

The OutOfMemory exception is thrown because of the size of the logger object, as it is never written to file or DB until the process ends.

Steps To Reproduce

1. Misconfigure Amazon S3 or generate a file that fails consistently to upload in the attachments folder.
2. Schedule and execute Amazon S3 Migration Process with Max Time set to 0 (infinite).
3. Wait for the OOM, this could take a long time.

Note: Alternatively you can limit the amount of heap memory usage to 1gb and the OOM exception should happen earlier.

Proposed Solution

Exit the process with failed status if a threshold of consecutive errors is surpassed. For example, number of failed batches is higher than 300, that means potentially 3000 files have failed to upload, and the process should also fail because of this.

Additional Information