Openbravo Issue Tracking System - Modules
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0045213ModulesIntegrations with Azure Blob Storagepublic2020-10-13 12:172020-10-20 07:41
Reporteralostale 
Assigned Tocberner 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version 
Merge Request Status
Regression date
Regression introduced by commit
Regression level
Review Assigned To
Support ticket
OBNetwork customer
Regression introduced in release
Summary0045213: Azure: OOM migrating attachements if missconfigured
DescriptionHaving a missconfiguration in Azure Storage module and trying to migrate thousands of attachments results in an OOM.

The problem occurs when trying to append to the process log the errors received for each of the attachments that is tried to be migrated.
Steps To ReproduceIn an instance with thousands of attachments stored in file system:
1. Configure Azure incorrectly
2. Schedule migration process:

2020-10-08 19:05:12,753 [DefaultQuartzScheduler_Worker-2] ERROR org.quartz.core.ErrorLogger - Job (OB_QUARTZ_GROUP.82ACCBF860F94F688604B7917F3A6907 threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception.
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz-2.3.2.jar:?]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) [quartz-2.3.2.jar:?]
Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:3745) ~[?:?]
        at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172) ~[?:?]
        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538) ~[?:?]
        at java.lang.StringBuilder.append(StringBuilder.java:174) ~[?:?]
        at org.openbravo.scheduling.ProcessLogger.log(ProcessLogger.java:45) ~[classes/:?]
        at org.openbravo.service.integration.azure.storage.process.MigrateProcess.migrateAttachments(MigrateProcess.java:169) ~[classes/:?]
        at org.openbravo.service.integration.azure.storage.process.MigrateProcess.doExecute(MigrateProcess.java:110) ~[classes/:?]
        at org.openbravo.service.db.DalBaseProcess.execute(DalBaseProcess.java:85) ~[classes/:?]
        at org.openbravo.scheduling.DefaultJob.execute(DefaultJob.java:65) ~[classes/:?]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202) ~[quartz-2.3.2.jar:?]
        ... 1 more
Proposed SolutionBefore starting the batch process, ensure configuration is valid and fail properly in case it is not.
Additional Information
TagsNo tags attached.
Relationships
related to defect 0045246 new Triage Platform Conn OOM exception if misconfigured or files fail to upload consistently 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2020-10-13 12:17alostaleNew Issue
2020-10-13 12:17alostaleAssigned To => platform
2020-10-13 12:17alostaleModules => Core
2020-10-13 12:17alostaleTriggers an Emergency Pack => No
2020-10-13 12:24alostaleProjectOpenbravo ERP => Modules
2020-10-13 12:25alostaleCategoryA. Platform => Integrations with Azure Blob Storage
2020-10-14 15:07hgbotNote Added: 0123680
2020-10-14 17:07cbernerAssigned Toplatform => cberner
2020-10-19 12:41cbernerRelationship addedrelated to 0045246
2020-10-20 07:41hgbotResolutionopen => fixed
2020-10-20 07:41hgbotStatusnew => closed
2020-10-20 07:41hgbotNote Added: 0123784
2020-10-20 07:41hgbotNote Added: 0123785
2020-10-20 07:41hgbotNote Added: 0123786

Notes
(0123680)
hgbot   
2020-10-14 15:07   
Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/merge_requests/4 [^]
(0123784)
hgbot   
2020-10-20 07:41   
Directly closing issue as related merge request is already approved.

Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage [^]
Changeset: f5f52378edd222f27ba661c075c74e0eee341e46
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 2020-10-16T11:16:36+02:00
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/commit/f5f52378edd222f27ba661c075c74e0eee341e46 [^]

Fixes ISSUE-45213: OOM migrating attachments if misconfigured

An OutOfMemory exception is thrown if the azure configuration is not
correct when executing MigrateProcess. This was happening because of an
infinite loop when processing batches of files for upload.
It has been fixed by setting a threshold of consecutive errors allowed,
if this threshold is exceeded, MigrateProcess fails. The threshold has
been set to 300.
Note that the errors must be consecutive, if a eventually suceeds
to upload, the current errors counter is reset.

---
M src/org/openbravo/service/integration/azure/storage/process/MigrateProcess.java
---
(0123785)
hgbot   
2020-10-20 07:41   
Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage [^]
Changeset: b32c3084d5b4ff17d86f10a64690f71f2280b0a0
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 2020-10-19T15:17:14+02:00
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/commit/b32c3084d5b4ff17d86f10a64690f71f2280b0a0 [^]

Related to ISSUE-45213: Add logger for Openbravo on MigrateProcess process

Previously MigrateProcess was only logging to the process logger, this
resulted in it not showing its current status until the process ended.
Adding also an log4j logger, allows to have logs in Tomcat and see the
current status of the Migration process or if there's any error or warning.

---
M src/org/openbravo/service/integration/azure/storage/process/MigrateProcess.java
---
(0123786)
hgbot   
2020-10-20 07:41   
Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.service.integration.azure.storage/-/merge_requests/4 [^]