web stats
SFTP error due to file size - trying to set up an alert. - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 12-02-2019, 09:14 AM
Brad_AZ Brad_AZ is offline
Mirth Newb
 
Join Date: Mar 2018
Location: Arizona, USA
Posts: 23
Brad_AZ is on a distinguished road
Default SFTP error due to file size - trying to set up an alert.

We have some issues with large files and SFTP. While others are looking at a solution, I was trying to get some kind of alert as opposed to just checking the logs. The error I get is:

Quote:
[2019-12-02 10:05:00,040] ERROR (com.mirth.connect.connectors.file.FileReceiver:43 1): Unable to dispatch message to channel XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXX. File: smb://SERVERNAME001P/Files/App/Outbound/Charge_20191202.txt
java.io.IOException: File Charge_20191202.txt is too large. Unable to read files greater than 2147483647 bytes.
I can't get an alert to fire, because (I'm guessing) that this is failing before it gets to the channel. Can I alert on the FileReciever? I'm looking, but can't get that sorted. We are on 3.8.1 with Platinum support and extensions.

(And we are trying to get smaller files sent, but that is a battle in itself)

Thanks!
__________________
SIG|1|Brad|Mirth Certified Interface Analyst^Cancer Treatment Centers of America

Last edited by Brad_AZ; 12-02-2019 at 09:17 AM.
Reply With Quote
  #2  
Old 12-03-2019, 07:31 AM
kirbykn2's Avatar
kirbykn2 kirbykn2 is offline
Mirth Guru
 
Join Date: Sep 2014
Location: Michigan
Posts: 607
kirbykn2 is on a distinguished road
Default

Just to verify, are you trying to pick files up from an SFTP location? If so, can you set the maximum limit on the file size on the source tab?
__________________
Best,

Kirby

Mirth Certified|Epic Bridges Certified|Cloverleaf Level 2 Certified

Appliance Version 3.11.4
Mirth Connect Version 3.8.0
Java Version 1.6.0_45-b06
Java (64 bit) Version 1.6.0_45-b06
Java 7 (64 bit) Version 1.7.0_151-b15
Java 8 (64 bit) Version 1.8.0_181-b13
PostgreSQL Version 9.6.8
Reply With Quote
  #3  
Old 12-03-2019, 09:00 AM
Brad_AZ Brad_AZ is offline
Mirth Newb
 
Join Date: Mar 2018
Location: Arizona, USA
Posts: 23
Brad_AZ is on a distinguished road
Default

Sorry - we generically call our channels SFTP if they are either sending or picking up files via SFTP.

This is a file pickup that SFTPs to a destination. So this issue is a file reader using smb where it can't ingest the file due to size.

File size is set to 0 min, and Ignore Maximum is selected.
__________________
SIG|1|Brad|Mirth Certified Interface Analyst^Cancer Treatment Centers of America
Reply With Quote
  #4  
Old 12-03-2019, 09:06 AM
kirbykn2's Avatar
kirbykn2 kirbykn2 is offline
Mirth Guru
 
Join Date: Sep 2014
Location: Michigan
Posts: 607
kirbykn2 is on a distinguished road
Default

is this a valid file you expect to be able to process? If not, can you set the maximum file size to something appropriate?

Does this file contain a batch of messages that you can parse using the batch feature in Mirth?

Also, review the Mirth logs for errors.
__________________
Best,

Kirby

Mirth Certified|Epic Bridges Certified|Cloverleaf Level 2 Certified

Appliance Version 3.11.4
Mirth Connect Version 3.8.0
Java Version 1.6.0_45-b06
Java (64 bit) Version 1.6.0_45-b06
Java 7 (64 bit) Version 1.7.0_151-b15
Java 8 (64 bit) Version 1.8.0_181-b13
PostgreSQL Version 9.6.8
Reply With Quote
  #5  
Old 12-03-2019, 09:45 AM
Brad_AZ Brad_AZ is offline
Mirth Newb
 
Join Date: Mar 2018
Location: Arizona, USA
Posts: 23
Brad_AZ is on a distinguished road
Default

Quote:
Originally Posted by kirbykn2 View Post
is this a valid file you expect to be able to process? YES If not, can you set the maximum file size to something appropriate?

Does this file contain a batch of messages that you can parse using the batch feature in Mirth NO

Also, review the Mirth logs for errors.
So the file size is a point of contention between the producers of the file and us, but it is a legit file. However, we might set the max to just below what Mirth can accept - then at least we should be able to capture the error at the channel level and alert on it.

The only error I found was the original which included "Unable to read files greater than 2147483647 bytes.". So I will make the max just under that and lobby to have them send us smaller files. Maybe as them to switch to daily as opposed to weekly.

Thanks!~
__________________
SIG|1|Brad|Mirth Certified Interface Analyst^Cancer Treatment Centers of America
Reply With Quote
  #6  
Old 12-03-2019, 11:02 AM
agermano agermano is offline
Mirth Guru
 
Join Date: Apr 2017
Location: Indiana, USA
Posts: 1,100
agermano is on a distinguished road
Default

The issue is that the message needs to be converted to a byte[] at some point, and the max size for an array in Java is Integer.MAX_VALUE (2147483647.)

If it is a delimited file that you can read as a batch, it will actually stream the file and you should be ok since no single message will go over the 2GB mark.

The only other way to stream and split the file would be to process it yourself in a javascript reader.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 07:09 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Mirth Corporation