web stats
Unable to manage Retries with FTP FileWriter - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 02-02-2012, 02:14 PM
Thierry DUBREU Thierry DUBREU is offline
What's HL7?
 
Join Date: Aug 2011
Posts: 3
Thierry DUBREU is on a distinguished road
Default Unable to manage Retries with FTP FileWriter

Hi,

when the FTP destination server is down or due to the network we get the following error. We have all permissions on the destination directory. If I resend this messages, it works fine, but the order of HL7 messages is not respected, this is a major feature.

ERROR-403: File connector error
ERROR MESSAGE: Error writing file
java.net.SocketException: Connection reset


It's impossible to retry sending automatically the message before processing the next message in case of error.

Could you add to the FileWriter two additionnal parameters to manage retries.
- maxRetries : number of retries
- RetryWaitingTime : waiting time between 2 retries

see post : http://www.mirthcorp.com/community/f...ead.php?t=5581

Most integration engines have this functionnality.

Thanks for your response
Reply With Quote
  #2  
Old 02-02-2012, 03:20 PM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,124
narupley is on a distinguished road
Default

Mirth does provide this functionality for its TCP, LLP, HTTP, and Web Service Senders. I'm sure different people will have different views on this, but FTP by its very nature is just dumping files into a remote directory either one at a time or in daily batches, etc. So, it's not really tailored to be used as a live feed where order needs to be preserved.

That said, there's an easy way around it in Mirth. Set up an inbound and outbound channel. The inbound channel sends to the outbound channel via LLP, with persistent queues turned on. Then, the outbound channel writes the file via FTP, and if it was successfully written, it can create a custom response variable which can then be sent back to the inbound channel.

I've attached a couple of sample channels to show you what I mean. Here's the code I used in the postprocessor; feel free to alter it as you see fit:

Code:
if ($r('FTP_TEST_OUT').getStatus() == 'SUCCESS') {
	var msg = new XML(SerializerFactory.getHL7Serializer().toXML(messageObject.getRawData()));
	var ack = <HL7Message/>;
	ack.MSH['MSH.1'] = msg.MSH['MSH.1'].toString();
	ack.MSH['MSH.2'] = msg.MSH['MSH.2'].toString();
	ack.MSH['MSH.3'] = msg.MSH['MSH.5'].copy();
	ack.MSH['MSH.4'] = msg.MSH['MSH.6'].copy();
	ack.MSH['MSH.5'] = msg.MSH['MSH.3'].copy();
	ack.MSH['MSH.6'] = msg.MSH['MSH.4'].copy();
	ack.MSH['MSH.7']['MSH.7.1'] = DateUtil.getCurrentDate('yyyyMMddHHmmss');
	ack.MSH['MSH.9']['MSH.9.1'] = msg.MSH['MSH.9']['MSH.9.1'].toString();
	ack.MSH['MSH.9']['MSH.9.2'] = msg.MSH['MSH.9']['MSH.9.2'].toString();
	ack.MSH['MSH.9']['MSH.9.3'] = 'ACK';
	ack.MSH['MSH.10'] = msg.MSH['MSH.10'].copy();
	ack.MSH['MSH.11'] = msg.MSH['MSH.11'].copy();
	ack.MSH['MSH.12'] = msg.MSH['MSH.12'].copy();
	ack.MSA['MSA.1']['MSA.1.1'] = 'AA';
	ack.MSA['MSA.2']['MSA.2.1'] = msg.MSH['MSH.10']['MSH.10.1'].toString();
	ack.MSA['MSA.3']['MSA.3.1'] = 'File sent successfully';
	responseMap.put('ACK',ResponseFactory.getSuccessResponse(SerializerFactory.getHL7Serializer().fromXML(ack)));
}
return;
Attached Files
File Type: xml FTP_TEST_IN.xml (5.3 KB, 131 views)
File Type: xml FTP_TEST_OUT.xml (7.9 KB, 109 views)
Reply With Quote
  #3  
Old 04-15-2012, 05:14 PM
glenn71 glenn71 is offline
OBX.3 Kenobi
 
Join Date: Nov 2010
Location: Sydney Australia
Posts: 129
glenn71 is on a distinguished road
Default

This is great info from narupley.. I am going to steal some of these ideas to retain msg persistency & retries in some of my LLP feeds.

If you are after a file writer that has retries here is one that works for me:

Step 1:
Create a function in code templates so you can re-use this on other channels:
Code:
function ftpWrite(ftpserver, ftpport, ftpuser, ftppass, ftppassive, ftptimeout, 
						ftpfile, ftppath, ftpappendfile, ftpByteArray,
						maxRetries, reconnectWaitSeconds) {

	// Do the ftp connection and send file
	var returnBoolean = false;
	var connected = false;
	var retryCount = 0;

	// setup ftp connection
	while (! connected && retryCount++ < maxRetries) {

		if (retryCount > 1) {
			// not the first attempt wait a bit, maybe it will come back online shortly
			java.lang.Thread.sleep(reconnectWaitSeconds * 1000);
		}

		try {

			ftpConn = new Packages.com.mirth.connect.connectors.file.filesystems.FtpConnection(ftpserver, 
												ftpport, ftpuser, ftppass, ftppassive, ftptimeout);
		
			if ( ftpConn == null || ! ftpConn.isConnected() ) {
				connected = false;
			} else {
				connected = true;
			}
		} catch(e) {
				logger.error("Ftp connection attempt failed for server:  " + ftpserver
					+ ".  Reconnection attempt #" + retryCount + " of " + maxRetries
					+ ", waiting "
					+ reconnectWaitSeconds + " seconds before retry.");
				connected = false;
		}
	}

	// if ftp connection was successful then write the file to the ftp server
	try {
		if ( connected ) {
		
			ftpConn.writeFile(ftpfile, ftppath, ftpappendfile, ftpByteArray);
			returnBoolean = true; 
			ftpConn.destroy();
		} else {
			logger.error("Unable to connect to ftp server:  " + ftpserver
					+ ".  Max retry limit exceeded. ");
		}
	} catch(e) {
			logger.error("ftp connection OK, however file write failed. Server: " 
						+ ftpserver + ", Filename: " + ftpfile );
	}

	// return the status of the function
	return returnBoolean;

}
Step 2:
In your channel, create a destination "JAVASCRIPT WRITER" and re-use this code with your params.
Code:
var ftpserver = "myftpserver";
var ftpport = 0;				// zero for default ftp port
var ftpuser = "myuser";
var ftppass = "mypass";
var ftppassive = true;			// ftp connection type
var ftpappendfile = true;		// append or overwrite
var ftptimeout = 10000;		// in milliseconds
var ftpfile = "ftpfilename-" + DateUtil.getCurrentDate('yyyy-M-d') + ".txt";
var ftppath = "myfilepath/";
var maxRetries = 1500;
var reconnectWaitSeconds = 60;
var ftpSuccess = false;

// build string that is going to be written to ftp file
// it needs to be stored in a byteArray
var msgString = new java.lang.String( $('resultString') );
var ftpByteArray = msgString.getBytes();

// call function that writes ftp file
ftpSuccess = ftpWrite(ftpserver, ftpport, ftpuser, ftppass, ftppassive, ftptimeout, 
						ftpfile, ftppath, ftpappendfile, ftpByteArray,
						maxRetries, reconnectWaitSeconds);

if ( ! ftpSuccess ) {

	logger.error("Error writing ftp file with message: " + msgString);
	responseMap.put(connector, ResponseFactory.getFailureResponse("Error writing ftp file with message: " + msgString));
}
In the above case it retries 1500 times and waits 1min between retries, plus it logs the failure in mirth. Since I monitor mirth this works fine for me in practice as the msg has to be stuck for 25 hours before it goes on to try the next one.. we monitor 24/7 so this should never happen.

p.s. $('resultString') is the string of data I am passing from a transformer.. so it's the data I am wanting to write to the ftp server.
Reply With Quote
  #4  
Old 05-31-2012, 06:45 AM
tactics tactics is offline
Mirth Newb
 
Join Date: Jan 2012
Posts: 8
tactics is on a distinguished road
Default Automatic reprocess of errored messages

Another solution to automatically reprocess errored message is proposed here:



http://www.mirthcorp.com/community/f...5487#post25487


.
Reply With Quote
  #5  
Old 05-31-2012, 07:39 AM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,124
narupley is on a distinguished road
Default

Quote:
Originally Posted by tactics View Post
Another solution to automatically reprocess errored message is proposed here:



http://www.mirthcorp.com/community/f...5487#post25487


.
I'll just copy my response here just in case:

Quote:
Originally Posted by narupley View Post
This is a great alternative!

It looks like this is tailored specifically for channels that are using both a vanilla File Reader (reading from local directory) and a File Writer. So, if you used a File Reader that's polling via FTP from a remote server, that method wouldn't work (though you could make it work with some extra code). Also, what if you had a different source connector, like an LLP Listener?

In your postprocessor, instead of placing the file back in a source directory, another way to simulate this is simply to re-route the message directly to your channel:

Code:
if ($r('Destination 1').getStatus() != 'SUCCESS')
	router.routeMessageByChannelId(channelId,messageObject.getRawData(),true);
However, it's important to note that in general, this method is basically a rotating queue. That is, if the order of messages is important (as it certainly is in most registration and lab HL7 feeds) then you would probably want to stick to a persistent queue simulation that isn't rotating.

Just my two cents!

EDIT: Also, as far as overwriting statistics goes, this might help: http://www.mirthcorp.com/community/f...13&postcount=4
Reply With Quote
  #6  
Old 06-04-2012, 05:38 AM
tactics tactics is offline
Mirth Newb
 
Join Date: Jan 2012
Posts: 8
tactics is on a distinguished road
Default

Hi Narupley,
thanks for your tip.

In order to decrement the error statistics, I added this to my code:



var channelStatisticsController = Packages.com.mirth.connect.server.controllers.Chan nelStatisticsController.getInstance();
channelStatisticsController.decrementErrorCount(ch annelId);
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 01:50 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation