web stats
Multiple source folder SFTP connection - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 05-21-2019, 04:29 AM
gojoshi gojoshi is offline
OBX.1 Kenobi
 
Join Date: Jul 2013
Posts: 42
gojoshi is on a distinguished road
Default Multiple source folder SFTP connection

Wondering if there is a good way to connect to multiple sftp SOURCE folders? Please note again, this is for a source connection, and not the destination.

For example, the SFTP folder were structured as follows:

SFTP/Main/FolderA and SFTP/Main/FolderB


The mirth channel should pull files every 24 hours. And if there are files in SFTP/Main/FolderA, pull files from that folder. If not, then pull files from SFTP/Main/FolderB.

Any help on a good way to do this would be appreciated.
Reply With Quote
  #2  
Old 05-21-2019, 05:55 AM
odo odo is offline
OBX.3 Kenobi
 
Join Date: Feb 2017
Location: Luxembourg
Posts: 141
odo is on a distinguished road
Default

If FolderA and FolderB are the only subfolders of Main, simply use SFTP/Main as file location and check the "Include All Subdirectories"-Option of the sftp-connector.
Reply With Quote
  #3  
Old 05-21-2019, 06:54 AM
gojoshi gojoshi is offline
OBX.1 Kenobi
 
Join Date: Jul 2013
Posts: 42
gojoshi is on a distinguished road
Default

@ODO - The issue with that is that it will end up reading from both folders. We need to make sure that it tries to read from the first folder before it reads from the second folder. If there are no files in the first folder, then, and ONLY THEN, should it read from the second folder.
Reply With Quote
  #4  
Old 05-23-2019, 12:24 AM
odo odo is offline
OBX.3 Kenobi
 
Join Date: Feb 2017
Location: Luxembourg
Posts: 141
odo is on a distinguished road
Default

Use the fileDirectory attribute in the source map to determine where the file is coming from. The pollComplete attribute will tell you if the current message is the last one that has been polled.


I would try the following:

  1. create a source filter that writes the current message in a globalChannelMap variable e.g. messageToProcess.
    • If fileDirectory does not yet exist and fileDirectory=folder B ==> messageToProcess=currentMessage, otherwise do nothing.
    • If fileDirectory=folder A ==> messageToProcess=yourMessage.
  2. If current message has the pollComplete-flag set ==> send messageToProcess to destination, otherwise skip.
Reply With Quote
  #5  
Old 05-23-2019, 09:32 AM
agermano agermano is offline
Mirth Guru
 
Join Date: Apr 2017
Location: Indiana, USA
Posts: 1,008
agermano is on a distinguished road
Default

If you're comfortable working in javascript (I think there may be some examples in this forum of using either the mirth SftpConnection class or the Jsch library directly) you could use a javascript reader instead of a file reader.

This would allow you to check if directory A contains file before looking to directory B.

If the number of files are few or the filesizes are small, you could read all of the files from your target directory in the javascript reader and create messages similar to how the file reader would. The major differences would be (1) you need to be able to fit all of the files in memory at once, and (2) you need to manage deleting or moving the source files yourself after reading them. You might want to consider waiting to delete or move until the post-processor. There could be cases if you are deleting immediately after reading that you have an error that causes a loss of data before it's committed to the database.

Otherwise, if the total file size is expected to be large, you could do a directory listing in the javascript reader, and pass the path to each file as the message. Then in the transformer you could download and replace the path in the message with the actual contents.
Reply With Quote
Reply

Tags
sftp, sftp connection

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 05:11 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation