web stats
File reader processes files in random order - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 10-12-2006, 07:07 PM
nshaik nshaik is offline
Mirth Guru
 
Join Date: Sep 2006
Posts: 302
nshaik
Default File reader processes files in random order

=========
Mirth v1.1

Channel: File Reader to Database
=========

File reader randomly picks the files from the "Directory to read" folder.

Q: How to setup Mirth to process oldest files first (FIFO - First-In-First-Out order)?

More Details
=========

Setup Mirth to read files from a folder which had about 22,000 message files with datetime ranging from 10/10/06 3:30 PM to 10/12/06 6:00 PM. The messages were expected to be processed with oldest file first (Operating system file create date).

In this specific test case, it should process files from 10/10/06, 10/11/06, and then 10/12/06. But Mirth randomly picks the files from the "Directory to read" folder and process them (the files are getting moved in random order in the "Move-to Directory" folder).

This would make the updates incorrect. For e.g., A31 message for patient update on 10/10/06 and 10/12/06 for the same patient. If the 10/12/06 A31 message gets processed first and then 10/10/06, the patient data would be incorrect.

NOTE: This problem may not occur when the files are getting processed in real-time i.e., as and when they arrive. But when Mirth HL7 server is down due to hardware / network problems, the HL7 host would be sending the messages and they get accumulated in the target folder. Later, when the Mirth HL7 box is turned back on and is up and running, it needs to process the messages in the order they arrived.

Thanks in advance!





Reply With Quote
  #2  
Old 10-13-2006, 10:45 AM
chrisl chrisl is offline
Mirth Employee
 
Join Date: Aug 2006
Posts: 967
chrisl is an unknown quantity at this point
Default Re: File reader processes files in random order

This is a good idea - can you add this as an issue to the JIRA? (www.mirthproject.org/jira).

Thanks!
__________________
Chris Lang
Reply With Quote
  #3  
Old 10-14-2006, 06:18 AM
nshaik nshaik is offline
Mirth Guru
 
Join Date: Sep 2006
Posts: 302
nshaik
Default Re: File reader processes files in random order

Chris,

Added this problem to the issue tracker.

As I understand, a possible solution would be an option to process a static list. Details below:

Suggestions:
==================================
1. SHORT TERM TEMPORARY WORKAROUND

Utility / Script to move files in sorted order (by file-date) from a TEMP folder to the INBOUND folder with a time delay (in seconds)
==================================

* Stop Mirth
* Rename current inbound folder to temp and create a new empty inbound folder
This is basically to save time in moving large set of files to a temporary folder.
* Start Mirth
* Write a utility / script which will take TimeDelayInSeconds as parameter:
Move the files in sorted order (oldest file first) from temporary folder to
INBOUND folder with the specified delay

This should take care of the problem of processing the files in the order received.

=======================
2. A Possible Solution
=======================

* Mirth Admin to have the following option for "File Reader"
[Check box] Use static list of files

* Display message on ?Deploy Channel?
You selected "Use static list of files". Mirth will work only with the current
snap-shot of files and ignore other files that come in later.
Are you sure you want to continue?
* Process the static list of files in the order received (FIFO)
* mirth.log file to show

************************************************** *********
--> [Datetime stamp] Processing static list of files...

--> Total no. of files in the static list: [nnnn]

--> First file in list: [File name with the file create date] (oldest file in the list)

--> Last file in list: [File name with the file create date] (latest file in the list)

--> IMPORTANT:
--> Please log back to Admin module and turn "Use static list of files" option off.
--> Redeploy channel to continue normal processing.
************************************************** *********
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
seeming random lockup on file read 1.8. phc Support 3 02-19-2009 01:23 PM
File Reader to read files from Network Share baziz Support 3 12-09-2008 11:28 AM
How to let file reader pick up certain files? yywind Support 9 11-15-2008 10:59 PM
1.7.1 file reader/file writer channel steverhusker Support 2 08-12-2008 08:53 AM
FTP reader for csv files jerchap General Discussion 2 05-05-2008 09:51 AM


All times are GMT -8. The time now is 06:13 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation