web stats
Performance Issue with SQL to HL7 using Mirth Connect - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 12-09-2010, 06:02 AM
dkwang dkwang is offline
What's HL7?
 
Join Date: Dec 2010
Posts: 1
dkwang is on a distinguished road
Default Performance Issue with SQL to HL7 using Mirth Connect

Hi,

We are using Mirth Connect 1.8 to pull 1 million records from SQL 2005 then convert each record into a HL7 message. The conversion is done in mostly JavaScript for some light transformation logic along with message builder function. The processing, however, took nearly 24 hours. We need to look for ways to improve the performance and like to know if there is any better way of doing this task.

Note:
- The JavaScript section is very straight forward with no further look up but a few if statements.
- We are pulling 1000 records per 2 seconds and updating a single field on the record. (mark it as pulled)
Reply With Quote
  #2  
Old 12-10-2010, 11:24 PM
mnowlin mnowlin is offline
OBX.3 Kenobi
 
Join Date: Sep 2006
Posts: 141
mnowlin is an unknown quantity at this point
Default

1000 messages every two seconds seems a bit aggressive to me... Doing the math, you're trying to process a million records in around 33 minutes - if the job is being stretched to almost 24 hours, something is seriously overloaded.

I suggest backing off the load to maybe 50 or 100 messages every two seconds, and even that could be pushing the limit, depending on your platform. You may want to even start at 10 messages a second and ramp things up, monitoring the processing rate, until you see the TPS rate go the wrong way. There is sometimes a fine line between maximum throughput and grinding to a halt...
Reply With Quote
Reply

Tags
hl7, performance, sql

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 06:37 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation