web stats
Large CSV process as batch very slow - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Thread Tools Display Modes
Old 02-11-2016, 08:36 AM
gonasiri gonasiri is offline
What's HL7?
Join Date: Nov 2015
Posts: 1
gonasiri is on a distinguished road
Default Large CSV process as batch very slow

I have an interface which reads a csv file and wirted the data to database.
I'm trying to process a 14 MB CSV file as a batch process and it's taking a long time to process this file, as it goes row after row and it has 50,000 rows.

Is there any way to speed up the process?

Reply With Quote
Old 02-11-2016, 08:42 AM
narupley's Avatar
narupley narupley is offline
Mirth Employee
Join Date: Oct 2010
Posts: 7,124
narupley is on a distinguished road

Please post the channel here so we can better troubleshoot. First thing is, you could turn on the source queue. Then, does the order in which specific rows are processed by the Database Writer matter? If not, then you can turn on the destination queue and crank up the queue threads. After that to increase throughput further, if you have any source or destination filtering/transformation, you can also move all of that logic into the destination filter/transformer, and then enable "Include Filter/Transformer" on the advanced queue settings.
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.

- How do I foo?
- You just bar.
Reply With Quote

batch process, csv

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

All times are GMT -8. The time now is 06:38 PM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation