web stats
Mirth increasing number of open threads on Windows - Mirth Community

Go Back   Mirth Community > Mirth Connect > Development

Reply
 
Thread Tools Display Modes
  #1  
Old 11-17-2008, 10:29 AM
rampratapa rampratapa is offline
OBX.1 Kenobi
 
Join Date: May 2008
Posts: 45
rampratapa
Default Mirth increasing number of open threads on Windows

Hi,
I have the file reader and some javascript in Source Translator and destination translator. Also, I have some custom java code calls within the javascript. I am running the Mirth Server on Windows 2003 server.

The following are the observations using the windows trask manager

Before starting the Mirth Server the Thread Count : 493
After Starting the Mirth Servwr Thread Count : 560
After 40 minutes the Thread count is : 670
After two hours the Thread count is : 1097

We are not processing any messages only with that code we are running the Mirth Channels.

It appears that some threads are not closed.

Any one has any idea how to find the issue. I increased the thread count using the following parameter

wrapper.java.additional.1=-Xss100k as I was getting OutOfMemorey issue after 4500 threads. However the above increase in the thread may be the issue.

Can any one help to resolve the issue or How to narrow down the issue.

Thank you for the help

Thanks
RAm
Reply With Quote
  #2  
Old 11-17-2008, 10:48 AM
jbartels jbartels is offline
Mirth Guru
 
Join Date: Oct 2006
Posts: 727
jbartels is on a distinguished road
Default Re:Mirth increasing number of open threads on Windows

If you remove the calls to your custom java code does the thread count keep increasing?

What version is your JRE?

Can you export your channel and post it here?

I'm thinking that it would be best to get the custom code out of the picture and try to reproduce this with just Mirth.
__________________
Jon Bartels

Zen is hiring!!!!
http://consultzen.com/careers/
Talented healthcare IT professionals wanted. Engineers to sales to management.
Good benefits, great working environment, genuinely interesting work.
Reply With Quote
  #3  
Old 11-17-2008, 11:48 AM
rampratapa rampratapa is offline
OBX.1 Kenobi
 
Join Date: May 2008
Posts: 45
rampratapa
Default Re:Mirth increasing number of open threads on Windows

I am using the following JRE
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

I am going to remove the code ( Java Native Code and run the process). Do you think the native code runs on a seperated thread? Do we need to do any additional thing to clean-up in the java code>

Please let me know

I will post the result soon after my testing.

Thanks
Ram
Reply With Quote
  #4  
Old 11-17-2008, 07:10 PM
nshaik nshaik is offline
Mirth Guru
 
Join Date: Sep 2006
Posts: 302
nshaik
Default Re:Mirth increasing number of open threads on Windows

You might want to try with JRE 1.5.

Mirth 1.7.1 Installation note:
Quote:
Note that Java 1.5 is required. Java 1.6 can be used, however it is not recommended nor supported
Hope it helps!
Reply With Quote
  #5  
Old 11-18-2008, 10:37 AM
jbartels jbartels is offline
Mirth Guru
 
Join Date: Oct 2006
Posts: 727
jbartels is on a distinguished road
Default Re:Mirth increasing number of open threads on Windows

rampratapa wrote:
Quote:
I am going to remove the code ( Java Native Code and run the process). Do you think the native code runs on a seperated thread? Do we need to do any additional thing to clean-up in the java code?
I don't know, but having code that isn't part of Mirth running makes it harder to figure out what the problem might be.

If that code is removed and the threading issue goes away, then the problem is either in that code or how it is used. If the code is removed and the problem is still there, then its an issue with Mirth.
__________________
Jon Bartels

Zen is hiring!!!!
http://consultzen.com/careers/
Talented healthcare IT professionals wanted. Engineers to sales to management.
Good benefits, great working environment, genuinely interesting work.
Reply With Quote
  #6  
Old 11-18-2008, 10:49 AM
rampratapa rampratapa is offline
OBX.1 Kenobi
 
Join Date: May 2008
Posts: 45
rampratapa
Default Re:Mirth increasing number of open threads on Windows

Thank you for the response. At present I am wokring on removing the java calls from javascript and try to run the test so that I can located the issue.

I will post my results as soon as I complete my testing.

Thank you for the help

Thanks
Ram
Reply With Quote
  #7  
Old 11-18-2008, 01:11 PM
rampratapa rampratapa is offline
OBX.1 Kenobi
 
Join Date: May 2008
Posts: 45
rampratapa
Default Re:Mirth increasing number of open threads on Windows

It is the JavaScript calling java causing the thread issue. I have commented the java calling code and tested the channel and it is working fine.

However if you call a java code from JavaScript then it is not closing the threades.

Case : I have a channel with FileReader and Channel Writer. The is channel reades batch and split the batch into individual requests and feed to other channel with a single request. The following JavaScript code was written in the Source Transformer

---- JAVA SCRIPT --START---
var inMessage = messageObject.getRawData();
logger.error("Started Processing Batch:" + inMessage );
try{

var batchSplitter = Packages.com.service.impl.MirthFacade();
var splittedBatch = batchSplitter.splitDocument(inMessage);

logger.error("Completed Java calling");
var OrignialFileName = $c('originalFilename');
channelMap.put("OrignialFileName",OrignialFileName );
var trackingId = 0 ;
if (OrignialFileName == null){
OrignialFileName ="";
}
if (splittedBatch != null){
logger.error("In Message" + inMessage + "OrignialFileName" + OrignialFileName);
header = "<?xml version\="1.0" encoding\="UTF-8"\?> <message> <http.request>/</http.request> <http.version>HTTP/1.1</http.version> <Host>localhost:8091</Host> <Content-Length>5815</Content-Length> <http.method>POST</http.method> <payload>";
trackingStartTag = "</payload><trackingId>"
trailer = " </trackingId> <batchId>" + OrignialFileName+"</batchId><User-Agent>Jakarta Commons-HttpClient/3.0.1</User-Agent></message>";

logger.debug("Incoming Message" + inMessage );
logger.info("Incoming Message Received" + splittedBatch.size() );
// route single request to Individual Batch
for (var index=0; index < splittedBatch.size(); index++) {
trackingId++;
payloadMessage = header + splittedBatch.get(index) + trackingStartTag + trackingId + trailer;
logger.error("routed Message" + payloadMessage );
router.routeMessage("Channel 2", payloadMessage,true,true);
}
logger.info("Batch Was Routed Channel 2" + OrignialFileName);

}else {
channelMap.put("ERRMESSAGE",inMessage);
channelMap.put("ERROR","FATAL");
}


}catch (ex) {
channelMap.put("ERRMESSAGE",ex);
channelMap.put("ERROR","FATAL");
logger.error(ex);
}
return true;

---- JAVA SCRIPT --END---

MirthFacade is POJO and splitDocument is method that splits the document into indvidual messages.

When I put 20 files to this channel the mirth creates 20 threads and is not closing the threads even after the process completed splitting and routing the messages. If Add another 20, it is creating 20 more threads now they became 40, so it is increasing the threads.

NOTE : The java code is not opening any connections and it is pure POJO.

Please let me know what you think

Thank you for the help

Thanks
Ram
Reply With Quote
  #8  
Old 11-18-2008, 02:08 PM
jbartels jbartels is offline
Mirth Guru
 
Join Date: Oct 2006
Posts: 727
jbartels is on a distinguished road
Default Re:Mirth increasing number of open threads on Windows

Ram,

I'm finding this very interesting. Thanks for posting about it. I'm still at work, but when I get home I'll play around with it and see if I can reproduce the issue.

Since removing the call to Java code resolves the threading problem that means that the root-cause is:

1) The Java code that is being called
2) Not closing out/destroying the Java objects created (not sure on this, my knowledge of the Java lifecycle is limited)
3) Mirth (maybe Rhino or Mule) doesn't handle the code correctly

If it were #1, you should be able to get the threading issue to happen again by just calling that Java code from another application.

I think if it were #2 or #3 the issue would show up with a simple Java class. Like a "Hello World" class would cause it to break just the same as a "Do lots of buisness logic" class.
__________________
Jon Bartels

Zen is hiring!!!!
http://consultzen.com/careers/
Talented healthcare IT professionals wanted. Engineers to sales to management.
Good benefits, great working environment, genuinely interesting work.
Reply With Quote
  #9  
Old 11-18-2008, 02:49 PM
jacobb jacobb is offline
Mirth Employee
 
Join Date: Aug 2006
Location: Irvine, CA
Posts: 1,218
jacobb is an unknown quantity at this point
Default Re:Mirth increasing number of open threads on Windows

I ran a test to verify this and didn't get the same results. I used jconsole to monitor the number of live threads, by starting Mirth with the VM argument -Dcom.sun.management.jmxremote and then starting jconsole.

My JavaScript step did the following:

var a = Packages.com.webreach.mirth.util.StringUtil();
a.convertLFtoCR("asdfsdf");

A pretty simple test that also calls a Java class with a static method. After running a couple hundred messages through, the peak thread count never went above 71, and the live thread count seemed to stay around 66 and now rests at 61. Let me know if you have different results, jbartels.
__________________
Jacob Brauer
Director, Software Development
NextGen Healthcare

Reply With Quote
  #10  
Old 11-18-2008, 07:13 PM
rampratapa rampratapa is offline
OBX.1 Kenobi
 
Join Date: May 2008
Posts: 45
rampratapa
Default Re:Mirth increasing number of open threads on Windows

jacobb,
Thank you for the quick test. I was using Non-Static methods ( instance methods) by getting the Instance using Spring Factory. So I have attached Spring Framework for business objects so that I can get them based on the context. I am continuing my research by deleting the Spring Framework and then see the results. Then I will introduce Spring Framework and see the results.

I will post the results as soon as I complete my testing.

Once again thank you for all your help

Thanks
Ram


UPDATE : 9:18 PM CST

I have deleted Spring Factory and it is working fine. I am using Process Explorer to track number of threads Mirth Opened for Processing. When it started it is opening 50 threads and once I put the load they are between 60 to 70 threades.

It is working fine now with Pure Java Class.

Thank you all of you for the help. Now I am going to work with Spring Framework and let you know what was the problem that is causing the thread leak.

Thanks
Ram

Post edited by: rampratapa, at: 11/18/2008 20:22

UPDATE : 11/19/2008

I am able to identify the issue. If the class has instance variables ( class level variables) and it is holding a object reference then Mirth is not releasing the thread and therefore the thread count increase over period of time. When I moved those to local to method then it started working fine.

I was under the view that if the javascript in the transformer or filter executed all the variable in that including the Java Class varibale will become out of scode and the thread is free from resources. I think it is not the case. However I am able to fix the issue and Spring FrameWork is working fine with Mirth.

I would like to thank you all for helping me in this issue.

Thank you
Ram

Post edited by: rampratapa, at: 11/19/2008 09:39
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Mirth intrerrupts Windows shutdown smakarsky Support 9 11-04-2008 07:41 AM
Does mirth has ability to send X number of the OBX julia1231 Support 2 06-25-2008 07:13 AM
SIIM 2008 Open Source Plugfest starring Mirth sween General Discussion 0 05-21-2008 08:34 AM
1.3.2 Administrator launch time increasing rapidly MBT766 Support 2 07-10-2007 01:40 PM
Mirth 1.3 - File Open dialog is unable to select directory nshaik Support 1 12-19-2006 11:18 AM


All times are GMT -8. The time now is 11:37 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation