Diagnose a Job That Does Not Complete

Follow

A Job may not seem to complete in JAMS and will appear to continue to execute without completion. This can be caused by a number of things: 

  • An interactive program that assumes that there is always someone there to read a dialog box and click "OK" or "Cancel". 
  • When these programs are run by a batch Job scheduler (like JAMS), the dialog is usually not displayed where a user can see it.
  • A process that has not completed on the target machine, the process has not sent output back for JAMS to interpret.
  • Also, if the output of the Job is being redirected to an external log file.  Since nothing is being returned to STDOUT, the Job continues to run as there is no information for JAMS to interpret as well.

Process Explorer would be best suited to diagnose these issues. Process Explorer is a free Microsoft tool that can be downloaded from: http://technet.microsoft.com/en-us/sysinternals/bb896653

If a problem is suspected, the first step is to look at what process(es) are running.  

The Process Explorer screen shot below shows JAMS Jobs running on a local JAMS Scheduler and running via the JAMS Executor Service or JAMS Agent service.

The lines beneath JAMSExecutor.exe are Jobs being executed on the local server by the local JAMS Scheduler. The lines beneath JAMSAgent.exe are Jobs being executed on the local server by a JAMSScheduler on a different server.

Notice the yellow JAMSHostV2.exe line? This is a Job that uses the "Routine" type of Execution Method. This includes PowerShell, Workflow and others. The two lines below that, cmd.exe and Ping.exe are from a single Job that uses the Command Execution Method which has a Type of Batch.

The Type of the Job's Execution Method is a key piece of information that needs to be known so that it can be investigated. A Type of Routine will show JAMSHost.exe, JAMSHostV2.exe, JAMSHost32 or JAMSHostV232.exe. A Type of Batch will have something else, whatever is specified in the Application or Command in the Execution Method definition.

Finding the process for the Job can tell if the application is still executing, also if it is getting any sort of CPU or IO activity. If there are many executing Jobs, the log file can be viewed via the JAMS Monitor view to find the Process ID of the Job's initial process.

Sometimes, users can see exactly why the Job in question is hung. In some cases a Job fails but, the Visual Studio debugger will intercept the failure and display a dialog box asking if the user wants to debug the process. Usually, this dialog cannot be seen by anyone.

Just-In-Time debugging is usually not something that users want on a server that is used for executing batch Jobs. Users can disable just-in-time debugging by deleting the following registry keys:
  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug\Debugger
  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\DbgManagedDebugger

 

On a 64-bit Operating System delete the following registry keys as well:
  • HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug\Debugger

  • HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\DbgManagedDebugger

If a user is concerned about deleting these keys, users can also rename the key so that they may always be reverted back.

Sometimes a user may have one particular Job that just won't complete. Every time the user tries to run it, it runs until manual intervention. These Jobs are possibly opening a dialog box and waiting for a response. To really see what's going on, log on to the console of the server where the Job is going to run using the Windows account that the Job will execute under.

Temporarily change the Job's Execution Method to enable Interactive Mode, as seen here:


Submit the Job and wait for a dialog to appear.
Have more questions? Submit a request

Comments