Recently we faced one issue in production that Sitecore job hangs or gets stuck. This issue is blocking other jobs and all our search index re-build jobs were in the queue. End users were not getting optimal search experience and our latest published content was not showing on the live site. This is not happening frequently, it happened twice in the last 6 months. After a lot of googling, we found recycle application pool is the only solution. Sitecore support told us currently there is no way to kill individual jobs. Sitecore support still analyzing dumps and logs to find the root cause. But there is still an open problem, how do we know when jobs get stuck? We can check running jobs on the “Jobs.aspx” page but how do we know, when to check? If we don’t respond quickly it could crash the site.
We decided at least, we should get an alert when jobs get stuck. We need a task scheduler, which will run in a certain interval and check if there are any long-running jobs are there. Then it will use Sitecore EXM to send an email. In the beginning, we thought, we will read the recipient list from a config, then we thought we can use Sitecore List Manager. We will add recipient emails in there and from the back-end, we will read the list and shoot emails. Then we ended up, can just create an email campaign in Sitecore EXM and add the list from the list manager as a recipient.
At first we have to create a list in the Sitecore list manager and add our recipients in the list:
Second task is to create an email and add the list in the recipients section and also keep a note of the item Id.:
Next step is to create a Command in Sitecore Tasks and in the type section enter the method name to invoke:
Then create a scheduler and set the running intervals. In the item field enter the Email Campaign Id – which we got from previous step:
Now schduler and email are ready, we need to shoot the email from the back-end based on long running jobs duration. We will need “SendingManagerFactory” and “ExmCampaignService”.
var sendingManagerFactory = ServiceLocator.ServiceProvider.GetService<ISendingManagerFactory>();
var campaignService = ServiceLocator.ServiceProvider.GetService<IExmCampaignService>();
Then we will read the Email Campaign based on the item parameter we are passing in the scheduler.
MessageItem emailCampaign = campaignService.GetMessageItem(itemArray.Length > 0 ? itemArray[0].ID.ToGuid() : new Guid());
Then we will get the jobs from the “JobManager” and check if they are running more than 20 minutes and shooting the email.
var jobs = Sitecore.Jobs.JobManager.GetJobs();
foreach (var job in jobs)
{
if (job.Status.State == Sitecore.Jobs.JobState.Running)
{
var timeDiff = System.DateTime.UtcNow - job.QueueTime;
if (timeDiff.Minutes > 20)
{
ISendingManager sendingManager = sendingManagerFactory.GetAsyncSendingManager(emailCampaign);
sendingManager.SendMessage();
break;
}
}
}
Thanks
MH