Level: Information

Message: Inside hosted service - for nsq


Message: Started the consumer service and handler for NSQ


Message: Notification event with message ID 1846f510192c3000 received: {"TemplateId":45,"EntityType":"visitor_management","EventType":null,"NotifyRoleId":355,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"visitorManagementId":"22","orgId":"251"}},"ActorContext":{"Employee":[5698],"createdby":[267]},"Metadata":null}


Message: Started processing notification for message ID 1846f510192c3000


Message: Looking up the notification handler for message ID 1846f510192c3000


Message: Event handling in process for message ID 1846f510192c3000


Message: Dispatching the notifications for message ID 1846f510192c3000


Message: Successfully processed notification for message ID 1846f510192c3000


Message: Started processing notification for message ID 1846f6de762c3000


Message: Looking up the notification handler for message ID 1846f6de762c3000


Message: Event handling in process for message ID 1846f6de762c3000


Message: Dispatching the notifications for message ID 1846f6de762c3000


Message: Successfully processed notification for message ID 1846f6de762c3000


Message: Started processing notification for message ID 1846f6f71f6c3000


Message: Looking up the notification handler for message ID 1846f6f71f6c3000


Message: Event handling in process for message ID 1846f6f71f6c3000


Message: Dispatching the notifications for message ID 1846f6f71f6c3000


Message: Successfully processed notification for message ID 1846f6f71f6c3000


Message: Started processing notification for message ID 1846f7a93eec3000


Message: Looking up the notification handler for message ID 1846f7a93eec3000


Message: Event handling in process for message ID 1846f7a93eec3000


Message: Dispatching the notifications for message ID 1846f7a93eec3000


Message: Successfully processed notification for message ID 1846f7a93eec3000


Message: Started processing notification for message ID 1846f7a9402c3000


Message: Looking up the notification handler for message ID 1846f7a9402c3000


Message: Event handling in process for message ID 1846f7a9402c3000


Message: Dispatching the notifications for message ID 1846f7a9402c3000


Message: Successfully processed notification for message ID 1846f7a9402c3000


Message: Started processing notification for message ID 1846f7b6816c3000


Message: Looking up the notification handler for message ID 1846f7b6816c3000


Message: Event handling in process for message ID 1846f7b6816c3000


Message: Dispatching the notifications for message ID 1846f7b6816c3000


Message: Successfully processed notification for message ID 1846f7b6816c3000


Message: Started processing notification for message ID 1846f7b681ec3000


Message: Looking up the notification handler for message ID 1846f7b681ec3000


Message: Event handling in process for message ID 1846f7b681ec3000


Message: Dispatching the notifications for message ID 1846f7b681ec3000


Message: Successfully processed notification for message ID 1846f7b681ec3000


Message: Started processing notification for message ID 1846f7b6826c3000


Message: Looking up the notification handler for message ID 1846f7b6826c3000


Message: Event handling in process for message ID 1846f7b6826c3000


Message: Dispatching the notifications for message ID 1846f7b6826c3000


Message: Successfully processed notification for message ID 1846f7b6826c3000


Message: Started processing notification for message ID 1846f7cb86ec3000


Message: Looking up the notification handler for message ID 1846f7cb86ec3000


Message: Event handling in process for message ID 1846f7cb86ec3000


Message: Dispatching the notifications for message ID 1846f7cb86ec3000


Message: Successfully processed notification for message ID 1846f7cb86ec3000


Message: Started processing notification for message ID 1846f7d3abac3000


Message: Looking up the notification handler for message ID 1846f7d3abac3000


Message: Event handling in process for message ID 1846f7d3abac3000


Message: Dispatching the notifications for message ID 1846f7d3abac3000


Message: Successfully processed notification for message ID 1846f7d3abac3000


Message: Started processing notification for message ID 1846f7dbe5ac3000


Message: Looking up the notification handler for message ID 1846f7dbe5ac3000


Message: Event handling in process for message ID 1846f7dbe5ac3000


Message: Dispatching the notifications for message ID 1846f7dbe5ac3000


Message: Successfully processed notification for message ID 1846f7dbe5ac3000


Message: Started processing notification for message ID 1846f7dbe66c3000


Message: Looking up the notification handler for message ID 1846f7dbe66c3000


Message: Event handling in process for message ID 1846f7dbe66c3000


Message: Dispatching the notifications for message ID 1846f7dbe66c3000


Message: Successfully processed notification for message ID 1846f7dbe66c3000


Message: Notification event with message ID 1846fe3431ec3000 received: {"TemplateId":45,"EntityType":"visitor_management","EventType":null,"NotifyRoleId":355,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"visitorManagementId":"24","orgId":"251"}},"ActorContext":{"Employee":[5698],"createdby":[267]},"Metadata":null}


Message: Started processing notification for message ID 1846fe3431ec3000


Message: Looking up the notification handler for message ID 1846fe3431ec3000


Message: Event handling in process for message ID 1846fe3431ec3000


Message: Dispatching the notifications for message ID 1846fe3431ec3000


Message: Successfully processed notification for message ID 1846fe3431ec3000


Message: Inside hosted service - for nsq


Message: Started the consumer service and handler for NSQ


Message: Notification event with message ID 1847040b6eec3000 received: {"TemplateId":51,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":"AssignTo","EntityContext":{"Parameters":{"alertId":"2015","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2015","EmailFooter":"OQSHA Team","UserName":" ","Changes":"","description":"desc"}},"ActorContext":{"AssignTo":[23019,14361]},"Metadata":null}


Message: Started processing notification for message ID 1847040b6eec3000


Message: Looking up the notification handler for message ID 1847040b6eec3000


Message: Event handling in process for message ID 1847040b6eec3000


Message: Dispatching the notifications for message ID 1847040b6eec3000


Message: Successfully processed notification for message ID 1847040b6eec3000


Message: Notification event with message ID 1847040b6fec3000 received: {"TemplateId":52,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":"AssignTo","EntityContext":{"Parameters":{"alertId":"2015","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2015","EmailFooter":"OQSHA Team","UserName":" ","Changes":"","description":"desc"}},"ActorContext":{"AssignTo":[23019,14361]},"Metadata":null}


Message: Started processing notification for message ID 1847040b6fec3000


Message: Looking up the notification handler for message ID 1847040b6fec3000


Message: Event handling in process for message ID 1847040b6fec3000


Message: Dispatching the notifications for message ID 1847040b6fec3000


Message: Successfully processed notification for message ID 1847040b6fec3000


Message: Notification event with message ID 1847046854ec3000 received: {"TemplateId":45,"EntityType":"visitor_management","EventType":null,"NotifyRoleId":355,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"visitorManagementId":"25","orgId":"251"}},"ActorContext":{"Employee":[9106],"createdby":[267]},"Metadata":null}


Message: Started processing notification for message ID 1847046854ec3000


Message: Looking up the notification handler for message ID 1847046854ec3000


Message: Event handling in process for message ID 1847046854ec3000


Message: Dispatching the notifications for message ID 1847046854ec3000


Message: Successfully processed notification for message ID 1847046854ec3000


Message: Notification event with message ID 184704fc72ac3000 received: {"TemplateId":45,"EntityType":"visitor_management","EventType":null,"NotifyRoleId":355,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"visitorManagementId":"26","orgId":"251"}},"ActorContext":{"Employee":[4297],"createdby":[267]},"Metadata":null}


Message: Started processing notification for message ID 184704fc72ac3000


Message: Looking up the notification handler for message ID 184704fc72ac3000


Message: Event handling in process for message ID 184704fc72ac3000


Message: Dispatching the notifications for message ID 184704fc72ac3000


Message: Successfully processed notification for message ID 184704fc72ac3000


Message: Notification event with message ID 18470724816c3000 received: {"TemplateId":50,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"alertId":"2016","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016","EmailFooter":"OQSHA Team","UserName":"Giselle Angle","description":"desc"}},"ActorContext":null,"Metadata":null}


Message: Started processing notification for message ID 18470724816c3000


Message: Looking up the notification handler for message ID 18470724816c3000


Message: Event handling in process for message ID 18470724816c3000


Message: Sending message ID 18470724816c3000 to DLQ


Message: Dead letter envelope for DLQ message: {"SourceTopic":"notification.orchestrator","SourceChannel":"notification-orchestrator","FailedAtUtc":"2026-09-07T12:45:30.4128824Z","Attempts":1,"ErrorType":"OQSHA.Notification.Orchestrator.Exceptions.PermanentOrchestratorException","ErrorMessage":"No recipients received for message ID 18470724816c3000","StackTrace":"   at OQSHA.Notification.Orchestrator.Services.UsersService.ResolveRecipientsAsync(String messageId, Int32 orgId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Services\\UsersService.cs:line 84\r\n   at OQSHA.Notification.Orchestrator.Services.UsersService.GetRecipientsWithDetailsAsync(String messageId, Int32 orgId, NotificationEventDto request, NotificationTemplatesDbo notificationTemplate) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Services\\UsersService.cs:line 134\r\n   at OQSHA.Notification.Orchestrator.NotificationHandlers.SafetyAlertsNotificationHandler.HandleTemplateBasedAlertNotificationAsync(String messageId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\NotificationHandlers\\SafetyAlertsNotificationHandler.cs:line 75\r\n   at OQSHA.Notification.Orchestrator.NotificationHandlers.SafetyAlertsNotificationHandler.ProcessEventAsync(String messageId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\NotificationHandlers\\SafetyAlertsNotificationHandler.cs:line 45\r\n   at OQSHA.Notification.Orchestrator.Routing.EventRouter.ExecuteAsync(String messageId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Routing\\EventRouter.cs:line 38\r\n   at OQSHA.Notification.Orchestrator.Worker.OrchestratorMessageHandler.HandleMessage(IMessage message) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Worker\\OrchestratorMessageHandler.cs:line 44","OriginalBody":"{\"TemplateId\":50,\"EntityType\":\"safety_alerts\",\"EventType\":null,\"NotifyRoleId\":null,\"NotifyUserId\":null,\"NotifyDynamicActor\":null,\"EntityContext\":{\"Parameters\":{\"alertId\":\"2016\",\"orgId\":\"264\",\"alertLink\":\"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016\",\"EmailFooter\":\"OQSHA Team\",\"UserName\":\"Giselle Angle\",\"description\":\"desc\"}},\"ActorContext\":null,\"Metadata\":null}"}


Message: Sent message ID 18470724816c3000 to DLQ


Message: Notification event with message ID 1847072481ec3000 received: {"TemplateId":49,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"alertId":"2016","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016","EmailFooter":"OQSHA Team","UserName":"Giselle Angle","description":"desc"}},"ActorContext":null,"Metadata":null}


Message: Started processing notification for message ID 1847072481ec3000


Message: Looking up the notification handler for message ID 1847072481ec3000


Message: Event handling in process for message ID 1847072481ec3000


Message: Sending message ID 1847072481ec3000 to DLQ


Message: Dead letter envelope for DLQ message: {"SourceTopic":"notification.orchestrator","SourceChannel":"notification-orchestrator","FailedAtUtc":"2026-09-07T12:45:30.4271903Z","Attempts":1,"ErrorType":"OQSHA.Notification.Orchestrator.Exceptions.PermanentOrchestratorException","ErrorMessage":"No recipients received for message ID 1847072481ec3000","StackTrace":"   at OQSHA.Notification.Orchestrator.Services.UsersService.ResolveRecipientsAsync(String messageId, Int32 orgId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Services\\UsersService.cs:line 84\r\n   at OQSHA.Notification.Orchestrator.Services.UsersService.GetRecipientsWithDetailsAsync(String messageId, Int32 orgId, NotificationEventDto request, NotificationTemplatesDbo notificationTemplate) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Services\\UsersService.cs:line 134\r\n   at OQSHA.Notification.Orchestrator.NotificationHandlers.SafetyAlertsNotificationHandler.HandleTemplateBasedAlertNotificationAsync(String messageId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\NotificationHandlers\\SafetyAlertsNotificationHandler.cs:line 75\r\n   at OQSHA.Notification.Orchestrator.NotificationHandlers.SafetyAlertsNotificationHandler.ProcessEventAsync(String messageId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\NotificationHandlers\\SafetyAlertsNotificationHandler.cs:line 45\r\n   at OQSHA.Notification.Orchestrator.Routing.EventRouter.ExecuteAsync(String messageId, NotificationEventDto request) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Routing\\EventRouter.cs:line 38\r\n   at OQSHA.Notification.Orchestrator.Worker.OrchestratorMessageHandler.HandleMessage(IMessage message) in C:\\GitLab-Runner\\builds\\27n6lYj0J\\0\\incident-reporter\\incident-reporter-api\\IncidentReporter\\OQSHA.Notification.Orchestrator\\Worker\\OrchestratorMessageHandler.cs:line 44","OriginalBody":"{\"TemplateId\":49,\"EntityType\":\"safety_alerts\",\"EventType\":null,\"NotifyRoleId\":null,\"NotifyUserId\":null,\"NotifyDynamicActor\":null,\"EntityContext\":{\"Parameters\":{\"alertId\":\"2016\",\"orgId\":\"264\",\"alertLink\":\"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016\",\"EmailFooter\":\"OQSHA Team\",\"UserName\":\"Giselle Angle\",\"description\":\"desc\"}},\"ActorContext\":null,\"Metadata\":null}"}


Message: Sent message ID 1847072481ec3000 to DLQ


Message: Notification event with message ID 1847072bad2c3000 received: {"TemplateId":52,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":"AssignToSectionIncharge","EntityContext":{"Parameters":{"alertId":"2016","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016","EmailFooter":"OQSHA Team","UserName":" ","Changes":"","description":"desc"}},"ActorContext":{"AssignToDsoAso":[23019],"AssignToSectionIncharge":[14361]},"Metadata":null}


Message: Started processing notification for message ID 1847072bad2c3000


Message: Looking up the notification handler for message ID 1847072bad2c3000


Message: Event handling in process for message ID 1847072bad2c3000


Message: Dispatching the notifications for message ID 1847072bad2c3000


Message: Successfully processed notification for message ID 1847072bad2c3000


Message: Notification event with message ID 1847072badac3000 received: {"TemplateId":51,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":"AssignToSectionIncharge","EntityContext":{"Parameters":{"alertId":"2016","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016","EmailFooter":"OQSHA Team","UserName":" ","Changes":"","description":"desc"}},"ActorContext":{"AssignToDsoAso":[23019],"AssignToSectionIncharge":[14361]},"Metadata":null}


Message: Started processing notification for message ID 1847072badac3000


Message: Looking up the notification handler for message ID 1847072badac3000


Message: Event handling in process for message ID 1847072badac3000


Message: Dispatching the notifications for message ID 1847072badac3000


Level: Error

Message:
Dispatch failed for message ID 1847072badac3000 at attempt NsqSharp.Core.ErrNotConnected: not connected
   at NsqSharp.Producer.Connect()
   at NsqSharp.Producer.sendCommandAsync(Command cmd, Chan`1 doneChan, Object[] args)
   at NsqSharp.Producer.MultiPublishAsync(String topic, IEnumerable`1 bodies, Object[] args)
   at OQSHA.Notification.Orchestrator.Publishing.NotificationDispatchPublisher.DispatchAsync(List`1 dispatchRequests, CancellationToken ct) in C:\GitLab-Runner\builds\27n6lYj0J\0\incident-reporter\incident-reporter-api\IncidentReporter\OQSHA.Notification.Orchestrator\Publishing\NotificationDispatchPublisher.cs:line 35
   at OQSHA.Notification.Orchestrator.Routing.EventRouter.ExecuteAsync(String messageId, NotificationEventDto request) in C:\GitLab-Runner\builds\27n6lYj0J\0\incident-reporter\incident-reporter-api\IncidentReporter\OQSHA.Notification.Orchestrator\Routing\EventRouter.cs:line 45
   at OQSHA.Notification.Orchestrator.Worker.OrchestratorMessageHandler.HandleMessage(IMessage message) in C:\GitLab-Runner\builds\27n6lYj0J\0\incident-reporter\incident-reporter-api\IncidentReporter\OQSHA.Notification.Orchestrator\Worker\OrchestratorMessageHandler.cs:line 44 and exception: {ex.Message}


Message: Requeued message ID 1847072badac3000


Message: Successfully processed notification for message ID 1847072badac3000


Message: Notification event with message ID 1847073ff06c3000 received: {"TemplateId":51,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":"AssignTo","EntityContext":{"Parameters":{"alertId":"2016","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016","EmailFooter":"OQSHA Team","UserName":" ","Changes":"Assigning to User","description":"desc"}},"ActorContext":{"AssignToDsoAso":[23019],"AssignToSectionIncharge":[14361],"AssignTo":[15941]},"Metadata":null}


Message: Started processing notification for message ID 1847073ff06c3000


Message: Looking up the notification handler for message ID 1847073ff06c3000


Message: Event handling in process for message ID 1847073ff06c3000


Message: Dispatching the notifications for message ID 1847073ff06c3000


Message: Successfully processed notification for message ID 1847073ff06c3000


Message: Notification event with message ID 1847073ff0ac3000 received: {"TemplateId":52,"EntityType":"safety_alerts","EventType":null,"NotifyRoleId":null,"NotifyUserId":null,"NotifyDynamicActor":"AssignTo","EntityContext":{"Parameters":{"alertId":"2016","orgId":"264","alertLink":"https://sandbox.oqsha.com/#/safety-alerts-v2/edit-safety-alert-v2/2016","EmailFooter":"OQSHA Team","UserName":" ","Changes":"Assigning to User","description":"desc"}},"ActorContext":{"AssignToDsoAso":[23019],"AssignToSectionIncharge":[14361],"AssignTo":[15941]},"Metadata":null}


Message: Started processing notification for message ID 1847073ff0ac3000


Message: Looking up the notification handler for message ID 1847073ff0ac3000


Message: Event handling in process for message ID 1847073ff0ac3000


Message: Dispatching the notifications for message ID 1847073ff0ac3000


Message:
Dispatch failed for message ID 1847073ff0ac3000 at attempt NsqSharp.Core.ErrNotConnected: not connected
   at NsqSharp.Producer.Connect()
   at NsqSharp.Producer.sendCommandAsync(Command cmd, Chan`1 doneChan, Object[] args)
   at NsqSharp.Producer.MultiPublishAsync(String topic, IEnumerable`1 bodies, Object[] args)
   at OQSHA.Notification.Orchestrator.Publishing.NotificationDispatchPublisher.DispatchAsync(List`1 dispatchRequests, CancellationToken ct) in C:\GitLab-Runner\builds\27n6lYj0J\0\incident-reporter\incident-reporter-api\IncidentReporter\OQSHA.Notification.Orchestrator\Publishing\NotificationDispatchPublisher.cs:line 35
   at OQSHA.Notification.Orchestrator.Routing.EventRouter.ExecuteAsync(String messageId, NotificationEventDto request) in C:\GitLab-Runner\builds\27n6lYj0J\0\incident-reporter\incident-reporter-api\IncidentReporter\OQSHA.Notification.Orchestrator\Routing\EventRouter.cs:line 45
   at OQSHA.Notification.Orchestrator.Worker.OrchestratorMessageHandler.HandleMessage(IMessage message) in C:\GitLab-Runner\builds\27n6lYj0J\0\incident-reporter\incident-reporter-api\IncidentReporter\OQSHA.Notification.Orchestrator\Worker\OrchestratorMessageHandler.cs:line 44 and exception: {ex.Message}


Message: Requeued message ID 1847073ff0ac3000


Message: Successfully processed notification for message ID 1847073ff0ac3000


Message: Notification event with message ID 184709c16dac3000 received: {"TemplateId":45,"EntityType":"visitor_management","EventType":null,"NotifyRoleId":355,"NotifyUserId":null,"NotifyDynamicActor":null,"EntityContext":{"Parameters":{"visitorManagementId":"27","orgId":"251"}},"ActorContext":{"Employee":[5698],"createdby":[267]},"Metadata":null}


Message: Started processing notification for message ID 184709c16dac3000


Message: Looking up the notification handler for message ID 184709c16dac3000


Message: Event handling in process for message ID 184709c16dac3000


Message: Dispatching the notifications for message ID 184709c16dac3000


Message: Successfully processed notification for message ID 184709c16dac3000