1. After Duplicating a resource WITH relationships updated as well
2. Go to the newly created resource
3. Double Click on nested resource on the design surface
4. Edit it
5. Open the resource in the explorer that should have been edited
6. Observe it is not edited (And the original one is)

Hello Team.

I'd like to bring an error to your attention and for your consideration. We are currently observing this error on Warewolf 2.8.6.16 and it is causing the VM that is hosting the microservice to become unstable.

Context:

On Monday 2024-07-01 we deployed code to our LeadsServices UAT environment. Because changes were made to the secure.config (workflow was set to public), we had to restart the service in order for the changes in the secure.config to take effect. The VM restart was performed at around 16:15. After the restart, Sai from DevOps observed, using PRTG monitoring, a couple of red flags. One of them was the number of concurrent Warewolf connections. 

Now because this is a relatively new MicroService, and UAT environment, there wasn't any processes running in the background. I'm attaching the warewolf log to support this and for your review and consideration. The red flags only disappeared after 15mins - even tho there were no transactions / workflows processing on the microservice. wareWolf-Server-Leads-UAT.zip

However, what we have found is that when we restart a Production VM, one that is busy with a lot of workflows executing at the same time, these red flags take much longer to disappear. We also observed that workflows or APIs would take much longer to execute during this time. For example a Post Tool that calls an API that normally takes around 3 seconds would take much much longer - often to the point of timing out. And when this happens, Architecture and DevOps decides to switch off the VM as a precautionary measure - which means we are sitting without a service in Production.

If we consider the time we observed the red flags in PRTG, and we look at the corresponding time in the Warewolf Server logs - it appears that the following error may be the culprit:

[Header]
2024-07-01 16:15:02,101 ERROR - [Warewolf Error] - Dev2.ServerLifecycleManager
System.Runtime.InteropServices.COMException (0x8007045B): A system shutdown is in progress. (Exception from HRESULT: 0x8007045B)
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at System.Management.ManagementScope.InitializeGuts(Object o)
at System.Management.ManagementScope.Initialize()
at System.Management.ManagementObjectSearcher.Initialize()
at System.Management.ManagementObjectSearcher.Get()
at Dev2.ServerLifecycleManager.GetNumberOfCores()
at Dev2.ServerLifecycleManager.TrackUsage(UsageType usageType, IExecutionLogPublisher logger)
at Dev2.ServerLifecycleManager.CleanupServer()
[Footer]

Please feel free to reach out should you need more information. Let's understand and solve together.

Regards,

Wynand

Hi

I am currently running warewolf on a VM with 8 vCPUs, I want to know how do I get insight into how many workflows are currently running and utilizing threads and how many are being queued to run

The reason I would like this insight is to be able to accurately monitor performance to ensure that I have enough vCPUs for the amount of workflows processing and also to know when to upscale or down scale correctly to be able to save costs.

My monitoring product that I am currently using is PRTG for monitoring my VM's

Im getting an error when using the post tool to a server that requires the Accept header:

"The 'Accept header must be modified using the appropriate property or method.
Parameter name: name"

The API setup was working properly before and this error started when I think the API started using the header on their side. Either way, its a coding issue on our end.

This is one Im calling: Investec programable banking, SA PB Account Information : Transfer Multiple v2

Its probably too dificult to setup on your end so happy to help with this where needed.

1. Have the ability in the Warewolf Studio to see the resource consumption down to individual named workflow for memory used, CPU threads used etc.
2. Have the ability to integrate 3rd party monitoring tools such as PRTG with the above in order to identify which named workflow is consuming all the resources assigned to the VM or Kubernetes POD.
3. Have the ability to set percentage threshold alerts on resources assigned to the VM or Kubernetes POD.
4. Have the ability to email alerts when thresholds are breached.

Scenario 

when warewolf get started it just shows simple message that "warewolf started..." but not provide squences of events generated with warewolf start 

Why we are looking this ?

Case 1 : 

with containerize warewolf , sometimes warewolf server is not starting on some time and it laggs to get started. that's where we are aware about why it is not starting /where it is stuck 

Case 2 : 

we also notice that with high amount of data in queue, triggers are started and take to get load, yet warewolf server has not fully started or container has not fully initialized  

Case 3 : 

when there are certain numbers of warewolf has been developed in warewolf instance like more than 800 workflows and it takes time to get started 

to resolve all the above cases , we need detail level logs of warewolf then it will help us to resolve or fix issue with our microservices . 

Good day,

I recently monitored RabbitMQ while the Ad Hoc Journey was executing. After the Ad Hoc Journey finished executing (confirmed by looking at the ServerLog) I can still see unack'ed messages in the relevant Rabbit queue.

I've seen it happen on a number of occasions and I'm wondering if the PreFetch function while running multiple containers (12 in the example below) doesn't lock some messages (21 as per the example below) as per screenshot below: 

How do I deploy from a batch file or command line rather than the Studio?

I am getting the following error when turning queues off. The queue was at a concurrency set as 1 then when I changed it to 0 and saved it errored in the Warewolf log.


2022-11-06 17:28:10,817 INFO - [Warewolf Info] - Save Trigger Queue Service
2022-11-06 17:28:11,323 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\52e4dba3-73fa-4267-877a-fa4f493a1511.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,332 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\667e5af4-c5a8-461b-b03d-c5683fe1fca0.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,340 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\9923dc51-c138-4944-a7a4-06cb468f2e86.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,343 INFO - [00000000-0000-0000-0000-000000000000] - Trigger restarting 'f00a48f4-5348-490b-add4-3991c578c717'
2022-11-06 17:28:11,359 ERROR - [ at System.Diagnostics.Process.Kill()
at Warewolf.OS.ProcessMonitor.Kill()] - Access is denied
2022-11-06 17:28:11,371 INFO - [WarewolfLogger.exe] - Logging Server OnError, Error details:Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
2022-11-06 17:28:11,372 INFO - [WarewolfLogger.exe] - 11/6/2022 5:28:11 PM [Debug] Error while reading System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - --- End of inner exception stack trace ---
2022-11-06 17:28:11,379 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2022-11-06 17:28:11,379 INFO - [WarewolfLogger.exe] - at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
2022-11-06 17:28:11,390 INFO - [Warewolf Info] - queue process died: ProfileCustomerBackBook(f00a48f4-5348-490b-add4-3991c578c717)

In the for-each, there is a new record set being appended from a scalar variable output. 

And then I attempt to sum up the values in the recordset. It fails with an invalid calculate field. Please see the debug output - there should only be one record in the recordset.