Good day,

I recently monitored RabbitMQ while the Ad Hoc Journey was executing. After the Ad Hoc Journey finished executing (confirmed by looking at the ServerLog) I can still see unack'ed messages in the relevant Rabbit queue.

I've seen it happen on a number of occasions and I'm wondering if the PreFetch function while running multiple containers (12 in the example below) doesn't lock some messages (21 as per the example below) as per screenshot below: 

I am getting the following error when turning queues off. The queue was at a concurrency set as 1 then when I changed it to 0 and saved it errored in the Warewolf log.


2022-11-06 17:28:10,817 INFO - [Warewolf Info] - Save Trigger Queue Service
2022-11-06 17:28:11,323 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\52e4dba3-73fa-4267-877a-fa4f493a1511.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,332 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\667e5af4-c5a8-461b-b03d-c5683fe1fca0.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,340 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\9923dc51-c138-4944-a7a4-06cb468f2e86.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,343 INFO - [00000000-0000-0000-0000-000000000000] - Trigger restarting 'f00a48f4-5348-490b-add4-3991c578c717'
2022-11-06 17:28:11,359 ERROR - [ at System.Diagnostics.Process.Kill()
at Warewolf.OS.ProcessMonitor.Kill()] - Access is denied
2022-11-06 17:28:11,371 INFO - [WarewolfLogger.exe] - Logging Server OnError, Error details:Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
2022-11-06 17:28:11,372 INFO - [WarewolfLogger.exe] - 11/6/2022 5:28:11 PM [Debug] Error while reading System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - --- End of inner exception stack trace ---
2022-11-06 17:28:11,379 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2022-11-06 17:28:11,379 INFO - [WarewolfLogger.exe] - at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
2022-11-06 17:28:11,390 INFO - [Warewolf Info] - queue process died: ProfileCustomerBackBook(f00a48f4-5348-490b-add4-3991c578c717)

In the for-each, there is a new record set being appended from a scalar variable output. 

And then I attempt to sum up the values in the recordset. It fails with an invalid calculate field. Please see the debug output - there should only be one record in the recordset.

Hi Team, there is one suggestion to improve RabbitMQ consumer performance. We can divide the load with multiple consumer by specifying prefetch count value. If we are not specifying prefetch value that means single consumer have unlimited buffer means single consumer will pick all the data from queue.

Prefetch meaning "prefetch simply controls how many messages the broker allows to be outstanding at the consumer at a time. When set to 10, this means the broker will send 10 message, wait for the ack, then send the next."

References : 
https://www.rabbitmq.com/consumer-prefetch.html
https://www.cloudamqp.com/blog/how-to-optimize-the-rabbitmq-prefetch-count.html
https://stackoverflow.com/questions/65201334/rabbit-mq-prefetch-undestanding

Suggestion : Add prefetch configuration in trigger section of warewolf studio (i.e like trigger concurrency)

Please let me know if you need more help to understand.

PriceIncreaseByMandate.bite
newjourney (2).log

In local environment, sometime it gives "Resume node not found" error. but After restart of warewolf service. it is working fine. Price increase journey have 2 suspend tool in workflow but both are working fine in local environment. But on Dev environment it is failing Randomly. Out of 10 it is failing almost 7 times. I am providing here, workflow and server log.

Yes it new workflow.

Hangfir giving below error: 

{"FailedAt":"2022-04-18T05:28:31.2801372Z","ExceptionType":"System.InvalidOperationException","ExceptionMessage":"Error resuming. ServiceAction is null for Resource ID:ae105df3-9dc5-437e-ba21-e93dd6a0f356","ExceptionDetails":"System.InvalidOperationException: Error resuming. ServiceAction is null for Resource ID:ae105df3-9dc5-437e-ba21-e93dd6a0f356 ---> System.Exception: Error resuming. ServiceAction is null for Resource ID:ae105df3-9dc5-437e-ba21-e93dd6a0f356\r\n --- End of inner exception stack trace ---\r\n at HangfireServer.ResumptionAttribute.OnPerformResume(PerformingContext context) in Y:\\WOLF-INST-COMREL\\Dev\\Warewolf.HangfireServer\\ResumptionAttribute.cs:line 94\r\n at Hangfire.Profiling.ProfilerExtensions.InvokeAction[TInstance](InstanceAction`1 tuple)\r\n at Hangfire.Profiling.SlowLogProfiler.InvokeMeasured[TInstance,TResult](TInstance instance, Func`2 action, String message)\r\n at Hangfire.Profiling.ProfilerExtensions.InvokeMeasured[TInstance](IProfiler profiler, TInstance instance, Action`1 action, String message)\r\n at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation)"}

Hi,

I have noticed that All the values of recordset are not passing into child workflow for each loop. I have checked this same behavior into the Select & Apply tool and it's working fine.

Please check the sample attached workflows & issue into screenshot.

forloop_recordset_all_values_not_pass_issue....png
ForLoopIssue-MainWorkflow (1).bite
ForLoopIssue-ChildWorkflow (1).bite

Gandalf 5 months ago

It looks like the For Each is getting confused with 2 different recordset inputs: User(*) and Benefit(*)

Gandalf 5 months ago

Change For Each to use No. of Executes = 3 and see different result. This is from the tool handeling the replacement of * differently and incorectly.

We have checked and found that the sequence of BenifitID in User Payload is incorrect so we changed the sequence and checked the result. Output is coming properly.

Attached are the Edited Workflows.

ForLoopIssue-ChildWorkflow (2).bite
ForLoopIssue-MainWorkflow (2).bite

There is a problem with the suspend tool. I have two workflows created in 2.6.6.0, one works fine and the other does not.

Works fine: SuspendToolTest

Does not work: suspendtest

suspendtest.bite
SuspendToolTest.bite

Please see attached workflow. When passing the following JSON as input:

{
"dateTime": "2020-04-14T09:20:18.982Z"
}

Even though the JSON is valid and I have specified the DateTime Input Format correctly when I debug I get an error that the Input String is not in the correct format. However when I 'View in Browser' it does not show an error. Furthermore the debug input of the tool does not show the input due to the error.

Date Time Issue.bite

PS. I got this to work by making the input format my current system format as it turned out, internally the Newtonsoft de-serializer interprets the DateTime into the system format. This is however hidden from user and would be difficult to find.

I think showing the Debug Input values would help the user see what value was actually being used and then they could adjust the format based on that. There is a case for saying that the format should not have changed without me explicitly doing so.

Hello,

is there any limitation on the RabbitMQ Publish Module? 

In my case, I have one recordset with about 60000 different numbers. Now I want to publish this to RabbitMQ. But I only got a message rate of max. 20/s. That's not so fast. With other tools I reached over 20000 per second. 

During the job runs, I've checked the system performance. 
CPU: < 25%

RAM:  2,5Gb / 8Gb

How much messages can WareWolf send to RabbitMQ?

Thanks!

What javascript execution engine is used? I have having issues with scripts as some functions are not working properly.

Problem 1:

"testString".endsWith("g");

is valid javascript and works with node and browsers. This should return true.

However running this as a script in a workflow results in an error due to it not being a function.

Problem 2:

If I use variables in the script the strings are not escaped.

Variable:

[[testPath]] = "C:\test"

In script:

"[[testPath]]" => "c:est"

Suggestions:

I think it would be very powerful to use node.js as the javascript execution as it is so widely used. Edge.js is a c# library that allows node to easily be executed from .Net. I haven't researched it much but may be an option.

As far as escaping strings you could predefine them all on a javascript object that the script runs in. Instead of using [[varName]] we could just use $ww.varName if we use $ww as the example object that Warewolf populates. This would take care of string escaping problems and even allow complex objects to be manipulated as expected while coding javascript.