When executing a workflow in Warewolf that retrieves data from a SQL Server database and generates a JSON output, the application modifies the data types in the JSON payload. This behavior leads to inconsistencies that break PowerApps integration, as PowerApps requires consistent data types for proper functioning.

The key issues observed are:

1. String Fields:
   • String fields (FldVarchar) are sometimes output as numbers when the data from SQL is numeric (e.g., 1 instead of "1"). This results in inconsistent data types, causing errors when PowerApps processes the JSON.
2. Float Fields:
   • Float values (FldFloat) are being formatted with a locale-specific decimal separator (",") instead of the JSON standard dot ("."). For example, 1.123 in SQL is output as "1,123", which PowerApps cannot interpret as a valid numeric value.
3. Null Values:
   • Null values in the database are being output as empty strings ("") in the JSON payload for both string and numeric fields. PowerApps expects numeric fields to use null to signify missing data, and empty strings cause parsing errors or unexpected behavior.

Steps to Reproduce:

1. Create a stored procedure in SQL Server that returns a dataset containing strings, numbers, floats, and nulls:

   sqlCopyEditCreate Procedure usp_GetValues AS
   Declare @TempTable TABLE
   (
       FldVarchar varchar(10),
       FldNumber int,
       FldFloat float
   )
   Insert into @TempTable Values('String', 1, 1.123), ('1', 1, 1.123), (null, null, null)
   Select * from @TempTable
   

2. Use the Warewolf Low Code tool to create a workflow that calls the stored procedure using the SQL Server Database connector.
3. Map the output fields (FldVarchar, FldNumber, FldFloat) to a recordset in the workflow.
4. Run the workflow and observe the JSON output.
5. Attempt to integrate the JSON output with PowerApps.

Expected Behavior:

• FldVarchar should consistently output string values (e.g., "String" or "1").
• FldFloat should use a dot (".") as the decimal separator (e.g., 1.123 instead of "1,123").
• Null values should be represented as null in the JSON payload, not as empty strings ("").

Actual Behavior:

• FldVarchar alternates between string and numeric data types depending on the SQL value, breaking PowerApps' ability to process the field.
• FldFloat uses a comma as the decimal separator, which PowerApps cannot parse as a number.
• Null values are converted to empty strings, causing errors or unexpected behavior in PowerApps workflows.

Impact on PowerApps:
PowerApps relies on consistent and standard JSON data types for proper functionality. The inconsistent handling of data types in Warewolf is breaking workflows and causing significant delays in application development and deployment.

Proposed Solution:

1. Ensure that Warewolf outputs consistent data types in JSON:
   • Always serialize strings as strings.
   • Maintain numeric values in their standard JSON representation (e.g., dot as the decimal separator).
   • Represent null values correctly as null.
2. Provide an option in Warewolf to enforce strict type adherence when generating JSON outputs.

Attachments:

• Screenshot of the Warewolf workflow.
• Example of the problematic JSON output:

  jsonCopyEdit{
    "Values": [
      {
        "FldFloat": "1,123",
        "FldNumber": 1,
        "FldVarchar": "String"
      },
      {
        "FldFloat": "1,123",
        "FldNumber": 1,
        "FldVarchar": 1
      },
      {
        "FldFloat": "",
        "FldNumber": "",
        "FldVarchar": ""
      }
    ]
  }
  

• Expected JSON output:

  jsonCopyEdit{
    "Values": [
      {
        "FldFloat": 1.123,
        "FldNumber": 1,
        "FldVarchar": "String"
      },
      {
        "FldFloat": 1.123,
        "FldNumber": 1,
        "FldVarchar": "1"
      },
      {
        "FldFloat": null,
        "FldNumber": null,
        "FldVarchar": null
      }
    ]
  }

1. After Duplicating a resource WITH relationships updated as well
2. Go to the newly created resource
3. Double Click on nested resource on the design surface
4. Edit it
5. Open the resource in the explorer that should have been edited
6. Observe it is not edited (And the original one is)

Hello Team.

I'd like to bring an error to your attention and for your consideration. We are currently observing this error on Warewolf 2.8.6.16 and it is causing the VM that is hosting the microservice to become unstable.

Context:

On Monday 2024-07-01 we deployed code to our LeadsServices UAT environment. Because changes were made to the secure.config (workflow was set to public), we had to restart the service in order for the changes in the secure.config to take effect. The VM restart was performed at around 16:15. After the restart, Sai from DevOps observed, using PRTG monitoring, a couple of red flags. One of them was the number of concurrent Warewolf connections. 

Now because this is a relatively new MicroService, and UAT environment, there wasn't any processes running in the background. I'm attaching the warewolf log to support this and for your review and consideration. The red flags only disappeared after 15mins - even tho there were no transactions / workflows processing on the microservice. wareWolf-Server-Leads-UAT.zip

However, what we have found is that when we restart a Production VM, one that is busy with a lot of workflows executing at the same time, these red flags take much longer to disappear. We also observed that workflows or APIs would take much longer to execute during this time. For example a Post Tool that calls an API that normally takes around 3 seconds would take much much longer - often to the point of timing out. And when this happens, Architecture and DevOps decides to switch off the VM as a precautionary measure - which means we are sitting without a service in Production.

If we consider the time we observed the red flags in PRTG, and we look at the corresponding time in the Warewolf Server logs - it appears that the following error may be the culprit:

[Header]
2024-07-01 16:15:02,101 ERROR - [Warewolf Error] - Dev2.ServerLifecycleManager
System.Runtime.InteropServices.COMException (0x8007045B): A system shutdown is in progress. (Exception from HRESULT: 0x8007045B)
at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)
at System.Management.ManagementScope.InitializeGuts(Object o)
at System.Management.ManagementScope.Initialize()
at System.Management.ManagementObjectSearcher.Initialize()
at System.Management.ManagementObjectSearcher.Get()
at Dev2.ServerLifecycleManager.GetNumberOfCores()
at Dev2.ServerLifecycleManager.TrackUsage(UsageType usageType, IExecutionLogPublisher logger)
at Dev2.ServerLifecycleManager.CleanupServer()
[Footer]

Please feel free to reach out should you need more information. Let's understand and solve together.

Regards,

Wynand

Hi

I am currently running warewolf on a VM with 8 vCPUs, I want to know how do I get insight into how many workflows are currently running and utilizing threads and how many are being queued to run

The reason I would like this insight is to be able to accurately monitor performance to ensure that I have enough vCPUs for the amount of workflows processing and also to know when to upscale or down scale correctly to be able to save costs.

My monitoring product that I am currently using is PRTG for monitoring my VM's

Im getting an error when using the post tool to a server that requires the Accept header:

"The 'Accept header must be modified using the appropriate property or method.
Parameter name: name"

The API setup was working properly before and this error started when I think the API started using the header on their side. Either way, its a coding issue on our end.

This is one Im calling: Investec programable banking, SA PB Account Information : Transfer Multiple v2

Its probably too dificult to setup on your end so happy to help with this where needed.

1. Have the ability in the Warewolf Studio to see the resource consumption down to individual named workflow for memory used, CPU threads used etc.
2. Have the ability to integrate 3rd party monitoring tools such as PRTG with the above in order to identify which named workflow is consuming all the resources assigned to the VM or Kubernetes POD.
3. Have the ability to set percentage threshold alerts on resources assigned to the VM or Kubernetes POD.
4. Have the ability to email alerts when thresholds are breached.

Scenario 

when warewolf get started it just shows simple message that "warewolf started..." but not provide squences of events generated with warewolf start 

Why we are looking this ?

Case 1 : 

with containerize warewolf , sometimes warewolf server is not starting on some time and it laggs to get started. that's where we are aware about why it is not starting /where it is stuck 

Case 2 : 

we also notice that with high amount of data in queue, triggers are started and take to get load, yet warewolf server has not fully started or container has not fully initialized  

Case 3 : 

when there are certain numbers of warewolf has been developed in warewolf instance like more than 800 workflows and it takes time to get started 

to resolve all the above cases , we need detail level logs of warewolf then it will help us to resolve or fix issue with our microservices . 

Good day,

I recently monitored RabbitMQ while the Ad Hoc Journey was executing. After the Ad Hoc Journey finished executing (confirmed by looking at the ServerLog) I can still see unack'ed messages in the relevant Rabbit queue.

I've seen it happen on a number of occasions and I'm wondering if the PreFetch function while running multiple containers (12 in the example below) doesn't lock some messages (21 as per the example below) as per screenshot below: 

How do I deploy from a batch file or command line rather than the Studio?

I am getting the following error when turning queues off. The queue was at a concurrency set as 1 then when I changed it to 0 and saved it errored in the Warewolf log.


2022-11-06 17:28:10,817 INFO - [Warewolf Info] - Save Trigger Queue Service
2022-11-06 17:28:11,323 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\52e4dba3-73fa-4267-877a-fa4f493a1511.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,332 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\667e5af4-c5a8-461b-b03d-c5683fe1fca0.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,340 ERROR - [Warewolf Error] - TriggersCatalog - Load - C:\ProgramData\Warewolf\Triggers\Queue\9923dc51-c138-4944-a7a4-06cb468f2e86.bite
System.Security.Cryptography.CryptographicException: Key not valid for use in specified state.

at System.Security.Cryptography.ProtectedData.Unprotect(Byte[] encryptedData, Byte[] optionalEntropy, DataProtectionScope scope)
at Warewolf.Security.Encryption.DpapiWrapper.Decrypt(String cipher)
at Dev2.Runtime.Hosting.TriggersCatalog.LoadQueueTriggerFromFile(String filename)
at Dev2.Runtime.Hosting.TriggersCatalog.Load()
2022-11-06 17:28:11,343 INFO - [00000000-0000-0000-0000-000000000000] - Trigger restarting 'f00a48f4-5348-490b-add4-3991c578c717'
2022-11-06 17:28:11,359 ERROR - [ at System.Diagnostics.Process.Kill()
at Warewolf.OS.ProcessMonitor.Kill()] - Access is denied
2022-11-06 17:28:11,371 INFO - [WarewolfLogger.exe] - Logging Server OnError, Error details:Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
2022-11-06 17:28:11,372 INFO - [WarewolfLogger.exe] - 11/6/2022 5:28:11 PM [Debug] Error while reading System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2022-11-06 17:28:11,378 INFO - [WarewolfLogger.exe] - --- End of inner exception stack trace ---
2022-11-06 17:28:11,379 INFO - [WarewolfLogger.exe] - at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
2022-11-06 17:28:11,379 INFO - [WarewolfLogger.exe] - at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
2022-11-06 17:28:11,390 INFO - [Warewolf Info] - queue process died: ProfileCustomerBackBook(f00a48f4-5348-490b-add4-3991c578c717)