Querying Audit History

Audit history is a great out of box feature in model-driven apps. However, querying audit history is a bit tricky. Unfortunately commonly used querying mechanisms like Power Automate CDS connectors, LinQ, or simply FetchXml doesn’t support it. This post will discuss options we have and sample code for it.

Options

  1. Using SDK messages, RetrieveRecordChangeHistoryRequest & RetrieveRecordChangeHistoryResponse, covered in this post 
  2. Using Kingswaysoft’s Integration Toolkit for D365 (not covering in this post)

Scenario

I will query audit history for contact entity records and read audit details for its email address attribute. Audit details are available under these four heads:

  • Changed date
  • Changed field
  • Old value
  • New value

If auditing is enabled, this code will work for almost any entity and attributes.

How it works

We need ids (GUID) of entities and using those we will query audit history. I m using a fetchxml query to retrieve ids, but it can be mechanism of your choice depending on implementation and requirement.

       var targetEntites_query = @"<fetch {0}>
                                <entity name='contact'>                               
                                 </entity>
                                 </fetch>";     

Generally we know FetchXml can return maximum 5000 entities, but this code will handle and return even if there are more then 5000 records in the result.

public List<Entity> RetrieveAllRecords(string fetch)
        {
            var moreRecords = false;
            int page = 1;
            var cookie = string.Empty;
            List<Entity> Entities = new List<Entity>();
            do
            {
                var xml = string.Format(fetch, cookie);
                var collection = CrmClient.RetrieveMultiple(new FetchExpression(xml));

                if (collection.Entities.Count >= 0) Entities.AddRange(collection.Entities);

                moreRecords = collection.MoreRecords;
                if (moreRecords)
                {
                    page++;
                    cookie = string.Format("paging-cookie='{0}' page='{1}'", System.Security.SecurityElement.Escape(collection.PagingCookie), page);
                }
            } while (moreRecords);

            return Entities;
        }

Tip: FetchXml query must have {0} if query will return more then 5000 records. Additional columns can be added in fetch if required.

Next, I m looping through these ids and read audit history for records using this code:

 public AuditDetailCollection GetAuditHistory(string entityLogicalName, Guid recordId)
        {           
            var changeRequest = new RetrieveRecordChangeHistoryRequest();
            changeRequest.Target = new EntityReference(entityLogicalName, recordId);
            var changeResponse = (RetrieveRecordChangeHistoryResponse)this.CrmClient.Execute(changeRequest);             
            return changeResponse.AuditDetailCollection;
        }

Above function returns AuditDetailCollection which has a collection of AuditDetails. One Audit detail represents one entry in audit history. Please note audit history records are in the same order as they appear in UI (descending).

Every audit details record will have a changed date, and collection of new and old values with field names which we will need to loop through and read.

Below is code to accomplish this:

             //Collection of entities for which we are going to read audit history
            var AllTargetEnteties = this.RetrieveAllRecords(targetEntites_query);


            foreach (var targetComplaint in AllTargetEnteties)
            {
                //Now pass id(guid) of record with entity name to retrieve audit history 
                var audit_history_entries = this.GetAuditHistory(targetComplaint.LogicalName, targetComplaint.Id);


                foreach (AuditDetail auditDetail in audit_history_entries.AuditDetails)
                {

                    if ((auditDetail.GetType())?.Name == "AttributeAuditDetail")                     
                    {
                        //Below code reads Changed Date
                        var changeDate = auditDetail.AuditRecord.GetAttributeValue<DateTime>("createdon");

                        var newValueEntity = ((AttributeAuditDetail)auditDetail)?.NewValue;
                        if (newValueEntity.Attributes.Count > 0)
                        {
                            {
                                foreach (var attrNewValue in newValueEntity?.Attributes)
                                {
                                    //Here we will need to match attribute name to read new value.
                                    //In this case I m reading emailaddress1
                                    if (attrNewValue.Key == "emailaddress1")
                                    {                                        
                                        var newEmailAddress = attrNewValue.Value;
                                        //Custom Logic for New Value here

                                    }
                                }
                            }
                        }


                        var oldValueEntity = ((AttributeAuditDetail)auditDetail)?.OldValue;
                        if (oldValueEntity.Attributes.Count > 0)
                        {
                            foreach (var attrOldValue in oldValueEntity?.Attributes)
                            {
                                //Here we will need to match attribute name to read old value.
                                //In this case I m reading emailaddress1
                                if (attrOldValue.Key == "emailaddress1")
                                {
                                    var oldEmailAddress = attrOldValue.Value;
                                    //Custom logic for Old value will be here

                                }
                            }

                        }

                    }
                }
            }

Let’s Connect

 twIcon lnIcon fbicon

Can’t connect to D365 if system clock is wrong

Probably title of this post summaries it well, but there is some detail worth explanation. I was developing an Azure Function app which was basically integrating data between D365 CE and another system. The major functionality of this application was complete and tested. Two weeks back when I came back from leave I found application (which I was developing), can’t connect to CE now. I was using Microsoft.Xrm.Tooling assembly with the following code but now it started returning null:

public static IOrganizationService GetOrganizationService(ref TraceWriter log)
{
IOrganizationService _orgService = null;
string connectionstring = ConfigurationManager.AppSettings["connectionstring"].ToString();
CrmServiceClient conn = new Microsoft.Xrm.Tooling.Connector.CrmServiceClient(connectionstring);
_orgService = (IOrganizationService)conn.OrganizationWebProxyClient != null ? (IOrganizationService)conn.OrganizationWebProxyClient : (IOrganizationService)conn.OrganizationServiceProxy;
return _orgService;
}

CrmServiceClient has two very useful properties LastCRMError and LastCRMException which were showing this error message:

Unable to Login to Dynamics CRMOrganizationWebProxyClient is nullOrganizationServiceProxy is nullOrganizationServiceProxy is null

Tried looking for solutions and found the following suggestions:

  1. In connection string try Orgnaization unique name instead of friendly name
  2. May be assembly version is not compatible or code/ connection string should be written differently
  3. Use ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12; before connection

but none of these worked in my case. I contacted Microsoft support and they confirmed there was no such update or roll out that might have caused this but the good thing is they were still willing to help 🙂

I tried different types of applications, environments and versions but no luck.

In last while testing the following code the error was a little more meaningful:


IServiceManagement orgServiceManagement = ServiceConfigurationFactory.CreateManagement(new Uri("https://myCrmInstance.crm5.dynamics.com/XRMServices/2011/Organization.svc"));

AuthenticationCredentials authCredentials = new AuthenticationCredentials();
authCredentials.ClientCredentials.UserName.UserName = “user@email.com”;
authCredentials.ClientCredentials.UserName.Password = “*********”;
AuthenticationCredentials tokenCredentials = orgServiceManagement.Authenticate(authCredentials);

OrganizationServiceProxy organizationProxy = new OrganizationServiceProxy(orgServiceManagement, tokenCredentials.SecurityTokenResponse);
Entity contact = new Entity(“contact”);
contact.Attributes[“firstname”] = “Yawer”;
contact.Attributes[“lastname”] = “Iqbal”;
var contactId = organizationProxy.Create(contact);

“The security timestamp is invalid because its creation time (‘2019-04-01T12:30:45.790Z’) is in the future. Current time is ‘2019-04-01T12:24:29.185Z’ and allowed clock skew is ’00:05:00′.”

Getting hint about time I found time on my machine is 6 minutes behind. What caused this time change I still don’t know but since it was the difference of just 6 minutes I couldn’t notice this change. Corrected time and everything started working as it was. I thought to do a little experiment and moved clock 6 minutes ahead of current time but this time error was different:

An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.

and inner exception:

An error occurred when verifying security for the message.

Hope this sharing will save someone’s time.

Please feel free to share this post, leave comments or suggestions if there are any. If you like to remain informed about future posts please follow me.

Enjoy this beautiful day of your 365 life 🙂

#D365 #D365CE #MSDynamics

Unexpected exception from plug-in (Execute): System.MissingMethodException: Method not found: ‘!!0[] System.Array.Empty()’.

For a demo I was writing a workflow activity and got the error:

Unexpected exception from plug-in (Execute): WorkFlowComponents.DeleteUnwantedPotentialLeads: System.MissingMethodException: Method not found: '!!0[] System.Array.Empty()'.

It was happening becasue my assembly version was not supported by Dynamics 365. As per this document Dynamcis 365 supports .Net Framework 4.5.2.

Corrected target .Net framework, redeployed assembly and same code started working as expected.

Hope it will help.

Ftp to CRM using Azure Function

FTPTOCRM

New to Azure Function?

Function is serverless offering form Azure. Serverless computing is a way to write code without need to manage infrastructure, application dependencies and other required resources. Even for scaling Azure will take care of them. Once we know the environment using Function is easy, login to Azure portal write or deploy code and start using it.

An Azure function is simply a function written in C#, Java, JavaScript, F#, Python or PHP. A function can be executed manually or scheduled to run automatically. The third way to execute a function is through triggers. A trigger can be another Azure service or something which has no link with Azure. Some Azure services which can trigger a function are Cosmos DB, Event Hubs, App Service, Storage queues, blobs, Service Bus queues or topics. Functions are also available for Logic Apps, Power Apps, Microsoft Flow and outside Azure over HTTP.

About Demo

This function will be scheduled to run once in 24 hours. From ftp server it will read a CSV file (data for lead entity) and pass it to CRM. Functions can be coded and published from Visual Studio or directly in Azure portal, for this demo I will use later approach.

FtpFun_0CreatefunApp

Create Function App and Function

To create a function we need to create a function app. If you don’t want to use existing resource group and storage feel free to create new.

Open function app and add a function in it:


In Schedule add 0 0 12 * * *. This cron expression will trigger this function in midnight at 12 O’clock. See this for more details about cron expression.

App Settings

Let’s add CRM connection string, FTP URL and credentials as application settings to avoid hard coding.

Sample of application settings:

Key Value Comments
FtpId UserId  Ftp account user id
FtpPassword P@ssword  Ftp account password
FtpAddress  ftp://ftp.domain.com/full.csv Ftp address with file name
Connectionstring AuthType = Office365; Url = https://crminstace.crm6.dynamics.com/;

UserName=crmuser@domain.com;

Password=crmP@ssword

This is example with Dynamics 365, no need to surround with single or double quote.

Code

FtpFun_10CreatefunApp

using System;
using System.Configuration;
using System.IO;
using System.Net;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Tooling.Connector;
public static void Run(TimerInfo myTimer, TraceWriter log)
{
log.Info($”Execution Started. : {DateTime.Now} “);
Stream fileStream = null;
string[] fileContentToWriteToCRM;
IOrganizationService org;
string ftpId = ConfigurationManager.AppSettings[“Ftpid”].ToString();
string ftpAddress = ConfigurationManager.AppSettings[“ftpAddress”].ToString();
string ftpPassword = ConfigurationManager.AppSettings[“ftpPassword”].ToString();
#region Read Ftp File(s)
FtpWebRequest ftpReq = (System.Net.FtpWebRequest)System.Net.FtpWebRequest.Create(ftpAddress);
ftpReq.Credentials = new NetworkCredential(ftpId, ftpPassword);
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ftpReq.EnableSsl = false;
WebResponse tmpRes = ftpReq.GetResponse();
fileStream = tmpRes.GetResponseStream();
#endregion
#region ProcessData
TextReader tmpReader = new StreamReader(fileStream);
var txtData = tmpReader.ReadToEnd();
fileContentToWriteToCRM = txtData.Split(new string[] { “\r\n” }, StringSplitOptions.RemoveEmptyEntries);
#endregion
#region CRM Data Posting
string connectionstring = ConfigurationManager.AppSettings[“connectionstring”];
CrmServiceClient conn = new Microsoft.Xrm.Tooling.Connector.CrmServiceClient(connectionstring);
org = (IOrganizationService)conn.OrganizationWebProxyClient != null ? (IOrganizationService)conn.OrganizationWebProxyClient : (IOrganizationService)conn.OrganizationServiceProxy;
log.Info($”CRM connection established”);
log.Info($”Looping to move data to CRM”);
foreach (var row in fileContentToWriteToCRM)
{
var rowvalues = row.Split(‘,’);
Entity lead = new Entity(“lead”);
lead.Attributes[“subject”] = rowvalues[0].ToString();
lead.Attributes[“firstname”] = rowvalues[1].ToString();
lead.Attributes[“lastname”] = rowvalues[2].ToString();
var id = org.Create(lead);
log.Info($”Lead created in CRM with GUID : {id} “);
}
log.Info($”Loop Ended moved all data to CRM “);
#endregion
}

Adding Dependencies

Since code use assemblies from CRM SDK, we will add these to our code. With Azure Function it is achieved using project.json file. Add project.json file if it is not already there and then add NuGet packages in it.

FtpFun_11CreatefunApp

FtpFun_12CreatefunApp

{
“frameworks”: {
“net46”:{
“dependencies”: {
“Microsoft.CrmSdk.CoreAssemblies”: “9.0.0.0”,
“Microsoft.CrmSdk.XrmTooling.CoreAssembly”:”9.0.0.7″
}
}
}
}
Let’s save it, run the function if it is not already running and see how we go.

Result

FtpFun_13CreateFunApp

Hope this help, enjoy your 365 day 🙂

Cheers

About Me 🙂

I m an IT consultant working in Melbourne Australia. I solve business problems using Microsoft technologies (Dynamics 365, Office 365, Azure, Flow, Power Apps, Power BI). I m involved in community activities and I blog at http://www.crmtechie.com/

I love to get connected with people working in IT, providing solutions or who just like Microsoft technologies. To get in touch please follow my blog, and connect through Linkedin, Twitter or Facebook

Blog: http://www.crmtechie.com/

Twitter: @YawerIqbal

Linkedin: YawerIqbal

Facebook: Yawer.Iqbal

ConditionExpression error “expected argument(s) of type ‘System.Guid’ but received ‘System.Guid[]”

Today while writing query expression I was getting error:

Condition for attribute ‘sf_contract.contractid’: expected argument(s) of type ‘System.Guid’ but received ‘System.Guid[]’.

We can pass a single or multiple values in ConditionExpression using appropriate comparison operator(s). In this example ConditionExpression was comparing multiple GUID values (stored in object array) with an attribute using “IN” comparison operator.

ConditionExpression have a few constructors, and I was trying to use one that accepts params object[] as argument but it didn’t work as per expectation.

This error was gone when I changed my collection to attribute’s base type, so basically I changed object[] to Guid[]. Hope this explanation will save someone’s time.

QueryExpression error, “The formatter threw an exception while trying to deserialize the message”

Today I wrote a QueryExpression which was throwing exception “The formatter threw an exception while trying to deserialize the message”.

The complete error message was:

“The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://schemas.microsoft.com/xrm/2011/Contracts/Services:query. The InnerException message was ‘Error in line 1 position 2016. Element ‘http://schemas.microsoft.com/2003/10/Serialization/Arrays:anyType&#8217; contains data from a type that maps to the name ‘http://schemas.microsoft.com/xrm/2011/Contracts:EntityReference&#8217;. The deserializer has no knowledge of any type that maps to this name. Consider changing the implementation of the ResolveName method on your DataContractResolver to return a non-null value for name ‘EntityReference’ and namespace ‘http://schemas.microsoft.com/xrm/2011/Contracts&#8217;.’. Please see InnerException for more details.”

I was using ‘EntityReference’ in a condition, and error message gives hint that somehow there is an issue with ‘EntityReference’ format, and it is not being deserialized correctly. To use ‘EntityReference’ in condition we need to pass its ‘GUID’, and not complete ‘EntityReference’ object.

In the following QueryExpression, the condition will produce a similar exception which can be avoided by passing GUID instead of passing ‘EntityReference’ object as parameter.

QueryExpression query = new QueryExpression();
query.EntityName = “new_document”;
query.ColumnSet = new ColumnSet(“new_name”);
query.Criteria.AddCondition(“new_authorid”, ConditionOperator.Equal,currentDoc.GetAttributeValue<EntityReference>(“new_authorid”));
EntityCollection results =org.RetrieveMultiple(query);