Querying Audit History

Audit history is a great out of box feature in model-driven apps. However, querying audit history is a bit tricky. Unfortunately commonly used querying mechanisms like Power Automate CDS connectors, LinQ, or simply FetchXml doesn’t support it. This post will discuss options we have and sample code for it.

Options

  1. Using SDK messages, RetrieveRecordChangeHistoryRequest & RetrieveRecordChangeHistoryResponse, covered in this post 
  2. Using Kingswaysoft’s Integration Toolkit for D365 (not covering in this post)

Scenario

I will query audit history for contact entity records and read audit details for its email address attribute. Audit details are available under these four heads:

  • Changed date
  • Changed field
  • Old value
  • New value

If auditing is enabled, this code will work for almost any entity and attributes.

How it works

We need ids (GUID) of entities and using those we will query audit history. I m using a fetchxml query to retrieve ids, but it can be mechanism of your choice depending on implementation and requirement.

       var targetEntites_query = @"<fetch {0}>
                                <entity name='contact'>                               
                                 </entity>
                                 </fetch>";     

Generally we know FetchXml can return maximum 5000 entities, but this code will handle and return even if there are more then 5000 records in the result.

public List<Entity> RetrieveAllRecords(string fetch)
        {
            var moreRecords = false;
            int page = 1;
            var cookie = string.Empty;
            List<Entity> Entities = new List<Entity>();
            do
            {
                var xml = string.Format(fetch, cookie);
                var collection = CrmClient.RetrieveMultiple(new FetchExpression(xml));

                if (collection.Entities.Count >= 0) Entities.AddRange(collection.Entities);

                moreRecords = collection.MoreRecords;
                if (moreRecords)
                {
                    page++;
                    cookie = string.Format("paging-cookie='{0}' page='{1}'", System.Security.SecurityElement.Escape(collection.PagingCookie), page);
                }
            } while (moreRecords);

            return Entities;
        }

Tip: FetchXml query must have {0} if query will return more then 5000 records. Additional columns can be added in fetch if required.

Next, I m looping through these ids and read audit history for records using this code:

 public AuditDetailCollection GetAuditHistory(string entityLogicalName, Guid recordId)
        {           
            var changeRequest = new RetrieveRecordChangeHistoryRequest();
            changeRequest.Target = new EntityReference(entityLogicalName, recordId);
            var changeResponse = (RetrieveRecordChangeHistoryResponse)this.CrmClient.Execute(changeRequest);             
            return changeResponse.AuditDetailCollection;
        }

Above function returns AuditDetailCollection which has a collection of AuditDetails. One Audit detail represents one entry in audit history. Please note audit history records are in the same order as they appear in UI (descending).

Every audit details record will have a changed date, and collection of new and old values with field names which we will need to loop through and read.

Below is code to accomplish this:

             //Collection of entities for which we are going to read audit history
            var AllTargetEnteties = this.RetrieveAllRecords(targetEntites_query);


            foreach (var targetComplaint in AllTargetEnteties)
            {
                //Now pass id(guid) of record with entity name to retrieve audit history 
                var audit_history_entries = this.GetAuditHistory(targetComplaint.LogicalName, targetComplaint.Id);


                foreach (AuditDetail auditDetail in audit_history_entries.AuditDetails)
                {

                    if ((auditDetail.GetType())?.Name == "AttributeAuditDetail")                     
                    {
                        //Below code reads Changed Date
                        var changeDate = auditDetail.AuditRecord.GetAttributeValue<DateTime>("createdon");

                        var newValueEntity = ((AttributeAuditDetail)auditDetail)?.NewValue;
                        if (newValueEntity.Attributes.Count > 0)
                        {
                            {
                                foreach (var attrNewValue in newValueEntity?.Attributes)
                                {
                                    //Here we will need to match attribute name to read new value.
                                    //In this case I m reading emailaddress1
                                    if (attrNewValue.Key == "emailaddress1")
                                    {                                        
                                        var newEmailAddress = attrNewValue.Value;
                                        //Custom Logic for New Value here

                                    }
                                }
                            }
                        }


                        var oldValueEntity = ((AttributeAuditDetail)auditDetail)?.OldValue;
                        if (oldValueEntity.Attributes.Count > 0)
                        {
                            foreach (var attrOldValue in oldValueEntity?.Attributes)
                            {
                                //Here we will need to match attribute name to read old value.
                                //In this case I m reading emailaddress1
                                if (attrOldValue.Key == "emailaddress1")
                                {
                                    var oldEmailAddress = attrOldValue.Value;
                                    //Custom logic for Old value will be here

                                }
                            }

                        }

                    }
                }
            }

Let’s Connect

 twIcon lnIcon fbicon

Can’t connect to D365 if system clock is wrong

Probably title of this post summaries it well, but there is some detail worth explanation. I was developing an Azure Function app which was basically integrating data between D365 CE and another system. The major functionality of this application was complete and tested. Two weeks back when I came back from leave I found application (which I was developing), can’t connect to CE now. I was using Microsoft.Xrm.Tooling assembly with the following code but now it started returning null:

public static IOrganizationService GetOrganizationService(ref TraceWriter log)
{
IOrganizationService _orgService = null;
string connectionstring = ConfigurationManager.AppSettings["connectionstring"].ToString();
CrmServiceClient conn = new Microsoft.Xrm.Tooling.Connector.CrmServiceClient(connectionstring);
_orgService = (IOrganizationService)conn.OrganizationWebProxyClient != null ? (IOrganizationService)conn.OrganizationWebProxyClient : (IOrganizationService)conn.OrganizationServiceProxy;
return _orgService;
}

CrmServiceClient has two very useful properties LastCRMError and LastCRMException which were showing this error message:

Unable to Login to Dynamics CRMOrganizationWebProxyClient is nullOrganizationServiceProxy is nullOrganizationServiceProxy is null

Tried looking for solutions and found the following suggestions:

  1. In connection string try Orgnaization unique name instead of friendly name
  2. May be assembly version is not compatible or code/ connection string should be written differently
  3. Use ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12; before connection

but none of these worked in my case. I contacted Microsoft support and they confirmed there was no such update or roll out that might have caused this but the good thing is they were still willing to help 🙂

I tried different types of applications, environments and versions but no luck.

In last while testing the following code the error was a little more meaningful:


IServiceManagement orgServiceManagement = ServiceConfigurationFactory.CreateManagement(new Uri("https://myCrmInstance.crm5.dynamics.com/XRMServices/2011/Organization.svc"));

AuthenticationCredentials authCredentials = new AuthenticationCredentials();
authCredentials.ClientCredentials.UserName.UserName = “user@email.com”;
authCredentials.ClientCredentials.UserName.Password = “*********”;
AuthenticationCredentials tokenCredentials = orgServiceManagement.Authenticate(authCredentials);

OrganizationServiceProxy organizationProxy = new OrganizationServiceProxy(orgServiceManagement, tokenCredentials.SecurityTokenResponse);
Entity contact = new Entity(“contact”);
contact.Attributes[“firstname”] = “Yawer”;
contact.Attributes[“lastname”] = “Iqbal”;
var contactId = organizationProxy.Create(contact);

“The security timestamp is invalid because its creation time (‘2019-04-01T12:30:45.790Z’) is in the future. Current time is ‘2019-04-01T12:24:29.185Z’ and allowed clock skew is ’00:05:00′.”

Getting hint about time I found time on my machine is 6 minutes behind. What caused this time change I still don’t know but since it was the difference of just 6 minutes I couldn’t notice this change. Corrected time and everything started working as it was. I thought to do a little experiment and moved clock 6 minutes ahead of current time but this time error was different:

An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.

and inner exception:

An error occurred when verifying security for the message.

Hope this sharing will save someone’s time.

Please feel free to share this post, leave comments or suggestions if there are any. If you like to remain informed about future posts please follow me.

Enjoy this beautiful day of your 365 life 🙂

#D365 #D365CE #MSDynamics

Quick Tip: How to modify view FetchXML

We can easily create or modify a view in Dynamics 365, this basically generates and saves view FetchXML. It is a great feature but we don’t have control to modify FetchXML. The good news is it can be done without writing code. “View Designer” is another useful XrmToolBox plugin which allows editing of view FetchXML, save and publish it back. Thanks, Andre Margono (MVP) for sharing this. I hope this quick tip was helpful.

#MSDyn365 #MsDynCRM

Enjoy your day of 365 life.

About Me 🙂

I m an IT consultant working in Melbourne Australia. I solve business problems using Microsoft technologies (Dynamics 365, Office 365, Azure, Flow, Power Apps, Power BI). I m involved in community activities and I blog at http://www.crmtechie.com/

I love to get connected with people working in IT, providing solutions or who just like Microsoft technologies. To get in touch please follow my blog, and connect through Linkedin, Twitter or Facebook

Blog: http://www.crmtechie.com/

Twitter: @YawerIqbal

Linkedin: YawerIqbal

Facebook: Yawer.Iqbal

Unexpected exception from plug-in (Execute): System.MissingMethodException: Method not found: ‘!!0[] System.Array.Empty()’.

For a demo I was writing a workflow activity and got the error:

Unexpected exception from plug-in (Execute): WorkFlowComponents.DeleteUnwantedPotentialLeads: System.MissingMethodException: Method not found: '!!0[] System.Array.Empty()'.

It was happening becasue my assembly version was not supported by Dynamics 365. As per this document Dynamcis 365 supports .Net Framework 4.5.2.

Corrected target .Net framework, redeployed assembly and same code started working as expected.

Hope it will help.

Ftp to CRM using Azure Function

FTPTOCRM

New to Azure Function?

Function is serverless offering form Azure. Serverless computing is a way to write code without need to manage infrastructure, application dependencies and other required resources. Even for scaling Azure will take care of them. Once we know the environment using Function is easy, login to Azure portal write or deploy code and start using it.

An Azure function is simply a function written in C#, Java, JavaScript, F#, Python or PHP. A function can be executed manually or scheduled to run automatically. The third way to execute a function is through triggers. A trigger can be another Azure service or something which has no link with Azure. Some Azure services which can trigger a function are Cosmos DB, Event Hubs, App Service, Storage queues, blobs, Service Bus queues or topics. Functions are also available for Logic Apps, Power Apps, Microsoft Flow and outside Azure over HTTP.

About Demo

This function will be scheduled to run once in 24 hours. From ftp server it will read a CSV file (data for lead entity) and pass it to CRM. Functions can be coded and published from Visual Studio or directly in Azure portal, for this demo I will use later approach.

FtpFun_0CreatefunApp

Create Function App and Function

To create a function we need to create a function app. If you don’t want to use existing resource group and storage feel free to create new.

Open function app and add a function in it:


In Schedule add 0 0 12 * * *. This cron expression will trigger this function in midnight at 12 O’clock. See this for more details about cron expression.

App Settings

Let’s add CRM connection string, FTP URL and credentials as application settings to avoid hard coding.

Sample of application settings:

Key Value Comments
FtpId UserId  Ftp account user id
FtpPassword P@ssword  Ftp account password
FtpAddress  ftp://ftp.domain.com/full.csv Ftp address with file name
Connectionstring AuthType = Office365; Url = https://crminstace.crm6.dynamics.com/;

UserName=crmuser@domain.com;

Password=crmP@ssword

This is example with Dynamics 365, no need to surround with single or double quote.

Code

FtpFun_10CreatefunApp

using System;
using System.Configuration;
using System.IO;
using System.Net;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Tooling.Connector;
public static void Run(TimerInfo myTimer, TraceWriter log)
{
log.Info($”Execution Started. : {DateTime.Now} “);
Stream fileStream = null;
string[] fileContentToWriteToCRM;
IOrganizationService org;
string ftpId = ConfigurationManager.AppSettings[“Ftpid”].ToString();
string ftpAddress = ConfigurationManager.AppSettings[“ftpAddress”].ToString();
string ftpPassword = ConfigurationManager.AppSettings[“ftpPassword”].ToString();
#region Read Ftp File(s)
FtpWebRequest ftpReq = (System.Net.FtpWebRequest)System.Net.FtpWebRequest.Create(ftpAddress);
ftpReq.Credentials = new NetworkCredential(ftpId, ftpPassword);
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
ftpReq.EnableSsl = false;
WebResponse tmpRes = ftpReq.GetResponse();
fileStream = tmpRes.GetResponseStream();
#endregion
#region ProcessData
TextReader tmpReader = new StreamReader(fileStream);
var txtData = tmpReader.ReadToEnd();
fileContentToWriteToCRM = txtData.Split(new string[] { “\r\n” }, StringSplitOptions.RemoveEmptyEntries);
#endregion
#region CRM Data Posting
string connectionstring = ConfigurationManager.AppSettings[“connectionstring”];
CrmServiceClient conn = new Microsoft.Xrm.Tooling.Connector.CrmServiceClient(connectionstring);
org = (IOrganizationService)conn.OrganizationWebProxyClient != null ? (IOrganizationService)conn.OrganizationWebProxyClient : (IOrganizationService)conn.OrganizationServiceProxy;
log.Info($”CRM connection established”);
log.Info($”Looping to move data to CRM”);
foreach (var row in fileContentToWriteToCRM)
{
var rowvalues = row.Split(‘,’);
Entity lead = new Entity(“lead”);
lead.Attributes[“subject”] = rowvalues[0].ToString();
lead.Attributes[“firstname”] = rowvalues[1].ToString();
lead.Attributes[“lastname”] = rowvalues[2].ToString();
var id = org.Create(lead);
log.Info($”Lead created in CRM with GUID : {id} “);
}
log.Info($”Loop Ended moved all data to CRM “);
#endregion
}

Adding Dependencies

Since code use assemblies from CRM SDK, we will add these to our code. With Azure Function it is achieved using project.json file. Add project.json file if it is not already there and then add NuGet packages in it.

FtpFun_11CreatefunApp

FtpFun_12CreatefunApp

{
“frameworks”: {
“net46”:{
“dependencies”: {
“Microsoft.CrmSdk.CoreAssemblies”: “9.0.0.0”,
“Microsoft.CrmSdk.XrmTooling.CoreAssembly”:”9.0.0.7″
}
}
}
}
Let’s save it, run the function if it is not already running and see how we go.

Result

FtpFun_13CreateFunApp

Hope this help, enjoy your 365 day 🙂

Cheers

About Me 🙂

I m an IT consultant working in Melbourne Australia. I solve business problems using Microsoft technologies (Dynamics 365, Office 365, Azure, Flow, Power Apps, Power BI). I m involved in community activities and I blog at http://www.crmtechie.com/

I love to get connected with people working in IT, providing solutions or who just like Microsoft technologies. To get in touch please follow my blog, and connect through Linkedin, Twitter or Facebook

Blog: http://www.crmtechie.com/

Twitter: @YawerIqbal

Linkedin: YawerIqbal

Facebook: Yawer.Iqbal

How to view plugins deployment summary ?

Sometimes we need to know when last time a plugin was updated. This information is helpful usually before exporting assemblies from one environment to other.
In similar way few times we may need to know who updated a particular assembly last time ?

The following LINQ or Fetch Xml query will return this summary:

(from p in PluginAssemblySet
orderby p.ModifiedOn descending
select new
{
PluginName = p.Name, ModifiedOn = p.ModifiedOn.Value.ToLocalTime(),
ModifiedBy= p.ModifiedBy.Name, Role=p.ModifiedBy.LogicalName,Version=p.Version
}).Dump();

 

 

ConditionExpression error “expected argument(s) of type ‘System.Guid’ but received ‘System.Guid[]”

Today while writing query expression I was getting error:

Condition for attribute ‘sf_contract.contractid’: expected argument(s) of type ‘System.Guid’ but received ‘System.Guid[]’.

We can pass a single or multiple values in ConditionExpression using appropriate comparison operator(s). In this example ConditionExpression was comparing multiple GUID values (stored in object array) with an attribute using “IN” comparison operator.

ConditionExpression have a few constructors, and I was trying to use one that accepts params object[] as argument but it didn’t work as per expectation.

This error was gone when I changed my collection to attribute’s base type, so basically I changed object[] to Guid[]. Hope this explanation will save someone’s time.

QueryExpression error, “The formatter threw an exception while trying to deserialize the message”

Today I wrote a QueryExpression which was throwing exception “The formatter threw an exception while trying to deserialize the message”.

The complete error message was:

“The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://schemas.microsoft.com/xrm/2011/Contracts/Services:query. The InnerException message was ‘Error in line 1 position 2016. Element ‘http://schemas.microsoft.com/2003/10/Serialization/Arrays:anyType&#8217; contains data from a type that maps to the name ‘http://schemas.microsoft.com/xrm/2011/Contracts:EntityReference&#8217;. The deserializer has no knowledge of any type that maps to this name. Consider changing the implementation of the ResolveName method on your DataContractResolver to return a non-null value for name ‘EntityReference’ and namespace ‘http://schemas.microsoft.com/xrm/2011/Contracts&#8217;.’. Please see InnerException for more details.”

I was using ‘EntityReference’ in a condition, and error message gives hint that somehow there is an issue with ‘EntityReference’ format, and it is not being deserialized correctly. To use ‘EntityReference’ in condition we need to pass its ‘GUID’, and not complete ‘EntityReference’ object.

In the following QueryExpression, the condition will produce a similar exception which can be avoided by passing GUID instead of passing ‘EntityReference’ object as parameter.

QueryExpression query = new QueryExpression();
query.EntityName = “new_document”;
query.ColumnSet = new ColumnSet(“new_name”);
query.Criteria.AddCondition(“new_authorid”, ConditionOperator.Equal,currentDoc.GetAttributeValue<EntityReference>(“new_authorid”));
EntityCollection results =org.RetrieveMultiple(query);

The ‘orderBy’ expression is limited to invoking the ‘ ‘ parameter

LINQ is widely used in projects and it makes querying very handy. Using LINQ for querying CRM data has some limitations though, some of the standard LINQ features are not supported to use with CRM.

One such limitation is when we are joining entities and also sorting results. To use orderby, we are limited to the first entity only (the left side entity in join). So the query in Example 1 works perfectly fine:

Example 1:

var results = from c in ContactSet
join a in AccountSet
on c.ContactId equals a.PrimaryContactId.Id
orderby c.FullName
select new
{
c.FullName,
c.Address1_City,
a.Name,
a.Address1_Name
};

But trying to sort by columns of other entity is not supported. So if we try Example 2 it will throw the following NotSupportedException

The ‘orderBy’ expression is limited to invoking the ‘c’ parameter.

Exaple 2:

var results = from c in ContactSet
join a in AccountSet
on c.ContactId equals a.PrimaryContactId.Id
orderby a.Name
select new
{
c.FullName,
c.Address1_City,
a.Name,
a.Address1_Name
};

Of course if you run similar quires directly on SQL Server database they will work fine.

Solution:

If you are getting this exception there are two possible solutions:

  • Modify your quey as per Example 1 and put orderby columns’s entity firs or on the left side of join
  • For queries involving orderby from multiple entities, select data, store it in memory and use a separate LINQ query to sort it.
    This MSDN document list limitations of LINQ for CRM

Enjoy working with CRM 🙂

Parent child lookup fields in Dynamic CRM

Showing parent-child records in drop-down lists is a common requirement of business applications.

Some examples:

  1. Manager > Employees
  2. Country > State or Province > City
  3. Account category > Account sub-category > Account number

In Dynamics CRM same behaviour can be implemented by using multiple lookup fields with “Related Records Filtering” which is an out of box CRM feature. For this demo consider an entity “Car Finance Application”, which needs some related details like car brand, model and variant.

a

 

 

Steps:

  • Create the following entities and their relationship as described below:
    • Brand
    • Model, it has N:1 relationship with Brand
    • Variant, it has N:1 relationship with Model
  • Add brand, model and variant lookup fields in consumer entity which in this case is CarFinanceApplication:
  • Add brand, model and variant fields in the form
  • Double click on Model lookup field, and do the “Related Records Filtering”  configurations as shown below:  
  • Double click on Variant lookup field, and do the “Related Records Filtering”  configurations as below:

 

  • Save changes + build customizations + test working of lookup field

Form and lookup fields have been configured and selecting brand will show relevant models. As user selects a model, variant look up will list relevant records but there is a catch. If user changes brand or model after selecting variant, values in three lookup fields will not be in sync. Saving record in such scenarios will basically save  incorrect data  and you will agree many times even user will not realise this. An easy approach to fix this issue can be to reset model and variant look up fields when brand field changes and reset variant lookup field when model field changes.

  • From same form window add the following JS as a code library:
function ClearLookUp() {
for (var counter = 0; counter < arguments.length; counter++) {
if (typeof(arguments[counter]) === "string") {
var field = Xrm.Page.data.entity.attributes.get(arguments[counter]);
if (field != null && field.getValue() != null)
field.setValue(null);
}
}
}
  • Double click on “Brand” lookup field and add “ClearLookUp” javascript function on its “onChange” event. Pass execution context and name of lookup field(s) as comma-separated arguments which will need to reset.

 

  • Double click on “Model” lookup field and call “ClearLookUp” function on its “onChange” event. Pass execution context and name of lookup field(s) as a comma-separated argument.
  • That’s it, test your work and go enjoy a coffee if you like 🙂

Considerations:

  • Any number of lookup fields can be configured with this approach
  • Javascript function is a sort of generic, it can be configured with any lookup field without modification. Also just by calling once it can reset any number of lookup fields, the only requirement will be to pass the name of lookup fields as comma-separated arguments.

Credit:

Before this post, I found a similar implementation by Rashan which I have improved.