CI/CD & Test Automation for Dynamics 365 in Azure DevOps/VSTS – Part 1

In this blog series, we will explore building out DevOps processes and practices for Dynamics 365 Customer Engagement (CE) by utilizing Wael Hamez MSCRM Build tools. In this first blog, we will cover the version control for Solutions.

What is DevOps?

DevOps is a new term emerging from the collision of two major related trends. The first was also called “agile infrastructure” or “agile operations”; it sprang from applying Agile and Lean approaches to operations work. The second is a much-expanded understanding of the value of collaboration between development and operations staff throughout all stages of the development lifecycle when creating and operating a service, and how important operations has become in our increasingly service-oriented world (cf. Operations: The New Secret Sauce).

Problem Statement

I have started working on CI/CD when I was assigned as a Platform Engineer for a Dynamics 365 CE implementation project. At that time, I had a few key challenges with Dynamics 365 CE. I have listed those below:

1. Solution files were manually extracted and imported to target as a deployment process

2. No Unit testing or validation for deployed solution

3. Multiple deployment process is followed between release environments. For example, in Dev and Sit environment, the solution was migrated manually, and in UAT, Pre-Prod and Prod environment DB compare was applied to promote changes

4. Master data were mutually entered in each environment

5. Multiple developers working in the same organizations overwriting the changes.

Before we start solving the problem, let us take a moment to define some of the mandatory steps which we need to follow.

1. All the source code must be in source control. For example, Plugin Code, Custom workflows, actions, Warehouses (HTML, js, etc) Solution file, master Data, and user roles, etc.

2. Check-in regularly and every change should trigger the commit process

3. Commit process should be short and validates the committed component(Gated-Checkin)

4. Track changes and rollback as needed

Pre-Requisites

1. Visual Studio 2017/2015

2. Visual Studio Team Services (VSTS)/ Azure DevOps

3. Dynamics 365 CE(online)

4. PowerShell

5. Solution Packager

6. Dynamics 365 Build Tools by Wael Hamze(https://marketplace.visualstudio.com/items?itemName=WaelHamze.xrm-ci-framework-build-tasks)

Here is a little bit of information regarding the environment. We are using Dynamics 365 (Online) v9.0 and VSTS/Azure DevOps for version control, build and deployment. I am using Visual Studio 2017 with the Microsoft Dynamics CRM SDK Templates extension installed.

I have also installed the Dynamics 365 Build Tools(By Wael Hamze) into our VSTS Organization so the tasks are available for all Build and Release Definitions.

Version control for solutions

Solutions in Dynamics 365 CE are in essence a package containing any customization we’ve done to our environment that we can export from one environment then import into various environments. When exported from an environment, solutions are in the form of a zip file. When that zip file is unzipped, the output directory contains folders for plugins, web resources, and any workflows we have made as well as XML files defining the schema of any customization we have done. In the zipped format, our schema definition is contained in one massive file. Consider this zip file as a binary, or in layman’s terms, a tidy package with a fancy bow, i.e. It may look nice but it’s not easy to see what’s inside and it’s a poor format for version control.

Solution packager is a tool that essentially takes our Dynamics 365 CE solution zip file and breaks it out into a logical folder structure by decomposing the contents. The resulting output shows a more granular view of our solution and is considerably more friendly for version control as you can see from the example screenshots below.

Dynamics 365 Instance Management

Below are the little regarding how we have managed the dynamics 365 instances for achieving this DevOps.

1. Dev environment: It is the true source for all the customization and configuration changes.

2. SIT: where you test all the functionality against different types of data by internal test users.

3. UAT: where you test all the functionality against different types of data by business users.

4. Production: Go Live Instance.

Dynamics CRM CICD process

The below diagram illustrates the basic flow of Dynamics 365 CE Workflow process.

In this part, we are going to see the highlighted one in detail.

The process followed by the developer is essentially these steps (pictured below), the two steps in the middle are handled by the SolutionExportDev.cmd script:

It will export the solution from source instance and unpack the solutions into the folders in BuildAutomation.Solutions.

1. Developers benefit using this process by making the solution deployment repeatable and predictable.

2. Developers can associate their work items with their check-ins

3. Very useful for tracking changes (traceability) and comparing solution files.

4. SolutionExport.ps1 – This will connect to the Dynamics 365 CE instance and export the Unmanaged and Managed solutions from that environment.

5. SolutionUnpack.cmd – This will extract the Unmanaged solution into the BuildAutomation.Solutions /BuildAutomation folder.  The Core Tools folder contains the Solution Packager tool from the Dynamics Core Tools NuGet package (pictured below)

Our solution consists of a CRM Deployment Packager project(BuildAutomation.SolutionPackager) and Solutions(BuildAutomation.Solutions) which is used as a container for the extracted files and folders of the solution.

We added a Scripts(SolutionUnpack.cmd) folder to the project (BuildAutomation.SolutionPackager) which contains some below PowerShell scripts.

Powershell script for exporting the solution from the source environment.

$solutionName ="BuildAutomation"
$SolutionFilePath="C:\Users\DBALASU\Source\Workspaces\BuildAutomation\BuildAutomation.Solutions\BuildAutomation"
$CRMSourceUserName=""
$CRMSourcePassword=""
$CRMDestinationUserName=""
$CRMDestinationPassword=""
$CRMSourceUrl="https://instancename.crm.dynamics.com"
$CRMDestinationUrl="https://instancename.crm5.dynamics.com"

Set-StrictMode -Version latest
function InstallRequiredModule{
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass -Force
$moduleName = “Microsoft.Xrm.Data.Powershell”
$moduleVersion = “2.7.2”
if (!(Get-Module -ListAvailable -Name $moduleName )) {
Write-host “Module Not found, installing now”
$moduleVersion
Install-Module -Name $moduleName -MinimumVersion $moduleVersion -Force
}
else
{
Write-host “Module Found”
}
}
function EstablishCRMConnection{
param(
[string]$CRMUserName,
[string]$CRMSecPasswd,
[string]$crmUrl)
Write-Host "UserId: $CRMUserName Password: $CRMSecPasswd CrmUrl: $crmUrl"
$CRMSecPasswdString = ConvertTo-SecureString -String $CRMSecPasswd -AsPlainText -Force
write-host "Creating credentials"
$Credentials = New-Object System.Management.Automation.PSCredential ($CRMUserName, $CRMSecPasswdString)
write-host "Credentials object created"
write-host "Establishing crm connection next"
$crm = Connect-CrmOnline -Credential $Credentials -ServerUrl $CrmUrl
write-host "Crm connection established"
return $crm
}

InstallRequiredModule

#Update Source CRM instance details below:
Write-Host "going to create source connection"
$CrmSourceConnectionString = EstablishCRMConnection -user "$CRMSourceUserName" -secpasswd "$CRMSourcePassword" -crmUrl "$CRMSourceUrl"
Write-Host "source connection created"
Set-CrmConnectionTimeout -conn $CrmSourceConnectionString -TimeoutInSeconds 1000

Write-Host "going to create destination connection"
$CrmSourceDestinationString = EstablishCRMConnection -user "$CRMDestinationUserName" -secpasswd "$CRMDestinationPassword" -crmUrl "$CRMDestinationUrl"
Write-Host "destination connection created"
Set-CrmConnectionTimeout -conn $CrmSourceDestinationString -TimeoutInSeconds 1000

Write-Host "Publishing Customizations in source environment"
Publish-CrmAllCustomization -conn $CrmSourceConnectionString
Write-Host "Publishing Completed in source environment."

Write-Host "Exporting Solution"
Export-CrmSolution -conn $CrmSourceConnectionString -SolutionName "$solutionName" -SolutionFilePath "$SolutionFilePath" -SolutionZipFileName "$solutionName.zip" 
Write-host "Solution Exported." 

1. $SolutionName – Name of the solution which you want to export the source instance.

2. $SolutionFilePath – Desired folder path to export the solution

3. $CRMSourceUserName – Username of the Dynamics 365 CE source instance

4. $CRMDestinationUserName – Username of the Dynamics 365 CE destination instance

5. $CRMDestinationPassword – Password of the Dynamics 365 CE destination instance

6. $CRMDestinationPassword – Password of the Dynamics 365 CE source instance

7. $CRMSourceUrl – URL of the Dynamics 365 CE source instance

8. $CRMDestinationUrl – URL of the Dynamics 365 CE destination instance.

After updating the values in the above script, you can copy and paste in Windows PowerShell and run it. This will establish the connection to the Dynamics 365 CE source instance and export the solution to the $SolutionFilePath(Desired folder path to export the solution). Once this is done then run the below command file to extract the solution file.

Command Script:

rem @echo off
set solution=%1
set folder=%2

if "%solution%"=="" set solution=BuildAutomation
if "%folder%"=="" set folder=%solution%
if "%auth%"=="" set auth=ifd
set managed=false
set packagetype=unmanaged
set projectdir=BuildAutomation.Solutions

tools\solutionpackager.exe /action:Extract /packagetype:%packagetype% /zipfile:..\%projectdir%\%folder%\%solution%.zip /folder:..\%projectdir%\%folder%\ /allowDelete:No
exit /b
:EXIT
@pause

 

Developers need to do below the configuration in visual studio(Run the command prompt from inside Visual Studio) to execute the SolutionUnPack.cmd

To make the tool available, add it to the external tools list. Here are the steps:

1. Open Visual Studio.

2. Select the Tools menu, and then choose External Tools.

1. On the External Tools dialog box, choose the Add button. A new entry appears.

1. Enter a Title for your new menu item such as Command Prompt.

2. In the Command field, specify the file you want to launch, such as SolutionUnPack.cmd

3. In the Arguments field, please leave it as blank.

4. Choose a value for the Initial directory field, such as Project Directory.

Example:

Title – Name of the external tools to run

Command – Provide the physical path to the command file

Initial directory – Provide the directory path of the SolutionUnPack.cmd file.

Use Output Window – Enable

Prompt for Arguments ­– Disable

Close on exit–Enable

1. Choose the OK button.

The new menu item is added, and you can access the command prompt from the Tools menu as follows:

To run this, please click on ExportCrmSolution. It will export the solution from source instance and unpack the solutions into the folders in BuildAutomation. Solutions.

Once the above step is done, the developer now has some manual steps to perform depending on the files/changes that were extracted from the solution:

1. Include any new file under the extracted folder Solutions\BuildAutomation (extracted) into the project- BuildAutomation. Solutions (so that they can be checked-in). NOTE: When doing the check-in, be sure to check the ‘Excluded Changes’ if additional files were detected and promote them to the ‘Included Changes’ as needed.

2. For any .xaml file (under Solutions\BuildAutomation\Workflows or Solutions\BuildAutomation \Entities\xxxx\Formulas), make sure that Build Action is ‘None’ (not ‘Page’ or ‘XamlAppDef’). If this step is skipped, you may get one of this compilation error (s): – Project file must include the .NET Framework assembly ‘WindowsBase, PresentationCore, PresentationFramework’ in the reference list. – Assembly “System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’ cannot be resolved”.

1. Ensure that DLL files under Solutions\..\PluginAssemblies are added to source control.

2. Undo all pending changes that do not specifically apply to your changes.

3. Associate the check-in with a work item(s), write a comment and check-in all required changes.

Note: Check-in to TFS will auto trigger the deployment process.

In my next blog, we will see how to pack the solution and deploy it in the Target instance using VSTS Build and Release definition.

If you are interested in this topic and would like to do some further self-study I encourage you to check out my blog on this.

#BuildAutomation #ContinuousDeployment #ContinuousIntegration #DevOps