Posts in Category: data

Keeping your Dynamics365 Audit Logs Tidy

Audits are a great way to see what has happened on records, what they did and more importantly who did it.

But if you have automated processes running that are pumping data into Dynamics because a field or two has changed, you might end up with an Audit History looking something like this.

In this case, an update was triggered, but it wasn’t until the fourth update that there was an actual difference in the data being changed.  Even then, when we sent the whole packet of data, we sent it all.

With only four feels you can already see this gets a little painful to follow.

Capture.PNGRead More


Accessing Data with LiteDb

In my last post I talked about how easy it was to create a database with LiteDb.

But accessing your data is even easier.

To start off, assuming we have no document (i.e., structure) associated with our data – we can create a generic table on the fly using the built in BSonDocument structure.

In this example, I first use my database to get a collection of documents under “FirstTable”.  If it’s your first time creating this document, don’t worry, it’ll be created for you, so no need to do any “CREATE” mumbo jumbo.

var FirstTable = database.GetCollection("FirstTable");

Now I’m going to create some data.  The simplest, most brain-dead way to do this is to create a Dictionary, make some columns and give them whatever values I want that are of type BsonValue

Dictionary<string, BsonValue> docs															

Read More


Getting Started with LiteDB

Every project you do will have data and eventually you’re going to need to store that data somewhere.

And config files are such a pain to manage (and also so early 2000s) so as much as possible I try to use a database or some unit of storage.

And like anything you build, once you start building it, it takes on a life of it’s own that you twist and turn and wrench into something useful until one day you look at it and go – maybe storing our configuration in a 25,000 line XML file wasn’t the best idea?

So if it’s going to be a bad idea down the road, why not start doing the right thing from the get go – enter LiteDb.

LiteDb is an incredibly easy to use,

Read More


Patterns on Bulk Data Migration

No code in this most, well maybe a bit.

I’ve been working on a number of data migrations over the past few years using a variety of tools, from off the shelf solutions to home grown, I’ve run everything in between.

All of them have their hits and misses and this isn’t to harp on one over the other as I continue to refine my approach in getting data into Dynamics in the most efficient way possible.

My current investigation into how to optimize this process has lead me to consider what the most important pieces for a solution to work are;

  1. Make it As Fast As Possible.
  2. Being able to handle Concurrent Connections (i.e., multi-thread and parallelize it until the cows come home).
  3. Accessible in the environment that the user operates in (read: if you’re

    Read More


File Source Encoding

I’ve been doing some migration from CSV files over the last few months that resulted in some interesting exercises in the cleansing of data before it gets to Dynamics.

For all of this work, the code was written using C# and the Dynamics SDK.

Encoding

I ran into a number of encoding problems in transferring data from the files into Dynamics. Initially, I thought Dynamics could not handle the character set but on closer inspection, it was the code that wasn’t loading it properly.  I tried a number of different Encoding formats but the one that addressed all scenarios as using the Encoding.Default and let the system decide which to use and when.

Therefore, when loading the data, my initial delimited looked as follows;
var data = File.ReadAllLines(file, Encoding.Default).Select(a => a.Split('\n').ToList());

Delimiters

Since all the data

Read More