Reading Notes #306

MVIMG_20171126_090346Suggestion of the week


Cloud


Programming


Databases



Reading Notes #305

AzureDatabricks

Cloud


Programming


Miscellaneous


Reading Notes #304

IMG_20171108_160315

Cloud


Programming


Databases


Podcast


Miscellaneous



Reading Notes #303

logo-glyphSuggestion of the week

  • Writing tests in Postman (joyce) - With all the connected things and all the API in our system, this post shows a brilliant and simple way to test all those external calls.

Cloud


Programming


Data


Miscellaneous




Reading Notes #302

Autumn

Cloud


Programming


Data


Miscellaneous




Reading Notes #301

300love

Programming


Data

Miscellaneous


  • My First Year as an MVP, part 1 (Jen Kuntz) - Interesting post. It feels soooooo familiar, and yet so far now. I look forward to meeting you in March fellow Canadian MVP.


Reading Notes #300

300loveCloud


Programming


Data


Miscellaneous



Lessons Learned with Power Bi and Dynamic DAX Expressions

Since its availability, I try to use Power Bi as often as I can. It is so convenient, to build visual to explain data coming from a ton of possible data source. And it's a breeze to share it with clients, or colleagues. However, this post is not an info-commercial about Power Bi, it's about sharing some challenges I got trying to prepare a report and how I fix it.

The Data Source


The context is simple, all transactions are in one table, and I have a second table with a little information related to clients. To that, I personally like to add a calendar table, because it simplifies my life.

Datamodel

For this report, it is very important to but a slicer by Client.

The Goal


I needed to have one report that shows for every customer three different Year To Date (YTD) total. The classic YTO, a YTD but when the beginning of the years is, in fact, when the client started is enrolment, and the last one was a rolling twelve.
It looks pretty simple, and in fact, it's not that complicated. Let's examine each total formula one by one.

Classic Year To Date Total


Before we get started, it's a good practice to reuse Mesure to simplify our formula, and to explicit expression. Let's create a Measure for the Total Sales, that will be used inside our other formulas.

TotalSales = SUM('Sales'[Total])

Now the Year To Date, is simple to add by adding a New Measure and entering the formula:

YDTClassic = TOTALYDT([TotalSales], 'Calendar'[Date])

If you activate the Preview feature of Power Bi, it could be even easier. Look for the button New Quick Measure and select the Total Year To Date, fill up the form and voila!

QuickMeasure

The generated formula looks a bit different because Power Bi managed the error in h expression.

TotalYTD = 
IF(
    ISFILTERED('Calendar'[Date]),
    ERROR("Time intelligence quick measures can only be grouped or filtered by the Power BI-provided date hierarchy."),
    TOTALYDT([TotalSales], 'Calendar'[Date].[Date])
)

Anniversary Year To Date Total


I spent more time then I was expecting on that one. Because in the Online DAX documentation it is said that the formula TOTALYDT accept a third parameter to specify the end of the year. So if I had only one client, with a fix anniversary date (or a fiscal year) this formula will work assuming, the special date is April 30th.
TOTALYDT([TotalSales], 'Calendar'[Date], "04-30")
However, in my case, the ending date changes at with every client. I'm sure right now you are thinking that's easy Frank just set a variable and that it! Well, it won't work. The thing is the formula is expecting a static literal, no variable aloud even if it returns a string.
The workaround looks at first a bit hard, but it's not that complex. We need to write our own YTD formula. Let's look at the code, and I will explain it after.

Anniversary YTD = 
VAR enddingDate =   LASTDATE(Company[EnrolmentDate])
VAR enddingMonth =  MONTH ( enddingDate )
VAR enddingDay =    DAY ( enddingDate )
VAR currentDate =   MAX ( Calendar[Date] )
VAR currentYear =   YEAR ( currentDate )
VAR enddingThisYear =   DATE ( currentYear, enddingMonth, enddingDay )
VAR enddingLastYear =   DATE ( currentYear - 1, enddingMonth, enddingDay )
VAR enddingSelected =   IF ( enddingThisYear < currentDate, enddingThisYear, enddingLastYear )
RETURN
    CALCULATE (
        [TotalSales] ,    
        DATESBETWEEN(Calendar[Date],enddingSelected,currentDate)    
    )
First lines are all variable's declaration. They are not required, but I found it easier to understand when things are very explicit. Since I'm slicing my report by companies putting the LASTDATE is just a way not avoid errors. It should have only one record. Then we extract year, month, and day.
The last variable enddingSelected identify if the anniversary (the end date) is pasted or not in the curent calendar year.
The calculate function is returning the TotalSales between the last anniversary date and today.

Rolling twelve Total


For the last formula, the rolling twelve we will re-use the previous code, but in a simpler way since the end date is always yesterday.

Rolling 12 Total = 
VAR todayDate = TODAY()
VAR todayMonth =    MONTH ( todayDate )
VAR todayDay =  DAY ( todayDate ) 
VAR todayYear = YEAR ( todayDate ) 
VAR enddingLastYear =   DATE ( todayYear - 1, todayMonth, TodayDayVar +1) 
RETURN
    CALCULATE (
        [TotalSales] ,
        DATESBETWEEN( Calendar[Date], enddingLastYear, todayDate)   
    )

Wrap it up


I definitely learned a few things with that Power Bi session, but it turns out to be pretty easy. Again, leave a comment or send me an email if you have any comments or questions I will be very happy to ear from you.


References


Reading Notes #299

azure-1Cloud


Programming


Databases


Miscellaneous



How to know when an Azure Function is running in a Staging slot

I love Azure functions; I think they are very useful in many scenarios. I use them very often when I want to extend some functionality of on existing systems, to avoid having to open the Pandora box (aka. code). Recently, I was involved on a project where we used the Azure Functions as a schedule task to process (and generate) a lot of data into a database. We used VSTS to deploy the solution in a staging slot, and everything was really great. If you are interested to learn more about it, see my previous post. In fact, it was working so well that the Azure Function was running and doing it's job even while in staging slots...

In this quick post, I will share two options to prevent an Azure Function of running while in Stagging slot.

Option 1: Kudu to the rescue


If you have created a few functions, you probably already know that you have access to some Environment variables. They are very useful. I was pretty sure one exists specifying the current slot, but I didn't know the name of it. Even more, I forgot that all environment variables are displayed in the Kudu interface. D'oh! Thanks to Bruce Chen, that answered my question on StackoverFlow and help me to remember it was all there.

To get to your environment variables list, go in portal.azure.com. Open your Azure function and select the main note (1). The from the right section select the Platform features tab. Then finaly, in the Development Tools section select the Advanced tools (Kudu).

WhereisKudu

That will open the Kudu interface in a new tab. You just need to select the Environment tab and you will see them!

EnvironmentVariables

Now one more thing before we are reading to go. At the time, this post is published, Function slots are still in preview so don't forget to activate it ;)

NeedActivatePreview

Let's create a simple Azure Function to show how it works.

using System;

public static void Run(TimerInfo myTimer, TraceWriter log)
{
    log.Info($"JustDemoFunc got triggered at: {DateTime.Now}");

    var slotName = System.Environment.GetEnvironmentVariable("APPSETTING_WEBSITE_SLOT_NAME", EnvironmentVariableTarget.Process);
    if (!string.Equals("production", slotName , StringComparison.OrdinalIgnoreCase))
    {
        log.Info($"{DateTime.Now:s} Function is in stagging.");
        return;
    }

    log.Info($"{DateTime.Now:s} Function is in Production is will be executed.");
}

This is a C# timer function. Every time it will be executed (every 5 minutes) it will write to log that it got triggered. Than with System.Environment.GetEnvironmentVariable("APPSETTING_WEBSITE_SLOT_NAME", EnvironmentVariableTarget.Process) it's grabbing our slot name. By default the name is Production so if it's something else... we get out. See the log when in production slot.

JustDemoFunc got triggered at: 9/24/2017 3:28:16 PM
2017-09-24T15:28:16 Function is in Production is will be executed.
Function completed (Success, Id=###, Duration=21ms)
And now the same code, but running in another slot, in this case named Stagging.
JustDemoFunc got triggered at: 9/24/2017 3:25:29 PM
2017-09-24T15:25:29 Function is in stagging.
Function completed (Success, Id=###, Duration=161ms)

Option 2: Sticky Setting it is


Another option is also possible using Application Settings. For that, simply add a new Setting, in this case named: ShouldItRun. Now let's jump into the code.

using System;
using System.Configuration;

public static void Run(TimerInfo myTimer, TraceWriter log)
{
    log.Info($"JustDemoFunc got triggered at: {DateTime.Now}");

    if (!Convert.ToBoolean(ConfigurationManager.AppSettings["ShouldItRun"]))
    {
        log.Info($"{DateTime.Now:s} Function is in stagging.");
        return;
    }

    log.Info($"{DateTime.Now:s} Function is in Production is will be executed.");
}

To be able to read your application setting you will need using System.Configuration. Then a quick call to ConfigurationManager.AppSettings["ShouldItRun"] will return the value of your setting. Once again, see the log from the production slot.

JustDemoFunc got triggered at: 9/24/2017 4:03:43 PM
2017-09-24T16:03:43 Function is in Production is will be executed.
Function completed (Success, Id=###, Duration=30ms)

And in the staging slot.

JustDemoFunc got triggered at: 9/24/2017 4:02:21 PM
2017-09-24T16:02:21 Function is in stagging.
Function completed (Success, Id=###, Duration=8ms)

I hope you enjoy this little quick hack.


Watch the video





Reading Notes #298

IMG_20170919_161146Cloud


Programming


Miscellaneous

  • #222: Patrick Lencioni—Getting Hiring Right (EntreLeadership Team) - It was my first episode of this podcast, but definitely not the last one. Very interesting speakers... nice books referenced... loved it.
  • Why Your Boss Makes You Punch a Time Clock (Suzanne Lucas Suzanne Lucas is a freelance writer who spent 10 years in corporate human resources, where she hired, fired, managed the numbers, and double-checked with t) - Most of us have to do timesheets... I'm sure that at one point, you asked yourself the reason about it. It's time read an answer, in this post.


Reading Notes #297

Jekyll_AppService

Suggestion of the week

  • How to uninstall Scrum (Erwin Verweij) - When you will read that post (because you must read it... Seriously), you will smile, giggle and maybe even laugh.


Cloud


Programming


Miscellaneous





Lessons learned when deploying multiple databases to Azure with VSTS

It's had been a while since I worked into Visual Studio Team Services (VSTS), and it was a real pleasure to get back in that area. For the solution I was working on, we need to keep the current database up and running while deploying a new version. For this purpose, we decided to append the release number to database name (ex: MyDatabase363). In our Build and Release processes, we needed to identify which databases are from the last release. In this post, I will show what I did using an inline PowerShell script to get that number and set it as an environment variable so it can be accessible by other tasks.

To get started let's add a Azure PowerShell task to our build definition. In this post, I use a build process but of course this is also valid for release process. To find the task quickly, use the search text box. I will add two of those, one to get the number, the second to validate that this value is now set as an environment variable and readable from other tasks.
AddPowerShellTask

Now it's time to set the first task. Fill-out all the properties and select Inline Script as the Script Type. It should look like this.

InlineScript

Let's examine the code.
Get last Release Number
$matchingResources = Find-AzureRmResource -ResourceNameContains "mydatabase" -ResourceType "Microsoft.Sql/servers/databases"

$lastRelease = 0

ForEach($resource in $matchingResources)
{
    if ($resource.ResourceName -match '(\d)+$') {
        if($lastRelease -lt $matches[0]){
            $lastRelease = $matches[0]
        }
    }
}
Write-Output "The last release number is:  $lastRelease"
Write-Output ("##vso[task.setvariable variable=lastReleaseNumber;]$lastRelease")
On the first line, I use the Azure PowerShell commandlet Find-AzureRmResource1 to get an array of all the databases currently online in my resource group that contains a specific string. In this case, it's the name of the database without the release number. Then I will loop through all returned resources and using a very simple Regex to extract the release number and keep the biggest one (the last release).

To close that script we have two outputs. The first one is to give feedback in the logs, because it's always good to have some information there. The second one look more complicated, but if you split it, it's easier to see what's happening. In fact, we are producing a VSTS (previously called Visual Studio Online this is why it's VSO) command to initialize a variable ##vso[task.setvariable variable=lastReleaseNumber;] And of course, assign to it our last release number $lastRelease

To validate that we really successfully found our last release number and assigned it to a variable, let's try to read it back but from another step. That will be easily done this code in the other step created:
Validate the last Release Number
$number = $env:lastReleaseNumber

Write-Output "Confirmation, the last Release Number is:  $number "
The only thing missing before we can run our test is to create that environment variable. To to it simply go in the Variables tab and add it there.

env-variable

It's all set, run the build and you should see something similar in your logs.

trace



References




Reading Notes #296

IMG_20170910_134750

Cloud


Databases


Miscellaneous




Reading Notes #295

Sketch002

Cloud


Programming


Databases




Reading Notes #294

MagPi60-Cover

Suggestion of the week


Cloud


Programming


Miscellaneous



Reading Notes #293

IMG_20170813_103816-EFFECTSCloud


Programming




Reading Notes #292

cloudheight


Suggestion of the week


Cloud


Programming


Databases


Miscellaneous



Reading Notes #291

Blue-container

Cloud


Programming


Miscellaneous




Reading Notes #290

SEcurityCenterCloud


Programming


Databases


Miscellaneous



Reading Notes #289

IMG_20170710_092837

Cloud


Programming


Databases

  • Migrating to Azure SQL Database (Gavin Payne) - Very interesting and complete post that regroups references and gives details about some of the alternatives when it's migration time.


Reading Notes #288

IMG_20170706_063020

Cloud




Programming




Reading Notes #287

July1_800

Cloud


Programming


Miscellaneous


Reading Notes #286

Docker-Hub-NETCloud


Programming



Reading Notes #285

IMG_20170614_194330Cloud

Programming

Suggestion of the week

Miscellaneous