Franky's Notes: microsoftazure

Showing posts with label microsoftazure. Show all posts

Tons of statistics and metrics for Microsoft Azure websites (not only Asp.Net)

Labels: ApplicationInsights, aspnet, azure, metrics, microsoftazure, nodejs, post, stastistics, website

Your website is finally online. That great, good job. But now, you have a lot of questions: How much visits do I have? Which part of the site is mostly visited? Does the site performs well? In this post, I will show you some tools that exist in Microsoft Azure that will help you to get some answers.

Request and Error

When you create a website in Azure, you automatically got some monitoring. Go on portal.azure.com (the preview portal), and select your website.

This will be useful to see how many requests and errors you got. You could create some alerts, by clicking the "+" sign, and you would be notified by e-mail if the number of request is greater than x, over the last hour.

Analytics

The first time you will click on this section all the instruction will be given to you.

To collect end-user usage analytics about your application, insert the following script into each page you want to track. Place this code immediately before the closing </headtag, and before any other scripts. Your first data will appear automatically in just a few seconds.

To get analytics for the whole web site, in an Asp.Net MVC site, a good place will be: \Views\Shared\_Layout.cshtml.

In a Ghost blog, if you are using the default theme, the file will be /content/themes/casper/default.hbs. Otherwise, just replace "casper" by your theme name in the path.

Once your website is re-deployed, re-open the website blade and click again on the Analytics graph. And you will be able to see a lot of information: Session per browser, information on page's views, slowest pages, details on sessions, etc.

When you click on a graph, the Metrics Explorer blade will be visible. On the top of this blade, you will have many different options to customize your results. You will be able to add a chart, change the time range, add some filter and even set some alerts.

Moreover, if you click on those Charts, tables or even on a row, you will have more details and options to fine-tune the result.

Application Insights

You thought it was enough? Well, Microsoft Azure still has one more tool for you, that will cover in this post: Application Insights. With this one, you will be able to see the health of your application by adding some tests, custom events, logs, errors, etc.
Adding Application Insights to your Asp.Net website, can easily be done via Visual Studio, like I explained in a previous post.

Since Azure is compatible with many different languages, chances are that you are using one of those. Let say you are running a node.js Ghost blog, how could you add Application Insights? By using website extensions. To add an extension you could use the Kudu interface. This interface is easy to access. In a browser, type the URL of the website but inject "scm" between the name of the application and the azurewebsites.net. Something like http://mybookmanager.scm.azurewebsites.net/

Once you are in the Kudu interface, click on the tab Site Extensions, then section Gallery. Add the Application Insights Extension.
It's also possible to add a website extension using the website blade.

This will gives you a lot of information already, but to add more customs metric in .Net by sure to add the Application Insights SDK to your porject. In node.js use the applicationinsights package from npm,

~Frank Boucher

References

Upgrade an Application Windows Azure OS Family

Labels: aspnet, azure, Azure Storage, Membership, microsoftazure, mvc, Nuget, OSFamily, post, sdk, sql, storage, Upgrade, visualstudio, windows

Recently I add to upgrade an web site running in Azure Webrole from Azure OS famille 1.6 to a more recent version. While the migration was not complicated I encounter some little particularity that I found could be interesting to share.

The Context

The website was a Visual Studio 2010 project using Azure SDK 1.6 and a library call AspNetProvider that was part of Microsoft's sample few years ago to manage session and membership. Using the AspNetProvider library the session was saved in Azure blob storage, and the membership was saved in an SQL database.

The Goal

The application must stay a Visual Studio 2010 project, but using the most-recent Azure SDK and Azure Storage Client as possible.

The Solution

Azure SDK 2.1
Azure.StorageClient 4.0
Universal Provider version 2.1
OS famille 4

The Journey

Migration from SDK 1.6 to SDK 2.1

Azure SDK version 2.1 is the higher version compatible with Visual Studio 2010. And can be downloaded from Microsoft's website. Once it is installed, just open the project in Visual Studio and right-click on the Azure Project. By clicking on the upgrade button the magic will happen. Some errors could stay but the hard work will be done for you.

Migration from AspNetProvider to UniversalProvider

we need to remove all reference to the AspNetProvider library. Just expand the resources node in the Solution Explorer and delete the reference. One thing important is that since we are using Visual Studio 2010 the latest version of the UniversalProvider we can use is 1.2. More recent version are using .Net 4.5 and this is not compatible with the present solution. To get the reference added to the project just execute the following Nugget command:

Install-Package UniversalProvider -version 1.2

Check the web.config file to clean the membership connections.

Migration of the Azure Storage Client

This one is the easiest, just remove the reference in the reference node and then execute the following Nugget Command:

Install-Package Azure.Storage.Client

Migration of the membership data

The AspNetProvider was using prefixed SQL tables: aspnet_user, aspnet_membership, etc. The new membership manager is using another sets of tables. We must migrate the data from one set to the other one. Here a SQL script that will to exactly that. The script can be run multiple times because it will only copie the unmoved data.

-- ========================================================
-- Description:    Migrate data from asp_* tables 
--                 to the new table used by Universal provider
-- ========================================================

DECLARE @CNT_NewTable AS INT
DECLARE @CNT_OldTable AS INT

-- --------------------------------------------------------
-- Applications -------------------------------------------

INSERT INTO dbo.Applications (ApplicationName, ApplicationId, Description)
    SELECT    n.ApplicationName, n.ApplicationId, n.Description 
    FROM    dbo.aspnet_Applications o 
    LEFT    JOIN dbo.Applications n ON o.ApplicationId = n.ApplicationId
    WHERE    n.ApplicationId IS NULL

SELECT @CNT_NewTable = Count(1) from dbo.Applications 
SELECT @CNT_OldTable = Count(1) from aspnet_Applications

PRINT 'Application Count: ' + CAST(@CNT_NewTable AS VARCHAR) + ' = ' + CAST(@CNT_OldTable AS VARCHAR)

-- -------------------------------------------------------- 
-- Roles --------------------------------------------------

INSERT INTO dbo.Roles (ApplicationId, RoleId, RoleName, Description)
SELECT    o.ApplicationId, o.RoleId, o.RoleName, o.Description 
FROM    dbo.aspnet_Roles o
LEFT JOIN dbo.Roles n ON o.RoleId = n.RoleId
WHERE n.RoleId IS NULL

SELECT @CNT_NewTable = Count(1) from dbo.Roles 
SELECT @CNT_OldTable = Count(1) from aspnet_Roles

PRINT 'Roles Count : ' + CAST(@CNT_NewTable AS VARCHAR) + ' = ' + CAST(@CNT_OldTable AS VARCHAR)

-- --------------------------------------------------------
-- Users --------------------------------------------------

INSERT INTO dbo.Users (ApplicationId, UserId, UserName, IsAnonymous, LastActivityDate)
SELECT o.ApplicationId, o.UserId, o.UserName, o.IsAnonymous, o.LastActivityDate 
FROM dbo.aspnet_Users o LEFT JOIN dbo.Users n ON o.UserId = n.UserID 
WHERE n.UserID IS NULL

SELECT @CNT_NewTable = Count(1) from dbo.Users 
SELECT @CNT_OldTable = Count(1) from aspnet_Users

PRINT 'Users count: ' + CAST(@CNT_NewTable AS VARCHAR) + ' >= ' + CAST(@CNT_OldTable AS VARCHAR)

-- --------------------------------------------------------
-- Memberships --------------------------------------------

INSERT INTO dbo.Memberships (ApplicationId, UserId, Password, 
PasswordFormat, PasswordSalt, Email, PasswordQuestion, PasswordAnswer, 
IsApproved, IsLockedOut, CreateDate, LastLoginDate, LastPasswordChangedDate, 
LastLockoutDate, FailedPasswordAttemptCount, 
FailedPasswordAttemptWindowStart, FailedPasswordAnswerAttemptCount, 
FailedPasswordAnswerAttemptWindowsStart, Comment) 

SELECT o.ApplicationId, o.UserId, o.Password, 
o.PasswordFormat, o.PasswordSalt, o.Email, o.PasswordQuestion, o.PasswordAnswer, 
o.IsApproved, o.IsLockedOut, o.CreateDate, o.LastLoginDate, o.LastPasswordChangedDate, 
o.LastLockoutDate, o.FailedPasswordAttemptCount, 
o.FailedPasswordAttemptWindowStart, o.FailedPasswordAnswerAttemptCount, 
o.FailedPasswordAnswerAttemptWindowStart, o.Comment 
FROM dbo.aspnet_Membership o
LEFT JOIN Memberships n ON  o.ApplicationId = n.ApplicationId
                      AND o.UserId = n.UserId
WHERE n.UserId IS NULL AND n.ApplicationId IS NULL


SELECT @CNT_NewTable = Count(1) from dbo.Memberships 
SELECT @CNT_OldTable = Count(1) from aspnet_Membership

PRINT 'Memberships count: ' + CAST(@CNT_NewTable AS VARCHAR) + ' >= ' + CAST(@CNT_OldTable AS VARCHAR)

-- -------------------------------------------------------
-- UsersInRoles ------------------------------------------
TRUNCATE TABLE dbo.UsersInRoles
INSERT INTO dbo.UsersInRoles SELECT * FROM dbo.aspnet_UsersInRoles


SELECT @CNT_NewTable = Count(1) from dbo.UsersInRoles 
SELECT @CNT_OldTable = Count(1) from aspnet_UsersInRoles

PRINT 'UsersInRoles count: ' + CAST(@CNT_NewTable AS VARCHAR) + ' >= ' + CAST(@CNT_OldTable AS VARCHAR)

Migration from OSFamilly 1 to 4

Open the file .cscfg and edit the OS Family attribute. It's in the ServiceConfiguration node.

<ServiceConfiguration servicename="MyApp" osFamily="4" osVersion="*" ...>

Wrapping up

The only step left is to deploy in the staging environment to see if everything is working as expected. would recommend also to plan to upgrade as soon as possible because the Azure SDK 2.1 official retirement date is November 2015. I hope this post could help you, even if you are migrating from and to a different version. Any comments, suggestions and/or questions are welcome.

~ Frank Boucher

How to be sure that your PowerShell script are doing what you think?

Labels: azure, bdd, microsoftazure, pester, post, PowerShell, unittest

Recently, I was required to write PowerShell script to automate some Microsoft Azure deployments. I was at first really happy to share those scripts because I knew other developers would use it. They would add functionalities and the library would grow. The idea was nice, but how to be sure that while adding some functionalities they don’t break something else? I didn’t know any unit test framework for Powershell, so I decided to do a quick search online. I was sure some kind of framework already exists. What I didn’t expect, however, was to find a framework that was really easy to use, complete and free! In this post, I will introduce you to Pester, a wonderful PowerShell framework available on github.

What is Pester?

Pester is a Business Development Driven (BDD) unit tests framework that implements a lot of functionalities like mock and exceptions management.

How to Install

The framework is available on github.com but it can also be downloaded from different repositories. My personal favorite is to download it from Chocolatey because it’s the most painless method. The Chocolatey’s command to install Pester is:

cinst pester

Before continuing, if you try a Pester command right now, you will probably have the error message: “The term ‘___’ is not recognized as the name of a cmdlet…”. One way to fix that is to add Pester to your Profile. This can be done by executing the following command.

{Import-Module "C:\Chocolatey\lib\pester.2.0.4\tools\Pester.psm1"} | New-Item -path $profile -type file -force;. $profile

Get Started

Now that Pester is available on your machine, let’s start by creating our first test. Pester can scaffold for you an empty script file and a test file. If you add Pester to an already existing library, don’t be scared, Pester will not override the existing files.

Open a PowerShell console and type the following command:

New-Fixture c:\dev\pesterDemo pesterDemocd c:\dev\pesterDemo

You can try the new test by invoking Pester with this command:

Invoke-Pester

As you can see a test already exists, and it failed.

Simple test

Let’s create a reel test that will read an XML file to extract a property. Here is the content of the XML file and the code for both the script and the test file.

manifest.xml

<service>
    <name>Employee Provider</name>
    <version>1.2</version>
</service>

pesterDemo.Tests.ps1

$here = Split-Path -Parent $MyInvocation.MyCommand.Path
$sut = (Split-Path -Leaf $MyInvocation.MyCommand.Path).Replace(".Tests.", ".")
. "$here\$sut"

Describe "#pesterDemo#" {

    Context "Test with reel XML document."{

        It "Loads a reel manifest." {
            $xml = getManifest ".\manifest.xml"
                $xml.OuterXMl | Should Be "<service><name>Employee Provider</name><version>1.2</version></service>"
        }

        It "Return the service name."{
            $sName = GetServiceName ".\manifest.xml"
                $sName | Should Be "Employee Provider"
        }

    }
}

pesterDemo.ps1

function GetManifest($manifestPath){
        if(Test-Path $manifestPath)
        {
            [xml]$xml = Get-Content $manifestPath
            return $xml
    }
    else{
        Throw "Error. ÃƒÂnvalid Manifest file path."
    }
}

function GetServiceName($manifestPath){
        $manifest = GetManifest($manifestPath)
        return $manifest.service.name
}

Result of the test

The script file contains two simple functions. The first one [GetManifest] versifies if the file exists and returns his content. Otherwise, an exception is thrown. The second function [GetServiceName] retreives the content of the XML file, and returns the value of the property [name] of the node [service].

The test file contains two tests. The first one Loads a reel manifest test that the manifest.xml file is correctly loaded and validates the content. The second test Return the service name, validate that the name of the service is the same as expected. We can now invoke the tests.

Invoke-Pester

As you can see, all tests succeeded and the output is really clear. Now what if the path passed in parameter doesn’t exist? Let assume that in our design, we wanted that the code to throw an exception. A good thing for us, Pester already has everything to manage exceptions.

Let’s do a test to validate this “requirement.” First, define a code block that calls [GetManifest] with a wrong path. Then pipe the result in the Pester command Should. If you add the filling code to the pesterDemo.tests.ps1, and invoke-Pester all tests should succeed.

It "Throw an exception when loads a manifest with invalid path." {
{$xml = GetManifest ".\WrongPath\manifest.xml" }  |  Should Throw
}

Test with Mock

Sometimes, we don’t want our tests to interact with reel components. That’s why mocks are so useful. Thankfully for us, creating a mock with Pester is a piece of cake. In all the previous tests, a real XML file was used. To make some validation on the content of the file, I could have as many different files as different scenarios, or simpler I could use some mock.

With Pester, we must set up a context where our mock will be present. Here is a test where I mock the XML file. As expected, any path will work since the file is not reel.

Context "Test with Mocks."{

    Mock GetManifest{return [xml]"<service><name>Employee Provider</name><version>1.2</version></service>"}

    It "Any path will works"{
        $xml =  GetManifest -MockWith ".\wrongPath\manifest.xml"
            $xml.OuterXMl | Should Be "<service><name>Employee Provider</name><version>1.2</version></service>"
    }
}

Wrapping up

Of course, this little introduction is not an exhaustive list of all the features of Pester. I hope that with these little code snippets, I gave you the drive you needed to test your PowerShell scripts. So next time you write this amazing script to deploy your app on Microsoft Azure, think of Pester.

References

[BDD]: BDD http://en.wikipedia.org/wiki/Behavior-driven_development
Pester: https://github.com/pester/Pester
Chocolatey: https://chocolatey.org/

~Frank

Pages

Tons of statistics and metrics for Microsoft Azure websites (not only Asp.Net)

Request and Error

Analytics

Application Insights

References

Upgrade an Application Windows Azure OS Family

The Context

The Goal

The Solution

The Journey

Migration from SDK 1.6 to SDK 2.1

Migration from AspNetProvider to UniversalProvider

Migration of the Azure Storage Client

Migration of the membership data

Migration from OSFamilly 1 to 4

Wrapping up

How to be sure that your PowerShell script are doing what you think?

What is Pester?

How to Install

Get Started

Simple test

Result of the test

Test with Mock

Wrapping up

References