Showing posts with label continuous integration. Show all posts
Showing posts with label continuous integration. Show all posts

Deploy to Azure Directly From the Repository with GitHub Actions


You hear about that new GitHub Actions. Or maybe you didn't but would like to add a continuous integration, continuous deployment (CI-CD) to your web application. In this post, I will show you how to add a CI-CD to deploy automatically to Azure using the GitHub Actions.

What are GitHub Actions


GitHub Actions are automated workflows to do things. One of these could be a CI-CD. Using a workflow you could decide to compile and execute some unit tests at every push or pull request (PR). Another workflow could be that you deploy that application.

In this article, I will deploy a .Net Core application in Azure. However, you can use any languages you would like and deploy anywhere you like... I just needed to pick one :)

Now, let's get started.

Step 1 - The Code.


We need some code in a GitHub repo. Create a GitHub repo, clone it locally. And your app in it. I created mine with dotnet new blazorserver -n cloud5minsdemo -o src. Then commit and push.

Step 2 - Define the workflow


We got the code, now it's time to define our workflow. I will be providing all the code snippets required for the scenario cover in this post, but there is tons of template ready to be used available directly from your GitHub repository! Let's have a look. From your repository click on the Action tab, and voila!


When I wrote this post, a lot of available templates assumed the Azure resources already existed and you and adding a CI-CD to the mixt to automated your deployment. It's great but in my case, I was building a brand new web site so those didn't fit my needs. This is why I created my own template. The workflow I created was inspired by Azure/webapps-deploy. And there a lot of information also available on Deploy to App Service using GitHub Actions.

Let's add our template to our solution. GitHub will look in the folder .github/workflows/ from the root of the repository. Then create a file with the extension .yml

Here the code for my dotnet.yml, as any YAML file the secret is in the indentation as it is whitespace sensitive:

on: [push,pull_request]

env:
  AZURE_WEBAPP_NAME: cloud5minsdemo   # set this to your application's name
  AZURE_GROUP_NAME: cloud5mins2

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:

    # checkout the repo
    - uses: actions/checkout@master
    
    - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version:  3.0.101


    # dotnet build and publish
    - name: Build with dotnet
      run: dotnet build ./src --configuration Release
    - name: dotnet publish 
      run: |
        dotnet publish ./src -c Release -o myapp 

    - uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - run: |
        az group create -n ${{ env.AZURE_GROUP_NAME }} -l eastus 
        az group deployment create -n ghaction -g ${{ env.AZURE_GROUP_NAME }} --template-file deployment/azuredepoy.json


    # deploy web app using Azure credentials
    - name: 'Azure webapp deploy'
      uses: azure/webapps-deploy@v1
      with:
        app-name: ${{ env.AZURE_WEBAPP_NAME }}
        package: './myapp' 

    # Azure logout 
    - name: logout
      run: |
        az logout


The Agent


There is a lot in there let's start by the first line. The on: is to define the trigger, in this case, the workflow will be trigger at every push or PR.

The env: is where you can declare variables. It's totally optional, but I think it will help then templates are more complex or simply to reuse them easily.

Then comes the jobs: definition. In this case, we will use the latest version of Ubuntu as our build agent. Of course, in a production environment, you should be more specify and select the OS that matches your needs. This job will have multiples steps defined in the, you guess it, steps: section/

We specify a branch to work with and set up our agent by:

uses: actions/setup-dotnet@v1
with:
  dotnet-version: 3.0.101

This is because I have a .Net Core project. For Node.js project, it would be

uses: actions/setup-node@v1
with:
  node-version: 10.x

And it would be a better idea to set the version as an environment variable to be able to change it quickly.

The next two instructions are really .Net Core focus as they will build and package the application into a folder myapp. Of course, in the "section" you could execute some unit test or any other validation that you may find useful.

The next section may be less obvious.

- uses: azure/login@v1
  with:
    creds: ${{ secrets.AZURE_CREDENTIALS }}

Access and Secrets


To have our GitHub Action to be able to create resources and deploy the code it needs to have access. The azure/login@v1 will let the Action login, using a Service Principal. In other words, we will create an authentication in the Azure Active Directory, with enough permission to do what we need.

Let's examine the following Azure CLI command:

`az ad sp create-for-rbac --name "c5m-Frankdemo" --role contributor --scopes /subscriptions/{subscription-id} --sdk-auth`

This will create a Service Principal named "c5m-Frankdemo" with the role "contributor" on the subscription specified. The role contributor can do mostly anything except granting permission.

Because no resources already existed the GitHub Action will require more permission. If you create the Resource Group outside of the CI-CD, you could limit the access only to this specific resource group. Using this command instead:

`az ad sp create-for-rbac --name "c5m-Frankdemo" --role contributor --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group} --sdk-auth`

The Azure CLI command will return a JSON. We will copy-paste this JSON into a GitHub secret. GitHub secrets encrypted secrets and allow you to store sensitive information, such as access tokens, in your repository. To access them go in the Settings of the repository and select Secrets from the left menu.


Click the Add a new secret button, and type AZURE_CREDENTIALS as the name. It could be anything, as long as you use that value in the YAML file describing the workflow. Put the JSON including the curly brackets in the Value textbox and click the save button.

Provisioning the Azure Resources


Now that the workflow has access we could execute some Azure CLI commands, but let's see what missing:

- run: |
    az group create -n ${{ env.AZURE_GROUP_NAME }} -l eastus 
    az group deployment create -n ghaction -g ${{ env.AZURE_GROUP_NAME }} --template-file deployment/azuredepoy.json --parameters myWebAppName=${{ env.AZURE_WEBAPP_NAME }}

The first command will create an Azure Resource Group, where all the resources will be created. The second one will deploy the website using an Azure Resource Manager (ARM) template. The --template-file deployment/azuredepoy.json tells us the template is a file named azuredeploy.json located in the folder deployment. Notice that the application name is passed to a parameter myWebAppName, using the environment variable.

An ARM template is simply a flat file that a lot like a JSON document. Use can use any text editor, I like doing mine with Visual Studio Code and two extensions: Azure Resource Manager Snippets, and Azure Resource Manager (ARM) Tools With those tools I can build ARM template very efficiently. For this template, we need a service plane and a web App. Here what the template looks like.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "myWebAppName": {
           "type": "string",
           "metadata": {
                "description": "WebAppName"
            }
        }
    },
    "variables": {},
    "resources": [
        {
            "name": "[parameters('myWebAppName')]",
            "type": "Microsoft.Web/sites",
            "apiVersion": "2016-08-01",
            "location": "[resourceGroup().location]",
            "tags": {
                "[concat('hidden-related:', resourceGroup().id, '/providers/Microsoft.Web/serverfarms/frankdemoplan')]": "Resource",
                "displayName": "[parameters('myWebAppName')]"
            },
            "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', 'frankdemoplan')]"
            ],
            "properties": {
                "name": "[parameters('myWebAppName')]",
                "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', 'frankdemoplan')]"
            }
        },
        {
            "name": "frankdemoplan",
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2018-02-01",
            "location": "[resourceGroup().location]",
            "sku": {
                "name": "F1",
                "capacity": 1
            },
            "tags": {
                "displayName": "frankdemoplan"
            },
            "properties": {
                "name": "frankdemoplan"
            }
        }
    ],
    "outputs": {},
    "functions": []
}

This template is simple, it only contains the two required resources: a service plan, and a web app. To learn more about the ARM Template you can read my other post or check out this excellent introduction in the documentation.

Once the template is created and saved in its folder.

The deployment


There are only two last steps to the YAML file: the deployment and logout. Let's have a quick look at the deployment.

# deploy web app using Azure credentials
- name: 'Azure webapp deploy'
  uses: azure/webapps-deploy@v1
  with:
    app-name: ${{ env.AZURE_WEBAPP_NAME }}
    package: './myapp' 


Now that we are sure the resources exist in Azure we can deploy the code. This will be done with azure/webapps-deploy@v1 that will take the package generated by dotnet into myapp. Since we are already authenticated there is no need to specify anything at this point.

Everything is ready for the deployment. You just need to commit and push (into master) and the GitHub Action will be triggered. You can follow the deployment by going into the Actions tab.



After a few minutes, the website should be available in Azure. This post only shows a very simple build and deployment, but you can do so many things with those GitHub Actions, like executing tasks or packaging a container... I would love to know how you use them. Leave a comment or reach out on social media.


If you prefer, I also did a video of this post:



~

How to Deploy your Azure Function Automatically with ARM template (4 different ways)

It's so nice to be able to add some serverless components in our solution to make them better in a snap. But how do we manage them? In this post, I will explain how to create an Azure resource manager (ARM) template to deploy any Azure Function and show how I used this structure to deploy an open-source project I've been working on these days.

Part 1 - The ARM template

An ARM template is a JSON file that describes our architecture. To deploy an Azure Function we need at least three recourses: a functionApp, a service plan, and a storage account.


The FunctionApp is, of course, our function. The service plan could be set as dynamic or describe the type of resource that will be used by your function. The storage account is where is our code.


In the previous image, you can see how those components interact more with each other. Inside the Function, we will have a list of properties. One of those properties will be the Runtime, for example, in the AZUnzipEverything demo, it will be dotnet. Another property will be the connection string to our storage account that is also part of our ARM template. Since that resource doesn't exist yet, we will need to use the dynamic code.

The Function node will contain a sub-resource of type storageAccount. This is where we will specify where is our code, so it cant be clone to Azure.

Building ARM for a Simple Function


Let's see a template for a simple Azure Function that doesn't require any dependency, and we will examine it after.

You can use any text editor to edit your ARM template. However, the bundle VSCode with the extensions Azure Resource Manager Tools and Azure Resource Manager Snippets is particularly efficient.
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {},
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.Storage/storageAccounts",
            "apiVersion": "2018-07-01",
            "name": "storageFunc",
            "location": "[resourceGroup().location]",
            "tags": {
                "displayName": "storageFunc"
            },
            "sku": {
                "name": "Standard_LRS"
            },
            "kind": "StorageV2"
        },
        {
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2018-02-01",
            "name": "servicePlan",
            "location": "[resourceGroup().location]",
            "sku": {
                "name": "Y1",
                "tier": "Dynamic"
            },
            "properties": {
                "name": "servicePlan",
                "computeMode": "Dynamic"
            },
            "tags": {
                "displayName": "servicePlan"
            }
        },
         {
              "apiVersion": "2015-08-01",
              "type": "Microsoft.Web/sites",
              "name": "functionApp",
              "location": "[resourceGroup().location]",
              "kind": "functionapp",
              "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', 'servicePlan')]",
                "[resourceId('Microsoft.Storage/storageAccounts', 'storageFunc')]"
              ],
              "properties": {
                "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', 'servicePlan')]",
                "siteConfig": {
                  "appSettings": [
                    {
                      "name": "AzureWebJobsDashboard",
                      "value": "[concat('DefaultEndpointsProtocol=https;AccountName=', 'storageFunc', ';AccountKey=', listKeys('storageFunc','2015-05-01-preview').key1)]"
                    },
                    {
                      "name": "AzureWebJobsStorage",
                      "value": "[concat('DefaultEndpointsProtocol=https;AccountName=', 'storageFunc', ';AccountKey=', listKeys('storageFunc','2015-05-01-preview').key1)]"
                    },
                    {
                      "name": "WEBSITE_CONTENTAZUREFILECONNECTIONSTRING",
                      "value": "[concat('DefaultEndpointsProtocol=https;AccountName=', 'storageFunc', ';AccountKey=', listKeys('storageFunc','2015-05-01-preview').key1)]"
                    },
                    {
                      "name": "WEBSITE_CONTENTSHARE",
                      "value": "storageFunc"
                    },
                    {
                      "name": "FUNCTIONS_EXTENSION_VERSION",
                      "value": "~2"
                    },
                    {
                      "name": "FUNCTIONS_WORKER_RUNTIME",
                      "value": "dotnet"
                    }
                  ]
                }
              },
              "resources": [
                  {
                      "apiVersion": "2015-08-01",
                      "name": "web",
                      "type": "sourcecontrols",
                      "dependsOn": [
                        "[resourceId('Microsoft.Web/sites/', 'functionApp')]"
                      ],
                      "properties": {
                          "RepoUrl": "https://github.com/FBoucher/AzUnzipEverything.git",
                          "branch": "master",
                          "publishRunbook": true,
                          "IsManualIntegration": true
                      }
                 }
              ]
            }
        
    ],
    "outputs": {}
}

The Storage Account


The first resources listed in the template is the Account Storage. There nothing specific about it.

The Service Plan


The service plan is the second resource in the list. It's important to notice that to be able to use the SKU Dynamic you will need at least the API version of apiVersion to be "2018-02-01". Then you specify the SKU.

    "sku": {
        "name": "Y1",
        "tier": "Dynamic"
    }

Of course, you can use the other SKU if you prefer.

The Function App


Final resources added to the mixt, and this is where all the pieces are getting together. It's important to notice that the other in which the resources are listed are not considered by Azure while deploying (it's only for us ;) ). To let Azure knows you need to add dependencies.

"dependsOn": [
    "[resourceId('Microsoft.Web/serverfarms', 'servicePlan')]",
    "[resourceId('Microsoft.Storage/storageAccounts', 'storageFunc')]"
]

This way the Azure Function will be created after the service plan and the storage account are available. Then in the properties we will be able to build the ConnectionString to the blob storage using a reference.

{
    "name": "AzureWebJobsDashboard",
    "value": "[concat('DefaultEndpointsProtocol=https;AccountName=', 'storageFunc', ';AccountKey=', listKeys('storageFunc','2015-05-01-preview').key1)]"
}

The last piece of the puzzle is the sub-resource sourcecontrol inside the FunctionApp. This will define where Azure should clone the code from and in which branch.

"resources": [
    {
        "apiVersion": "2015-08-01",
        "name": "web",
        "type": "sourcecontrols",
        "dependsOn": [
        "[resourceId('Microsoft.Web/sites/', 'functionApp')]"
        ],
        "properties": {
            "RepoUrl": "https://github.com/FBoucher/AzUnzipEverything.git",
            "branch": "master",
            "publishRunbook": true,
            "IsManualIntegration": true
        }
    }
]

To be sure that everything is fully automatic the properties publishRunbook and IsManualIntegration must be set as true. Otherwise, you will need to do a synchronization between your Git (in this case on GitHub), and the Git in Azure.

There is excellent documentation that explains many deferent scenarios to Automate resource deployment for your function app in Azure Functions

Azure Unzip Everything


To deploy the project AzUnzipEverything available on GitHub, I needed one more Azure Storage with pre-define containers (folders).


Of course, all the source code of both the Azure Function and the ARM template are available on GitHub, but let me highlight how the containers are defined from an ARM template.

"resources": [
    {
        "type": "blobServices/containers",
        "apiVersion": "2018-07-01",
        "name": "[concat('default/', 'input-files')]",
        "dependsOn": [
            "storageFiles"
        ],
        "properties": {
            "publicAccess": "Blob"
        }
    }
]

Just like with sourcecontrol, we will need to add a list of sub-resources to our storage account. The name MUST start by 'default/'.

Part 2 - Four Deployment Options

Now that we have a template that describes our needs we just need to deploy it. There are multiple ways it could be done, but let's see four of them.

Deploy from the Azure Portal


Navigate to the Azure Portal (https://azure.portal.com), from your favorite browser and search for "deploy a custom template" directly in the search bar located at the top of the screen (in the middle). Or go at https://portal.azure.com/#create/Microsoft.Template. One in the Custom deployment page, click on the link Build your own template in the editor. From there, you can copy-paste or upload your ARM template. You need to save it to see the real deployment form.


Deploy with a script


Would it be in PowerShell or in Azure CLI you can easily deploy your template with these two commands.

In Azure CLI

# create resource group
az group create -n AzUnzipEverything -l eastus

# deploy it
az group deployment create -n cloud5mins -g AzUnzipEverything --template-file "deployment\deployAzure.json" --parameters "deployment\deployAzure.parameters.json"  

In PowerShell

# create resource group
New-AzResourceGroup -Name AzUnzipEverything -Location eastus

# deploy it
New-AzResourceGroupDeployment -ResourceGroupName  AzUnzipEverything -TemplateFile deployment\deployAzure.json

Deploy to Azure Button


One of the best way to help people to deploy your solution in their Azure subscription is the Deploy to Azure Button.



You need to create an image link (in HTML or Markdown) to this to a special destination build in two-part.

The first one is a link to the Azure Portal:

https://portal.azure.com/#create/Microsoft.Template/uri/

And the second one is the location of your ARM template:

https%3A%2F%2Fraw.githubusercontent.com%2FFBoucher%2FAzUnzipEverything%2Fmaster%2Fdeployment%2FdeployAzure.json

However, this URL needs to be encoded. There is plenty of encoders online, but you can also do it from the terminal with the following command (A big thanks to @BrettMiller_IT who showed me this trick during one of my live streams).

[System.Web.HttpUtility]::UrlEncode("https://raw.githubusercontent.com/FBoucher/Not-a-Dog-Workshop/master/deployment/deployAzure.json")

Clicking the button will bring the user at the same page on the Azure Portal but in the user subscription.

Azure DevOps Pipeline

From the Azure DevOps portal (https://dev.azure.com), select your project and create a new Release Pipeline. Click on the + Add an artifact button to connect your Git repository.



Once it's added, you need to add a task the current job. Click on the link 1 job, 0 task (4). Now you just need to specify your Azure subscription, the name of the resource group and select the location of your ARM template inside your repository. To make the deployment automatic with each push in the repository, click that little lightning bolt and enable the Continuous deployment trigger.


Wrapping-up

Voila, you know have four different ways to deploy your Azure Function automatically. But don't take my word for it, try it yourself! If you need more details you can visit the project on GitHub or watch this video where I demo the content of this post.


Visual Studio Online is definitely more than text editor online!

5 powerful features that make Visual Studio online an indispensable tool

If you are a .Net developer then you know Visual Studio. Few months ago, Microsoft released a new version of this tool call Visual Studio 2013 Community edition. This version offers all the features and is completely free. Many of us already have downloaded and installed this version. Another version is also available, but doesn't have the popularity of his brothers... yet. Let's talk about Visual Studio Online.
 

Get Started

Did you know that Visual Studio Online is available for free? Go to http://www.visualstudio.com/ and create an free account that will included:


  • 5 FREE Basic user licenses
  • Unlimited team projects and private code repos
  • FREE work item tracking for all users
  • FREE 60 minutes/month of build
  • and much more.... 

  • 1- It's a Source Control

    It may be called Visual Studio Online, but it has all the features from Team Foundation Server (TFS).
    Let's create a new project. From your home page, under the tab Overview (selected by default) click on New. This will pop the form to create a new team project. One interesting thing to note is that both TFS and Git are supported as Version control system.
    For the post I will create my project MyBookManager using Git.

    Create_project_with_VSO

    2- It's an Agile board


    Now that we have a project, it's time to describe it. Visual Studio Online has an entire section just for collaborative work. From the home screen of your VSO click on _Users_ tab. Create your users, roles and manage their permissions.
    It's now time to time to describe all the features of the project. Click on _Overview_ tab to get back to Home. Select your project, then click on _Work_ tab. From here you will be able to create all the features you need, and split them in tasks and sub tasks; perfect to fill-up a backlog item list and plan your next sprint.
    Once a sprint is started, it's also from here that you will be able to see and interact with the Board.

    board

    From this board, cards can be move by dragging them in the column that reflected the current status of the work. Open the on-premise Visual Studio. From the bar Team Explorer click on the little plug, and select: “Select Team Project…”. If your VSO Server is not present, just add it, then select the your project (ex: MyBookManager).
    For this post I will create an empty Asp.Net MVC project. In the Models folder I add a new class Book with very basic code.

    public class Book
    {
         public int BookID { get; set; }
         public string Title { get; set; }
         public bool Read { get; set; }
    }
    

    Then by right-clicking on the folder Controllers select the Add new Scafolded Item option, and use that to create our CRUD controller and views for the Book model we just created. Let’s check-in the code, and go back to VSO. You should see the code.

    To give more visibility to the work’s status, you can create very nice chart and pinned them on the Home page. To add a new chart, go in the Work tab select Queries.  You can use the existing query or simply create a new one. To add a chart to the Home page right-click on the query and select Pin to Home page.

    Home

    3- It's a build / Deployment server


    create_build_step_1If it's one think that seem always complex for a developer is the automation build and deployment server setup. With VSO no need for all that complexity and resources, only follow a short wizard and your build machine will be setup in Azure ready to deploy as soon as you want.  You can setup an automatic build and deployment from the Azure Portal and Visual Studio. In this post, I will use Visual Studio.

    From the Team Explorer bar in Visual Studio, click on Builds then New Build Definition, to get the wizard.

    Build_step1

    First step, just put a name who makes sense.

    build_step_2

    Step two, we need to choose what will trigger the build. I select continuous integration, so the build should start at every check-in.

    Build_process

    When I create my project I select Git as a source control, so for the next step I will need to expand the Build process template zone, by clicking Show details.  Then from the list select GitContinuousDeploymentTemplate.12.xaml.

    The build definition is done, save and let it there for now.

    4- It's a Code Editor


    If it's one feature that you could expect from a tool like VSO is to be able to edit code online.  The online version is obviously less powerful then the on-premise one, but still you will it very convenient and easy to use.

    Let's change the button in the book list to be more modern. First, select Code tab.  Now let's find the Index.html file under the View folder. Once the file is selected, click on Edit button. It's time to add some bootstrap classes to transform the look of our button. As you can see even we are in a web  browser we have IntelliSense. 

    Edit_online

    Build_overall_processHit Save and it will commit our change. Since we create a continuous build, this one should kick-in and build and deploy our changes. We can see the status of our build by going in the Build tab in Visual Studio Online. It’s also possible to see it in the Home page by pinning it.
    When the build will be done, our change will be visible in the website. Of course, this build is very simple, unit test and more complexes processes could also be implemented.


    update_Result_after_auto_deployed



    5- It's a Test server


    Another great feature is the possibility to create a test plan for the quality assurance (QA) tester. You could create it via VSO as a test plan by going in the last tab Test. And create a checklist,  so they can follow it to test the application feature by feature.  You could also record the action with Microsoft Test Manager and then do a load test on the website by running simultaneously multiple tests on different instances in Azure.

    Microsoft_Test_Manager



    Verdict

    Visual Studio Online is a great tool and win to  be now more. What are the features you enjoy the most in VSO?


    ~Frank B

    References

  • http://www.visualstudio.com/