What I’ve learned moving from Azure WebJobs to Azure Function Apps

If you have been using Azure for some times, you might know about Azure WebJob, a great way to run your code in background on triggers and integrate with other services via bindings. Recently, Azure introduced a similar offering: Azure Function App. Function App is based on a very similar idea, where you can run your code in any supporting languages on certain triggers such as a Queue message or a HTTP request or a timer.

We have been using Azure WebJob for a couple of years to process our email/push notifications and to clean up of our database. WebJob served that purpose perfectly until our app growth reveals a big limitation: WebJob consumes all the resource of the web app itself. In theory, WebJob should be a separate background process and should not interfere with the web app. However, since it is inside the App Service, it utilizes the same CPU/Memory consumption constraint of the App Service Plan. We started getting downtime alert from our APIs because the WebJob is running too heavily, defeating the purpose of splitting up some processing to a separate process in the first place.

We look around for solutions. First of all, we tried scaling out the app service plan based on CPU and memory consumption. However, it was hard to set a proper rule to scale up and down, and the cost multiplied quite quickly. We started looking at moving to Azure Function App with consumption plan. On paper it looks great because we can completely separate our background processing and our API. We also does not have to waste computing resources for the WebJob dashboard as telemetry is now sent to Application Insights instead of Azure Storage (I heard the WebJob dashboard consumed App Service Plan’s CPU to perform certain indexing).

Unfortunately, the migration was not as smooth as we thought. We soon met many problems with our C# implementation:

  • If you are using Timer trigger in Function with consumption plan, remember that you have to use the CRON schedule format (0 0 */8 * * *), not the Timer format (08:00:00).
  • Using QueueTrigger, you’ll get an amazingly cryptic error message on Azure Dashboard: The binding type(s) ‘queueTrigger’ are not registered. Please ensure the type is correct and the binding extension is installed. Visual Studio generated a functions.json with binding queueTrigger and can be seen on the Azure portal. We also added queues section to host.json file. We have also installed Microsoft.Azure.WebJobs.Extensions.Storage to enable QueueTrigger in code. So what did we miss here? Turn out you have to add
    Microsoft.Azure.WebJobs.Script.ExtensionsMetadataGenerator nuget package as well, so that Visual Studio will generate extensions.json.
    This is not mentioned anywhere in the docs but only a small page on GitHub, which is very irritating.
  • ILogger is supported by the new WebJob SDK since the migration to .NET Core and also by Function SDK v2, which is great since our Functions can finally share code with ASP.NET Core. Unfortunately, you might get error “Cannot bind parameter ‘log’ to type ILogger. Make sure the parameter Type is supported by the binding.” Turn out you need to use the exact version of ILogger in the SDK (e.g. SDK 1.0.19 uses 2.1.0) (stackoverflow), which would be a big problem if one of the dependencies updates to higher versions.

There are also several limitations that is probably due to Function App being a quite young platform in Azure:

  • It seems that Function is actually based on WebJob behind the scene, so the features set of Function SDK in C# is only a subset of that of WebJob:
    • WebJob recently moved to .NET Standard and get the extremely convenient dependency injection. Meanwhile, there is no DI story in Function. This might be ok with small function, but ours get a lot of logic and we would like to have DI to get proper tests.
  • The current dashboard experience in Azure portal lacks the ability to replay the execution. This is not really critical for applications that have already been in production for a while as you should already have some retry mechanisms. However, it is quite convenient for development and small testing. Hopefully this will come soon in the future.

If you have any better solutions for those problems, please feel free to let me know in the comments.

Rethink database design with Microsoft Azure

I am definitely not a database expert, so what I am writing here is only from my experience migrating all my applications to Microsoft Azure. If you think that I am doing something wrong or you know a better way, please feel free to leave a comment, so that I can learn more about living in the Cloud.

In this blog post I will write a little bit about what I have been using for a very long time before moving to Azure. Then I will write about my experience with SQL Azure and Azure Table Storage, specifically the problems that I have when migrating from SQL Azure (or MS SQL in general) to Table Storage. You can go directly to the last part if you would like.

Read More

Steps to install WordPress on Azure Virtual Machine running Ubuntu

I have moved my blog to Azure Website for a while and I am quite happy with it. However, recently I have decided to move my blog to an Azure VM running Ubuntu.

I have to say Azure Website is very fast, convenient and easy to set up: you just have to create a new website, chose the WordPress template and go though the WordPress’s 5-minute setup. However, that require you to scale up to Shared, Basic or Standard plan (which cost at least 10 USD/month) to get some more advanced features such as setting custom domains. Besides, the free MySQL database by ClearDB provides you with only 20MB, and the cheapest upgrade is another 9 USD/month for 1GB (quite huge for my blog).

Recently, I happened to create a Ubuntu VM for my research. And most of the time the VM is idle. So instead of turning it on an shutting it down all the time to save cost, I decided to move my blog again to the VM, and buy a new domain for it: www.nguyenquyhy.com.

However, setting WordPress up on a Virtual Machine with Ubuntu 14.04 is not really as joyful as on an Azure Website. It cost me almost 3 days to solve all the issues, so I decided to note down all the steps I take here in case I need to move my blog again in the future, or someone else gets the same problems.

Read the steps

Some mid-night thought from the TechEd keynote

It is almost morning, but TechEd is so much exciting, so I cannot go to bed yet. Besides that, Visual Studio Update 2 is also installing so I cannot write codes either. So I will jot down all the excited features that I have just learned from the keynote.

Visual Studio Online API

Just one word: AWESOME! I have been waiting for this for too long.

Recently I have migrated all my project codes to Visual Studio Online from my previous Subversion provider. Moreover, with the extensive support for project management in Visual Studio Online, all my app’s backlogs and bugs are recorded here as well. The current VSO web portal has many great and advanced features; unfortunately, it lacks a responsive design for smartphones which is very important for me when most my development is for the phone and many of my tests are on the phone itself.

I always imagine about detecting a bug or thinking of a new feature when using the app and then being able to add it immediately to the backlog in a simple phone app (voice control could be cool as well, something like VSO record a bug for XXX app about …). It is now all possible with the introduction of VSO API.

Even more than that, the team has also released WebHooks as well. So not only can you poll your backlog for information, but you (and your code) can also know exactly when something has changed. Let’s say you can write some code to easily send your phone a toast when your backlog items are changed by other team members!

Azure API Management

This one is also a very nice feature that has just come to Azure. Basically this API Management is a proxy to your current API, where you can map new endpoints managed by API Management to your existing Web API endpoints. In addition, this proxy does provide you with so many useful facilities, for example, a developer console with example generation, testing support and a management console with throttling, caching, analytics, notification, security and many convenient functions for your APIs. You can also customize your developer console by changing LESS variable, adding posts, navigation items and widgets.

New & Exciting Microsoft

Microsoft has so many great products through out its history. However, with the new focus on services, Microsoft is now opening up many of its services (e.g. OneNote, OneDrive, VSO, Office 365, etc.), which is so nice for app developers and so nice for my imagination.

Let’s start some new projects right away!