Nikolai Thomassen - Data Engineer

Database migrations using FlyWay

Updated August 25, 2019August 24, 2019

In the world as a developer you have a big toolbox in front of you, constantly evolving. As a result it’s not always easy to navigate between everything and sometimes hard to find out where to start.

You may remember that I previously blogged about testing using the tSQLt framework. At the point I assumed that you were familiar with SQL Server Database Projects, having a state based deployment model. Today, however, I will give you an introduction on how to start with migration based deployments.

Scaling the Data Management Nodes for Azure Data Explorer

Updated April 2, 2019April 2, 2019

Is your Event Hub ingestion to Azure Data Explorer “throttling”? Do you have trouble increasing the processing of events received at the cluster? Scaling up or scaling out the cluster does not seem to resolve the issue? Then the data management nodes, running the endpoint might be the reason.

Azure Data Explorer Series, Part 1

Updated March 31, 2019March 31, 2019

Have you heard about Azure Data Explorer, formerly known as project Kusto? The new PaaS from Microsoft that enables fast and highly scalable data exploration? The service that empowers Azure Monitor and Time Series Insights?

Fear not! As of 7 February 2019 it was announced to be General Available, meaning that it’s the new kid on the block. I will therefore dive into the nitty-gritty details in a series of blog posts that hopefully gives you enough information to get started yourself.

Azure SQL unit testing with tSQLt using Azure DevOps

Updated February 27, 2019February 26, 2019

Working as a Data Engineer I often see that unit tests are written for the applications that are developed, but often forgotten about when doing database development. We would assume that it’s the de facto standard in 2019 if the target is to be fully DevOps as a development team.

There are many great third party tools out there that simplifies setting up the processes, most of them requires licenses. Therefore I will go back to the basics so that you cannot blame the license cost for not implementing database unit testing.

Azure Data Studio now support MFA

Updated February 3, 2019February 3, 2019

When I first learned about Azure Data Studio being available September 2018, I was excited for finally getting a cross platform alternative to SQL Server Management Studio that also is built on top of Visual Studio Code. And who doesn’t love Visual Studio Code?

Sadly the excitement dropped a bit when I realized that the only option for logging in to Azure SQL databases was by using SQL logins, but that has changed now!

Azure DevOps with Data Factory

Updated January 30, 2019January 20, 2019

In many organization a common misconception is that DevOps is about the tools we use, so let’s use a second to read the citation from Microsoft.

DevOps brings together people, processes, and technology, automating software delivery to provide continuous value to your users.
Microsoft

Of course this post will not be about what DevOps is and isn’t, but I think it’s important refresh the citation once in a while. This post will however focus on how the tooling can help you automate the deployment of Data Factory from development to test environment, that can easily be transitioned to any other environments you may have.