-
Learning Databricks – Accessing a Data Lake Using a Service Principal
Introduction One of the first tasks to learn when using Data bricks is how to access data contained in a data lake. I will be using Azure as my cloud provider in these posts and therefore my data lake will be Azure Data Lake (gen2). This post will show the necessary steps that you need…
-
Unstructured Data to Structured – JSON Normalisation
Introduction I had a job interview today and one of the topics that was raised was importing JSON based data (which can be considered as unstructured) into a table format (structured). I offered a few ways of how this could be done and I think answered the question appropriately, but the interviewer said a number…
-
Automate the Boring Stuff – DevOps
Introduction The fun part of being able to code is well… the coding right? (provided it works) but in reality so many developers spend a minority of their working day writing code, there are meetings, phone calls, lunch (ok that bit isn’t bad), and all the work around building, testing and deploying software. An approach…