Running Commentary

 This is the powerpoint presentation I used when giving a session to my team about Docker. Tried to start with fundamental concepts and go up to using Docker Compose.

This mini-project started as a necessity.  I get an email like this everyday, which gives daily numbers for different metrics. The problem with such an email is that it is difficult to keep track of or gauge the rate of growth of these metrics. If you want to know how what the previous day's numbers were you need to go to the previous day's email.  The better option any day is to have a graph that shows the values of the metrics every day. Something like this: With this goal I set out to make this system but with as less coding and expenses as possible.  First obvious step in the process was to get hold of the email and extract its text. The easiest way to do this is to use Azure Logic App.  Here is what my logic app looks like: When any new email arrives in my Outlook 365 account, with a subject that contains "System usage Stats as on", it passes the body of the email to an Azure function named 'EmailTextCleanerFunction' (code below), whose job is to clean an...

I deployed my webapp on Azure and found that the ease and simplicity of deploying and managing an Azure Webapp Service does not come cheap!  So next best option was to deploy it on Kubernetes using Azure Kubernetes Service. It involves a few more steps than webapp service, but then it is cheaper and if I want to move it to Google Container Engine, I don't have to break into sweat.  So here are the steps I followed: Created a new Resource Group named firstResourceGroup. az group create --name firstResourceGroup --location eastus Created new ACR in this new resource group. Let's call it 'firstContainerRegistry'.  az acr create --resource-group firstResourceGroup --name firstContainerRegistry --sku Basic On successful completion, the output is like this: {   "adminUserEnabled": false,   "creationDate": "2019-04-24T05:03:32.564208+00:00",   "id": "/subscriptions/1900743b-c1ab-48cd-9951-eb03f5c237...

If you are here I am assuming that you are looking to migrate the data from Cloud Datastore to Big Query because you want to do some analysis and are frustrated by limitations imposed by GQL (Google Query Language). First of all you need to create a backup of the data in datastore. Use the Datastore Admin tool provided by Google to take a backup and store it automatically in the Cloud Storage bucket. Select all the entities and press 'Backup Entities'. Give a backup name, select Google Cloud Storage as backup storage destination and specify a bucket name. Once the backup job is completed, you will see the backup listed. You can select a backup and press 'Info' and see the details (Entities are masked in the screenshot below). Go to the bucket mentioned in 'Handle' and you will see the file mentioned above. You will also see many more files with similar names, ending with .backup_info (e.g. ahRzfmpkYS1wZC1zbG8tc2FuZGJveHJBCxIcX0FFX0RhdGFzdG9yZUFk...

Signed URLs are pretty nifty feature given by Google Cloud Platform to let anyone access your cloud storage (bucket or any file in the bucket) without need to sign in. Official documentation gives step by step details as to how to read/write to the bucket using gsutil or through a program. This article will tell you how to upload a file to the bucket using curl so that any client which doesn't have cloud SDK installed can do this using a simple script. This command creates a signed PUT URL for your bucket. gsutil signurl -c 'text/plain' -m PUT serviceAccount.json gs://test_bucket_location Here is my URL: https://storage.googleapis.com/test_sl?GoogleAccessId=my-project-id@appspot.gserviceaccount.com&Expires=1490266627&Signature=UfKBNHWtjLKSBEcUQUKDeQtSQV6YCleE9hGG%2BCxVEjDOmkDxwkC%2BPtEg63pjDBHyKhVOnhspP1%2FAVSr%2B%2Fty8Ps7MSQ0lM2YHkbPeqjTiUcAfsbdcuXUMbe3p8FysRUFMe2dSikehBJWtbYtjb%2BNCw3L09c7fLFyAoJafIcnoIz7iJGP%2Br6gAUkSnZXgbVjr6wjN%2FIaudXIqA...

You would already be aware that there are multiple options given by Google Cloud Platform to store data. Here is  Google documentation  on when to use which option: Google recommends using Google Cloud Storage (GCS) to store static content like files/videos etc. There is something called 'Blobstore' as well which is also used to store such content but it is on the way to being deprecated. This page talks about using GCS to store images. Look at  this page  to understand basic requirements for setup of GCS. In the Cloud Store Browser below following buckets are already available. If you select any bucket, you would be able to see the objects created in it.  Here you can see the image file in the 'jda-pd-slo-sandbox.appspot.com' bucket. You won't be able to add/delete files or folder from the browser if you don't have proper access but through code (running with the service account) it should not be a problem. Objects on GCS are immutable so you ca...