Running Commentary

There are so few examples of Azure Functions implemented in Java and even official documentation is a little confusing, so I decided to write my own post explaining some of the points. So here is my function class: public class EventHubTriggerFunction { /** * This function will be invoked when an event is received from Event Hub. */ @FunctionName ( "EventHubTrigger-Java" ) public void run ( @EventHubTrigger (name = "message" , eventHubName = "my-event-hubs" , connection = "EventHubConnectionString" , consumerGroup = "$Default" ) String message, final ExecutionContext context ) { context.getLogger().info( "Java Event Hub trigger function executed." ); context.getLogger().info( "Message:" + message); } } The noteworthy thing in this snippet are the annotation attributes. name : Any name that can be given to the...

This mini-project started as a necessity.  I get an email like this everyday, which gives daily numbers for different metrics. The problem with such an email is that it is difficult to keep track of or gauge the rate of growth of these metrics. If you want to know how what the previous day's numbers were you need to go to the previous day's email.  The better option any day is to have a graph that shows the values of the metrics every day. Something like this: With this goal I set out to make this system but with as less coding and expenses as possible.  First obvious step in the process was to get hold of the email and extract its text. The easiest way to do this is to use Azure Logic App.  Here is what my logic app looks like: When any new email arrives in my Outlook 365 account, with a subject that contains "System usage Stats as on", it passes the body of the email to an Azure function named 'EmailTextCleanerFunction' (code below), whose job is to clean an...

Here is a simple example of how and when to use CyclicBarrier in Java.  

Here is a simple example of how and when to use CountDownLatch in Java.

Getting heapdump for our services is very important for doing any memory profiling to debug any leaks or performance issues. Unfortunately it is not that straight forward. Fortunately, it is not impossible either! We need to SSH in the container in which our service is running. Go to App Engine → Instances page and select your service from the 'services' dropdown. SSH into any of the instances: Ignore this and SSH into VM. Once you SSH, you should be able to see cloud shell as below. If you are logging in for the first time, there may be some questions asking for SSH key. You can leave the key blank and continue. This shell is the Linux VM on which our docker image is running. We need to find out more details about it before we can do anything with it.   There are a bunch of images running. We are interested in the first one ('us.gcr.io/.....') running in a container named 'gaeapp'. Now we need to go inside this docker c...

The best thing about Apache Beam ( B atch + Str eam ) is that multiple runners can be plugged in and same pipeline can be run using Spark, Flink or Google Cloud Dataflow. If you are a beginner like me and want to run a simple pipeline using Spark Runner then whole setup may be tad daunting. Start with Beam's WordCount examples  which help you quickstart with running pipelines using different types of runners. There are code snippets for running the same pipeline using different types of runners but here the code is running on your local system using Spark libraries which is good for testing and debugging pipeline. If you want to run the pipeline on a Spark cluster you need to do a little more work! Let's start by setting up a simple standalone single-node cluster on our local machine. Extending the cluster is as easy as running a command on another machine, which you want to add to cluster. Start with the obvious: install spark on your machine! (Remember to have Java a...

You would already be aware that there are multiple options given by Google Cloud Platform to store data. Here is  Google documentation  on when to use which option: Google recommends using Google Cloud Storage (GCS) to store static content like files/videos etc. There is something called 'Blobstore' as well which is also used to store such content but it is on the way to being deprecated. This page talks about using GCS to store images. Look at  this page  to understand basic requirements for setup of GCS. In the Cloud Store Browser below following buckets are already available. If you select any bucket, you would be able to see the objects created in it.  Here you can see the image file in the 'jda-pd-slo-sandbox.appspot.com' bucket. You won't be able to add/delete files or folder from the browser if you don't have proper access but through code (running with the service account) it should not be a problem. Objects on GCS are immutable so you ca...

Binary Heaps are one of the easiest and most popular means to implement priority queues. You can get a lot of tutorials explaining the algorithms for insertion, deletion of the maximum (or minimum) element and using the heap for implementing priority queue. Here is a simple Java code for these: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 import java.util.Arrays ; class BinaryHeapImpl { private int arr []; private int heapSize = 0 ; BinaryHeapImpl ( int size ){ ...

If you are reading this, I assume you are aware about Twitter APIs. If you are new to this API, I suggest you read the  API documentation  (which is quite good BTW!) This simple program of mine uses Twitter search API to get "positive" tweets for a search string. It aims to retrieve all the search results (tweets) and hence has to make multiple calls to REST API, for which it uses max_id (explained later). Let me give an overview of all the request parameters that we have specified in the URL (Twitter API URL): q=%23SearchString: #SearchString count=100: Number of tweets you want the API to return in one call. include_entities=false: Exclude details of entities in the JSON response. You can set it to true if you want. max_id: Since we want more than 100 tweets, we are invoking the API multiple times and giving the id of last tweet returned in previous invocation as 'max_id' and asking Twitter to give 100 tweets prior to this tweet.  e.g. First call ret...