Setting up parallel Spark2 installation with Cloudera and Jupyter Notebooks

Yes, this topic is far from new material. Especially if you consider Cloud tech stack evolution/change speed, it has been a long time since Apache Spark version 2 was introduced (26-07-2016, to be more precise). But moving into the cloud is not an easy solution for all companies, where data volumes can make such a move prohibitive. And in on-premises contexts, the speed of operational change is significantly slower.

This post summarizes the steps for deploying Apache Spark 2 alongside Spark 1 with Cloudera, and install python jupyter notebooks that can switch between Spark versions via kernels. Given that this is a very frequent setup in big data environments, thought I would make the life easier for “on-premise engineers”, and, hopefully, speed up things just a little bit.

Continue reading “Setting up parallel Spark2 installation with Cloudera and Jupyter Notebooks”

Check out video Understanding Using Temporal Cycle-Consistency Learning

It has been a long time since I last blogged. João and I have been busy getting things ready to launch a new project to streamline multi-cloud management that we have been working in the last couple of months. We will talk soon in more detail about it, so stay put.

Meanwhile just wanted to share this very interesting blog post from Google AI blog – video understanding using temporal cycle-consistency learning – where they propose a self-supervised learning method to classify different actions, postures, etc. in videos.

Continue reading “Check out video Understanding Using Temporal Cycle-Consistency Learning”