Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

So I was playing with AWS Lambdas, and wanted to setup SNS notification for that.

1

aws sns publish --topic-arn arn:aws:sns:eu-west-1:22834709971:my-awesome-sns-topic --subject 'Subject' --message 'Hello, hello!'

I recently came across an interesting problem/game called Game Of Life. A visual version of this is available here. Also, here is a beginners introduction to Game Of Life, or simply Life, written by Alex Bellos at Guardian.

It is a cellular automation problem created by John Horton Conway, a British mathematician in 1970, not a game you play to win or lose. This problem has a few rules that we need to follow.

Rules

1. A cell in the grid can be dead or alive.
2. A live cell with less than 2 live neighbours will die.
3. A live cell with 2 or 3 live neighbours will stay alive.
4. A live cell with more than 3 live neighbours will die.
5. A dead cell with exactly 3 neighbours will become alive.

These rules are pretty simple. Once we know the number of live neighbours of a given cell, we can determine it’s fate in a step. In this post I’m going to dicuss method to find neighbours of a given cell in a matrix.

In Game of Life, we take a seed grid and eventually get another grid as a output, by applying the rules to each cell.

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]

Setting up Airflow on AWS Linux was not direct, because of outdated default packages. For example I had trouble using setuid in Upstart config, because AWS Linux AMI came with 0.6.5 version of Upstart.

AMI Version: amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2 (ami-c51e3eb6)

Install gcc, python-devel, and python-setuptools

1

sudo yum install gcc-c++ python-devel python-setuptools

Upgrade pip

1

sudo pip install --upgrade pip

Install airflow using pip

1

sudo /usr/local/bin/pip install airflow[s3, hive, python]