Hortonworks Sandbox on Ubuntu using docker
In a previous post, I talked about installing Hortonworks Sandbox on a Mac machine. That was straight forward.
In this blog ’m going to talk about installing the Hortonworks Sandbox manually on a Linux VM.
A little background about Docker base device size #
The first time docker is started, it sets up a base device with a default size of 10GB. All future images and containers would be a snapshot of this base device.
Base size is the maximum size that a container/image can grow to. By default docker limit containers to 10G. In Devicemapper, new container/images take zero size and grow up to the maximum size. Changing the base size will not actually change the physical usage of containers unless they grow larger than 10 GB.
You can view this information with the following command
|
|
Since Hortonworks image is greater than 10GB, we need to increase the storage base size. We can do this by changing the storage-driver to overlay and increasing the dm.bazesize.
Increase the docker base device size on Ubuntu #
For this we need to set storag-opt=dm.basesize
and change storage-diver
to overlay
.
Assuming we have Systemd
based setup
- First create directory for customising docker service
|
|
- Edit
/etc/systemd/system/docker.service.d/docker.conf
|
|
|
|
Here the first line with empty ExecStart
is necessary to clear previous configurations.
Also note that we are increasing the dm.basesize
to 30G
IMPORTANT: Make sure there is enough space in /mnt/docker-data
Reload systemd daemon and restart docker #
After editing docker service definition for systemd we need to releoad systemd daemon and restart docker, for the new settings to take effect.
|
|
If you check sudo docker info
, Storage Driver should be overlay
and Docker Root Dir should be /mnt/docker-data
Now we have setup docker to work with Hortonworks Sandbox.
Download the Hortonworks Sandbox Docker image #
Download the docker image. It’s a huge one 12GB, use high bandwidth network :)
|
|
Download the Hortonworks Script #
|
|
Load the sandbox image #
|
|
This is a heavy task. Go get your coffee now because it will take some time to load the docker
image.
This step would fail if you did not increase the base device size.
After it is loaded you should see sandbox-hdp
in your docker images
list.
Update /etc/hosts file #
Add the following to /etc/hosts
|
|
Run the Hortonworks Sandbox startup script #
|
|
Again this will take some time. Once the script completes you should be able to access the sandbox.
Access Hortonworks Sandbox #
Go to https://sandbox.hortonworks.com:8888/
This Sandbox comes with a lot of components like Ambari, Ranger, Hive, Spark etc. installed and configured.