The current project I'm working on has a heavy dependency on Elasticsearch and I thought it would be good to walk through my setup using Docker. I wanted to manage Elasticsearch using Docker for a few reasons:
There are plenty of references on the web about setting up services using Docker, but I figured I would walk you through what I've been doing and how it's been working for me.
Before we get started I am using Elasticsearch 1.7 as you can see in my Docker file. I have not upgraded to 2.x and know my configure will have to change as Elasticsearch can no longer run as root. I want to keep as much control over the Docker install as I can so I built my own Dockerfile based on the official elasticsearch Dockerfile. I also wanted to build my own Dockerfile instead of using the Elasticsearch Dockerfile to ensure I can easily upgrade my Elasticsearch instance when I wanted. Since I've started the project I have upgraded 3 times. This has been very painless as I am still in development and it is easy to keep things moving. I haven't made many changes to the base file. Below is my Dockerfile for Elasticsearch.
## # # Dockerfile for our Elastic Search cluster. The dockerfile is based off of # https://github.com/dockerfile/elasticsearch. # # Pull base image. FROM ubuntu:14.04 ENV ES_PKG_NAME elasticsearch-1.7.3 # Install system level packages needed to install elastic search or dependancies. RUN apt-get -y install software-properties-common # Install Java. RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 \ select true | debconf-set-selections && \ add-apt-repository -y ppa:webupd8team/java && \ apt-get update && \ apt-get install -y oracle-java8-installer && \ rm -rf /var/lib/apt/lists/* && \ rm -rf /var/cache/oracle-jdk8-installer # Define commonly used JAVA_HOME variable ENV JAVA_HOME /usr/lib/jvm/java-8-oracle # Install Elasticsearch. RUN \ cd / && \ wget https://download.elasticsearch.org/elasticsearch/elasticsearch/$ES_PKG_NAME.tar.gz && \ tar xvzf $ES_PKG_NAME.tar.gz && \ rm -f $ES_PKG_NAME.tar.gz && \ mv /$ES_PKG_NAME /elasticsearch # Mount elasticsearch.yml config ADD config/elasticsearch.yml /elasticsearch/config/elasticsearch.yml # Expose ports. # - 9200: HTTP # - 9300: transport EXPOSE 9200 EXPOSE 9300
As you can see it is pretty much the same thing as the official Elasticsearch Dockerfile with a few very minor tweaks like the Elasticsearch version.
While in development I use docker compose to help manage the Docker container. It's a great tool to help you manage and control your containers. It becomes really useful when you are managing more than one container at a time. My docker-compose.yml file is very basic and sets the image, working directory, startup command, volumes and ports. I like to keep attributes that might change in my Docker compose file so I can make changes without having to rebuild the container.
elasticsearch: build: elasticsearch working_dir: /data command: /elasticsearch/bin/elasticsearch -Des.config=/data/config/elasticsearch.yml volumes: - /var/lib/docker/<app_name>/elasticsearch/data:/data - ./elasticsearch/config:/data/config ports: - "9200:9200" - "9300:9300"
Docker recommends using docker-compose in development and staging environments but not production.
One of the more important parts of the Dockerfile is making sure I have mounted my Elasticsearch configuration file. I've made a few small changes to the Elasticsearch config file. I recommend reading this article from loggly's blog as well as this optimization article on the Elasticsearch site. I found both of these extremely useful when configuring my server. I've simply set my cluster name details and made a few minor configuration tweaks. Since I haven't deployed to production and haven't been able to monitor performance I just wanted to make some of the changes recommended in the above articles. The configuration file will evolve as I deploy to production and am able to look at the server performance and metrics.
path: data: /data/data logs: /data/log plugins: /data/plugins work: /data/work # Cluster information cluster.name: <application_name> node.name: <application_name> # Security settings script.disable_dynamic: true # Index settings action.auto_create_index: false # Turn off swap to get a big speed increase. This will prevent the ES server # from swapping memory on the node. https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html#setup-configuration-memory bootstrap.mlockall: true # Disable deleting all indices from the api. action.disable_delete_all_indices: true indices.cluster.send_refresh_mapping: false
Overall I've found working with Elasticsearch and Docker to be really helpful. I love the fact that I can easily start and stop the service and have it isolated on my local machine. Doing full re-indexes and upgrades to Elasticsearch has become a lot easier with very minimal work.