Category: ElasticSearch

Enable Authentication in ElasticSearch

After you have configured your ElasticSearch endpoint, the next important step is to make it secure. If you have a standard installation of ElasticSearch probably you have your ES endpoint pointing to http://localhost:9200. In my case I do have this configuration and this is how I can query into a test index:

GET http://localhost:9200/test/persons/1 HTTP/1.1
User-Agent: Fiddler
Host: localhost:9200
Content-Length: 0

And ElasticSearch will return an HTTP Status 200 OK with my resultset:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Content-Length: 196

{"_index":"test","_type":"persons","_id":"1","_version":1,"found":true,"_source":{
   "first_name":"Raffaele",
   "last_name":"Garofalo",
   "city":"Izmir",
   "e_mail":"raffaeu@gmail.com"
}}

The major problem of the default installation of ES is that is totally unsecure.

  • First is running over an HTTP protocol, which means that all information sent are in clear text, including your authentication credentials 
  • Second, there is no authentication mechanism in place, which means that anybody can get the data from my endpoint

So let’s see how we can enable Authentication and how we can enable SSL to make our ES endpoint secure.

Enable Authentication with Shield

One of the features that I really love about ES is the plugin architecture. You can easily install on an existing ES endpoint any of the available plugins with a couple of lines of code and configure them at runtime.

One of the most important plugins available for ES is Shield. Shield is a plugin that provides plenty of options to secure your ES endpoint, just to mention few:

  • Enable Basic Authentication with Username and Password
  • Provide three different type of basic User [admin | power_user | guest]
  • Create custom roles with custom permissions
  • Enabling Message Authentication
  • Auditing
  • Custom authentication providers such as LDAP, OAuth, Active Directory and more

Note:
If you have multiple nodes for your cluster, in order to have Shield running correctly you must stop all nodes and install shield over all your nodes. This is the only option available so you must take offline your ES while installing Shield

In order to install Shield you should open an ElasticSearch SHELL, which in Windows means “open a DOS console and point to your /bin folder where the elasticsearch program is available”:

# browse the elastic search dir
$ CD C:\Program Files\elasticsearch-2.1.1\bin

Then you have to install two plugins, License and Shield. This is required because Shield is partially free, which means that:

When your license expires, Shield operates in a degraded mode where access to the Elasticsearch cluster health, cluster stats, and index stats APIs is blocked. Shield keeps on protecting your cluster, but you won’t be able to monitor its operation until you update your license. For more information, see License Expiration.

$ plugin install license
$ plugin install shield

Until you reboot your ES endpoint, you will not be able to have the Authentication enabled. After reboot, if you try to issue the previous request, you will get an HTTP 401 UNHAUTORIZED response:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Basic realm="shield"
Content-Type: application/json; charset=UTF-8
Content-Length: 357

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/test/persons/1]","header":{"WWW-Authenticate":"Basic realm=\"shield\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/test/persons/1]","header":{"WWW-Authenticate":"Basic realm=\"shield\""}},"status":401}

So the next step is to create an Admin user that we can use to start to query our ES endpoint:

#browse /bin/shield directory
$ esusers useradd es_admin -r admin

With the esusers useradd command we created a new user named es_admin with the param –r we defined the role admin (available roles are admin, power_user, guest).

Second step, we need to issue an authenticated request to ES using our account in the following way:

GET http://localhost:9200/test/persons/1 HTTP/1.1
User-Agent: Fiddler
Host: localhost:9200
Content-Length: 0
Authorization: Basic ZXNfYWRtaW46c2VjcmV0

Where the “strange” string is just username : password combination, converted by Fiddler in a 64 bit string.

All right, so now you have your ES secured and the basic user management in place but you have another problem. Now you are sending per each request a username and password over the web, so you must enable SSL communication on your ES endpoint to encrypt the communication.

Note:
If you want to create custom roles with custom access, you need to modify the file [elasticsearch]/config/shield/roles.yml. In this file you can have a look at the current roles [admin, power_user, user] and modify them or create new one. For example you may have a role that grant permissions only to a specific index (see multi-tenant architecture).
Remember, every time you modify a configuration file in ES you MUST restart the service otherwise ES will keep using the cached .yml configuration files.

Enable SSL on ElasticSearch

First of all you need a X.509 certificate for your domain. There is no shortcut here, if you have a domain where you host your ES endpoint you must purchase an SSL certificate from a CA authority that covers your domain. This procedure is a bit painful and verbose but when done it’s done for your entire ES installation. You can also create a cert file from your own Windows/Linux machine, for example, using OPENSSL or similar but remember that your certification file will not be signed by a known CA authority and you may encounter problems during production.

In my case I host ES in a public DNS and I have a Domain Certificate from a CA authority in the form of .pfx file. Using OPENSSL I create a new .pem file from my .pfx using this command:

openssl pkcs12 -in certificate.pfx -out certificate.pem -nodes

Anyway you must have access to a .pem certificate file in order to continue.

keytool -importcert -keystore node01.jks -file cacert.pem -alias my_ca

This command (keytool) will import the certificate into the Java Keystore Database, which contains a list of trusted certificates.

Next you need to generate a private key and a certificate for your node as following:

keytool -genkey -alias node01 -keystore node01.jks -keyalg RSA -keysize 2048 -validity 712 -ext san=dns:node01.example.com,ip:192.168.1.1

With this command we create a key and a public certificate valid for 712 days (which is the classic example shown on ElasticSearch documentation). This command will also prompt you for some questions that will be registered together with the certificate. The san attribute allows to specify the DOMAIN where you are hosting your ES endpoint.

So, right now you have a node’s certificate but it needs to be signed by a CA authority, actually the one who issued the certificate for you.

keytool -certreq -alias node01 -keystore node01.jks -file node01.csr -keyalg rsa -ext san=dns:node01.example.com,ip:192.168.1.1

Now we have a Certificate Signing Request .csr that we can send to our CA. The CA will sign it and returns to you a .crt file. You can also use OPENSSL and generate the .crt by yourself if you wish. At this point you have to import the .crt into your .jks database with the following command:

keytool -importcert -keystore node01.jks -file node01-signed.crt -alias node01

Now we can configure SHIELD to use SSL and HTTPS transport protocol. Just let ES knows where your keystore is located and which is the password for your keystore and for your certificate:

# ---------------------------------- SSL -----------------------------------
shield.ssl.keystore.path: C:\Program Files\elasticsearch-2.2.0\elasticsearch-2.2.0\config\cert\keystore.jks
shield.ssl.keystore.password: [keystore password]
shield.ssl.keystore.key_password: [certificate password]
shield.transport.ssl: true
shield.http.ssl: true

At this point your Elastic Search will be hosted still on port 9200 (if you didn’t override this setting) but available only over HTTPS.

Alternative way of creating a keystore

In my specific scenario I didn’t want to modify the certificate because my CA authority is quite strict so I didn’t have the option to import a .pem, modify and create a .csr and so on so I used an alternative way.

First of all I have created my keystore:

keytool -genkey -alias mydomain -keyalg RSA -keystore keystore.jks -keysize 2048

Then I have imported by .pfx file as is, without any modification:

keytool -importkeystore -srckeystore mypfxfile.pfx -srcstoretype pkcs12 
-destkeystore keystore.jks -deststoretype JKS

Then I created a .csr request for my DNS but I will not send the request to my CA authority:

keytool -certreq -alias mydomain -keystore keystore.jks -file mydomain.csr

And finally I moved the two files, .jks and .csr into the config directory of elastic search and configured elastic search using the keystore password and the original certificate password. It worked like a charm and I didn’t need to send a .csr and import a .crt.