Encrypted communication on Elastic server by setting up TLS and adjusting CORS permissions

Posted by Minxuan Sun
Minxuan Sun

In this tutorial, we will provide step-by-step instructions on how to encrypt communication among Elasticsearch, Kibana, and Logstash. After the setup, Elasticsearch and Kibana can be accessed by web browsers via https.

Before we start, make sure the following conditions are satisfied or agreed:

  • Elastic services are running with no issues on the server.
  • The directory layout for the Elastic stack is known. We use the directory layout of RPM in this tutorial. For further information, check this website: Directory layout of RPM
  • If shell prompt asks for additional options, press ENTER to choose the default one.
  • Using sudo before any command if Permission denied occurs
  • If any service fails to start, check the corresponding log. If the log indicates access denied for some files, make sure you have given the access permission for the files. Check this website for further information.
  • Some steps like creating directories, copying files, and editing files may be omitted.
  • Read the comments for the commands carefully. Make changes for the commands according to your situation.
  • Since we use self-sign CA in this tutorial, the web browser will show the certificate is not trusted in the case. It is ok, later I will write a tutorial on how to use third-party CA to make sure the certificate is trusted by the web browser. Please follow me to receive the latest notification about my articles.

 

Kibana, Elasticsearch, Logstash

Step 1: Using self-sign CA generate from Elasticsearch:

cd to the binary scripts directory. In my case, cd /usr/share/elasticsearch. Make sure you type your own domain name or IP address after CN= in the following commands

 

bin/elasticsearch-certutil ca --ca-dn CN=your_domain_name

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 -name "CN=your_domain_name,OU=Consulting Team,DC=your_domain_name,DC=com"

At this point, you have obtained two files: elastic-certificates.p12 and elastic-stack-ca.p12. Now we are going to set up passwords for built-in users. Make sure the passwords are recorded and protected in a safe place. Once the password is set up, it won’t allow being recovered. Use bin/elasticsearch-setup-passwords auto to let the script generate the passwords or use /bin/elasticsearch-setup-passwords interactive to customize the passwords for each build-in users by yourself.

Use the following command to create a directory certs under the conf directory and copy elastic-certificates.p12 to this directory.

 

mkdir -p /etc/elasticsearch/certs

cp /usr/share/elasticsearch/elastic-certificates.p12 /etc/elasticsearch/certs

 

Add the following lines in the Elasticsearch configuration file. In my case, the path is /etc/elasticsearch/elasticsearch.yml .

 

http.host: 0.0.0.0 # accept request from remote

 

xpack.security.enabled: true

 

xpack.security.transport.ssl.enabled: true

 

xpack.security.transport.ssl.verification_mode: certificate

 

xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12

 

xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

 

xpack.security.http.ssl.enabled: true

 

xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12

 

xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12

 

xpack.security.http.ssl.client_authentication: optional

 

Restart Elasticsearch service to make sure the above changes take effect and no error occurred at this point.

 

Step 2: Enable cross-origin resource sharing for Elasticsearch

Enable CORS so that a browser on another origin can send requests to Elasticsearch. Add the following lines in the Elasticsearch configuration file. In my case, the path is /etc/elasticsearch/elasticsearch.yml .

 

Attention, for the setting http.cors.allow-origin , it should be the URL of the website which you are giving permission to connect. In my case, I will use "https://ai.query.ai" .

 

http.cors.enabled: true

 

http.cors.allow-origin: "https://ai.query.ai"

 

http.cors.allow-methods : OPTIONS, HEAD, GET, POST, PUT, DELETE

 

http.cors.allow-headers : Authorization, X-Requested-With,X-Auth-Token,Content-Type, Content-Length

 

Step 3: Encrypt communication between Kibana and web browser:

cd to certs folder under the Elasticsearch conf directory. Run the following commands to obtain the key file and certificate file from elastic-certificates.p12 .

 

openssl pkcs12 -in /elastic-certificates.p12 -out newfile.crt.pem -clcerts -nokeys

 

penssl pkcs12 -in ./elastic-certificates.p12 -out newfile.key.pem -nocerts -nodes

 

At this point two new files are obtained: newfile.crt.pem and newfile.key.pem . Create a directory named certs under Kibana conf folder, and copy these two files to certs directory. Add the following lines to the Kibana configuration file to enable HTTP SSL for Kibana. In my case, it is located at /etc/kibana/kibana.yml

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.key: /etc/kibana/certs/newfile.key.pem

server.ssl.certificate: /etc/kibana/certs/newfile.crt.pem

 

Step 4: Encrypt communication between Kibana and Elasticsearch:

See step1, we’ve obtained the password for user Kibana. Add these lines to the Kibana configuration file to enable TLS between Elasticsearch and Kibana:

elasticsearch.hosts: ["https://localhost:9200"] # https, not http

 

elasticsearch.username: "kibana"

 

elasticsearch.password: "XXXXXXXXX" # the password for user kibana

 

elasticsearch.ssl.verificationMode: none

 

Restart Kibana service, access the Kibana UI on your web browser. Make sure the Kibana UI shows up and is connected by https.

 

Step 5: Encrypt communication between Logstash and Elasticsearch:

There are configurations related to Elasticsaerch in Logstash’s pipeline. Since we’ve enabled https and built-in users for Elasticsearch. We should do the following changes in every pipeline config file in order to make the connection to Elasticsearch via https.

First, create a directory called certs under Logstash conf directory. In my case, it is located at /etc/logstash . Then, copy newfile.crt.pem to the certs directory.

Make changes on Elasticsearch output plugin for every Logstash pipeline

 

output {

 

elasticsearch {

 

hosts => "https://localhost:9200" #make sure it's https

 

index => "nginx-%{+YYYY.MM.dd}"

 

document_type => "nginx_logs"

 

user => logstash_system

 

password => xxxxxxxxx # obtained by step1

 

cacert => '/etc/logstash/certs/newfile.crt.pem' # generated from step3

 

ssl_certificate_verification => false

 

}

 

}

Restart Logstash in order for the changes to take effect.

 

All Done:

After the above changes, the communications among Elasticsearch, Kibana, and Logstash are encrypted. 

 

Interested in similar content? Follow our linkedin page!

 

Contact Us

If you like this content or have suggestions for other topics you’d like us to cover please let us know, we’d love to hear from you. You can reach us at contact@query.aiYou’re also free to follow us on linkedin and visit www.query.ai to subscribe to our updates.

Minxuan Sun

Written by Minxuan Sun

I am a software developer working at Query.AI ( http://query.ai ). Query.AI provides answers and insights on IT and cybersecurity data residing in log and SIEM platforms like Splunk and ELK / Elasticsearch.