.. _elk-operator:

ELK service
===========

Software Factory bundles an ELK stack based on `Opendistro for Elasticsearch`_
to ease searching through the logs artifacts of jobs. Once activated,
the console log of every build is exported through logstash and
then searchable via Kibana.

A Software Factory user might want to export more artifacts
than the job's console. Indeed a job may generate additional
log files. In that case a custom zuul *post-run* job must be defined.
In order to do so a user must refer
to :ref:`Export logs artifacts to logstash <zuul-artifacts-export-logstash>`.

.. _`Opendistro for Elasticsearch`: https://opendistro.github.io/

How to activate
---------------

These services are not deployed by default but can be activated by adding
the following components in */etc/software-factory/arch.yaml*:

.. code-block:: yaml

 - elasticsearch
 - logstash
 - job-logs-gearman-client
 - job-logs-gearman-worker
 - kibana

Then running:

.. code-block:: bash

 # sfconfig

The Kibana interface should be accessible via the Software Factory top menu under
the name Kibana.


Manual actions after installing Software Factory
------------------------------------------------

After the installation of Software Factory, or the activation of ELK,
you must connect to Kibana as the `admin` user and go to the
section "Discover" in Kibana menu to ensure that Kibana has
created the index pattern. This is a per-requisite before a regular
Kibana user (which has read-only access) can search through the jobs logs.

.. note::

   After the first CI job execution, The job console logs are sent to
   ElasticSearch through Logstash. Then Kibana will be able to create the
   index pattern when connected as `admin`.

The password for `admin` is set as a `elasticsearch_password`
entry in `secrets.yaml file`.

You can also use below command:

.. code-block:: bash

   awk '/elasticsearch_password/ {print $2}' /var/lib/software-factory/bootstrap-data/secrets.yaml


Managing internal users
-----------------------

There are few users created in Elasticsearch in Software Factory.
The full list of available users can be seen :ref:`here <Available users>`

Users information are stored in `internal_users.yml` file which is
located in `opendistro_security config directory`:

.. code-block:: none

   /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/

For each user, password has been generated by sfconfig tool.
Not encrypted passwords are stored in:

.. code-block:: none

   /var/lib/software-factory/bootstrap-data/secrets.yaml

More information about the file, you can find `here`_.

.. _`here`: https://opendistro.github.io/for-elasticsearch-docs/docs/security/configuration/yaml/#internal_usersyml


.. _Available users:

Available users
---------------

There are few users created  in Software Factory for Elasticsearch:

- admin - the superuser in Kibana. It has all permissions to manage the Kibana and Elasticsearch cluster
- kibanaserver - this user is used by Kibana service to connect to the Elasticsearch
- logstash - dedicated user to communicate logstash service to the Elasticsearch
- curator - user that is used by curator service to 'clean-up' the index
- kibana - a read-only user. This user shows on the login page

For each user, password has been generated by sfconfig tool. You can find
them in:

.. code-block:: none

   /var/lib/software-factory/bootstrap-data/secrets.yaml


Resetting password
------------------

The best way to change the user password is to change it in the
`secrets.yaml` file and re-run sfconfig tool.

To do that without the sfconfig tool, you need to generate it by using this
script:

.. code-block:: bash

   /usr/share/elasticsearch/plugins/opendistro_security/tools/hash.sh -p <MY PASSWORD>

The generated hash should be added into the `internal_users.yml` file as the
"hash" property for the user in `opendistro_security config directory`.

.. note::

   To apply that change, you need to run `securityadmin.sh` script.
   Don't forget to put unencrypted password into the `secrets.yaml`
   file before executing sfconfig tool!


securityadmin.sh script
-----------------------

To apply changes, you need to execute the `securityadmin script`_.
It is the most important step that you should not forget after doing some
changes.

In the Software Factory, the command below will setup the `securityadmin.sh`
script. What you need to do is to execute it on the Elasticsearch host.

.. code-block:: bash

   /usr/share/elasticsearch/plugins/opendistro_security/tools/securityadmin.sh   \
     -cd /usr/share/elasticsearch/plugins/opendistro_security/securityconfig/  \
     -icl -nhnv -cacert /etc/elasticsearch/certs/localCA.pem  \
     -cert /etc/elasticsearch/certs/elasticsearch-admin.crt  \
     -key /etc/elasticsearch/certs/elasticsearch-admin.key    \
     -h $(hostname)

.. _`securityadmin script`: https://opendistro.github.io/for-elasticsearch-docs/docs/security/configuration/generate-certificates/#run-securityadminsh


Tenants
-------

Software Factory is only configuring one tenant: `global`.
Multi-tenancy is disabled (check Kibana configuration file).


Manage indices
--------------

Query the list and usage of index using:

.. code-block:: bash

   curl http://elasticsearch:9200/_cat/indices?v


Delete old/unused index using:

.. code-block:: bash

   curl -X DELETE http://elasticsearch:9200/INDEX-NAME