Elastic Tutorials: Collects Docker metrics and logs using Elastic Agent

This Integration collects metrics and logs from Docker containers. The default data streams for metrics collection are: container, cpu, diskio, healthcheck, info, memory and network. The image metricset is not enabled by default. The container_logs data stream for containers’ logs collection is enabled by default.

Compatibility

The Docker module is currently tested on Linux and Mac with the community edition engine, versions 1.11 and 17.09.0-ce. It is not tested on Windows, but it should also work there.

Step 1 – Install Docker

Step 2 – Enable Integration

Step 3 – Work on Agent Configuration changes

$ cd /opt/Elastic/Agent
$ systemctl stop elastic-agent
# Replace the content of elastic-agent.yml which you recieved from Kibana.
$ vi elastic-agent.yml
$ systemctl restart elastic-agent
$ systemctl status elastic-agent

Step 4 – Create docker containers

$  docker run -itd ubuntu

Step 5 – Verify Containers

Running from within Docker

The docker Integration will try to connect to the docker socket, by default at unix:///var/run/docker.sock. If Elastic Agent is running inside docker, you'll need to mount the unix socket inside the container:

docker run -d \
  --name=metricbeat \
  --user=root \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  docker.elastic.co/beats/metricbeat:latest metricbeat -e \
  -E output.elasticsearch.hosts=["elasticsearch:9200"]

For log collection since the discovery of the containers happen automatically, again access to unix:///var/run/docker.sock will be needed so as Agent to be able to watch for Container events. In addition, access is required to the containers' logs files which by default follows the pattern of /var/lib/docker/containers/${docker.container.id}/*-json.log If Elastic Agent is running inside docker, you'll need to mount the logs' directory too inside the container:

docker run -d \
  --name=metricbeat \
  --user=root \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" \
  docker.elastic.co/beats/metricbeat:latest metricbeat -e \
  -E output.elasticsearch.hosts=["elasticsearch:9200"]

In all cases make sure that Agent has the proper permissions to access these files.

• Author
• Recent Posts

Follow me

Rajesh Kumar

Mentor for DevOps - DevSecOps - SRE - Cloud - Container & Micorservices at Software AG

Join my following certification courses...
- DevOps Certified Professionals (DCP)
- Site Reliability Engineering Certified Professionals (SRECP)
- Master in DevOps Engineering (MDE)
- DevSecOps Certified Professionals (DSOCP)
URL - https://www.devopsschool.com/certification/

My Linkedin - https://www.linkedin.com/in/rajeshkumarin

Follow me

Latest posts by Rajesh Kumar (see all)

• Best AI tools for Software Engineers - November 4, 2024
• Installing Jupyter: Get up and running on your computer - November 2, 2024
• An Introduction of SymOps by SymOps.com - October 30, 2024