Elasticsearch: Adding DigitalOcean Space as a Snapshot Repository

Step 1: Add DigitalOcean Spaces Credentials to Elasticsearch Keystore

Login to each node in the Elasticsearch Cluster and add DigitalOcean Spaces access and secret keys to the Elasticsearch keystore:

docker exec -it elasticsearch-docker-node-1 /usr/share/elasticsearch/bin/elasticsearch-keystore add s3.client.default.access_key
docker exec -it elasticsearch-docker-node-1 /usr/share/elasticsearch/bin/elasticsearch-keystore add s3.client.default.secret_key

Step 2: Restart Elasticsearch

Restart Elasticsearch for the changes to take effect:

curl -u "user:password" -X POST "http://localhost:9200/_nodes/reload_secure_settings" -H 'Content-Type: application/json' -d'
{
   "secure_settings_password":""
}'

or

docker-compose restart

Step 3: Register the Snapshot Repository

Register the snapshot repository by sending a PUT request:

curl -u "user:password" -X PUT "http://localhost:9200/_snapshot/test-elasticsearch-space?pretty" -H 'Content-Type: application/json' -d'
{
    "type": "s3",
    "settings": {
        "bucket": "test-elasticsearch-space",
        "base_path": "elk-snapshots",
        "endpoint" : "fra1.digitaloceanspaces.com"
    }
}
'

where:

• bucket – name of the storage bucket in DigitalOcean Spaces where snapshots will be stored.

• base_path – the directory path within the bucket where snapshots will be stored.

• endpoint – the regional endpoint for DigitalOcean Space

Expected output:

{
  "acknowledged" : true
}

Step 4: Verify Snapshot Repository

Check if the repository is registered:

curl -u "user:password" -X GET "http://localhost:9200/_snapshot?pretty"

Delete the Snapshot Repository in Elasticsearch

• all snapshot repositories:

  curl -u "user:password" -X DELETE "http://localhost:9200/_snapshot/*"

• specific repo:

  curl -u "user:password" -X DELETE "http://localhost:9200/_snapshot/test-elasticsearch-space"

Troubleshooting

• verify access and secret key:

  docker exec -it elasticsearch-docker-node-1 /usr/share/elasticsearch/bin/elasticsearch-keystore show s3.client.default.access_key
  docker exec -it elasticsearch-docker-node-1 /usr/share/elasticsearch/bin/elasticsearch-keystore show s3.client.default.secret_key

• mount a DigitalOcean Space to a Droplet, in order to check permissions and availability:

  • Install s3fs-fuse:

    sudo apt update
    sudo apt install -y s3fs
    s3fs --version

  • Store credentials in a file:

    echo "ACCESS_KEY:SECRET_KEY" > ~/.passwd-s3fs
    chmod 600 ~/.passwd-s3fs

  • Create a Mount Point:

    sudo mkdir /mnt/do-space

  • Mount the Space

    s3fs test-elasticsearch-space /mnt/do-space -o url=https://fra1.digitaloceanspaces.com -o use_path_request_style -o allow_other

  • Verify the Mount

    df -h
    ls /mnt/do-space
    mount

  • command for debugging:

    s3fs test-elasticsearch-space /mnt/do-space -o url=https://fra1.digitaloceanspaces.com -o use_path_request_style -o allow_other -o dbglevel=info -f -o curldbg