Snapshot management for elastic cloud enterprise

@toc

3 month 23 Number ,Elastic The latest 7.12 edition . In this version , The most important update is frozen tier Release . Compared with the previous version of cold tier（ About cold tier The details of the , You can check the previous blog posts ：Elastic Searchable snapshot A preliminary study of the function of 、Elastic Searchable snapshot A preliminary study of the function of Two （hot phase））, The biggest difference is that we can search the data directly in the object store , That is, we can keep the snapshot data in the object storage all the time Available online , By building a small-scale , A computing cluster with only basic storage , You can view the massive data saved in the snapshot ！ Achieve real separation of computing and storage , And greatly reduce the cost of looking up huge historical frozen data and improve the query efficiency .（ Refer to the official blog ： Search directly with the new frozen layer S3）

High energy picture ahead ：

Insert picture description here

A single node " mount "1PB data , Local disk usage 1.7%, Only a few computing resources and local storage resources can be used to query massive data .

To do that , There are several premises ：

• Need to have Elastic Of Enterprise Level subscription
• There are already available object stores for the snapshot repository

And when we use ECE（Elastic Cloud Enterprise） When creating a cluster , It has naturally met the first demand . What we have to do is pass ECE To manage our snapshot Repository , And build an index and searchable snapshot The relationship between , We will show you how to do this in a series of articles .

In this paper , Let's take a quick look ECE Snapshot management function on .

Processing snapshots

Snapshot for your Elasticsearch The index provides backup . When there are not enough zones to provide high availability , Snapshots can be used to recover from failures , You can also recover from accidental deletion .

Before you start

For your Elasticsearch The cluster enables snapshots and uses snapshots , You must have configured the repository . After configuring the snapshot Repository , Every time 30 Take a snapshot every minutes or at an interval you specify .

Be careful ： Snapshots back up only open indexes . If you close the index , The index will not be included in the snapshot , So you won't be able to restore data .

from Elastic Stack 7.6 Version start , Use Kibana To manage snapshots . stay Kibana in , You can set up other repositories to store snapshots , Rather than by Elastic Cloud Enterprise Currently managed repositories . You can view and delete snapshots , And configure snapshot lifecycle management （SLM） Policy to automatically execute when creating and deleting snapshots .

stay Elastic Cloud Enterprise in , You can view Snapshot list , You can also deploy from other sites in the same region Restore snapshot .

Configure snapshot warehouse

The snapshot repository is for the entire Elastic Cloud Enterprise Installation management , Create or manage snapshot repositories for Elasticsearch The cluster specifies the snapshot repository . After specifying the repository , By default, or the specified time interval, the default is every 30 Take a snapshot every minutes .

adopt Elastic The core repository plug-in provides support for the following repositories ：

• Amazon S3
• Microsoft Azure Storage
• Google cloud storage

Add snapshot warehouse configuration

You can be right here Elasticsearch Before the cluster performs any snapshot or restore operations , At least one snapshot repository configuration needs to be added to your Elastic Cloud Enterprise Installing .

To add a snapshot Repository ：

• Log in to Cloud UI.
• from “Platform” The menu , choice “Repositories”.
• single click "Add Repository" To add an existing snapshot Repository .
• Provide the name of the warehouse configuration .
• Select a supported warehouse type , Then specify the necessary settings ： - Amazon S3 To configure ： All warehouse options must be specified , Because there is no default . - region The area where the bucket is located . - bucket Name of the bucket used for the snapshot . - access key Access key for authentication . - secrect key The key used for authentication . - GCS To configure ： except S3 outside , Other configurations require the use of json How to configure ```json { "type": "gcs", "settings": { "bucket": "lex-demo-bucket", "client": "my_alternate_client" } } ```

-  Need to access GCP Of service account Configure an access credential 
-  And then through deployment -> Security, Configure the access credentials  ( Be careful , there client name , Must be with Security key The names inside match ：`my_alternate_client`)
  

   
   

     Insert picture description here 
   
  

  

   
   

     Insert picture description here 
   
  

Create a cluster and install the corresponding plug-ins

It can be installed at the time of creation ：

Insert picture description here

It can also be modified after installation ：

Insert picture description here

After the snapshot plug-in is installed , Can be in the cluster Kibana notice ECE Snapshot repository created for you （found-snapshots）：

Insert picture description here

You can back up the index to the snapshot repository , You can also register a new snapshot repository to distinguish ECE Automatic backup of

Create snapshot policy

after , stay Kibana Create snapshot policy on , Data can be backed up to ECE Created found-snapshots Warehouse

Insert picture description here

Be careful , there name, If it's named cloud-snapshot-policy, Can be in ECE Manually create snapshots on the interface of

Insert picture description here

View snapshots

We can do it in ECE View all available snapshots on the interface of ：

Insert picture description here

expectation

adopt ECE Manage a unified snapshot Repository , We can do , When multiple clusters generate snapshots through the warehouse , adopt searchable snapshot The function of , We can do this by building a small-scale , A computing cluster with only basic storage , You can view the massive data stored in the snapshot warehouse of multiple clusters ！