Elastic Stack 5.0 GA Release Rollover API and Shrink API 令人振奮的提供了兩個對於 time-based index 更友善的API

[後記:目前 Elastic Stack 5.0 已經 GA , 文章寫於 2016.07.01 Elastic Stack Release – 5.0.0-alpha4 的興奮 ]

Elastic Stack Release – 5.0.0-alpha4 令人振奮的提供了兩個對於 time-based index 更友善的API 。

New Rollover API and Shrink API makes managing indices for time-based event data much easier

Rollover Index


Rollover Index | Elasticsearch Reference [master] | Elastic
https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-rollover-index.html

The rollover index API rolls an alias over to a new index when the existing index is considered to be too large or too old.
可以使用這個 rollover api 定義時間(max_age)或者是  資料的筆數 (max_docs) ,當條件符合時,就會根據規則自己建立新的 index 。
PUT /logs-0001 
{
  "aliases": {
    "logs_write": {}
  }
}

POST logs_write/_rollover 
{
  "conditions": {
    "max_age":   "7d",
    "max_docs":  1000
  }
}
 
 1.  建立一個新的 index 並指定 alias 為 logs_write
 2.  設定 當 index 被建立了七天,或是文件數超過 1000 就會建立新的 index 。並且命名為 logs-0002
這裡是 POST 的 response
{
  "old_index": "logs-0001",
  "new_index": "logs-0002",
  "rolled_over": true, 
  "dry_run": false, 
  "conditions": { 
    "[max_age: 7d]": false,
    "[max_docs: 1000]": true
  }
}

index naming 的規格就是 使用 – 分隔 ,後面的數字會自行遞增。

Shrink index API

Shrink Index | Elasticsearch Reference [master] | Elastic
https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-shrink-index.html
另外一個 api 就是 shrink api
The shrink index API allows you to shrink an existing index into a new index with fewer primary shards.

如果你想變更 primary shard 的設定,可以去自動的去幫你建立一個新的 index 。
Shrinking works as follows:
  • First, it creates a new target index with the same definition as the source index, but with a smaller number of primary shards.
  • Then it hard-links segments from the source index into the target index. (If the file system doesn’t support hard-linking, then all segments are copied into the new index, which is a much more time consuming process.)
  • Finally, it recovers the target index as though it were a closed index which had just been re-opened.

這邊有個值得注意的地方就是shrink 過的index 每個shard 不能超過 2,147,483,519 documents  。

  • The index must not contain more than 2,147,483,519 documents in total across all shards that will be shrunk into a single shard on the target index as this is the maximum number of docs that can fit into a single shard.
首先,要先確定cluster 的 health 是 green , index 要被改成 read only ,且要把index的所有shard 搬到同一台node 上才能進行。
這兩個要求可以透過下面的 request 達到。
index.blocks.write 設成 true 是還是允許 delete 的這種改變metadata的操作。
PUT /my_source_index/_settings
{
  "settings": {
    "index.routing.allocation.require._name": "shrink_node_name", 
    "index.blocks.write": true 
  }
}
當 shard 搬移完成後,可以就可以使用 shrink api
POST my_source_index/_shrink/my_target_index
 
要觀察這些狀態,可以使用   > curl XGET ‘localhost:9200/_cat/recovery?v’
elasticsearch 加上了這兩個 api ,
對於 index time base 的資料的使用者,方便性又更加的提升了。
廣告

發表迴響

在下方填入你的資料或按右方圖示以社群網站登入:

WordPress.com Logo

您的留言將使用 WordPress.com 帳號。 登出 / 變更 )

Twitter picture

您的留言將使用 Twitter 帳號。 登出 / 變更 )

Facebook照片

您的留言將使用 Facebook 帳號。 登出 / 變更 )

Google+ photo

您的留言將使用 Google+ 帳號。 登出 / 變更 )

連結到 %s