概述
- 2019年04月25日 Elastic Stack 7.0.0 重磅发布
-
2019年10月02日 Elastic Stack 7.4.0 重磅发布
- 关系型数据库与 Elasticsearch 的抽象与类比
- 在7.0 版本之前,一个index 可以设置多个
Types
; - 7.0 以后一个索引只能创建一个
Type => "_doc"
- 在7.0 版本之前,一个index 可以设置多个
RDBMS | Elasticsearch |
---|---|
Table | Index(Type) |
Row | Document |
Column | Filed |
Schema | Mapping |
SQL | DSL |
关键特性
- 全新设计和导航……还有夜间模式
- Elasticsearch 集群协调迎来新时代
- 使用真实内存断路器提高节点弹性
- Elastic Stack 订阅 核心安全功能 免费、免费、免费!!
安装配置
- 详细的 安装配置说明可参考 Elasticsearch 5.x 集群配置
- Download 7.4.2
Elasticsearch
- 此处使用的版本为:
7.4.2
- 提示: OpenJDK 已经内置
$ES_PATH/jdk
JAVA (JVM)
系统配置
[root@jevic ~]# cat /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc unlimited
* hard nproc unlimited
es soft memlock unlimited
es hard memlock unlimited
[root@jevic ~]# cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.
* soft nproc 102400
root soft nproc unlimited
elasticsearch.yml
cluster.name: JevicTestDB
node.name: node194
node.master: true
node.data: true
path.data: /es-data1/data
bootstrap.memory_lock: true
network.host: 192.168.0.194
http.port: 9200
discovery.zen.minimum_master_nodes: 2
discovery.seed_hosts: ["node194", "node198", "node204"]
#discovery.seed_providers: unicast_hosts.txt
## 重试间隔
discovery.find_peers_interval: 1s
cluster.initial_master_nodes: ["node194", "node198", "node204"]
### sql插件支持
http.cors.enabled: true
http.cors.allow-origin: "*"
#### 必须停用xpack sql才可以使用elasticsearch-sql
xpack.sql.enabled: false
### 开启认证:
### 初始化认证: $ES_PATH/bin/elasticsearch-setup-passwords interactive
#xpack.security.enabled: true
Kibana
server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
### 开启认证
#xpack.security.enabled: true
#elasticsearch.username: "kibana"
#elasticsearch.password: "123456"
Logstash
- 收集 NGINX 日志
日志格式
log_format json '{"cip":"$remote_addr",'
'"timestamp":"$time_iso8601",'
'"rtime":$request_time,'
'"upres_time":$upstream_response_time,'
'"sbyte":$body_bytes_sent,'
'"host":"$http_host",'
'"request":"$request",'
'"scheme":"$scheme",'
'"length":"$content_length",'
'"server":"$upstream_addr",'
'"method":"$request_method",'
'"status":$status}';
pipe 配置
input{
file {
path => [ "/var/log/nginx/es_access.log" ]
start_position => "beginning"
codec => "json"
}
}
filter {
date {
match => [ "timestamp", "ISO8601" ]
target => "@timestamp"
}
mutate {
split => { "request" => " " }
add_field => { "url" => "%{[request][1]}"}
}
mutate {
remove_field => [ "path", "tags", "request"]
}
}
output {
#elasticsearch {
# hosts => [ "192.168.0.194:9200" ]
# index => "nginx-%{+YYYY.MM.dd}"
#}
stdout {
codec => rubydebug
}
}
- ps: filters-mutate
扩展阅读
- 如何使用 Elastic Stack 监测 Nginx Web 服务器
- 通过 Elasticsearch、Kibana 和 Beats 监测 Kafka
- 关于 elasticsearch-sql 和 cerebro 插件的安装部署请移步Github