首页 » 系统运维 » 日志系统 » 正文

CentOS7.3 64位安装部署ELK5.5.2

ELK不用过多介绍,分别是elasticsearch,logstash,kibana.

下面直入主题,开始安装。

 

环境:

系统:阿里云CentOS7.3 64位

配置:1核8G+100G磁盘(建议内存8G以上)

 

资料推荐:

官方书籍《Elasticsearch权威指南》中文版:

https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html

 

修改主机名为:

# vim /etc/hostname

修改为elk.chenxie.net

 

创建所需目录:

# mkdir /data/{app,software}

 

安装jdk1.8

# cd /data/software/

# wget http://qooco-software.oss-cn-beijing.aliyuncs.com/Java%E6%8C%81%E7%BB%AD%E9%9B%86%E6%88%90/jdk-8u77-linux-x64.gz

# tar -zxvf jdk-8u77-linux-x64.gz

# mv jdk1.8.0_77/ /data/app/

# cd /data/app/

# cd /data/app/

 

配置环境变量:

# vim /etc/profile

添加以下:

export JAVA_HOME=/data/app/java/

export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

 

安装elasticsearch

添加用户:

# useradd elasticsearch

 

# cd /data/software/

# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.2.tar.gz

# tar -zxvf elasticsearch-5.5.2.tar.gz

# mv elasticsearch-5.5.2 /data/app/

# cd /data/app/elasticsearch-5.5.2/

 

配置:

# vim /data/app/elasticsearch-5.5.0/config/elasticsearch.yml

cluster.name: qooco-elk
node.name: node-1
path.data: /data/es-data
path.logs: /data/es-logs
network.host: 172.16.1.251
http.port: 9200
discovery.zen.ping.unicast.hosts: ["172.16.1.251"]

 

创建用于存储es数据和日志的目录:

# mkdir /data/es-data

# mkdir /data/es-logs

 

# vim /etc/security/limits.conf

添加:

* – nofile 65536

* – memlock unlimited

 

# vim /etc/sysctl.conf

添加:

vm.max_map_count=262144

重启系统生效。


授权目录所有者为elasticsearch

# chown -R elasticsearch.elasticsearch /data/app/elasticsearch-5.5.2/

 

启动:

# su – elasticsearch

$ cd /data/app/elasticsearch-5.5.2/bin/

$ ./elasticsearch

 

验证:

# curl http://172.16.1.251:9200/
{
  "name" : "node-1",
  "cluster_name" : "chenxie-elk",
  "cluster_uuid" : "VcmKlt78RU6-TKysZVtiIQ",
  "version" : {
    "number" : "5.5.2",
    "build_hash" : "b2f0c09",
    "build_date" : "2017-08-14T12:33:14.154Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

 

安装nodejs环境用于安装head插件

# cd /data/software/

# wget http://cdn.npm.taobao.org/dist/node/v8.4.0/node-v8.4.0-linux-x64.tar.xz

# tar -xJf node-v8.4.0-linux-x64.tar.xz

# mv node-v8.4.0-linux-x64 /data/app/

 

配置环境变量:

# set for nodejs

export NODE_HOME=/data/app/node-v8.4.0-linux-x64/

export PATH=$NODE_HOME/bin:$PATH

 

安装cnpm:

# npm install -g cnpm –registry=https://registry.npm.taobao.org

 

安装grunt:

npm install -g grunt-cli

 

# grunt -version

grunt-cli v1.2.0

 

当我们启动grunt server后,命令窗口会被占用。

可以使用grunt server &让其在后台启动。或者再开一个终端窗口接着其他操作。

 

安装elasticsearch head插件

# cd /data/app/

来自官方步骤:

# git clone      git://github.com/mobz/elasticsearch-head.git

# cd elasticsearch-head

# npm install

# npm run start

open http://localhost:9100/

This will start a local webserver running on port 9100 serving elasticsearch-head

 

修改head的连接地址:

# vim /data/app/elasticsearch-head/head/_site/app.js

this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";

把localhost修改成你es的服务器地址,如:

this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://172.16.1.251:9200";

 

安装logstash

# cd /data/software/

# wget https://artifacts.elastic.co/downloads/logstash/logstash-5.5.2.tar.gz

# tar -zxvf logstash-5.5.2.tar.gz

# mv logstash-5.5.2 /data/app/

 

配置及使用待更新…

 

 

安装kibana

# cd /data/software/

# wget wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.2-linux-x86_64.tar.gz

# tar -zxvf kibana-5.5.2-linux-x86_64.tar.gz

# mv kibana-5.5.2-linux-x86_64 /data/app/

 

配置:

# cd /data/app/kibana-5.5.2-linux-x86_64/config/

# vim kibana.yml

server.port: 5601
server.host: "172.16.1.251"
elasticsearch.url: http://172.16.1.251:9200
kibana.index: ".kibana"

 

启动:

# cd /data/app/kibana-5.5.2-linux-x86_64/bin/

#./kibana

访问地址:http://172.16.1.251:5601/

 

安装Nginx用于kibana登录的权限认证

# cd /data/software/

# yum -y install gcc pcre-devel openssl openssl-devel make

# wget http://nginx.org/download/nginx-1.12.1.tar.gz

# tar -zxvf nginx-1.12.1.tar.gz

# cd nginx-1.12.1

# ./configure –prefix=/data/app/nginx –with-http_ssl_module –with-http_stub_status_module

# make && make install

 

配置:

# cat /data/app/nginx/conf/nginx.conf
#
worker_processes  4;
error_log  logs/error.log;

events {
    use epoll;
    worker_connections  65535;
}

worker_rlimit_nofile 102400;

http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format json   '{ "@timestamp": "$time_iso8601", '
                        '"@fields": { '
                        '"time_local": "$time_local", '
                        '"remote_addr": "$remote_addr", '
                        '"remote_user": "$remote_user", '
                        '"request_method": "$request_method", '
                        '"request": "$request", '
                        '"status": "$status", '
                        '"body_bytes_sent": "$body_bytes_sent", '
                        #'"request_body": "$request_body", '
                        '"http_referrer": "$http_referer", '
                        '"http_user_agent": "$http_user_agent", '
                        '"http_x_forwarded_for": "$http_x_forwarded_for", '
                        '"upstream_addr": "$upstream_addr", '
                        '"request_time": "$request_time", '
                        '"upstream_response_time": "$upstream_response_time" } }';

    access_log  logs/access.log  json;
    
    include vhost/*.conf; 

    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

 

# mkdir /data/app/nginx/conf/vhost/

# cat /data/app/nginx/conf/vhost/elk.chenxie.net.conf
#
server {
        listen       80;
        server_name     elk.chenxie.net;

        access_log      logs/elk.chenxie.net/elk.chenxie.net_access.log   json;
        error_log       logs/elk.chenxie.net/elk.chenxie.net_error.log;

        auth_basic "Restricted Access";
        auth_basic_user_file /data/app/nginx/conf/vhost/htpasswd.users;

        location / {
                proxy_pass http://172.16.1.251:5601;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection 'upgrade';
                proxy_set_header Host $host;
                proxy_cache_bypass $http_upgrade;
        }
}

 

添加用户:

# /data/app/nginx/conf/vhost/

# htpasswd  -c htpasswd.users alex

New password: 

Re-type new password: 

Adding password for user alex

 

启动nginx:

# /data/app/nginx/sbin/nginx

 

现在就可以使用80端口访问kibana并实现用户认证了,登录地址:http://172.16.1.251/

 

如有问题,欢迎指正…



发表评论

8 + 1 =