Installation and use of logstash

Logstash It is a powerful data processing tool , It can realize data transmission , Format processing , Format output , There are also powerful plug-in functions , Commonly used for log processing

explain ：logstash Collect diary for comparison Filebeat Very slowly , So collect diaries and use Filebeat, Don't have to logstash

One 、 download

        Address ：https://www.elastic.co/cn/downloads/logstash

Two 、 install

        Depend on the environment ：jdk1.8+

        Upload and unzip the installation package

        tar -xvf logstash-7.13.3-linux-x86_64.tar.gz

        

        

  3、 ... and 、 verification

        Get into bin perform logstash Start command , Let's go to the first logstash Example

        # bin/logstash -e 'input { stdin { } } output { stdout { } }'
        

        explain ： This example is to do any processing . Output what you input .

            After starting , Input hello And output

          

  Four 、 To configure

        The configuration has three parts as follows

         

        1、input File read and filter Handle

             【1】 create profile

                vim itcast-log.conf

                Content

                

input{
  file{
    path => "/opt/cc.log"
    start_position => "beginning"
  }
}

filter{
  mutate {
    split => {"message"=>"|"}
  }
}

output{
  stdout { codec => rubydebug }
}

                 【2】： establish cc.log, Prepare for the presentation

                        

                 【3】 start-up

                        command ：./bin/logstash -f itcast-log.conf 

                        

                 【4】 After startup, you will see that the log contents have been obtained , And has been processed

                         

         2、 Output to Elasticsearch

                Modify profile output modular ,hosts Is an array , Cluster configuration is separated by commas , Just add .

                output{
                    elasticsearch{
                        hosts => ["192.168.231.146:9200"]
                    }
                }

                 

                stay cc.log Add a new record to

                 

                  see （ View using kibana）