Monday, September 09, 2019

Setup the Kibana dashboards


PS D:\apps\metricbeat-7.3.1-windows-x86_64> .\install-service-metricbeat.ps1

Status   Name               DisplayName
------   ----               -----------
Stopped  metricbeat         metricbeat


PS D:\apps\metricbeat-7.3.1-windows-x86_64> .\metricbeat.exe modules enable system
Module system is already enabled


PS D:\apps\metricbeat-7.3.1-windows-x86_64> .\metricbeat setup
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards


PS D:\apps\metricbeat-7.3.1-windows-x86_64> .\metricbeat setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards


PS D:\apps\metricbeat-7.3.1-windows-x86_64> .\metricbeat setup -e `
>> -E output.logstash.enabled=false `
>> -E output.elasticsearch.hosts=['localhost:9200'] `
>> -E setup.kibana.host=localhost:5601

2019-09-09T16:43:49.594-0400    INFO    instance/beat.go:606    Home path: [D:\apps\metricbeat-7.3.1-windows-x86_64] Config path: [D:\apps\metricbeat-7.3.1-windows-x86_64] Data path: [D:\apps\metricbeat-7.3.1-windows-x86_64\data] Logs path: [D:\apps\metricbeat-7.3.1-windows-x86_64\logs]
2019-09-09T16:43:49.595-0400    INFO    instance/beat.go:614    Beat ID: 55ae7657-ff44-44bb-bf74-9960bd638650
2019-09-09T16:43:49.632-0400    INFO    [beat]  instance/beat.go:902    Beat info       {"system_info": {"beat": {"path": {"config": "D:\\apps\\metricbeat-7.3.1-windows-x86_64", "data": "D:\\apps\\metricbeat-7.3.1-windows-x86_64\\data", "home": "D:\\apps\\metricbeat-7.3.1-windows-x86_64", "logs": "D:\\apps\\metricbeat-7.3.1-windows-x86_64\\logs"}, "type": "metricbeat", "uuid": "55ae7657-ff44-44bb-bf74-9960bd638650"}}}
2019-09-09T16:43:49.632-0400    INFO    [beat]  instance/beat.go:911    Build info      {"system_info": {"build": {"commit": "a4be71b90ce3e3b8213b616adfcd9e455513da45", "libbeat": "7.3.1", "time": "2019-08-19T19:33:03.000Z", "version": "7.3.1"}}}
2019-09-09T16:43:49.634-0400    INFO    [beat]  instance/beat.go:914    Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.12.4"}}}
2019-09-09T16:43:49.671-0400    INFO    [beat]  instance/beat.go:918    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-09-07T18:49:28.22-04:00","name":"alex-win","ip":["fe80::b101:30ee:4cb:b72d/64","169.254.183.45/16","10.0.75.1/24","fe80::48a3:3e7f:6f2a:599e/64","169.254.89.158/16","fe80::4d94:82c9:a03b:d9dc/64","169.254.217.220/16","fe80::e0fb:3f9e:3ac0:174a/64","10.0.0.11/24","::1/128","127.0.0.1/8","fe80::c424:9412:b2b9:a826/64","192.168.160.113/28"],"kernel_version":"10.0.18362.329 (WinBuild.160101.0800)","mac":["f4:8e:38:b9:ab:21","00:15:5d:00:14:0c","16:53:30:8a:04:e5","26:53:30:8a:04:e5","94:53:30:8a:04:e5","00:15:5d:4b:14:d0"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"18362.329"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"a3d236c4-fca7-42ef-8ae4-5a253c1a1671"}}}
2019-09-09T16:43:49.675-0400    INFO    [beat]  instance/beat.go:947    Process info    {"system_info": {"process": {"cwd": "D:\\apps\\metricbeat-7.3.1-windows-x86_64", "exe": "D:\\apps\\metricbeat-7.3.1-windows-x86_64\\metricbeat.exe", "name": "metricbeat.exe", "pid": 13136, "ppid": 11296, "start_time": "2019-09-09T16:43:48.701-0400"}}}
2019-09-09T16:43:49.675-0400    INFO    instance/beat.go:292    Setup Beat: metricbeat; Version: 7.3.1
2019-09-09T16:43:49.678-0400    INFO    [index-management]      idxmgmt/std.go:178      Set output.elasticsearch.index to 'metricbeat-7.3.1' as ILM is enabled.
2019-09-09T16:43:49.679-0400    INFO    elasticsearch/client.go:170     Elasticsearch url: http://localhost:9200
2019-09-09T16:43:49.680-0400    INFO    [publisher]     pipeline/module.go:97   Beat name: alex-win
2019-09-09T16:43:49.682-0400    INFO    elasticsearch/client.go:170     Elasticsearch url: http://localhost:9200
2019-09-09T16:43:49.689-0400    INFO    elasticsearch/client.go:743     Attempting to connect to Elasticsearch version 7.3.1
2019-09-09T16:43:49.739-0400    INFO    [index-management]      idxmgmt/std.go:252      Auto ILM enable success.
2019-09-09T16:43:49.858-0400    INFO    [index-management]      idxmgmt/std.go:265      ILM policy successfully loaded.
2019-09-09T16:43:49.858-0400    INFO    [index-management]      idxmgmt/std.go:394      Set setup.template.name to '{metricbeat-7.3.1 {now/d}-000001}' as ILM is enabled.
2019-09-09T16:43:49.860-0400    INFO    [index-management]      idxmgmt/std.go:399      Set setup.template.pattern to 'metricbeat-7.3.1-*' as ILM is enabled.
2019-09-09T16:43:49.865-0400    INFO    [index-management]      idxmgmt/std.go:433      Set settings.index.lifecycle.rollover_alias in template to {metricbeat-7.3.1 {now/d}-000001} as ILM is enabled.
2019-09-09T16:43:49.866-0400    INFO    [index-management]      idxmgmt/std.go:437      Set settings.index.lifecycle.name in template to {metricbeat-7.3.1 {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2019-09-09T16:43:49.874-0400    INFO    template/load.go:169    Existing template will be overwritten, as overwrite is enabled.
2019-09-09T16:43:50.005-0400    INFO    template/load.go:108    Try loading template metricbeat-7.3.1 to Elasticsearch
2019-09-09T16:43:50.172-0400    INFO    template/load.go:100    template with name 'metricbeat-7.3.1' loaded.
2019-09-09T16:43:50.172-0400    INFO    [index-management]      idxmgmt/std.go:289      Loaded index template.
2019-09-09T16:43:50.175-0400    INFO    [index-management]      idxmgmt/std.go:300      Write alias successfully generated.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2019-09-09T16:43:50.181-0400    INFO    kibana/client.go:117    Kibana url: http://localhost:5601
2019-09-09T16:43:50.391-0400    INFO    kibana/client.go:117    Kibana url: http://localhost:5601
2019-09-09T16:43:52.632-0400    INFO    add_cloud_metadata/add_cloud_metadata.go:347    add_cloud_metadata: hosting provider type not detected.
2019-09-09T16:44:35.888-0400    INFO    instance/beat.go:776    Kibana dashboards successfully loaded.
Loaded dashboards



 

Saturday, September 07, 2019

Elastic Stack

Switch my elastic stash tutorial to Elastic site Getting started page.

Need to install MetricBeat.


Friday, September 06, 2019

Kibana Dev Tools console

Try to figure out what the following message for.

#! Deprecation: [types removal] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id}).


Thursday, September 05, 2019

Elastic Stack

Begin my elk tutorial.  (I found the tutorial of elastic search "type" was outdated.)

Anyway, download
elasticsearch-7.3.1-windows-x86_64.zip
kibana-7.3.1-windows-x86_64.zip
logstash-7.3.1.zip

Then unzip in d:\apps

I have them up
  • Elastic Search > localhost:9200
    To start, double click
    D:\apps\elasticsearch-7.3.1\bin\elasticsearch.bat

  • Kibana > localhost:5601
    To start double click
    D:\apps\kibana-7.3.1-windows-x86_64\bin\kibana.bat
  • logstash > localhost:9600
    Go to D:\apps\logstash-7.3.1
    In cmd, type bin\logstash -e "input {stdin{}} output{stdout{}}"




Monday, September 02, 2019

git first check in

Create a project in the github.com.
Let say I create a project "rss-admin"

PS D:\dvp\rssnews\rssproject> git init
Initialized empty Git repository in D:/dvp/rssnews/rssproject/.git/
PS D:\dvp\rssnews\rssproject> echo "#rss-admin" >> README.md
PS D:\dvp\rssnews\rssproject> git add --all
PS D:\dvp\rssnews\rssproject> git commit -m "first commit"
[master (root-commit) d3131a5] first commit
 31 files changed, 316 insertions(+)
 create mode 100644 README.md
 create mode 100644 db.sqlite3
 create mode 100644 manage.py
 create mode 100644 rss/__init__.py
 create mode 100644 rss/__pycache__/__init__.cpython-37.pyc
 ...
PS D:\dvp\rssnews\rssproject> git remote add origin https://github.com/ccapeng/rss-admin.git
PS D:\dvp\rssnews\rssproject> git remote -v
origin  https://github.com/ccapeng/rss-admin.git (fetch)
origin  https://github.com/ccapeng/rss-admin.git (push)
PS D:\dvp\rssnews\rssproject> git push -u origin master
...



Some notes :
1. Avoid to check python runtime file (*.pyc), before commit.
Create .gitignore like

__pycache__/
*.pyc
/env
*.bak

2. what if I enter the wrong git repository?
Try to correct it :
git remote set-url origin https://github.com/ccapeng/rss-admin.git