Monitoring Casual in Kyoto / Sensu + Graphite

Transcript

1. Sensu & Graphite Moden Monitoring Tool .POJUPSJOH
   $BTVBM
   5BML
   *O
   ,ZPUP
   JEZ@VVLJ

2. id:y_uuki

3. 2011 / 08 ͸ͯͳΠϯλʔϯ

4. 2012 / 09 ΞϧόΠτࢯ

5. 2013 / 07

6. 2013 / 09

7. 2013 / 11

8. 2013 / 12

9. None

10. In Kyoto

11. Monitoring

12. 
    NFUSJDT
    DPMMFDUJPO Today
    NFUSJDT
    WJTVBMJTBUJPO health check alert notification

13. Monitoring Tool

14. Monitoring Tool Requirement ✓ Scalable & Robust ✓ Pluggable ✓

    Programmable ✓ Friendly to immutable infrastructure ✓ Simple 

15. Case 1

16. None

17. ? Scalable & Robust - Munin 2.0 ? ✓ Pluggable

    - munin-node plugin ✗ Programmable - API ͳ͠ ✗ Friendly to immutable infrastructure - ઃఆϑΝΠϧʹ؂ࢹϊʔυͷϦετ͕ඞཁ (serf ͰͳΜͱ͔ͳΔ) ✗ Simple - because of RRDtool

18. Case 2

19. Collectd + Cacti + RRDtool

20. ✗ Scalable & Robust - because of RRDtool (cannot replicate)

    ✓ Pluggable - collectd input / output plugin (like fluentd) ✗ Programmable - Cacti ͸ API ͳ͠ ✗ Friendly to immutable infrastructure - Must configure node list ✗ Simple - because of RRDtool

21. Case 3

22. Graduate from

23. (SBQIJUF Scalable Realtime Graphing

24. Graphite Data Flow Graphite Webapp Carbon Whisper Collector TCP HTTP

    metrics Graph JSON Store Get Graphite

25. 3&45
    "1*
    - Graph rendering - JSON ! )JHI
    *OEFQFOEFODZ
    "SDIJUFDUVSF
    - Webapp

    Frontend - Storage

26. Replication / Sharding Carbon Relay Whisper metrics Carbon Whisper Carbon

    Whisper Carbon

27. Many Better Dashboards Graphite Webapp Carbon Whisper Collector TCP HTTP

    metrics JSON Store Get Graphite Dashboard Graphiti, Gdash, Pencil, Graphene…

28. 4FOTV A monitoring framework that aims! to be simple, malleable,

    and scalable

29. .POJUPSJOH
    3PVUFS
    - Checks plugin: ex. Apache is up or down

    - Handlers plugin: ex. sending alert e-mails ! )JHI
    *OEFQFOEFODZ
    "SDIJUFDUVSF
    - RabbitMQ - Redis - JSON

30. Sensu Data Flow sensu- client sensu- client RabbitMQ sensu server

    Redis sensu- client run checks checks run handlers store notification checks check request
31. None
32. None

33. More Detail • https://speakerdeck.com/joemiller/introduction- to-sensu • https://www.slideshare.net/jeremy_carroll/ sensu-14485155 • https://speakerdeck.com/joemiller/practical-

    examples-with-sensu-monitoring-framework

34. (SBQIJUF 4FOTV

35. Sensu & Graphite Data Flow sensu- client sensu- client sensu-

    client RabbitMQ sensu server Redis Graphite Webapp Carbon Whisper

36. ✓ Scalable & Robust - Maybe ✓ Pluggable - sensu

    plugin & sensu handler ✓ Programmable - Graphite: REST API & Carbon TCP API! - Sensu: Events API & Checks API … etc ✓ Friendly to Immutable Infrastructure - ؂ࢹϊʔυͷઃఆ͍Βͣ ✓ Simple - Sensu: Ruby, Redis, RabbitMQ, JSON! - Graphite: REST API

37. Graphite & RabbitMQ Integration

38. Sensu & Graphite Data Flow sensu- client sensu- client sensu-

    client RabbitMQ sensu server Redis Graphite Webapp Carbon Whisper

39. Stack Visualisation Storage Collector Collector Agent .VOJO $BDUJ $PMMFDUE (SBQIJUF

    4FOTV 33%UPPM

40. Other Solution Storage

41. OpenTSDB Kairos InfluxDB

42. Other Solution Collector

43. fluentd?

44. Integration with other tools

45. log health Infrastructure Information metrics Dynamic Static Host, Service, Role,

    Network, Location …

46. log health Integration Viewer metrics Dynamic Static Integration View Host,

    Service, Role, Network, Location …

47. log health Host, Service, Role, Network, Location … Integration with

    API metrics Dynamic Static Integration View JSON

48. Monitoring Tool Requirement ✓ Scalable & Robust ✓ Pluggable ✓

    Programmable ✓ Friendly to immutable infrastructure ✓ Simple 

49. ҟৗͳ౒ྗ͕ඞཁͳπʔϧʹݻࣥͤͣʹ ےͷྑ͍πʔϧΛબΜͰ͍͖͍ͨ

50. Graduate from

51. Sensu & Graphite Moden Monitoring Tool Monitoring Casual Talk In

    Kyoto id:y_uuki

52. Thanks