MetricSifter：クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024

Transcript

1. MetricSifterɿΫϥ΢υΞϓϦέʔγϣϯʹ͓͚Δ ނোՕॴಛఆͷޮ཰ԽͷͨΊͷ ଟมྔ࣌ܥྻσʔλͷಛ௃ྔ࡟ݮ FIT2024 τοϓίϯϑΝϨϯεηογϣϯ7-3 ηΩϡϦςΟɾωοτϫʔΫ ௶಺ ༎थɹ௽ా തจʢ͘͞ΒΠϯλʔωοτʣ 2024೥9݄6೔

2. 2 1. Introduction 2. Failure-oriented Feature Reduction Framework 3. Evaluation

   4. Conclusion ໨࣍ Tsubouchi, Y., Tsuruta, H.: MetricSifter: Feature Reduction of Multivariate Time Series Data for Efficient Fault Localization in Cloud Applications, IEEE Access, Vol.12, pp.37398-37417 (2024). ʲݪൃදͷॻࢽ৘ใʳ Ҏ߱ɺ[Tsubouchi+,ACCESS24]ͱදه

3. 3 1. Introduction 2. Failure-oriented Feature Reduction Framework 3. Experiment

   and Discussion 4. Conclusion Introduction

4. ςϨϝτϦγεςϜ ϝτϦΫε ࣌ܥྻͷ ਺஋σʔλ ΦϖϨʔλʔ 4 Ϋϥ΢υͷނোಛఆͷδϨϯϚ Ϋϥ΢υ Πϯλʔωοτ ΞϓϦέʔγϣϯ

   ෼ࢄγεςϜͱͯ͠ෳࡶԽ ো֐ͷ਍அͷͨΊͷ ςϨϝτϦ͕ॏཁͱͳ͍ͬͯΔ ࣗಈނোಛఆ[9-24] ಛఆࣗಈԽ ᶃ ϝτϦΫε਺͕૿େ ෆཁσʔλࠞೖʹΑΔ ಛఆਫ਼౓ͱ଎౓௿Լ δϨϯϚ ᶄ [25]

5. 5 δϨϯϚͷղফʹ޲͚ͯ ࣗಈނোಛఆ[9-24] ϝτϦΫε ࣌ܥྻͷ ਺஋σʔλ ΦϖϨʔλʔ ಛ௃࡟ݮ ڭࢣͳ͠ɾਖ਼ৗ࣌σʔλΛ൒ڭࢣͱ͢Δ ܰྔͳ౷ܭղੳɺػցֶशͳͲ

   ଟมྔ࣌ܥྻσʔλ ෆཁͳϝτϦΫεʢ࣌ܥྻʣͷ ݸ਺Λ࡟ݮ ࠜຊݪҼ ϥϯΩϯά [25,26]

6. 6 ࣌ܥྻதͷҟৗͷ༗ແʹண໨ طଘͷಛ௃࡟ݮͱͦͷ՝୊ ࣌ܥྻͷྨࣅੑ΍ؔ࿈ੑʹண໨ [14,23,26] [9,12,16,25] ୯Ұͷάϩʔόϧͳʮো֐ʯ ΁ͷؔ࿈ੑΛଊ͍͑ͨ ҟৗੑʹجͮ͘࡟ݮ ৑௕ੑʹجͮ͘࡟ݮ

   ো֐࣌ؒ֎ͷҟৗΛ࡟ ݮͰ͖ͳ͍ʢِӄੑʣ ো֐ؔ࿈ϝτϦΫεؒͰྨ ࣅ͢Δͱޡ࡟আʢِཅੑʣ ϝτϦΫεϩʔΧϧͷ ҟৗੑ΍৑௕ੑʹىҼ

7. 7 ؍࡯ͱԾఆ [Tsubouchi+,ACCESS24] FIGURE 1. ΑΓసࡌ ԣ࣠160͕ো֐ൃੜ࣌ࠁ ނোىҼͷมԽ఺͸ ͍ۙ࣌ؒʹݱΕΔ ؍࡯

   ϩʔΧϧΠϕϯτ มԽ఺͕࠷΋ภΔ࣌ؒൣғ͕ɺো֐ظؒͱͳΔ Ծఆ άϩʔόϧΠϕϯτ

8. 8 ఏҊɿো֐ࢦ޲ͷಛ௃࡟ݮ MetricSifter 2. άϩʔόϧΠϕϯτͱͯ͠ʮো֐ͷ࣌ؒൣғʯΛಛఆ͢Δ มԽ఺࣌ؒͷ෼෍ͷ࠷େͷๆ 1. ϩʔΧϧΠϕϯτͱͯ࣌͠ܥྻ͝ͱʹʮมԽ఺ʯΛݕग़͢Δ 3. ʮো֐ͷ࣌ؒൣғʯʹมԽ఺͕

   ͋Δ → อ࣋ ͳ͍ → ࡟আ t

9. 2. Failure-oriented Feature Reduction Framework

10. 10 MetricSifter͸ͲͷΑ͏ʹಈ࡞͢Δ͔ʁ [Tsubouchi+,ACCESS24] FIGURE 5. ΑΓసࡌ STEP 2:มԽ఺࣌ؒͷ෼෍ ΛجʹηάϝϯτΛ෼ׂ STEP

    1:࣌ܥྻ͝ͱʹɺ ނো༝དྷͷมԽ఺ީิ Λݕग़ STEP3: ࠷େີ౓ͷηά ϝϯτΛબ୒

11. 11 STEP 1: ୯มྔ࣌ܥྻͷมԽ఺ݕग़ ᶃ ίετؔ਺ɿݕग़͢ΔมԽͷछྨΛબ୒ ઃܭํ਑ɿυϝΠϯʹదͨ͠ɺมԽ఺ݕग़ͷطଘख๏[48]Λબ୒͢Δ ᶄ ୳ࡧ๏ɿมԽ఺Λ୳ͨ͢ΊͷΞϧΰϦζϜ ᶅ

    ϖφϧςΟ߲ɿݕग़͢ΔมԽ఺ͷ਺ʹ੍໿Λ͔͚Δ L2 ʢฏۉγϑτʣ PeltɿݫີղΛٻΊΔ͕৚݅෇͖ͰࢬמΓߴ଎ԽՄ BICʹج͖ͮώϡʔϦεςΟοΫʹܾఆɻͨͩ͠ಠࣗͷዞҙతͳ܎਺ Λ௥Ճɻ ω

12. 12 ᶃ ີ౓෼෍ͷਪఆ Χʔωϧີ౓ਪఆ๏ʢKDEʣΛ༻͍ͯ ཭ࢄܕͷ෼෍ີ౓Λੜ੒ STEP 2: มԽ఺ͷີ౓෼෍ਪఆͱηάϝϯςʔγϣϯ [Tsubouchi+,ACCESS24] FIGURE

    6. ΑΓసࡌ STEP 3: ࠷େͷηάϝϯτ ͱͯ͠બ୒ ᶄ ηάϝϯςʔγϣϯ ہॴ࠷খ఺ʹڥքઢΛҾ͘ ʢFig.6͸10ݸͷηάϝϯτʹ෼ׂʣ

13. 13 1. Introduction 2. Failure-oriented Feature Reduction Framework 3. Evaluation

    4. Conclusion Evaluation

14. 14 Q1: ಛ௃ྔ࡟ݮਫ਼౓͸Ͳͷఔ౓ྑ͍ͷ͔ʁ Q2: ނোಛఆੑೳΛͲͷఔ౓޲্ͤ͞Δ͔ʁ Q3: ύϥϝʔλʹͲͷఔ౓හײ͔ʁ (Parameter Sensitivity) Q4:

    ఏҊ๏ͷ֤STEP͕Ͳͷఔ౓ੑೳʹد༩͢Δ͔ʁ (Ablation Study) ධՁ

15. 15 σʔληοτ [Tsubouchi+,ACCESS24] TABLE 4. Λվม ߹੒σʔλ [58]Λ༻͍ͯো֐ΛγϛϡϨʔτ͠ ͨଟมྔ࣌ܥྻͱDAGΛੜ੒ɻ ࣮ূσʔλ

    ΞϓϦ αʔϏε਺ ނো਺ ϝτϦΫε਺ SS-small Sock Shop(SS) 7 90 64 SS-medium 184 SS-large 1312 TT-small Train Ticket(TT) 41 42 383 TT-medium 1349 TT-large 9458 ఆ൪ͷϕϯνϚʔ ΫΞϓϦʹɺCPU ·ͨ͸ϝϞϦͷա ৒࢖༻ނোΛ஫ೖ ͯ͠࠾औɻ ϊʔυ਺ Τοδ਺ 50 100 200 100 500 700 D50,100 sim D50,200 sim D200,500 sim D200,700 sim

16. 16 Q1: ಛ௃ྔ࡟ݮਫ਼౓͸Ͳͷఔ౓ྑ͍ͷ͔ʁ (c) Balanced accuracy [Tsubouchi+,ACCESS24] FIGURE 7. (c)

    ΑΓసࡌ MetricSifterͷฏۉਫ਼౓0.981Ͱ ࠷ྑ஋Λࣔͨ͠ɻ ৑௕࡟ݮάϧʔϓ͸ɺ૯ͯ͡ ௿είΞͱͳͬͨɻ ಺Ͱ࣌ܥྻ͕ྨࣅɾ૬ؔ ͢Δ΋ͷ͕࡟আ͞ΕΔͨΊɻ MA ∪ MB

17. PC+HT ϥϯμϜબ୒ 17 Q2: ނোಛఆੑೳΛͲͷఔ౓޲্ͤ͞Δ͔ʁ Ұ෦ൈਮ ૯߹ධՁɹ MetricSifter͕ ཧ૝ख๏ʹ ͍ۙਫ਼౓Λୡ੒

    ख๏ ਫ਼౓ උߟ Ideal 0.344 ཧ૝஋ MetricSifter 0.299 ࠷ྑ NSigma 0.241 ࣍఺ None 0.175 w /o ಛ௃࡟ݮ શނোಛఆ๏ͱͷ૊Έ߹ͤʹ ର͢Δtop-5ਫ਼౓ͷฏۉ஋

18. 18 Q2: ࣮ূσʔληοτ [Tsubouchi+,ACCESS24] FIGURE 11. (a) ΑΓҰ෦ൈਮͯ͠సࡌ -small SS

    64 metrics όʔ͕ਫ਼౓ ંΕઢ͕࣮ߦ࣌ؒ - top-5ਫ਼౓͸MetricSifter͕࠷ྑͰɺ࣮ߦޮ཰͸ҟৗੑ࡟ݮΑΓ΋ߴ͍ - ࣮ߦ࣌ؒ͸৑௕ੑ࡟ݮʢHDBS-SBD/HDBS-Rʣ͕࠷ྑ͕ͩਫ਼౓͸࠷΋௿͍

19. 19 Q2: ࣮ূσʔλৄࡉʢେن໛ >100 metricsʣ -medium SS -large SS -small

    TT -medium TT 184 metrics 1312 383 1349 [Tsubouchi+,ACCESS24] FIGURE 11. (b) ΑΓҰ෦ൈਮͯ͠సࡌ RCDͷΈ͕ݱ࣮తͳ࣌ؒ಺ʢ3600ඵҎ಺ʣͰॲཧΛऴ͑ͨ - ଞ͸ɺނোಛఆΞϧΰϦζϜʹฒྻੑ͕ͳ͍ͨΊ ϝτϦΫε਺>1000Ͱ͸ɺಛ௃࡟ݮͷ༗ແʹ͔͔ΘΒͣɺ ඇৗʹ௿͍ਫ਼౓ͱͳͬͨ

20. 20 1. Introduction 2. Failure-oriented Feature Reduction Framework 3. Experiment

    and Discussion 4. Conclusion Conclusion

21. 21 ɾಛ௃࡟ݮͷఆྔతͳൺֱධՁΛߦͬͨॳͷݚڀ ɾϩʔΧϧͷมԽ఺͔Βάϩʔόϧͳো֐Λଊ͑Δಛ௃࡟ݮMetricSifterΛఏҊɻ ɾ߹੒σʔλͰ͸ɺ0.981ͷ࠷ྑͷਖ਼ղ཰ͱͳΓɺނোಛఆਫ਼౓Λ24%޲্ɻ ɾ࣮ূσʔλͰ͸ނোಛఆͷਫ਼౓ͱޮ཰ͷ྆ํ·ͨ͸͍ͣΕ͔Λ޲্ͤͨ͞ɻ ·ͱΊ ɾΑΓੵۃతʹ࡟ݮ͢Δ໰୊Λఆٛ͠ɺ1000Ҏ্ͷϝτϦΫεΛ100ϝτϦ Ϋεఔ౓·Ͱ࡟ݮ͢Δ͜ͱΛ໨ࢦ͢ʢݱঢ়ͷ࡟ݮ཰͸40-60%ఔ౓ʣ ɾނোಛఆ๏ͷSOTA΍ଞͷެ։σʔληοτΛ༻͍ͯධՁ͢Δ ࠓޙͷݚڀ

    ίʔυͱσʔληοτɿhttps://github.com/ai4sre/metricsifter

22. Appendix

23. 23 ಛ௃ྔ࡟ݮͷ໰୊ఆٛ [Tsubouchi+,ACCESS24] FIGURE 2. ΑΓసࡌ ނোʢFaultʣൃੜޙɺϝτϦΫεཻ౓Ͱͷҟ ৗͷ఻ൖϞσϧ ɿ௚઀తʹӨڹ͕ݱΕͨϝτϦΫε ɿؒ઀తʹӨڹ͕ݱΕͨϝτϦΫε

    ɿແӨڹͷϝτϦΫε MA MB MC ো֐Λݕ஌ͨ͠ΒɺͰ͖ΔݶΓૣ͘ɺ Λಛఆ͢Δ͜ͱɻ MA ∪ MB ໰୊

24. 24 ɾϐΞιϯ૬ؔΑΓ΋ɺมԽ఺ͷ΄͏͕ো֐ൃੜ࣌ؒʹ෼෍͕ภΔ Root fault metricsͷϖΞϫΠζ෼ੳ [Tsubouchi+,ACCESS24] FIGURE 3.ΑΓసࡌ

25. 25 ಛ௃࡟ݮͷ࣮ݧ݁Ռʢ߹੒σʔληοτʣ

26. 26 ධՁࢦඪ ಛ௃࡟ݮ๏ ނোಛఆ๏ Specificity Recall Balanced Accuracy (BA) =

    (Specificity + Recall)/2 ຊυϝΠϯͷ ఆ൪ධՁࢦඪ AC@k AVG@5 top-kʹਖ਼ղؚ͕·ΕΔ͔ͷਫ਼౓ ( ) ͷࢉज़ฏۉ AC@k 1 ≤ j ≤ 5 ޡ࡟আ͍ͯ͠ͳ͍͔ʁ ա৒࡟ݮ͍ͯ͠ͳ͍͔ʁ ෼ྨ໰୊ҰൠͷධՁࢦඪ

27. 27 ɾਖ਼ৗੑ࡟ݮɿNSigma, BIRCH, K-S test, FluxInfer-AD ɾ৑௕ੑ࡟ݮɿHDBSCAN + SBD, HDBSCAN

    + ϐΞιϯ૬ؔ ɾཧ૝ख๏ɿIdealʢਖ਼ղ཰Balanced Accuracy͕100%ʣ ϕʔεϥΠϯ ಛ௃࡟ݮ๏ ނো ಛఆ๏ ɾϥϯμϜબ୒ʢRSʣ ɾҟৗ౓ϕʔεɿ -Diagnosis ɾҟৗ఻ൖϕʔεɿҼՌάϥϑߏங+είΞϦϯά ɾPC+PageRank, PC+HT, LiNGAM+PageRank, LinGAM + HT, RCD ϵ

28. 28 Q3: ύϥϝʔλʹͲͷఔ౓හײ͔ʁʢParameter Sensitivity) [Tsubouchi+,ACCESS24] FIGURE 9. ΑΓసࡌ : มԽ఺ݕ஌ͷϖφϧςΟ߲ͷ

    ॏΈ܎਺ʢSTEP 1ʣ ω 2.5ۙ๣ͰϐʔΫΛͱΓ஋ͷݮগ ʹහײͰ͋Δ ਫ਼౓΁ͷӨڹ͸௿͍ : ਪఆີ౓ؔ਺ͷฏ׈Խ܎਺ ʢSTEP 2ʣ h

29. 29 Q4: ఏҊ๏ͷ෦Ґ͕Ͳͷఔ౓ੑೳʹد༩͢Δ͔ʁ [Tsubouchi+,ACCESS24] FIGURE 10. ΑΓసࡌ ద੾ͳύϥϝʔλʔͰ͋Ε ͹ɺਫ਼౓ࠩ͸খ͍͞ STEP1ʢมԽ఺ݕग़ʣͷύϥ

    ϝʔλ ͕௿͍ͱਫ਼౓͕௿Լ ω ͔͠͠ɺSTEP2/3ʹΑΓਫ਼౓ Λճ෮Ͱ͖͍ͯΔ ߹੒ͷ͖Ε͍ͳσʔλͰ ͸ɺมԽ఺ݕग़ਫ਼౓͕ߴ͢ ͗ΔͨΊ

30. 30 1.؍࡯ʹجͮ͘Ծఆʹҧ൓͢ΔέʔεͰ͸ɺਫ਼౓͕௿Լ͢Δ 2.ࠜຊݪҼϝτϦΫεʹɺݕ஌ՄೳͳϨϕϧͷมԽ఺͕ͳ͍ ٞ࿦ɿMetricSifterͷݶք