Droot Internals

Transcript

1. %SPPU
   *OUFSOBMT JEZ@VVLJ
   ୈ̕ճ
   ίϯςφܕԾ૝Խͷ৘ใަ׵ձ!෱Ԭ

2. id:y_uuki @y_uuk1 ͸ͯͳ!ژ౎
   ΢ΣϒΦϖϨʔγϣϯΤϯδχΞ

3. IUUQZVVLJIBUFOBCMPHDPNFOUSZESPPU

4. TL;DR • ιϑτ΢ΣΞґଘ஍ࠈͷղܾͷͨΊʹ DockerΛ࢖͍͍ͨ • ຊ൪؀ڥͰDockerΛӡ༻͢Δͷ͸ͭΒ͍ • ʮBuild, Ship, Runʯͱ͍͏ίϯηϓτ͸޷͖

   • DockerΠϝʔδΛS3Λܦ༝ͯ͠഑෍͠ɺ chrootͰ࣮ߦ͢Δख๏ͷఏҊ

5. Ծ૝Խٕज़ )8Ծ૝Խ 04Ծ૝Խ ,7.
   9FO
   ʜ γεςϜίϯςφ
   -9$ ΞϓϦέʔγϣϯ ίϯςφ
   

   %PDLFS

6. ͳͥ(๻͸)DockerΛ࢖͏ͷ͔ • ✘ VMΑΓߴ଎ • ✘ Πϛϡʔλϒϧ • ✘ Φʔτεέʔϧ

   • ˚ ϙʔλϏϦςΟ • ◦ ϓϩάϥϚϒϧͳϗετ؀ڥ • ◦ ιϑτ΢ΣΞґଘ஍ࠈͷղܾ

7. ιϑτ΢ΣΞґଘ஍ࠈ • ͋Διϑτ΢ΣΞ͸͍͍ͨͯෳ਺ͷιϑτ ΢ΣΞʹґଘ͢Δ • ґଘઌͷιϑτ΢ΣΞ΋·ͨෳ਺ͷιϑτ ΢ΣΞʹґଘ͢Δ • ಉ͡؀ڥΛ࠶ݱ͢Δͷ͕೉͍͠ •

   BundlerͳͲΛ࢖ͬͯ΋CͷϥΠϒϥϦʹ ґଘ͢Δ͜ͱ΋

8. Docker • LinuxͷσΟετϦϏϡʔγϣϯ؀ڥ ͝ͱݻΊͯΠϝʔδԽ • /lib, /usr/bin, /etcͳͲͥΜͿ • Linux

   NamespacesͰಠཱͨ͠؀ڥΛ ࡞ͬͯΠϝʔδΛల։

9. DockerࠔΔ͜ͱ • Docker Engineͷෆ҆ఆ͞ • ωοτϫʔΫ·ΘΓͷύϑΥʔϚϯεྼԽ • ίϯςφͷΰϛ૟আ • ίϯςφͷແఀࢭσϓϩΠ

   • ίϯςφͷϩά؅ཧ • ίϯςφͷ؂ࢹ • ίϯςφͷσόοά • Docker Registryͷӡ༻

10. chroot

11. chroot ☓ Docker ͷΞΠσΞ 
    EPDLFS
    QVMM
    NZTRM
    
    $0/5"*/&3@*% EPDLFS
    DSFBUF
    NZTRM
    
    EPDLFS
    FYQPSU
    $0/5"*/&3@*%
    P
    NZTRMUBS
    
    NZTRMUBS
    Λ.Z42-Λಈ͔͍ͨ͠ϗετ΁ίϐʔ

    ͢Δɻ
    
    UBS
    YG[
    WBSDPOUBJOFSTNZTRMNZTRMUBS
    
    
    TVEP
    DISPPU
    WBSDPOUBJOFSTNZTRM
    NZTRME
12. None

13. ࠷ۙͷΞϓϦέʔγϣϯσϓϩΠ • git pull͕஗͍ • σϓϩΠαʔό͔Βͷrsync΋஗͍ • tarball σϓϩΠ •

    ੒Ռ෺ΛS3ͳͲʹ഑ஔ͠ɺσϓϩΠઌͰ s3 cp Ͱμ΢ϯϩʔυͯ͠ల։ • Serf/Consul • σϓϩΠαʔό͔ΒͷSSH͕஗͍ͨΊ

14. Droot

15. #VJME 4IJQ 3VO ESPPU
    SVO
    EPDLFS
    CVJME ESPPU
    FYQPSU EPDLFS
    EBFNPO 4UPSBHF
    
    4

    ESPPU
    
    EFQMPZ BXT
    T
    DQ BXT
    T
    DQ

16. %PDLFS %SPPU #VJME EPDLFS
    CVJME EPDLFS
    CVJME 3FHJTUSZ %PDLFS)VC %JTUSJCVUJPO ͳΜͰ΋Α͍
 "NB[PO
    4

    'JMF
    'PSNBU %PDLFS
    JNBHF ͳΜͰ΋Α͍
    FY
    UBSH[ $POUBJOFS -JOVY
    /BNFTQBDFT DISPPU

17. $ droot export • DockerΠϝʔδͷϑΝΠϧγεςϜΛtarܗࣜͰग़ྗ • جຊ͸ docker create &&

    docker export • gzip / aws cli ͱͷύΠϓʹΑΓɺtar.gzԽͯ͠S3ʹ഑ஔ • ϑΝΠϧγεςϜʹdrootઐ༻ͷ؀ڥม਺ϑΝΠϧ (/.drootenv) Λ࢓ࠐΉ 
    ESPPU
    FYQPSU
    EPDLFSpMFTBQQ
    c
    H[JQ
    DR
    
    c
    BXT
    T
    DQ
    
    TCVDLFUBQQUBSH[

18. $ droot deploy • ඪ४ೖྗ͔ΒtarΞʔΧΠϒΛಡΈࠐΈɺࢦఆ͠ ͨσΟϨΫτϦʹల։ • සൟʹߋ৽͞ΕΔίϯςφͷσϓϩΠΛ૝ఆ • rsync

    mode ͱ symlink mode 
    BXT
    T
    DQ
    TCVDLFUBQQUBSH[
    
    c
    HVO[JQ
    DR
    c
    ESPPU
    EFQMPZ
    SPPU
    WBSDPOUBJOFST

19. symlink ʹΑΔ atomic deploy • σϓϩΠࡁΈͷίϯςφ؀ ڥΛࠩ͠ସ͑Δඞཁ͕͋Δ • https://gist.github.com/ datagrok/3807742#file-

    symlink-replacement-md • symlink Λ rename(2)ɹ (mv -T) Ͱ੾Γସ͑Δ͜ͱ ʹΑΓΞτϛοΫʹσΟϨ ΫτϦΛࠩ͠ସ͑Δ 
    ᵓᴷᴷ
    BQQ
    
    BQQENBJO
    ᵓᴷᴷ
    BQQE
    
    
    
    
    ᵋᴷᴷ
    NBJO
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CJO
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CPPU
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    EFW
    
    
    
    
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CBDLVQ
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CJO
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CPPU
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    EFW
    
    
    
    
    
    
    
    
    

20. $ droot run • ࢦఆͨ͠σΟϨΫτϦΛchroot jailͱ࣮ͯ͠ߦ • σόΠεϑΝΠϧͷ࡞੒ (/dev/null, /dev/zeroͳͲ)

    • ϗετͷ /etc/group, /etc/resolve.confͳͲΛίϐʔ • bind mountͰϗετଆͷ೚ҙͷσΟϨΫτϦΛϚ΢ϯτ • Linux capabilities(7) ͰݖݶΛ཈੍ 
    TVEP
    ESPPU
    SVO
    DQ
    CJOE
    WBSMPH
    SPPU
    ɹɹWBSDPOUBJOFSTBQQ
    
    DPNNBOE

21. chroot(2) • ϓϩηεͷϧʔτσΟϨΫτϦΛมߋ • ϓϩηεͷઈରύεͷ୳ࡧى఺ͷมߋͷΈ • ϓϩηεΛੜ੒ͨ͠Γ͠ͳ͍ • ΧϨϯτσΟϨΫτϦ͸ͦͷ··ͳͷͰcrhootίʔ ϧޙʹchdir(“/“)͢Δ͜ͱ͕ଟ͍

    • jail؀ڥ֎ͷϑΝΠϧ΁షΒΕͨγϯϘϦοΫϦϯ Ϋ΁͸ΞΫηεͰ͖ͳ͍

22. BindϚ΢ϯτ • Linux 2.2͔Βಋೖ • σΟϨΫτϦ΍ϑΝΠϧΛଞͷҐஔ΁Ϛ΢ϯτ • chroot jail؀ڥ಺͔ΒϚ΢ϯτઌͷϑΝΠϧ΍σΟϨ ΫτϦ΁ΞΫηεͰ͖Δ

    • /var/containers/app/var/log ͱ͔ࢀর͢Δͷ໘౗ • mount -o bind /var/log /var/containers/app/var/log • ϗετͷ /var/log Λڞ༗

23. LinuxέʔύϏϦςΟ • chroot(2)͸ಛݖϓϩηεͰͳ͍ͱίʔϧͰ͖ͳ͍ • (ݫີʹ͸CAP_SYS_CHROOT) • ͔͠͠ɺεʔύʔϢʔβͰಈ͔͢ͷ͸ෆ҆ • εʔύʔϢʔβͰಈ͔ͭͭ͠ɺcapabilities(7)Ͱඞཁͳ ݖݶҎ֎Λམͱ͓ͯ͘͠

    • CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICE ΛڐՄ

24. Problems

25. • Docker (NamespacesΛ࢖ͬͨίϯςφ)΄Ͳͷɹ ϙʔλϏϦςΟ͸ͳ͍ • Dockerίϯςφͷ؀ڥม਺͕Ҿ͖ܧ͕Εͳ͍ • Dockerίϯςφ্ͷ user/group ͕σϓϩΠઌ

    ϗετʹ͍ͳ͍ ϙʔλϏϦςΟͷ໰୊

26. • ؀ڥม਺͸ϑΝΠϧͱͯ͠ӬଓԽ͞Εͳ͍ͷͰɺ Ұ୴ϑΝΠϧʹอଘ • droot export ͰҰ୴Dockerίϯςφͱͯ͠ىಈ͞ ͔ͤͯΒ env ίϚϯυͷ࣮ߦ݁ՌΛ

    /.drootenv ͱ ͯ͠อଘ • droot run Ͱ /.drootnenv ΛಡΈͩͯ͠؀ڥม਺Λ ෮ݩ • droot run —env Ͱ؀ڥม਺ͷ௥Ճɾ্ॻ͖Մೳ ؀ڥม਺ͷҾ͖ܧ͗

27. • User Namespaces಺Ͱ/etc/groupͳͲΛΈͯඞཁ ͳuser/grpupΛࣗಈ࡞੒ • ϓϩηεπϦʔߏ଄Λ͔͑ͨ͘ͳ͍ͷͰɺclone(2) Ͱ͸ͳ͘ chroot(2) ޙʹ unshare(2)

    ͢Δ • clone(2) ͩͱࢠϓϩηεΛੜ੒͢ΔͨΊɺεʔ ύʔόΠβϓϩηεͷ഑ԼͰdroot runͨ͠ͱ͖ ʹɺγάφϧ؅ཧ͕͏·͍͔͘ͳ͍͔΋ user/groupͷࣗಈ࡞੒(ະ࣮૷)

28. • ͜Ε͸͓ͦΒ͘ PID Namespacesͷ࿩ • https://lwn.net/Articles/532748/ • Namespaces௚Լͷϓϩηε͕pid 1ͱͯ͠ৼΔ෣͏ඞཁ͕Ͱ ͯ͘Δ

    • orphanϓϩηεͷճऩ͢Δඞཁ͕͋Δ

29. ίϯςφ͸ ࣗ෼Ͱ࡞ΕΔ

30. (PMBOH

31. • github.com/docker/docker/pkg • archive, devicemapper, fileutils, mount, symlink… • github.com/opencontainers/runc/libcontainer

    • Linux Namespaces·ΘΓ • https://github.com/syndtr/gocapability • LinuxέʔύϏϦςΟ • github.com/docker/engine-api • Docker APIΫϥΠΞϯτ ίϯςφπʔϧ޲͚ύοέʔδ

32. • ࣗ࡞ͷίϯςφπʔϧ Drootͷഎܠͱ࣮૷ • Build, Ship, RunΛ࣮ݱ͢Δୈ̏ͷιϑτ΢ΣΞ • droot export,

    droot deploy, droot run • DockerͰΠϝʔδΛ࡞ͬͯ chroot Ͱ࣮ߦ • ϙʔλϏϦςΟͷ໰୊ͱͦͷղܾ • ίϯςφ͸ࣗ෼Ͱ࡞ΕΔ ·ͱΊ

33. github.com/yuuki/droot