feat(adater): Implement go-micro adapter #214

Zhou-Haowei · 2020-08-25T14:46:00Z

Describe what this PR does / why we need it

This PR integrate the Adapter of go-micro

Does this pull request fix one issue?

This PR add integration for go-micro, Fixes #150

Describe how you did it

Add this feature via go-micro server wrapper and intercept client call

Describe how to verify it

Run tests inside the adapter/go_micro package

Special notes for reviews

CLAassistant · 2020-08-25T14:46:04Z

All committers have signed the CLA.

sczyh30 · 2020-08-26T02:38:59Z

Could you please resolve CI failure?

Zhou-Haowei · 2020-08-26T03:11:20Z

Hi, it seems that the Travis CI failure caused by the test of Circuit Breaker. I compared the log on Travis CI of both PR #213 and my PR indicates it. And the test result of the PR I submitted as follows. The following tests were performed on macOS 10.15.6 and with go version 1.15.

[Test Result]
GOROOT=/usr/local/Cellar/go/1.15/libexec #gosetup
GOPATH=/Users/zhouhaowei/go #gosetup
/usr/local/Cellar/go/1.15/libexec/bin/go test -c -o /private/var/folders/xg/nmsjbptn3db3kdf9mmp_3vv80000gn/T/___go_test_github_com_alibaba_sentinel_golang_adapter_go_micro github.com/alibaba/sentinel-golang/adapter/go_micro #gosetup
/usr/local/Cellar/go/1.15/libexec/bin/go tool test2json -t /private/var/folders/xg/nmsjbptn3db3kdf9mmp_3vv80000gn/T/___go_test_github_com_alibaba_sentinel_golang_adapter_go_micro -test.v
=== RUN   TestClientLimiter
INFO: log base directory is: /Users/zhouhaowei/logs/csp/
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, {"id":"go.micro.client","code":500,"detail":"service sentinel.test.server: not found","status":"Internal Server Error"}
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
    client_test.go:57: Got err when call, SentinelBlockError: FlowControl
--- PASS: TestClientLimiter (0.00s)
=== RUN   TestServerLimiter
2020-08-26 11:07:04  file=v2@v2.9.1/service.go:200 level=info Starting [service] sentinel.test.server
2020-08-26 11:07:04  file=grpc/grpc.go:864 level=info Server [grpc] Listening on [::]:49701
2020-08-26 11:07:04  file=grpc/grpc.go:697 level=info Registry [mdns] Registering node: sentinel.test.server-2d4203b1-8bc4-41c5-a0e0-0fe5f3bacf8f
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
--- PASS: TestServerLimiter (1.21s)
PASS

sczyh30 · 2020-08-26T03:15:12Z

The failure may occur here:

==================
WARNING: DATA RACE
Write at 0x00c0000bc9c0 by goroutine 56:
  github.com/alibaba/sentinel-golang/adapter/go_micro/proto.(*Response).Reset()
      /home/travis/gopath/src/github.com/alibaba/sentinel-golang/adapter/go_micro/proto/test.pb.go:61 +0x47
  github.com/golang/protobuf/proto.Unmarshal()
      /home/travis/gopath/pkg/mod/github.com/golang/protobuf@v1.4.0/proto/wire.go:57 +0x42
  github.com/micro/go-micro/v2/client/grpc.protoCodec.Unmarshal()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/codec.go:84 +0x81
  github.com/micro/go-micro/v2/client/grpc.(*protoCodec).Unmarshal()
      <autogenerated>:1 +0x7c
  github.com/micro/go-micro/v2/client/grpc.wrapCodec.Unmarshal()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/codec.go:66 +0xfe
  github.com/micro/go-micro/v2/client/grpc.(*wrapCodec).Unmarshal()
      <autogenerated>:1 +0xaf
  google.golang.org/grpc.recv()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:711 +0x125
  google.golang.org/grpc.(*csAttempt).recvMsg()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:885 +0x2cd
  google.golang.org/grpc.(*clientStream).RecvMsg.func1()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:736 +0x61
  google.golang.org/grpc.(*clientStream).withRetry()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:594 +0xe1
  google.golang.org/grpc.(*clientStream).RecvMsg()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:735 +0x167
  google.golang.org/grpc.invoke()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/call.go:73 +0x194
  google.golang.org/grpc.(*ClientConn).Invoke()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/call.go:37 +0x276
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).call.func2()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:163 +0x3da
Previous write at 0x00c0000bc9c0 by goroutine 144:
  github.com/alibaba/sentinel-golang/adapter/go_micro/proto.(*Response).Reset()
      /home/travis/gopath/src/github.com/alibaba/sentinel-golang/adapter/go_micro/proto/test.pb.go:61 +0x47
  github.com/golang/protobuf/proto.Unmarshal()
      /home/travis/gopath/pkg/mod/github.com/golang/protobuf@v1.4.0/proto/wire.go:57 +0x42
  github.com/micro/go-micro/v2/client/grpc.protoCodec.Unmarshal()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/codec.go:84 +0x81
  github.com/micro/go-micro/v2/client/grpc.(*protoCodec).Unmarshal()
      <autogenerated>:1 +0x7c
  github.com/micro/go-micro/v2/client/grpc.wrapCodec.Unmarshal()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/codec.go:66 +0xfe
  github.com/micro/go-micro/v2/client/grpc.(*wrapCodec).Unmarshal()
      <autogenerated>:1 +0xaf
  google.golang.org/grpc.recv()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:711 +0x125
  google.golang.org/grpc.(*csAttempt).recvMsg()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:885 +0x2cd
  google.golang.org/grpc.(*clientStream).RecvMsg.func1()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:736 +0x61
  google.golang.org/grpc.(*clientStream).withRetry()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:594 +0xe1
  google.golang.org/grpc.(*clientStream).RecvMsg()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:735 +0x167
  google.golang.org/grpc.invoke()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/call.go:73 +0x194
  google.golang.org/grpc.(*ClientConn).Invoke()
      /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/call.go:37 +0x276
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).call.func2()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:163 +0x3da
Goroutine 56 (running) created at:
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).call()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:156 +0x1020
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).call-fm()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:99 +0x11a
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).Call.func1()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:451 +0xa08
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).Call.func2()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:465 +0x3d
Goroutine 144 (running) created at:
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).call()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:156 +0x1020
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).call-fm()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:99 +0x11a
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).Call.func1()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:451 +0xa08
  github.com/micro/go-micro/v2/client/grpc.(*grpcClient).Call.func2()
      /home/travis/gopath/pkg/mod/github.com/micro/go-micro/v2@v2.9.1/client/grpc/grpc.go:465 +0x3d
==================
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:66: Simulate call finished
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    server_test.go:64: Got err when call, {"id":"go.micro.client","code":500,"detail":"SentinelBlockError: FlowControl","status":"Internal Server Error"}
    testing.go:954: race detected during execution of test
--- FAIL: TestServerLimiter (1.34s)
=== CONT  
    testing.go:954: race detected during execution of test
FAIL
coverage: 38.0% of statements
FAIL	github.com/alibaba/sentinel-golang/adapter/go_micro	1.446s

sczyh30 · 2020-08-26T03:15:59Z

And you may need to format your code and imports with gofmt/goimport.

Zhou-Haowei · 2020-08-26T03:18:22Z

Thanks for you review, I will check this and make a fix.

Zhou-Haowei · 2020-08-26T05:45:20Z

Fixes submitted

945 === RUN   TestClientLimiter
946 INFO: log base directory is: /home/travis/logs/csp/
947 === RUN   TestClientLimiter/success
948 === RUN   TestClientLimiter/success/second_fail
949 --- PASS: TestClientLimiter (0.00s)
950     --- PASS: TestClientLimiter/success (0.00s)
951         --- PASS: TestClientLimiter/success/second_fail (0.00s)
952 === RUN   TestServerLimiter
953 2020-08-26 05:37:46  file=v2@v2.9.1/service.go:200 level=info Starting [service] sentinel.test.server
954 2020-08-26 05:37:46  file=grpc/grpc.go:864 level=info Server [grpc] Listening on [::]:41089
955 2020-08-26 05:37:46  file=grpc/grpc.go:697 level=info Registry [mdns] Registering node: sentinel.test.server-ce3813aa-8d18-4e9f-ba04-339a24b4c8c3
956 === RUN   TestServerLimiter/success
957 === RUN   TestServerLimiter/success/second_fail
958 --- PASS: TestServerLimiter (1.21s)
959     --- PASS: TestServerLimiter/success (0.21s)
960         --- PASS: TestServerLimiter/success/second_fail (0.11s)
961 PASS
962 coverage: 44.3% of statements
963 ok  	github.com/alibaba/sentinel-golang/adapter/go_micro	2.322s	coverage: 44.3% of statements
964 ?   	github.com/alibaba/sentinel-golang/adapter/go_micro/proto	[no test files]

sczyh30 · 2020-08-26T06:28:02Z

You may need to re-organize your imports with goimports. Please pay attention to all CI failure message.

codecov-commenter · 2020-08-26T06:50:50Z

Codecov Report

Merging #214 into master will decrease coverage by 0.00%.
The diff coverage is 42.20%.

@@            Coverage Diff             @@
##           master     #214      +/-   ##
==========================================
- Coverage   42.41%   42.41%   -0.01%     
==========================================
  Files          82       85       +3     
  Lines        4406     4515     +109     
==========================================
+ Hits         1869     1915      +46     
- Misses       2297     2357      +60     
- Partials      240      243       +3

Impacted Files	Coverage Δ
adapter/go_micro/options.go	`37.93% <37.93%> (ø)`
adapter/go_micro/server.go	`41.02% <41.02%> (ø)`
adapter/go_micro/client.go	`46.34% <46.34%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 461e21b...d983c37. Read the comment docs.

Zhou-Haowei · 2020-08-26T06:54:04Z

Sorry for the format problems, I haven't lint the autogenerated proto files because of the quote "// Code generated by protoc-gen-micro. DO NOT EDIT." in the last build.

sczyh30

LGTM

sczyh30 · 2020-09-01T03:54:45Z

Thanks for contributing! I'll rearrange some of the code later.

feat(adater): Implement go-micro adapter

f9caa3a

sczyh30 added area/integrations Issue related to integrations with open-source components to-review PRs to review labels Aug 26, 2020

fix(Adapter): fix test cases and lint the code

7e05e15

feat(adapter): goimports lint autogenerate proto files

d983c37

sczyh30 self-requested a review August 26, 2020 07:01

sczyh30 approved these changes Sep 1, 2020

View reviewed changes

sczyh30 merged commit a28b746 into alibaba:master Sep 1, 2020

sczyh30 removed the to-review PRs to review label Sep 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(adater): Implement go-micro adapter #214

feat(adater): Implement go-micro adapter #214

Uh oh!

Zhou-Haowei commented Aug 25, 2020

Uh oh!

CLAassistant commented Aug 25, 2020 •

edited

Loading

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

codecov-commenter commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

sczyh30 left a comment

Uh oh!

sczyh30 commented Sep 1, 2020

Uh oh!

Uh oh!

feat(adater): Implement go-micro adapter #214

feat(adater): Implement go-micro adapter #214

Uh oh!

Conversation

Zhou-Haowei commented Aug 25, 2020

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

Uh oh!

CLAassistant commented Aug 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

sczyh30 commented Aug 26, 2020

Uh oh!

codecov-commenter commented Aug 26, 2020

Codecov Report

Uh oh!

Zhou-Haowei commented Aug 26, 2020

Uh oh!

sczyh30 left a comment

Choose a reason for hiding this comment

Uh oh!

sczyh30 commented Sep 1, 2020

Uh oh!

Uh oh!

CLAassistant commented Aug 25, 2020 •

edited

Loading