.NET Core 上使用 Jaeger 追蹤 gRPC 呼叫

隨著系統使用人數愈來愈多，架構也跟著愈來愈複雜，各種技術為了解決既有問題或是加快反應速度不斷推陳出新，服務的架構也從單一系統變成多層式設計再轉為微服務，但不變的還是需要 trace log 來解決 bug，為了解決日益困難的 trace 問題，2010 年 Google 發表了 Dapper, a Large-Scale Distributed Systems Tracing Infrastructure 論文，立下了分散式追蹤系統的根基，有很多系統就是基於該篇論文內容的實作，包含了 Zipkin、Jaeger ….etc

所在團隊需要一套可以 trace ASP.NET Core + gRPC 的軟體，過程中嘗試了 Zipkin、Jaeger、SkyWalking，最後選擇了 Jaeger，剛完成 .NET Core 上使用 Jaeger 追蹤 gRPC 呼叫的 POC, 立馬紀錄一下，不然我相信我一定記不了多久的XD

基本環境說明

macOS Mojave 10.14.3
Docker Engine - Community 18.09.2
Grpc 1.19.0
Grpc.Tools 1.19.0
Google.Protobuf 3.7.0
Google.Protobuf.Tools 3.7.0
OpenTracing.Contrib.Grpc 0.1.0
Jaeger 0.2.2
Microsoft.Extensions.Logging.Console 2.2.0

ProtoBuf 定義

message

syntax = "proto3";

package gRPC.Message;
option csharp_namespace = "gRPC.Message";
import "google/protobuf/timestamp.proto";

message HelloRequest{
    string Name=1;
    google.protobuf.Timestamp SendDate=2;
}

message GoodByeRequest{
    string Name=1;
}

message Response{
    bool IsSuccess=1;
    string ResponseMsg=2;
}

service

syntax = "proto3";

package gRPC.Message; 
option csharp_namespace = "gRPC.Message";
import "message.proto";

service gRPCService {
rpc SayHello(HelloRequest) returns (Response);
rpc SayGoodbye(GoodByeRequest) returns (Response);
}

建立 Jaeger 環境

透過 docker 建立 Jaeger 服務及後台

docker run --rm -d -p 6831:6831/udp -p 16686:16686 jaegertracing/all-in-one

建立 gRPC.Message 共用層

安裝 NuGet 套件

Grpc

Package Manager
```
Install-Package Grpc
```
.NET CLI
```
dotnet add package Grpc
```

Grpc.Tools

Package Manager
```
Install-Package Grpc.Tools
```
.NET CLI
```
dotnet add package Grpc.Tools
```

Google.Protobuf

Package Manager
```
Install-Package Google.Protobuf
```
.NET CLI
```
dotnet add package Google.Protobuf
```

OpenTracing.Contrib.Grpc

Package Manager

Install-Package OpenTracing.Contrib.Grpc

.NET CLI

dotnet add package OpenTracing.Contrib.Grpc

Jaeger

Package Manager
```
Install-Package Jaeger
```
.NET CLI
```
dotnet add package Jaeger
```

Microsoft.Extensions.Logging.Console

Package Manager

Install-Package Microsoft.Extensions.Logging.Console

.NET CLI

dotnet add package Microsoft.Extensions.Logging.Console

將 protobuf 轉為 C# class

/Users/`whoami`/.nuget/packages/grpc.tools/1.19.0/tools/macosx_x64/protoc -I /Users/`whoami`/.nuget/packages/google.protobuf.tools/3.7.0/tools/ -I ./proto/ --csharp_out src/gRPC.Message --grpc_out src/gRPC.Message ./proto/*.proto --plugin=protoc-gen-grpc=/Users/`whoami`/.nuget/packages/grpc.tools/1.19.0/tools/macosx_x64/grpc_csharp_plugin

加入 TracingHelper.cs

using Microsoft.Extensions.Logging;
using Jaeger;
using Jaeger.Samplers;

namespace gRPC.Message
{
    public static class TracingHelper
    {
        public static Tracer InitTracer(string serviceName, ILoggerFactory loggerFactory)
        {
            Configuration.SamplerConfiguration samplerConfiguration = new Configuration.SamplerConfiguration(loggerFactory)
                .WithType(ConstSampler.Type)
                .WithParam(1);

            Configuration.ReporterConfiguration reporterConfiguration = new Configuration.ReporterConfiguration(loggerFactory)
                .WithLogSpans(true);

            return (Tracer)new Configuration(serviceName, loggerFactory)
                .WithSampler(samplerConfiguration)
                .WithReporter(reporterConfiguration)
                .GetTracer();
        }
    }
}

建立 gRPC.Server

將 gRPC.Message 加入參考

加入 gRPCServiceImpl.cs 並繼承 gRPCService.gRPCServiceBase

using System.Reflection.Metadata.Ecma335;
using System.Threading.Tasks;
using gRPC.Message;
using Grpc.Core;

namespace gRPC.Server
{
    public class gRPCServiceImpl : gRPCService.gRPCServiceBase
    {
        public override Task<Response> SayHello(HelloRequest request, ServerCallContext context)
        {
            return Task.Run(() =>
            {
                return new Response
                {
                    IsSuccess = true,
                    ResponseMsg = $"Hi {request.Name} @ {request.SendDate.ToDateTime()} !!!"
                };
            });
        }

        public override Task<Response> SayGoodbye(GoodByeRequest request, ServerCallContext context)
        {
            return Task.Run(() =>
            {
                return new Response
                {
                    IsSuccess = true,
                    ResponseMsg = $"Bye,{request.Name}"
                };
            });
        }
    }
}

建立 gRPC interceptor 並啟動 gRPC server

using System;
using System.Threading.Tasks;
using gRPC.Message;
using Grpc.Core;
using Grpc.Core.Interceptors;
using Jaeger;
using Microsoft.Extensions.Logging;
using OpenTracing.Contrib.Grpc.Interceptors;

namespace gRPC.Server
{
    public class Program
    {
        private const string Server = "localhost";
        const int Port = 50051;


        public static async Task Main(string[] args)
        {
            ILoggerFactory loggerFactory = new LoggerFactory().AddConsole();
            var serviceName = "server";
            Tracer tracer = TracingHelper.InitTracer(serviceName, loggerFactory);
            ServerTracingInterceptor tracingInterceptor = new ServerTracingInterceptor(tracer);
                
                
            Grpc.Core.Server server = new Grpc.Core.Server
            {
                Services = {gRPCService.BindService(new gRPCServiceImpl()).Intercept(tracingInterceptor)},
                Ports = {new ServerPort(Server, Port, ServerCredentials.Insecure)}
            };
            server.Start();

            Console.WriteLine($"Greeter server listening on server {server} and port {Port}");
            Console.WriteLine("Press any key to stop the server...");
            Console.ReadKey();

            await server.ShutdownAsync();
        }
    }
}

建立 gRPC.Client

將 gRPC.Message 加入參考

建立 gRPC interceptor 並啟動 gRPC server

using System;
using System.Threading.Tasks;
using gRPC.Message;
using Google.Protobuf.WellKnownTypes;
using Grpc.Core;
using Grpc.Core.Interceptors;
using Jaeger;
using Microsoft.Extensions.Logging;
using OpenTracing.Contrib.Grpc.Interceptors;

namespace gRPC.Client
{
    public class Program
    {
        const string Server = "localhost";
        const int Port = 50051;

        public static async Task Main(string[] args)
        {
            var loggerFactory =new LoggerFactory().AddConsole();
            var serviceName = "client";
            Tracer tracer = TracingHelper.InitTracer(serviceName, loggerFactory);
            ClientTracingInterceptor tracingInterceptor = new ClientTracingInterceptor(tracer);


            Channel channel = new Channel($"{Server}:{Port}", ChannelCredentials.Insecure);

            var client = new gRPCService.gRPCServiceClient(channel.Intercept(tracingInterceptor));
            string user = "yowko";

            var reply = client.SayHello(new HelloRequest {Name = user,SendDate = DateTime.UtcNow.ToTimestamp()});
            Console.WriteLine("Greeting: " + reply.ResponseMsg);


            var response = client.SayGoodbye(new GoodByeRequest() {Name = user});
            Console.WriteLine("Response: " + response.ResponseMsg);

            await channel.ShutdownAsync();
            Console.WriteLine("Press any key to exit...");
            Console.ReadKey();
        }
    }
}

實際使用

使用 http://localhost:16686 開啟 Jaeger UI
依 Service 及 Operation 來搜尋
搜尋結果
trace 細節
包含單一 service 內容及 service 使用順序

心得

前面提到嘗試了 Zipkin、Jaeger、SkyWalking，其中 Zipkin 因為在 interceptor 的處理相對複雜而出局;SkyWalking 使用的組件較多、問題也較多下決定放棄，以使用的難易度來看，Jaeger 相較 Zipkin 與 SkyWalking 下，至少輕了二個量級：架構容易、使用方式也簡單

完成程式碼內容請參考 yowko/dotnetCore-gRPC-Jaeger

文章目錄