slog: Handler chaining, fanout, routing, failover, load balancing...

Design workflows ofsloghandlers:

• fanout:distributelog.Recordto multipleslog.Handlerin parallel
• pipeline:rewritelog.Recordon the fly (eg: for privacy reason)
• routing:forwardlog.Recordto all matchingslog.Handler
• failover:forwardlog.Recordto the first availableslog.Handler
• load balancing:increase log bandwidth by sendinglog.Recordto a pool ofslog.Handler

Here a simple workflow with both pipeline and fanout:

---

^{Sponsored by:}

Cloud-native search engine for observability - An OSS alternative to Splunk, Elasticsearch, Loki, and Tempo.

---

See also:

• slog-multi:slog.Handlerchaining, fanout, routing, failover, load balancing...
• slog-formatter:slogattribute formatting
• slog-sampling:slogsampling policy
• slog-mock:slog.Handlerfor test purposes

HTTP middlewares:

• slog-gin:Gin middleware forsloglogger
• slog-echo:Echo middleware forsloglogger
• slog-fiber:Fiber middleware forsloglogger
• slog-chi:Chi middleware forsloglogger
• slog-http:net/httpmiddleware forsloglogger

Loggers:

• slog-zap:Asloghandler forZap
• slog-zerolog:Asloghandler forZerolog
• slog-logrus:Asloghandler forLogrus

Log sinks:

• slog-datadog:Asloghandler forDatadog
• slog-betterstack:Asloghandler forBetterstack
• slog-rollbar:Asloghandler forRollbar
• slog-loki:Asloghandler forLoki
• slog-sentry:Asloghandler forSentry
• slog-syslog:Asloghandler forSyslog
• slog-logstash:Asloghandler forLogstash
• slog-fluentd:Asloghandler forFluentd
• slog-graylog:Asloghandler forGraylog
• slog-quickwit:Asloghandler forQuickwit
• slog-slack:Asloghandler forSlack
• slog-telegram:Asloghandler forTelegram
• slog-mattermost:Asloghandler forMattermost
• slog-microsoft-teams:Asloghandler forMicrosoft Teams
• slog-webhook:Asloghandler forWebhook
• slog-kafka:Asloghandler forKafka
• slog-nats:Asloghandler forNATS
• slog-parquet:Asloghandler forParquet+Object Storage
• slog-channel:Asloghandler for Go channels

🚀 Install

go get github.com/samber/slog-multi

Compatibility:go >= 1.21

No breaking changes will be made to exported APIs before v2.0.0.

⚠️Use this library carefully, log processing can be very costly (!)

💡 Usage

GoDoc:https://pkg.go.dev/github.com/samber/slog-multi

Broadcast:slogmulti.Fanout()

Distribute logs to multipleslog.Handlerin parallel.

import(
slogmulti"github.com/samber/slog-multi"
"log/slog"
)

funcmain() {
logstash,_:=net.Dial("tcp","logstash.acme:4242")// use github.com/netbrain/goautosocket for auto-reconnect
stderr:=os.Stderr

logger:=slog.New(
slogmulti.Fanout(
slog.NewJSONHandler(logstash,&slog.HandlerOptions{}),// pass to first handler: logstash over tcp
slog.NewTextHandler(stderr,&slog.HandlerOptions{}),// then to second handler: stderr
//...
),
)

logger.
With(
slog.Group("user",
slog.String("id","user-123"),
slog.Time("created_at",time.Now()),
),
).
With("environment","dev").
With("error",fmt.Errorf("an error")).
Error("A message")
}

Stderr output:

time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg= "A message" user.id=user-123 user.created_at=2023-04-10T14:00:0.000000+00:00 environment=dev error= "an error"

Netcat output:

{
"time":"2023-04-10T14:00:0.000000+00:00",
"level":"ERROR",
"msg":"A message",
"user":{
"id":"user-123",
"created_at":"2023-04-10T14:00:0.000000+00:00"
},
"environment":"dev",
"error":"an error"
}

Routing:slogmulti.Router()

Distribute logs to all matchingslog.Handlerin parallel.

import(
slogmulti"github.com/samber/slog-multi"
slogslack"github.com/samber/slog-slack"
"log/slog"
)

funcmain() {
slackChannelUS:=slogslack.Option{Level:slog.LevelError,WebhookURL:"xxx",Channel:"supervision-us"}.NewSlackHandler()
slackChannelEU:=slogslack.Option{Level:slog.LevelError,WebhookURL:"xxx",Channel:"supervision-eu"}.NewSlackHandler()
slackChannelAPAC:=slogslack.Option{Level:slog.LevelError,WebhookURL:"xxx",Channel:"supervision-apac"}.NewSlackHandler()

logger:=slog.New(
slogmulti.Router().
Add(slackChannelUS,recordMatchRegion("us")).
Add(slackChannelEU,recordMatchRegion("eu")).
Add(slackChannelAPAC,recordMatchRegion("apac")).
Handler(),
)

logger.
With("region","us").
With("pool","us-east-1").
Error("Server desynchronized")
}

funcrecordMatchRegion(regionstring)func(ctxcontext.Context,rslog.Record)bool{
returnfunc(ctxcontext.Context,rslog.Record)bool{
ok:=false

r.Attrs(func(attrslog.Attr)bool{
ifattr.Key=="region"&&attr.Value.Kind()==slog.KindString&&attr.Value.String()==region{
ok=true
returnfalse
}

returntrue
})

returnok
}
}

Failover:slogmulti.Failover()

List multiple targets for aslog.Recordinstead of retrying on the same unavailable log management system.

import(
"net"
slogmulti"github.com/samber/slog-multi"
"log/slog"
)


funcmain() {
// ncat -l 1000 -k
// ncat -l 1001 -k
// ncat -l 1002 -k

// list AZs
// use github.com/netbrain/goautosocket for auto-reconnect
logstash1,_:=net.Dial("tcp","logstash.eu-west-3a.internal:1000")
logstash2,_:=net.Dial("tcp","logstash.eu-west-3b.internal:1000")
logstash3,_:=net.Dial("tcp","logstash.eu-west-3c.internal:1000")

logger:=slog.New(
slogmulti.Failover()(
slog.HandlerOptions{}.NewJSONHandler(logstash1,nil),// send to this instance first
slog.HandlerOptions{}.NewJSONHandler(logstash2,nil),// then this instance in case of failure
slog.HandlerOptions{}.NewJSONHandler(logstash3,nil),// and finally this instance in case of double failure
),
)

logger.
With(
slog.Group("user",
slog.String("id","user-123"),
slog.Time("created_at",time.Now()),
),
).
With("environment","dev").
With("error",fmt.Errorf("an error")).
Error("A message")
}

Load balancing:slogmulti.Pool()

Increase log bandwidth by sendinglog.Recordto a pool ofslog.Handler.

import(
"net"
slogmulti"github.com/samber/slog-multi"
"log/slog"
)

funcmain() {
// ncat -l 1000 -k
// ncat -l 1001 -k
// ncat -l 1002 -k

// list AZs
// use github.com/netbrain/goautosocket for auto-reconnect
logstash1,_:=net.Dial("tcp","logstash.eu-west-3a.internal:1000")
logstash2,_:=net.Dial("tcp","logstash.eu-west-3b.internal:1000")
logstash3,_:=net.Dial("tcp","logstash.eu-west-3c.internal:1000")

logger:=slog.New(
slogmulti.Pool()(
// a random handler will be picked
slog.HandlerOptions{}.NewJSONHandler(logstash1,nil),
slog.HandlerOptions{}.NewJSONHandler(logstash2,nil),
slog.HandlerOptions{}.NewJSONHandler(logstash3,nil),
),
)

logger.
With(
slog.Group("user",
slog.String("id","user-123"),
slog.Time("created_at",time.Now()),
),
).
With("environment","dev").
With("error",fmt.Errorf("an error")).
Error("A message")
}

Chaining:slogmulti.Pipe()

Rewritelog.Recordon the fly (eg: for privacy reason).

funcmain() {
// first middleware: format go `error` type into an object {error: "*myCustomErrorType", message: "could not reach https://a.b/c" }
errorFormattingMiddleware:=slogmulti.NewHandleInlineMiddleware(errorFormattingMiddleware)

// second middleware: remove PII
gdprMiddleware:=NewGDPRMiddleware()

// final handler
sink:=slog.NewJSONHandler(os.Stderr,&slog.HandlerOptions{})

logger:=slog.New(
slogmulti.
Pipe(errorFormattingMiddleware).
Pipe(gdprMiddleware).
//...
Handler(sink),
)

logger.
With(
slog.Group("user",
slog.String("id","user-123"),
slog.String("email","user-123"),
slog.Time("created_at",time.Now()),
),
).
With("environment","dev").
Error("A message",
slog.String("foo","bar"),
slog.Any("error",fmt.Errorf("an error")),
)
}

Stderr output:

{
"time":"2023-04-10T14:00:0.000000+00:00",
"level":"ERROR",
"msg":"A message",
"user":{
"id":"*******",
"email":"*******",
"created_at":"*******"
},
"environment":"dev",
"foo":"bar",
"error":{
"type":"*myCustomErrorType",
"message":"an error"
}
}

Custom middleware

Middleware must match the following prototype:

typeMiddlewarefunc(slog.Handler) slog.Handler

The example above uses:

• a custom middleware,see here
• an inline middleware,see here

Note:WithAttrsandWithGroupmethods of custom middleware must return a new instance, instead ofthis.

Inline middleware

An "inline middleware" (aka. lambda), is a shortcut to middleware implementation, that hooks a single method and proxies others.

// hook `logger.Enabled` method
mdw:=slogmulti.NewEnabledInlineMiddleware(func(ctxcontext.Context,levelslog.Level,nextfunc(context.Context,slog.Level)bool)bool{
// [...]
returnnext(ctx,level)
})

// hook `logger.Handle` method
mdw:=slogmulti.NewHandleInlineMiddleware(func(ctxcontext.Context,recordslog.Record,nextfunc(context.Context,slog.Record)error)error{
// [...]
returnnext(ctx,record)
})

// hook `logger.WithAttrs` method
mdw:=slogmulti.NewWithAttrsInlineMiddleware(func(attrs[]slog.Attr,nextfunc([]slog.Attr) slog.Handler) slog.Handler{
// [...]
returnnext(attrs)
})

// hook `logger.WithGroup` method
mdw:=slogmulti.NewWithGroupInlineMiddleware(func(namestring,nextfunc(string) slog.Handler) slog.Handler{
// [...]
returnnext(name)
})

A super inline middleware that hooks all methods.

Warning: you would rather implement your own middleware.

mdw:=slogmulti.NewInlineMiddleware(
func(ctxcontext.Context,levelslog.Level,nextfunc(context.Context,slog.Level)bool)bool{
// [...]
returnnext(ctx,level)
},
func(ctxcontext.Context,recordslog.Record,nextfunc(context.Context,slog.Record)error)error{
// [...]
returnnext(ctx,record)
},
func(attrs[]slog.Attr,nextfunc([]slog.Attr) slog.Handler) slog.Handler{
// [...]
returnnext(attrs)
},
func(namestring,nextfunc(string) slog.Handler) slog.Handler{
// [...]
returnnext(name)
},
)

🤝 Contributing

• Ping me on twitter@samuelberthe(DMs, mentions, whatever:))
• Fork theproject
• Fixopen issuesor request new features

Don't hesitate;)

#Install some dev dependencies
make tools

#Run tests
maketest
#or
make watch-test

👤 Contributors

💫 Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2023Samuel Berthe.

This project isMITlicensed.