GitHub - samber/slog-formatter: 🚨 slog: Attribute formatting (original) (raw)

slog: Attribute formatting

Common formatters for slog library + helpers for building your own.

Handlers:

NewFormatterHandler: main handler
NewFormatterMiddleware: compatible with slog-multi middlewares
RecoverHandlerError: catch panics and error from handlers

Common formatters:

TimeFormatter: transforms a time.Time into a readable string
UnixTimestampFormatter: transforms a time.Time into a unix timestamp.
TimezoneConverter: set a time.Time to a different timezone
ErrorFormatter: transforms a go error into a readable error
HTTPRequestFormatter: transforms a *http.Request into a readable object
HTTPResponseFormatter: transforms a *http.Response into a readable object
PIIFormatter: hide private Personal Identifiable Information (PII)
IPAddressFormatter: hide ip address from logs
FlattenFormatterMiddleware: returns a formatter middleware that flatten attributes recursively

Custom formatter:

Format: pass any attribute into a formatter
FormatByKind: pass attributes matching slog.Kind into a formatter
FormatByType: pass attributes matching generic type into a formatter
FormatByKey: pass attributes matching key into a formatter
FormatByFieldType: pass attributes matching both key and generic type into a formatter
FormatByGroup: pass attributes under a group into a formatter
FormatByGroupKey: pass attributes under a group and matching key, into a formatter
FormatByGroupKeyType: pass attributes under a group, matching key and matching a generic type, into a formatter

See also:

slog-multi: slog.Handler chaining, fanout, routing, failover, load balancing...
slog-formatter: slog attribute formatting
slog-sampling: slog sampling policy
slog-mock: slog.Handler for test purposes

HTTP middlewares:

slog-gin: Gin middleware for slog logger
slog-echo: Echo middleware for slog logger
slog-fiber: Fiber middleware for slog logger
slog-chi: Chi middleware for slog logger
slog-http: net/http middleware for slog logger

Loggers:

slog-zap: A slog handler for Zap
slog-zerolog: A slog handler for Zerolog
slog-logrus: A slog handler for Logrus

Log sinks:

slog-datadog: A slog handler for Datadog
slog-betterstack: A slog handler for Betterstack
slog-rollbar: A slog handler for Rollbar
slog-loki: A slog handler for Loki
slog-sentry: A slog handler for Sentry
slog-syslog: A slog handler for Syslog
slog-logstash: A slog handler for Logstash
slog-fluentd: A slog handler for Fluentd
slog-graylog: A slog handler for Graylog
slog-quickwit: A slog handler for Quickwit
slog-slack: A slog handler for Slack
slog-telegram: A slog handler for Telegram
slog-mattermost: A slog handler for Mattermost
slog-microsoft-teams: A slog handler for Microsoft Teams
slog-webhook: A slog handler for Webhook
slog-kafka: A slog handler for Kafka
slog-nats: A slog handler for NATS
slog-parquet: A slog handler for Parquet + Object Storage
slog-channel: A slog handler for Go channels

🚀 Install

go get github.com/samber/slog-formatter

Compatibility: go >= 1.21

No breaking changes will be made to exported APIs before v2.0.0.

⚠️ Warnings:

in some case, you should consider implementing slog.LogValuer instead of using this library.
use this library carefully, log processing can be very costly (!)

🚀 Getting started

The following example has 3 formatters that anonymize data, format errors and format user. 👇

import ( slogformatter "github.com/samber/slog-formatter" "log/slog" )

formatter1 := slogformatter.FormatByKey("very_private_data", func(v slog.Value) slog.Value { return slog.StringValue("***********") }) formatter2 := slogformatter.ErrorFormatter("error") formatter3 := slogformatter.FormatByType(func(u User) slog.Value { return slog.StringValue(fmt.Sprintf("%s %s", u.firstname, u.lastname)) })

logger := slog.New( slogformatter.NewFormatterHandler(formatter1, formatter2, formatter3)( slog.NewTextHandler(os.Stdout, nil), ), )

err := fmt.Errorf("an error") logger.Error("a message", slog.Any("very_private_data", "abcd"), slog.Any("user", user), slog.Any("err", err))

// outputs: // time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

💡 Spec

GoDoc: https://pkg.go.dev/github.com/samber/slog-formatter

NewFormatterHandler

Returns a slog.Handler that applies formatters to.

import ( slogformatter "github.com/samber/slog-formatter" "log/slog" )

type User struct { email string firstname string lastname string }

logger := slog.New( slogformatter.NewFormatterHandler(formatter1, formatter2, formatter3)( slog.NewTextHandler(os.StdErr, nil), ), )

err := fmt.Errorf("an error") logger.Error("a message", slog.Any("very_private_data", "abcd"), slog.Any("user", user), slog.Any("err", err))

// outputs: // time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

NewFormatterMiddleware

Returns a slog-multi middleware that applies formatters to.

import ( slogformatter "github.com/samber/slog-formatter" slogmulti "github.com/samber/slog-multi" "log/slog" )

formattingMiddleware := slogformatter.NewFormatterHandler(formatter1, formatter2, formatter3) sink := slog.NewJSONHandler(os.Stderr, slog.HandlerOptions{})

logger := slog.New( slogmulti. Pipe(formattingMiddleware). Handler(sink), )

err := fmt.Errorf("an error") logger.Error("a message", slog.Any("very_private_data", "abcd"), slog.Any("user", user), slog.Any("err", err))

// outputs: // time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

RecoverHandlerError

Returns a slog.Handler that recovers from panics or error of the chain of handlers.

import ( slogformatter "github.com/samber/slog-formatter" slogmulti "github.com/samber/slog-multi" "log/slog" )

recovery := slogformatter.RecoverHandlerError( func(ctx context.Context, record slog.Record, err error) { // will be called only if subsequent handlers fail or return an error log.Println(err.Error()) }, ) sink := NewSinkHandler(...)

logger := slog.New( slogmulti. Pipe(recovery). Handler(sink), )

err := fmt.Errorf("an error") logger.Error("a message", slog.Any("very_private_data", "abcd"), slog.Any("user", user), slog.Any("err", err))

// outputs: // time=2023-04-10T14:00:0.000000+00:00 level=ERROR msg="a message" error.message="an error" error.type="*errors.errorString" user="John doe" very_private_data="********"

TimeFormatter

Transforms a time.Time into a readable string.

slogformatter.NewFormatterHandler( slogformatter.TimeFormatter(time.DateTime, time.UTC), )

UnixTimestampFormatter

Transforms a time.Time into a unix timestamp.

slogformatter.NewFormatterHandler( slogformatter.UnixTimestampFormatter(time.Millisecond), )

TimezoneConverter

Set a time.Time to a different timezone.

slogformatter.NewFormatterHandler( slogformatter.TimezoneConverter(time.UTC), )

ErrorFormatter

Transforms a Go error into a readable error.

import ( slogformatter "github.com/samber/slog-formatter" "log/slog" )

logger := slog.New( slogformatter.NewFormatterHandler( slogformatter.ErrorFormatter("error"), )( slog.NewTextHandler(os.Stdout, nil), ), )

err := fmt.Errorf("an error") logger.Error("a message", slog.Any("error", err))

// outputs: // { // "time":"2023-04-10T14:00:0.000000+00:00", // "level": "ERROR", // "msg": "a message", // "error": { // "message": "an error", // "type": "*errors.errorString" // "stacktrace": "main.main()\n\t/Users/samber/src/github.com/samber/slog-formatter/example/example.go:108 +0x1c\n" // } // }

HTTPRequestFormatter and HTTPResponseFormatter

Transforms *http.Request and *http.Response into readable objects.

import ( slogformatter "github.com/samber/slog-formatter" "log/slog" )

logger := slog.New( slogformatter.NewFormatterHandler( slogformatter.HTTPRequestFormatter(false), slogformatter.HTTPResponseFormatter(false), )( slog.NewJSONHandler(os.Stdout, nil), ), )

req, _ := http.NewRequest(http.MethodGet, "https://api.screeb.app", nil) req.Header.Set("Content-Type", "application/json") req.Header.Set("X-TOKEN", "1234567890")

res, _ := http.DefaultClient.Do(req)

logger.Error("a message", slog.Any("request", req), slog.Any("response", res))

PIIFormatter

Hides private Personal Identifiable Information (PII).

IDs are kept as is. Values longer than 5 characters have a plain text prefix.

import ( slogformatter "github.com/samber/slog-formatter" "log/slog" )

logger := slog.New( slogformatter.NewFormatterHandler( slogformatter.PIIFormatter("user"), )( slog.NewTextHandler(os.Stdout, nil), ), )

logger. With( slog.Group( "user", slog.String("id", "bd57ffbd-8858-4cc4-a93b-426cef16de61"), slog.String("email", "foobar@example.com"), slog.Group( "address", slog.String("street", "1st street"), slog.String("city", "New York"), slog.String("country", "USA"), slog.Int("zip", 12345), ), ), ). Error("an error")

// outputs: // { // "time":"2023-04-10T14:00:0.000000+00:00", // "level": "ERROR", // "msg": "an error", // "user": { // "id": "bd57ffbd-8858-4cc4-a93b-426cef16de61", // "email": "foob*******", // "address": { // "street": "1st **", // "city": "New ", // "country": "", // "zip": "**" // } // } // }

IPAddressFormatter

Transforms an IP address into "********".

import ( slogformatter "github.com/samber/slog-formatter" "log/slog" )

logger := slog.New( slogformatter.NewFormatterHandler( slogformatter.IPAddressFormatter("ip_address"), )( slog.NewTextHandler(os.Stdout, nil), ), )

logger. With("ip_address", "1.2.3.4"). Error("an error")

// outputs: // { // "time":"2023-04-10T14:00:0.000000+00:00", // "level": "ERROR", // "msg": "an error", // "ip_address": "*******", // }

FlattenFormatterMiddleware

A formatter middleware that flatten attributes recursively.

import ( slogformatter "github.com/samber/slog-formatter" slogmulti "github.com/samber/slog-multi" "log/slog" )

logger := slog.New( slogmulti. Pipe(slogformatter.FlattenFormatterMiddlewareOptions{Separator: ".", Prefix: "attrs", IgnorePath: false}.NewFlattenFormatterMiddlewareOptions()). Handler(slog.NewJSONHandler(os.Stdout, nil)), )

logger. With("email", "samuel@acme.org"). With("environment", "dev"). WithGroup("group1"). With("hello", "world"). WithGroup("group2"). With("hello", "world"). Error("A message", "foo", "bar")

// outputs: // { // "time": "2023-05-20T22:14:55.857065+02:00", // "level": "ERROR", // "msg": "A message", // "attrs.email": "samuel@acme.org", // "attrs.environment": "dev", // "attrs.group1.hello": "world", // "attrs.group1.group2.hello": "world", // "foo": "bar" // }

Format

Pass every attributes into a formatter.

slogformatter.NewFormatterHandler( slogformatter.Format(func(groups []string, key string, value slog.Value) slog.Value { // hide everything under "user" group if lo.Contains(groups, "user") { return slog.StringValue("****") }

    return value
}),

)

FormatByKind

Pass attributes matching slog.Kind into a formatter.

slogformatter.NewFormatterHandler( slogformatter.FormatByKind(slog.KindDuration, func(value slog.Value) slog.Value { return ... }), )

FormatByType

Pass attributes matching generic type into a formatter.

slogformatter.NewFormatterHandler( // format a custom error type slogformatter.FormatByType[*customError](func(err *customError) slog.Value { return slog.GroupValue( slog.Int("code", err.code), slog.String("message", err.msg), ) }), // format other errors slogformatter.FormatByType[error](func(err error) slog.Value { return slog.GroupValue( slog.Int("code", err.Error()), slog.String("type", reflect.TypeOf(err).String()), ) }), )

⚠️ Consider implementing slog.LogValuer when possible:

type customError struct { ... }

func (customError) Error() string { ... }

// implements slog.LogValuer func (customError) LogValue() slog.Value { return slog.StringValue(...) }

FormatByKey

Pass attributes matching key into a formatter.

slogformatter.NewFormatterHandler( slogformatter.FormatByKey("abcd", func(value slog.Value) slog.Value { return ... }), )

FormatByFieldType

Pass attributes matching both key and generic type into a formatter.

slogformatter.NewFormatterHandler( slogformatter.FormatByFieldType[User]("user", func(u User) slog.Value { return ... }), )

FormatByGroup

Pass attributes under a group into a formatter.

slogformatter.NewFormatterHandler( slogformatter.FormatByGroup([]{"user", "address"}, func(attr []slog.Attr) slog.Value { return ... }), )

FormatByGroupKey

Pass attributes under a group and matching key, into a formatter.

slogformatter.NewFormatterHandler( slogformatter.FormatByGroupKey([]{"user", "address"}, "country", func(value slog.Value) slog.Value { return ... }), )

FormatByGroupKeyType

Pass attributes under a group, matching key and matching a generic type, into a formatter.

slogformatter.NewFormatterHandler( slogformatter.FormatByGroupKeyType[string]([]{"user", "address"}, "country", func(value string) slog.Value { return ... }), )

🤝 Contributing

Ping me on twitter @samuelberthe (DMs, mentions, whatever :))
Fork the project
Fix open issues or request new features

Don't hesitate ;)

Install some dev dependencies

make tools

Run tests

make test

or

make watch-test

👤 Contributors

💫 Show your support

Give a ⭐️ if this project helped you!

📝 License

This project is MIT licensed.