Http client io reader changed file Magic Number (original) (raw)

November 26, 2025, 2:14am 1

I encountered a very strange situation.

this is the code

func LoadHttpImage(url string) (image.Image, error) {
    client := http.Client{
        Timeout: 5 * time.Second,
    }
    resp, err := client.Get(url)
    if err != nil {
        return nil, err
    }
    defer func() {
        err = resp.Body.Close()
    }()
    if resp.StatusCode != http.StatusOK {
        err = fmt.Errorf("status code %d", resp.StatusCode)
    }
    if err != nil {
        return nil, err
    }
    pic, _, err := image.Decode(resp.Body)
    if err != nil {
        fmt.Println(err)
    }
    return pic, nil
}

and the test case is

LoadHttpImage("https://fb.fenbike.cn/api/tarzan/images/17d6a6a52635d89.png?width=700")

the std io print error :“image: unknown format”

After debugging the code, I found that the problem was probably in this location(package: image/format.go:72)：

// sniff determines the format of r's data.
func sniff(r reader) format {
    formats, _ := atomicFormats.Load().([]format)
    for _, f := range formats {
        b, err := r.Peek(len(f.magic))
        if err == nil && match(f.magic, b) {
            return f
        }
    }
    return format{}
}

It appears to iterate through all file formats using magic numbers.

However, when I use Postman to view the binary stream of this file, it is indeed a PNG file.

What’s even stranger is that only this one file is like this. Could you please help me check if this is a bug in Go?

radovskyb (Benjamin Radovsky) November 26, 2025, 4:21am 2

Hey,

This took me a few minutes before I realised why this one was actually pretty confusing. Anyway, if you’re interested, I can let you know how I got to working it out, but pretty much, I noticed that the response encoding wasn’t always the same and neither was the response body size.

The issue is that it’s mostly returning a Brotli encoded response and once in a while it will actually return gzip I believe, so it was working here and there for me, but when it returns as Brotli encoded, it needs to be decoded before image/png will work on it.

Here’s a working example for you.

package main

import (
    "fmt"
    "image"
    "io"
    "log"
    "net/http"

    // Need this to decode into png for it's init side effects.
    _ "image/png"

    "github.com/andybalholm/brotli"
)

type BrotliTransport struct {
    Base http.RoundTripper
}

type brotliReadCloser struct {
    io.Reader
    Closer io.Closer
}

func (brc brotliReadCloser) Close() error {
    return brc.Closer.Close()
}

func (t *BrotliTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    req.Header.Set("Accept-Encoding", "br, gzip, deflate")

    base := t.Base
    if base == nil {
        base = http.DefaultTransport
    }

    resp, err := base.RoundTrip(req)
    if err != nil {
        return nil, err
    }

    if resp.Header.Get("Content-Encoding") == "br" {
        brReader := brotli.NewReader(resp.Body)
        resp.Body = brotliReadCloser{
            Reader: brReader,
            Closer: resp.Body,
        }
        resp.Header.Del("Content-Encoding")
    }

    return resp, nil
}

func LoadHTTPImage(url string) (image.Image, error) {
    client := &http.Client{
        Transport: &BrotliTransport{
            Base: http.DefaultTransport,
        },
    }

    resp, err := client.Get(url)
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

    img, _, err := image.Decode(resp.Body)
    if err != nil {
        return nil, err
    }
    return img, nil
}

func main() {
    i, err := LoadHTTPImage("https://fb.fenbike.cn/api/tarzan/images/17d6a6a52635d89.png?width=700")
    if err != nil {
        log.Fatalln(err)
    }

    fmt.Println(i.Bounds())
}

LIANGQI0811 (Theodore) November 26, 2025, 5:56am 3

think you very mach!

I didn’t notice the Brotli encoding issue.

radovskyb (Benjamin Radovsky) November 26, 2025, 5:58am 4

No problem

It was pretty subtle, and I easily could have missed it when checking the response headers.

Have a lovely day!