GitHub - pieterclaerhout/go-waitgroup: A sync.WaitGroup with error handling and concurrency control (original) (raw)

go-waitgroup

How to use

An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrency.

Using it is just like a normal sync.WaitGroup. The only difference is the initialisation. When you use waitgroup.NewWaitGroup, you have the option to specify it's size.

Any int which is bigger than 0 will limit the number of concurrent goroutines. If you specify -1 or 0, all goroutines will run at once (just like a plain sync.WaitGroup).

package main

import ( "fmt" "net/http"

"github.com/pieterclaerhout/go-waitgroup"

)

func main() {

urls := []string{
    "https://www.easyjet.com/",
    "https://www.skyscanner.de/",
    "https://www.ryanair.com",
    "https://wizzair.com/",
    "https://www.swiss.com/",
}

wg := waitgroup.NewWaitGroup(3)

for _, url := range urls {
    wg.BlockAdd()
    go func(url string) {
        defer wg.Done()
        fmt.Printf("%s: checking\n", url)
        res, err := http.Get(url)
        if err != nil {
            fmt.Println("Error: %v")
        } else {
            defer res.Body.Close()
            fmt.Printf("%s: result: %v\n", url, err)
        }
    }(url)
}

wg.Wait()
fmt.Println("Finished")

}

Using closures

There is also a way to use function closures to make it even more readable:

package main

import ( "fmt" "net/http"

"github.com/pieterclaerhout/go-waitgroup"

)

func main() {

urls := []string{
    "https://www.easyjet.com/",
    "https://www.skyscanner.de/",
    "https://www.ryanair.com",
    "https://wizzair.com/",
    "https://www.swiss.com/",
}

wg := waitgroup.NewWaitGroup(3)

for _, url := range urls {

    urlToCheck := url
    wg.Add(func() {
        fmt.Printf("%s: checking\n", urlToCheck)
        res, err := http.Get(urlToCheck)
        if err != nil {
            fmt.Println("Error: %v")
        } else {
            defer res.Body.Close()
            fmt.Printf("%s: result: %v\n", urlToCheck, err)
        }
    })

}

wg.Wait()
fmt.Println("Finished")

}

Handling errors

If you want to handle errors, there is also an ErrorGroup. This uses the same principles as a normal WaitGroup with a small twist.

First of all, you can only add functions which returns just an error.

Second, as soon as one of the queued items fail, the rest will be cancelled:

package main

import ( "context" "fmt" "os"

"github.com/pieterclaerhout/go-waitgroup"

)

func main() {

ctx := context.Background()

wg, ctx := waitgroup.NewErrorGroup(ctx, tc.size)
if err != nil {
    fmt.Println("Error: %v")
    os.Exit(1)
}

wg.Add(func() error {
    return nil
})

wg.Add(func() error {
    return errors.New("An error occurred")
})

if err := wg.Wait(); err != nil {
    fmt.Println("Error: %v")
    os.Exit(1)
}

}

You can also add multiple functions in one step:

package main

import ( "context" "errors" "fmt" "os"

"github.com/pieterclaerhout/go-waitgroup"

)

func main() {

ctx := context.Background()

wg, ctx := waitgroup.NewErrorGroup(ctx, tc.size)
if err != nil {
    fmt.Println("Error: %v")
    os.Exit(1)
}

wg.Add(
    func() error {
        return nil
    },
    func() error {
        return errors.New("An error occurred")
    },
)

if err := wg.Wait(); err != nil {
    fmt.Println("Error: %v")
    os.Exit(1)
}

}