download a large file, show a progress bar, and handle ctrl-c

last updated: Oct 20, 2023

I had cause to figure out how to do this today. The only third-party tool is schollz/progressbar, which I used in the most absolutely basic way, straight from its manual.

Updated: version 2

// create a context that will cancel on interrupt. defer'ing stop
// guarantees that when the function exits nothing will be listening
// for the signal any longer
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()

// prepare the download. Note that we're now using `NewRequestWithContext`
src := "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml"
uri := fmt.Sprintf("%s-%s.bin", src, name)
req := must(http.NewRequestWithContext(ctx, "GET", uri, nil))
resp := must(http.DefaultClient.Do(req))
defer resp.Body.Close()

// download to a `<filename>.part` file until the download is successfully complete
inProgressDownloadName := outputFile + ".part"
out := must(os.Create(inProgressDownloadName))
defer os.Remove(inProgressDownloadName)

bar := progressbar.DefaultBytes(
	resp.ContentLength,
	fmt.Sprintf("downloading %s model", yellow(name)),
)

// Check for context.canceled, so that we don't output an unsightly error if
// a user cancels the program. If it's any other error, handle as normal
_, err := io.Copy(io.MultiWriter(out, bar), resp.Body)
if err != nil {
	if err != context.Canceled {
		fmt.Println(err)
		panic(err)
	} else {
		os.Exit(1)
	}
}

// Download complete. rename the <filename>.part file -> <filename>
must_(os.Rename(inProgressDownloadName, outputFile))

fmt.Println(yellow("download complete"))

version 2 in context

version 1

The procedure for handling interrupt is to:

src := "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml"
uri := fmt.Sprintf("%s-%s.bin", src, name)
req := must(http.NewRequest("GET", uri, nil))
resp := must(http.DefaultClient.Do(req))
defer resp.Body.Close()

out := must(os.Create(outputFile))
defer out.Close()

bar := progressbar.DefaultBytes(
	resp.ContentLength,
	fmt.Sprintf("downloading %s model", yellow(name)),
)

// handle a sigint while we're downloading
done := make(chan bool)
go func() {
	sigchan := make(chan os.Signal, 1)
	signal.Notify(sigchan, os.Interrupt, syscall.SIGTERM)
	select {
	case <-sigchan:
		// ignore errors here, we've been interrupted and we're on
		// best-effort at this point. Try to remove the partial download
		out.Close()
		os.Remove(outputFile)
		os.Exit(1)
	case <-done:
		// the download finished, remove the handler and continue
		signal.Stop(sigchan)
		return
	}
}()

must(io.Copy(io.MultiWriter(out, bar), resp.Body))

// tell the interrupt handler we finished the download, it doesn't need to
// run any longer
done <- true

fmt.Printf("%s\n", yellow("download complete"))

You can see it in context here

(if you were curious, must is a function that accepts a value and an error, and returns the value if there wasn't an error:

// must accepts a value and an error, and returns the value if the error is
// nil. Otherwise, prints the error and panics
func must[T any](t T, err error) T {
	if err != nil {
		fmt.Println(err)
		panic(err)
	}
	return t
}

I'd label my usage of it as experimental, but I like it so far. Somewhat similar to zig's try.)

↑ up