I/O: Readers, Writers & Files

io.Reader

type Reader interface {
    Read(p []byte) (n int, err error)
}

One method. Fill the supplied slice with up to len(p) bytes; return how many you wrote and any error. io.EOF at the end is normal, not a real error.

Implementers: os.File, net.Conn, bytes.Buffer, strings.Reader, http.Response.Body, gzip readers, base64 decoders, and dozens more.

io.Writer

type Writer interface {
    Write(p []byte) (n int, err error)
}

The dual of Reader. Drain the slice into the underlying sink. Same huge family of implementers, most types implement both.

io.Closer

type Closer interface {
    Close() error
}

Anything that holds a resource (file descriptor, socket) implements Closer. Pair with defer x.Close() as soon as you acquire.

Composing the trio

The standard library composes these into composite interfaces:

type ReadCloser  interface { Reader; Closer }
type WriteCloser interface { Writer; Closer }
type ReadWriter  interface { Reader; Writer }

Functions accept the smallest interface that suffices. A function that just consumes bytes accepts io.Reader: meaning it works with files, sockets, in-memory buffers, HTTP bodies, and tests that pass strings.NewReader("..."). Massive flexibility.

Reading and writing files

Read whole file (small)

b, err := os.ReadFile("config.yaml")
if err != nil { /* handle */ }
fmt.Println(string(b))

Write whole file

err := os.WriteFile("out.txt", []byte("hello\n"), 0o644)

Stream (large file)

f, err := os.Open("big.csv")
if err != nil { /* handle */ }
defer f.Close()

// f satisfies io.Reader, pass it anywhere a Reader is wanted

Create / append

f, _ := os.Create("out.txt")                    // truncates
f, _ := os.OpenFile("log.txt", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
defer f.Close()
fmt.Fprintln(f, "line")

bufio: line-by-line is buffered

f, _ := os.Open("data.txt")
defer f.Close()

scan := bufio.NewScanner(f)
for scan.Scan() {
    line := scan.Text()        // one line at a time, no trailing newline
    // ...
}
if err := scan.Err(); err != nil { /* handle */ }

bufio.Scanner handles the buffering and line splitting. For other tokenizations (words, runes, custom), use scan.Split(bufio.ScanWords).

io.Copy & pipes

// Copy file → file
src, _ := os.Open("a.txt")
dst, _ := os.Create("b.txt")
defer src.Close(); defer dst.Close()

n, err := io.Copy(dst, src)        // returns bytes copied
fmt.Printf("wrote %d bytes\n", n)

Same pattern downloads a URL to disk:

resp, _ := http.Get("https://example.com/data.zip")
defer resp.Body.Close()

f, _ := os.Create("data.zip")
defer f.Close()

io.Copy(f, resp.Body)

This is the magic of io.Reader/io.Writer: the same function that copies file → file also copies HTTP → file, because both ends are just Reader/Writer.

Useful helpers

• io.ReadAll(r): slurp a Reader into a byte slice.
• io.LimitReader(r, n): wraps r to read at most n bytes.
• io.MultiWriter(a, b): write to multiple writers at once (great for tee'ing logs).
• io.MultiReader(r1, r2): concatenate readers.
• io.Discard: a Writer that drops everything (like /dev/null).
• strings.NewReader(s), bytes.NewReader(b): Reader over a string or []byte.
• bytes.Buffer: both a Reader and a Writer; great for tests.

Check your understanding

Practice exercises

package main

import (
    "bufio"
    "fmt"
    "os"
)

func main() {
    f, err := os.Open(os.Args[1])
    if err != nil { fmt.Fprintln(os.Stderr, err); os.Exit(1) }
    defer f.Close()

    scan := bufio.NewScanner(f)
    scan.Split(bufio.ScanWords)
    n := 0
    for scan.Scan() { n++ }
    fmt.Println(n)
}