Unable to use streaming Reader with bufio.NewScanner #131
Description
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.20 windows/amd64
What operating system and processor architecture are you using (go env)?
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\User\AppData\Local\go-build
set GOENV=C:\Users\User\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\User\Dev\PSDumpTool\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\User\AppData\Local\Temp\go-build599287788=/tmp/go-build -gno-record-gcc-switches
What did you do?
example_file.zst is a 12GB file which extracts to an ~130GB ndjson (new line delimeted json) file
package main
import (
"bufio"
"github.com/DataDog/zstd"
"log"
"os"
)
func main() {
file, err := os.Open("example_file.zst")
if err != nil {
log.Printf("Couldn't open file: %v\n", err)
return
}
defer file.Close()
fileScanner := bufio.NewScanner(zstd.NewReader(file))
fileScanner.Split(bufio.ScanLines)
for fileScanner.Scan() {
log.Printf("%s\n", fileScanner.Text())
}
}What did you expect to see?
I expected to see all lines to be printed in order
What did you see instead?
The program starts and exists without printing any lines or errors.