Mastering Golang Series - Part 1 - How to Read Data From The Terminal and a File

The Go programming language has gained tremendous popularity in the last few years. More and more technology companies have adopted Go as their main programing language. For developers, being proficient in Go is a big plus for their career advancement. Like every other programming language, Go has lots of its own tricks and gotcha moments. Besides, for a single task, there are probably multiple ways in Go to generate the same result. But it is usually not obvious which way is optimal. The Mastering Golang Series is written to help Go developers navigate the Go library labyrinth and make them better equipped before going to the battleground. For each article in the Mastering Golang Series, I will mainly focus on one specific topic and dig deep to find some parts unknown. I would suggest the readers check out my introduction article on Go: A Brief Introduction to The Go Programming Language and proceed from there.

Reading Data From The Terminal

There are multiple build-in methods in Golang for reading data from the terminal. In the following session, you will be introduced to the bufio and fmt packages, as well as the corresponding functions for reading data. Package bufio implements buffered I/O. It wraps an io.Reader or io.Writer object, creating another object (Reader or Writer) that also implements the interface but provides buffering and some help for textual I/O. Package fmt implements formatted I/O with functions analogous to C's printf and scanf. The format 'verbs' are derived from C's but are simpler.

bufio.NewReader(os.Stdin)

NewReader returns a new Reader whose buffer has the default size (bufio's default buffer size is 4K). By putting os.Stdin as the variable in the NewReader function, we indicate the Reader to read from the terminal. We will show how we can use the ReadString and the ReadLine methods to read data from the terminal.

func (b *Reader) ReadLine() (line []byte, isPrefix bool, err error)

ReadLine does a buffered read up to a line terminator. It handles either \n or \r\n, and returns just the line without the new line character.

// A reader is created which can be called multiple times
reader := bufio.NewReader(os.Stdin)

// Prompt and read
fmt.Print("Enter text: ") // enter "Emma"

line, isPrefix, err := reader.ReadLine()

if err != nil {  // check errorpanic(err)
}

fmt.Println(line) // prints [69 109 109 97]

fmt.Println(string(line)) // prints Emma

If we enter "Emma" as our test input, the line variable from the reader.Readline() call is a byte array representation of the input string, and we can use string(line) to convert the byte array into the original string. As you can see, ReadLine is a low-level line-reading primitive. Most callers should use ReadBytes('\n') or ReadString('\n') instead or use a Scanner.

func (b *Reader) ReadBytes(delim byte) ([]byte, error)

ReadBytes reads until the first occurrence of delim in the input, returning a slice containing the data up to and including the delimiter.

// A reader is created which can be called multiple times
reader := bufio.NewReader(os.Stdin)
// Prompt and read
fmt.Print("Enter text: ") // enter "Emma"

line, err := reader.ReadBytes('\n')

if err != nil {  // check errorpanic(err)
}

fmt.Println(line) // prints [69 109 109 97 10]

fmt.Println(string(line)) // prints Emma

As you can see, ReaderBytes returns the byte array of the input string, as well as the error message. But we still need to convert the byte array to the original string most times. Fortunately, there is a ReadString method which returns the original input string.

func (b *Reader) ReadString(delim byte) (string, error)

ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. 

// A reader is created which can be called multiple times
reader := bufio.NewReader(os.Stdin)
// Prompt and read
fmt.Print("Enter text: ") // enter "Emma"

str, err := reader.ReadString('\n')

if err != nil {  // check errorpanic(err)
}

fmt.Println(str) // prints Emma

fmt.Println([]byte(str)) // prints [69 109 109 97 10]

ReadString does return the original input string, but it is still not ideal because the delimiter (usually new line character) is also included in the string. If the delimiter is not desired, we need to make extra efforts to get rid of the delimiter. We need a method which takes in only the content without the delimiter. The NewScanner function in the bufio package is the solution.

bufio.NewScanner(os.Stdin)

NewScanner returns a new Scanner to read from os.Stdin.

func NewScanner(r io.Reader) *Scanner
func (s *Scanner) Scan() bool
func (s *Scanner) Text() string

scanner := bufio.NewScanner(os.Stdin)

fmt.Print("Enter Text: ")

// Scans a line from Stdin(Console)
scanner.Scan()

// Holds the string that scanned
text := scanner.Text()

fmt.Println(text) // prints "Emma"

fmt.Println([]byte(text)) // prints [69 109 109 97]

Scan advances the Scanner to the next token, which will then be available through the Bytes or Text method. It returns false when the scan stops, either by reaching the end of the input or an error. After Scan returns false, the Err method will return any error that occurred during scanning, except that if it was io.EOF, Err will return nil.

Text returns the most recent token generated by a call to Scan as a newly allocated string holding its bytes.

As you can see, the scanner.Text function returns the original content without the delimiter. If the one-line input is space-separated variables, we can easily use the string.Split function to convert the one-line string into a string array.

scanner := bufio.NewScanner(os.Stdin)

fmt.Print("Enter text: ") // enter "John Tom Jerry"

scanner.Scan()

text := scanner.Text()

names := strings.Split(text, " ") // names is [John, Tom, Jerry]

fmt.Println(names)

When reading data from the terminal, we often encounter the scenario where the input consists of multiple lines and the input ends with an empty line. We can easily combine a for loop of Scan and Text to read the input properly:

    // To create dynamic arrayvar arr []string
    scanner := bufio.NewScanner(os.Stdin)
    for {
        fmt.Print("Enter Text: ")

        // Scans a line from Stdin(Console)
        scanner.Scan()

        // Holds the string that scanned
        text := scanner.Text()

        if len(text) != 0 {
            fmt.Println(text)
            arr = append(arr, text)
        } else {
            break
        }

    }
    // Use collected inputs
    fmt.Println(arr)

fmt.Scanln

func Scanln(a ...interface{}) (n int, err error)

Scanln scans text read from standard input, storing successive space-separated values into successive arguments. It stops scanning at a newline and after the final item there must be a newline or EOF. Scanln can only take one line from standard input, and it is not suitable for multiple-line input.

var text1, text2 string

fmt.Print("Enter text: ")

fmt.Scanln(&text1, &text2) // enter "Emma laughs"

fmt.Println(text1) // prints "Emma"

fmt.Println(text2) // prints "laughs"

In general, always try to use the bufio.NewScanner for collecting input from the console. There are multiple ways to do the job but Scanner is originally intended to do the job.

Reading Data From a File

ioutil.ReadFile

If you just want the whole content in the file as a string, then the simple solution is to use the ReadFile function from the io/ioutil package. This function returns a slice of bytes which you can easily convert to a string.

func ReadFile(filename string) ([]byte, error)

ReadFile reads the file named by filename and returns the contents. A successful call returns err == nil, not err == EOF. Because ReadFile reads the whole file, it does not treat an EOF from Read as an error to be reported.

data, err := ioutil.ReadFile("file.txt") // just pass the file name

if err != nil {  // check errorpanic(err)
}

fmt.Println(data) // print the content as 'bytes'

str := string(data) // convert content to a 'string'

fmt.Println(str) // print the content as a 'string'

Note if the text file is in the same directory as your source code file, you can just pass the file name as the variable of the ReadFile function.

Readfile is nice. But you'll often want more control over how and what parts of a file are read. For these tasks, start by `Open`ing a file to obtain an `os.File` value.

f, err := os.Open("file.txt")

if err != nil {
   panic(err)
}

scanner := bufio.NewScanner(f)

for scanner.Scan() {           
   fmt.Println(scanner.Text())  
}

By default, scanner advances the token by newline, but of course you can customize how scanner should tokenise your file. Again, we use the bufio.NewScanner function to create a Scanner which uses an `os.File` value as the variable.

Besides bufio.NewScanner, there are many other ways to customize how to read a file. I will leave it to the readers for further exploration. For now, bufio.NewScanner is good enough.

Conclusions

By now, you should be comfortable with reading data from either a file or the console. I hope you enjoy my articles in the Mastering Golang Series. Keep learning! Keep growing!