Taras Bunyk

Embedding files in Go

Published: 2021-11-28T21:08:07.000Z

Go has a nice property - it compiles to single executable, that could just run. Which does not have depencencies to other packages, does not require runtime or interpreter.

But many of programs could not be built just in Go, and use multiple of different languages that are better fit for their tasks. For example in web you would probably use a lot of SQL, JavaScript, CSS, HTML, JSON, etc... Even desktop apps could use some embedded XML or GLSL, or even CSS for styling.

Yo dawg, we heard you like coding

There are two ways to put such code in your projects: as string literals in Go code, or as separate files. Separate files have advantage of having better modularity, and being more readable (less indentation, and it's easier to highlight file with extension .sql as SQL).

But text files in project that does not contain Go code have huge disadvantage - they are not included into the executable built by compiler, and need to be loaded during runtime. So you need to add them to Docker image, and distribute docker image instead. Or distribute your program in some other package format, that includes both executable and files.

Four years ago I have built a package & tool to fix that problem and embed text files into go executable as strings: https://github.com/bunyk/require

In that library you could call require.File("filename.txt"), and it will return content of that file as a string, without actually opening it. There is a hardcode utility that generates go code for each require.File call you have in your codebase to work.

But today I noticed two interesting lines in code of Neoray (My GUI of choice for Neovim):

//go:embed shader.glsl
var EmbeddedShaderSources string

Turns out that since Go 1.16, embed functionality is in standard library. Written by Russ Cox and reviewed by Rob Pike (commit). So, obviously it has better design than my package.

Small downside it has, is that you could use it only with global variables. Using embed directive in function gave me cannot apply to var inside func compilation error. And it does not work with constants, so you need to be careful to not mutate value yourself.

But, that are not really reasons to not deprecate my library. To quote fortune program used as an example:

One of my most productive days was throwing away 1000 lines of code. — Ken Thompson

package main

import (
	_ "embed"

//go:embed fortunes.txt
const contents string

func main() {
	fortunes := strings.Split(contents, "\n")