Writing to files using buffers in Zig

In this post we’re going to go through the steps below:

1. Create a new file to write into.
2. Construct a buffer to temporarily hold our bytes in memory.
3. Get our writer interface.
4. Write bytes to the file via the writer interface.

There’s working code at the end of this post if you want to skip to that.

Create a file

#

Let’s start with the first step.

const std = @import("std");

pub fn main() !void {
  const file = try std.fs.cwd().createFile("example.txt", .{});
  defer file.close();
}

There’s not a whole lot going on here and should be straight forward.

Construct a buffer

#

Next we need a buffer. I’ll show two examples, a buffer on the stack, and a buffer on the heap. We’re going to proceed with the stack buffer but I want to show examples of both and two rules of thumb on how to choose when to use them.

Stack buffer

#

A buffer on the stack has a compile-time known size. Auto-growing buffers isn’t a good idea here and not something built into zig because of one of zig’s rules, no hidden allocations. Zig has predictable memory usage and deterministic behaviors because it prefers explicitness in order to keep the costs visible.

const std = @import("std");

pub fn main() !void {
  const file = try std.fs.cwd().createFile("example.txt", .{});
  defer file.close();

+ var buf: [2048]u8 = undefined;
}

Heap buffer

#

We’re using a GeneralPurposeAllocator¹ for learning purposes. A buffer on the heap has an unknown size at compile-time and can auto-grow to fit your needs with the constraints being available resources on the machine your code is running on.

const std = @import("std");

pub fn main() !void {
  const file = try std.fs.cwd().createFile("example.txt", .{});
  defer file.close();

+ var gpa = std.heap.GeneralPurposeAllocator(.{}){};
+ defer _ = gpa.deinit();
+
+ const allocator = gpa.allocator();
+
+ const buf = try allocator.alloc(u8, 2048);
+ defer allocator.free(buf);
}

Choosing between stack and heap buffers

#

When writing with a buffer what you’re trying to balance is syscalls vs memory. Syscalls have a performance impact and memory has a resource consumption impact. The smaller your buffer, the more syscalls are required, the larger your buffer the more memory is required (though fewer syscalls).

If your buffer size is known at compile-time or up to 16kb is enough for your use-case, then stack buffer is simpler and faster. If you don’t know how much buffer size you need or you know you’ll need more than 16kb, then buffer on the heap is likely your best bet.

We’ll continue with a stack buffer.

Get a writer interface

#

Let’s get our writer interface. Here, the caller owns the buffer, the writer borrows the buffer. To get our interface we first need to pass a pointer to our owned buffer, to our writer. This type of interface is called an intrusive interface, we’ll go over that in a different blog post.

const std = @import("std");

pub fn main() !void {
  const file = try std.fs.cwd().createFile("example.txt", .{});
  defer file.close();

  var buf: [2048]u8 = undefined;

  var file_writer = file.writer(&buf);
  const writer_iface: *std.Io.Writer = &file_writer.interface;
}

Write bytes to the file

#

When writing, bytes are copied into the buffer until it fills, at which point the buffer flushes to the file via the interface and the buffer pointer resets to the 0 index. Writing continues as many times as needed and a flush will write any remaining partially filled buffer to the file. This is why you don’t forget to flush².

const std = @import("std");

pub fn main() !void {
  const file = try std.fs.cwd().createFile("example.txt", .{});
  defer file.close();

  var buf: [2048]u8 = undefined;

  var file_writer = file.writer(&buf);
  const writer_iface: *std.Io.Writer = &file_writer.interface;

  try writer_iface.writeAll("this is some slice of bytes to store in a file\n");
  try writer_iface.flush();
}

Our final working code compiles and writes this is some slice of bytes to store in a file to the file example.txt in your current working directory.