SIGN IN SIGN UP
//! Builds of the Zig compiler are distributed partly in source form. That
//! source lives here. These APIs are provided as-is and have absolutely no API
//! guarantees whatsoever.
progress towards semantic error serialization Introduces std.zig.ErrorBundle which is a trivially serializeable set of compilation errors. This is in the standard library so that both the compiler and the build runner can use it. The idea is they will use it to communicate compilation errors over a binary protocol. The binary encoding of ErrorBundle is a bit problematic - I got a little too aggressive with compaction. I need to change it in a follow-up commit to use some indirection in the error message list, otherwise iteration is too unergonomic. In fact it's so problematic right now that the logic getAllErrorsAlloc() actually fails to produce a viable ErrorBundle because it puts SourceLocation data in between the root level ErrorMessage data. This commit has a simplification - redundant logic for rendering AST errors to stderr has been removed in favor of moving the logic for lowering AST errors into AstGen. So even if we get parse errors, the errors will get lowered into ZIR before being reported. I believe this will be useful when working on --autofix. Either way, some redundant brittle logic was happily deleted. In Compilation, updateSubCompilation() is improved to properly perform error reporting when a sub-compilation object fails. It no longer dumps directly to stderr; instead it populates an ErrorBundle object, which gets added to the parent one during getAllErrorsAlloc(). In package fetching code, instead of dumping directly to stderr, it now populates an ErrorBundle object, and gets properly reported at the CLI layer of abstraction.
2023-02-23 16:18:43 -07:00
pub const ErrorBundle = @import("zig/ErrorBundle.zig");
pub const Server = @import("zig/Server.zig");
pub const Client = @import("zig/Client.zig");
2019-03-02 16:46:04 -05:00
pub const Token = tokenizer.Token;
pub const Tokenizer = tokenizer.Tokenizer;
stage2: *WIP*: rework ZIR memory layout; overhaul source locations The memory layout for ZIR instructions is completely reworked. See zir.zig for those changes. Some new types: * `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating each instruction independently, there is now a Tag and 8 bytes of data available for all ZIR instructions. Small instructions fit within these 8 bytes; larger ones use 4 bytes for an index into `extra`. There is also `string_bytes` so that we can have 4 byte references to strings. `zir.Inst.Tag` describes how to interpret those 8 bytes of data. - This is shared by all `Block` scopes. * `Module.WipZirCode`: represents an in-progress `zir.Code`. In this structure, the arrays are mutable, and get resized as we add/delete things. There is extra state to keep track of things. This struct is stored on the stack. Once it is finished, it produces an immutable `zir.Code`, which will remain on the heap for the duration of a function's existence. - This is shared by all `GenZir` scopes. * `Sema`: represents in-progress semantic analysis of a `zir.Code`. This data is stored on the stack and is shared among all `Block` scopes. It is now the main "self" argument to everything in the file that was previously named `zir_sema.zig`. Additionally, I moved some logic that was in `Module` into here. `Module.Fn` now stores its parameter names inside the `zir.Code`, instead of inside ZIR instructions. When the TZIR memory layout reworking time comes, codegen will be able to reference this data directly instead of duplicating it. astgen.zig is (so far) almost entirely untouched, but nearly all of it will need to be reworked to adhere to this new memory layout structure. I have no benchmarks to report yet, as I am still working through compile errors and fixing various things that I broke in this branch. Overhaul of Source Locations: Previously we used `usize` everywhere to mean byte offset, but sometimes also mean other stuff. This was error prone and also made us do unnecessary work, and store unnecessary bytes in memory. Now there are more types involved into source locations, and more ways to describe a source location. * AllErrors.Message: embrace the assumption that files always have less than 2 << 32 bytes. * SrcLoc gets more complicated, to model more complicated source locations. * Introduce LazySrcLoc, which can model interesting source locations with very little stored state. Useful for avoiding doing unnecessary work when no compile errors occur. Also, previously, we had `src: usize` on every ZIR instruction. This is no longer the case. Each instruction now determines whether it even cares about source location, and if so, how that source location is stored. This requires more careful work inside `Sema`, but it results in fewer bytes stored on the heap, without compromising accuracy and power of compile error messages. Miscellaneous: * std.zig: string literals have more helpful result values for reporting errors. There is now a lower level API and a higher level API. - side note: I noticed that the string literal logic needs some love. There is some unnecessarily hacky code there. * cut & pasted some TZIR logic that was in zir.zig to ir.zig. This probably broke stuff and needs to get fixed. * Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't think this quite how this code will be organized. Need some more careful planning about how to implement structs, unions, enums. They need to be independent Decls, just like a top level function.
2021-03-15 23:38:38 -07:00
pub const string_literal = @import("zig/string_literal.zig");
2022-08-31 13:36:48 +03:00
pub const number_literal = @import("zig/number_literal.zig");
2022-10-29 00:14:51 -07:00
pub const primitives = @import("zig/primitives.zig");
pub const isPrimitive = primitives.isPrimitive;
pub const Ast = @import("zig/Ast.zig");
pub const AstGen = @import("zig/AstGen.zig");
pub const Zir = @import("zig/Zir.zig");
pub const Zoir = @import("zig/Zoir.zig");
pub const ZonGen = @import("zig/ZonGen.zig");
pub const system = @import("zig/system.zig");
pub const CrossTarget = @compileError("deprecated; use std.Target.Query");
pub const BuiltinFn = @import("zig/BuiltinFn.zig");
pub const AstRlAnnotate = @import("zig/AstRlAnnotate.zig");
pub const LibCInstallation = @import("zig/LibCInstallation.zig");
pub const WindowsSdk = @import("zig/WindowsSdk.zig");
pub const LibCDirs = @import("zig/LibCDirs.zig");
pub const target = @import("zig/target.zig");
pub const llvm = @import("zig/llvm.zig");
2019-03-02 16:46:04 -05:00
// Character literal parsing
pub const ParsedCharLiteral = string_literal.ParsedCharLiteral;
pub const parseCharLiteral = string_literal.parseCharLiteral;
2022-08-31 13:36:48 +03:00
pub const parseNumberLiteral = number_literal.parseNumberLiteral;
// Files needed by translate-c.
pub const c_builtins = @import("zig/c_builtins.zig");
pub const c_translation = @import("zig/c_translation.zig");
pub const SrcHasher = std.crypto.hash.Blake3;
pub const SrcHash = [16]u8;
2024-02-26 22:26:19 -07:00
pub const Color = enum {
/// Determine whether stderr is a terminal or not automatically.
auto,
/// Assume stderr is not a terminal.
off,
/// Assume stderr is a terminal.
on,
pub fn get_tty_conf(color: Color) std.io.tty.Config {
return switch (color) {
.auto => std.io.tty.detectConfig(std.io.getStdErr()),
.on => .escape_codes,
.off => .no_color,
};
}
pub fn renderOptions(color: Color) std.zig.ErrorBundle.RenderOptions {
return .{
.ttyconf = get_tty_conf(color),
2024-02-26 22:26:19 -07:00
};
}
};
/// There are many assumptions in the entire codebase that Zig source files can
/// be byte-indexed with a u32 integer.
pub const max_src_size = std.math.maxInt(u32);
pub fn hashSrc(src: []const u8) SrcHash {
var out: SrcHash = undefined;
SrcHasher.hash(src, &out, .{});
stage2: blaze the trail for std lib integration This branch adds "builtin" and "std" to the import table when using the self-hosted backend. "builtin" gains one additional item: ``` pub const zig_is_stage2 = true; // false when using stage1 backend ``` This allows the std lib to do conditional compilation based on detecting which backend is being used. This will be removed from builtin as soon as self-hosted catches up to feature parity with stage1. Keep a sharp eye out - people are going to be tempted to abuse this. The general rule of thumb is do not use `builtin.zig_is_stage2`. However this commit breaks the rule so that we can gain limited start.zig support as we incrementally improve the self-hosted compiler. This commit also implements `fullyQualifiedNameHash` and related functionality, which effectively puts all Decls in their proper namespaces. `fullyQualifiedName` is not yet implemented. Stop printing "todo" log messages for test decls unless we are in test mode. Add "previous definition here" error notes for Decl name collisions. This commit does not bring us yet to a newly passing test case. Here's what I'm working towards: ```zig const std = @import("std"); export fn main() c_int { const a = std.fs.base64_alphabet[0]; return a - 'A'; } ``` Current output: ``` $ ./zig-cache/bin/zig build-exe test.zig test.zig:3:1: error: TODO implement more analyze elemptr zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty ``` So the next steps are clear: * Sema: improve elemptr * AstGen: implement structInitExpr
2021-04-08 19:05:05 -07:00
return out;
}
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
pub fn srcHashEql(a: SrcHash, b: SrcHash) bool {
return @as(u128, @bitCast(a)) == @as(u128, @bitCast(b));
stage2: rewire the frontend driver to whole-file-zir * Remove some unused imports in AstGen.zig. I think it would make sense to start decoupling AstGen from the rest of the compiler code, similar to how the tokenizer and parser are decoupled. * AstGen: For decls, move the block_inline instructions to the top of the function so that they get lower ZIR instruction indexes. With this, the block_inline instruction index combined with its corresponding break_inline instruction index can be used to form a ZIR instruction range. This is useful for allocating an array to map ZIR instructions to semantically analyzed instructions. * Module: extract emit-h functionality into a struct, and only allocate it when emit-h is activated. * Module: remove the `decl_table` field. This previously was a table of all Decls in the entire Module. A "name hash" strategy was used to find decls within a given namespace, using this global table. Now, each Namespace has its own map of name to children Decls. - Additionally, there were 3 places that relied on iterating over decl_table in order to function: - C backend and SPIR-V backend. These now have their own decl_table that they keep populated when `updateDecl` and `removeDecl` are called. - emit-h. A `decl_table` field has been added to the new GlobalEmitH struct which is only allocated when emit-h is activated. * Module: fix ZIR serialization/deserialization bug in debug mode having to do with the secret safety tag for untagged unions. There is still an open TODO to investigate a friendlier solution to this problem with the language. * Module: improve deserialization of ZIR to allocate only exactly as much capacity as length in the instructions array so as to not waste space. * Module: move `srcHashEql` to `std.zig` to live next to the definition of `SrcHash` itself. * Module: re-introduce the logic for scanning top level declarations within a namespace. * Compilation: add an `analyze_pkg` Job which is used to kick off the start of semantic analysis by doing the equivalent of `_ = @import("std");`. The `analyze_pkg` job is unconditionally added to the work queue on every update(), with pkg set to the std lib pkg. * Rename TZIR to AIR in a few places. A more comprehensive rename will come later.
2021-04-26 20:41:07 -07:00
}
stage2: blaze the trail for std lib integration This branch adds "builtin" and "std" to the import table when using the self-hosted backend. "builtin" gains one additional item: ``` pub const zig_is_stage2 = true; // false when using stage1 backend ``` This allows the std lib to do conditional compilation based on detecting which backend is being used. This will be removed from builtin as soon as self-hosted catches up to feature parity with stage1. Keep a sharp eye out - people are going to be tempted to abuse this. The general rule of thumb is do not use `builtin.zig_is_stage2`. However this commit breaks the rule so that we can gain limited start.zig support as we incrementally improve the self-hosted compiler. This commit also implements `fullyQualifiedNameHash` and related functionality, which effectively puts all Decls in their proper namespaces. `fullyQualifiedName` is not yet implemented. Stop printing "todo" log messages for test decls unless we are in test mode. Add "previous definition here" error notes for Decl name collisions. This commit does not bring us yet to a newly passing test case. Here's what I'm working towards: ```zig const std = @import("std"); export fn main() c_int { const a = std.fs.base64_alphabet[0]; return a - 'A'; } ``` Current output: ``` $ ./zig-cache/bin/zig build-exe test.zig test.zig:3:1: error: TODO implement more analyze elemptr zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty ``` So the next steps are clear: * Sema: improve elemptr * AstGen: implement structInitExpr
2021-04-08 19:05:05 -07:00
pub fn hashName(parent_hash: SrcHash, sep: []const u8, name: []const u8) SrcHash {
var out: SrcHash = undefined;
var hasher = SrcHasher.init(.{});
stage2: blaze the trail for std lib integration This branch adds "builtin" and "std" to the import table when using the self-hosted backend. "builtin" gains one additional item: ``` pub const zig_is_stage2 = true; // false when using stage1 backend ``` This allows the std lib to do conditional compilation based on detecting which backend is being used. This will be removed from builtin as soon as self-hosted catches up to feature parity with stage1. Keep a sharp eye out - people are going to be tempted to abuse this. The general rule of thumb is do not use `builtin.zig_is_stage2`. However this commit breaks the rule so that we can gain limited start.zig support as we incrementally improve the self-hosted compiler. This commit also implements `fullyQualifiedNameHash` and related functionality, which effectively puts all Decls in their proper namespaces. `fullyQualifiedName` is not yet implemented. Stop printing "todo" log messages for test decls unless we are in test mode. Add "previous definition here" error notes for Decl name collisions. This commit does not bring us yet to a newly passing test case. Here's what I'm working towards: ```zig const std = @import("std"); export fn main() c_int { const a = std.fs.base64_alphabet[0]; return a - 'A'; } ``` Current output: ``` $ ./zig-cache/bin/zig build-exe test.zig test.zig:3:1: error: TODO implement more analyze elemptr zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty ``` So the next steps are clear: * Sema: improve elemptr * AstGen: implement structInitExpr
2021-04-08 19:05:05 -07:00
hasher.update(&parent_hash);
hasher.update(sep);
hasher.update(name);
hasher.final(&out);
return out;
}
pub const Loc = struct {
line: usize,
column: usize,
/// Does not include the trailing newline.
source_line: []const u8,
pub fn eql(a: Loc, b: Loc) bool {
return a.line == b.line and a.column == b.column and std.mem.eql(u8, a.source_line, b.source_line);
}
};
pub fn findLineColumn(source: []const u8, byte_offset: usize) Loc {
var line: usize = 0;
var column: usize = 0;
var line_start: usize = 0;
var i: usize = 0;
while (i < byte_offset) : (i += 1) {
switch (source[i]) {
'\n' => {
line += 1;
column = 0;
line_start = i + 1;
},
else => {
column += 1;
},
}
}
while (i < source.len and source[i] != '\n') {
i += 1;
}
return .{
.line = line,
.column = column,
.source_line = source[line_start..i],
};
}
pub fn lineDelta(source: []const u8, start: usize, end: usize) isize {
var line: isize = 0;
if (end >= start) {
for (source[start..end]) |byte| switch (byte) {
'\n' => line += 1,
else => continue,
};
} else {
for (source[end..start]) |byte| switch (byte) {
'\n' => line -= 1,
else => continue,
};
}
return line;
}
pub const BinNameOptions = struct {
root_name: []const u8,
target: std.Target,
output_mode: std.builtin.OutputMode,
link_mode: ?std.builtin.LinkMode = null,
version: ?std.SemanticVersion = null,
};
/// Returns the standard file system basename of a binary generated by the Zig compiler.
pub fn binNameAlloc(allocator: Allocator, options: BinNameOptions) error{OutOfMemory}![]u8 {
const root_name = options.root_name;
const t = options.target;
switch (t.ofmt) {
.coff => switch (options.output_mode) {
.Exe => return std.fmt.allocPrint(allocator, "{s}{s}", .{ root_name, t.exeFileExt() }),
.Lib => {
const suffix = switch (options.link_mode orelse .static) {
.static => ".lib",
.dynamic => ".dll",
};
return std.fmt.allocPrint(allocator, "{s}{s}", .{ root_name, suffix });
},
.Obj => return std.fmt.allocPrint(allocator, "{s}.obj", .{root_name}),
},
.elf, .goff, .xcoff => switch (options.output_mode) {
.Exe => return allocator.dupe(u8, root_name),
.Lib => {
switch (options.link_mode orelse .static) {
.static => return std.fmt.allocPrint(allocator, "{s}{s}.a", .{
t.libPrefix(), root_name,
}),
.dynamic => {
if (options.version) |ver| {
return std.fmt.allocPrint(allocator, "{s}{s}.so.{d}.{d}.{d}", .{
t.libPrefix(), root_name, ver.major, ver.minor, ver.patch,
});
} else {
return std.fmt.allocPrint(allocator, "{s}{s}.so", .{
t.libPrefix(), root_name,
});
}
},
}
},
.Obj => return std.fmt.allocPrint(allocator, "{s}.o", .{root_name}),
},
.macho => switch (options.output_mode) {
.Exe => return allocator.dupe(u8, root_name),
.Lib => {
switch (options.link_mode orelse .static) {
.static => return std.fmt.allocPrint(allocator, "{s}{s}.a", .{
t.libPrefix(), root_name,
}),
.dynamic => {
if (options.version) |ver| {
return std.fmt.allocPrint(allocator, "{s}{s}.{d}.{d}.{d}.dylib", .{
t.libPrefix(), root_name, ver.major, ver.minor, ver.patch,
});
} else {
return std.fmt.allocPrint(allocator, "{s}{s}.dylib", .{
t.libPrefix(), root_name,
});
}
},
}
},
.Obj => return std.fmt.allocPrint(allocator, "{s}.o", .{root_name}),
},
.wasm => switch (options.output_mode) {
.Exe => return std.fmt.allocPrint(allocator, "{s}{s}", .{ root_name, t.exeFileExt() }),
.Lib => {
switch (options.link_mode orelse .static) {
.static => return std.fmt.allocPrint(allocator, "{s}{s}.a", .{
t.libPrefix(), root_name,
}),
.dynamic => return std.fmt.allocPrint(allocator, "{s}.wasm", .{root_name}),
}
},
.Obj => return std.fmt.allocPrint(allocator, "{s}.o", .{root_name}),
},
.c => return std.fmt.allocPrint(allocator, "{s}.c", .{root_name}),
.spirv => return std.fmt.allocPrint(allocator, "{s}.spv", .{root_name}),
.hex => return std.fmt.allocPrint(allocator, "{s}.ihex", .{root_name}),
.raw => return std.fmt.allocPrint(allocator, "{s}.bin", .{root_name}),
.plan9 => switch (options.output_mode) {
.Exe => return allocator.dupe(u8, root_name),
2022-08-18 18:58:28 -07:00
.Obj => return std.fmt.allocPrint(allocator, "{s}{s}", .{
root_name, t.ofmt.fileExt(t.cpu.arch),
2022-08-18 18:58:28 -07:00
}),
.Lib => return std.fmt.allocPrint(allocator, "{s}{s}.a", .{
t.libPrefix(), root_name,
2022-08-18 18:58:28 -07:00
}),
},
}
}
pub const SanitizeC = enum {
off,
trap,
full,
};
zig build system: change target, compilation, and module APIs Introduce the concept of "target query" and "resolved target". A target query is what the user specifies, with some things left to default. A resolved target has the default things discovered and populated. In the future, std.zig.CrossTarget will be rename to std.Target.Query. Introduces `std.Build.resolveTargetQuery` to get from one to the other. The concept of `main_mod_path` is gone, no longer supported. You have to put the root source file at the module root now. * remove deprecated API * update build.zig for the breaking API changes in this branch * move std.Build.Step.Compile.BuildId to std.zig.BuildId * add more options to std.Build.ExecutableOptions, std.Build.ObjectOptions, std.Build.SharedLibraryOptions, std.Build.StaticLibraryOptions, and std.Build.TestOptions. * remove `std.Build.constructCMacro`. There is no use for this API. * deprecate `std.Build.Step.Compile.defineCMacro`. Instead, `std.Build.Module.addCMacro` is provided. - remove `std.Build.Step.Compile.defineCMacroRaw`. * deprecate `std.Build.Step.Compile.linkFrameworkNeeded` - use `std.Build.Module.linkFramework` * deprecate `std.Build.Step.Compile.linkFrameworkWeak` - use `std.Build.Module.linkFramework` * move more logic into `std.Build.Module` * allow `target` and `optimize` to be `null` when creating a Module. Along with other fields, those unspecified options will be inherited from parent `Module` when inserted into an import table. * the `target` field of `addExecutable` is now required. pass `b.host` to get the host target.
2023-12-02 21:51:34 -07:00
pub const BuildId = union(enum) {
none,
fast,
uuid,
sha1,
md5,
hexstring: HexString,
pub fn eql(a: BuildId, b: BuildId) bool {
const Tag = @typeInfo(BuildId).@"union".tag_type.?;
zig build system: change target, compilation, and module APIs Introduce the concept of "target query" and "resolved target". A target query is what the user specifies, with some things left to default. A resolved target has the default things discovered and populated. In the future, std.zig.CrossTarget will be rename to std.Target.Query. Introduces `std.Build.resolveTargetQuery` to get from one to the other. The concept of `main_mod_path` is gone, no longer supported. You have to put the root source file at the module root now. * remove deprecated API * update build.zig for the breaking API changes in this branch * move std.Build.Step.Compile.BuildId to std.zig.BuildId * add more options to std.Build.ExecutableOptions, std.Build.ObjectOptions, std.Build.SharedLibraryOptions, std.Build.StaticLibraryOptions, and std.Build.TestOptions. * remove `std.Build.constructCMacro`. There is no use for this API. * deprecate `std.Build.Step.Compile.defineCMacro`. Instead, `std.Build.Module.addCMacro` is provided. - remove `std.Build.Step.Compile.defineCMacroRaw`. * deprecate `std.Build.Step.Compile.linkFrameworkNeeded` - use `std.Build.Module.linkFramework` * deprecate `std.Build.Step.Compile.linkFrameworkWeak` - use `std.Build.Module.linkFramework` * move more logic into `std.Build.Module` * allow `target` and `optimize` to be `null` when creating a Module. Along with other fields, those unspecified options will be inherited from parent `Module` when inserted into an import table. * the `target` field of `addExecutable` is now required. pass `b.host` to get the host target.
2023-12-02 21:51:34 -07:00
const a_tag: Tag = a;
const b_tag: Tag = b;
if (a_tag != b_tag) return false;
return switch (a) {
.none, .fast, .uuid, .sha1, .md5 => true,
.hexstring => |a_hexstring| std.mem.eql(u8, a_hexstring.toSlice(), b.hexstring.toSlice()),
};
}
pub const HexString = struct {
bytes: [32]u8,
len: u8,
/// Result is byte values, *not* hex-encoded.
pub fn toSlice(hs: *const HexString) []const u8 {
return hs.bytes[0..hs.len];
}
};
/// Input is byte values, *not* hex-encoded.
/// Asserts `bytes` fits inside `HexString`
pub fn initHexString(bytes: []const u8) BuildId {
var result: BuildId = .{ .hexstring = .{
.bytes = undefined,
.len = @intCast(bytes.len),
} };
@memcpy(result.hexstring.bytes[0..bytes.len], bytes);
return result;
}
/// Converts UTF-8 text to a `BuildId`.
pub fn parse(text: []const u8) !BuildId {
if (std.mem.eql(u8, text, "none")) {
return .none;
} else if (std.mem.eql(u8, text, "fast")) {
return .fast;
} else if (std.mem.eql(u8, text, "uuid")) {
return .uuid;
} else if (std.mem.eql(u8, text, "sha1") or std.mem.eql(u8, text, "tree")) {
return .sha1;
} else if (std.mem.eql(u8, text, "md5")) {
return .md5;
} else if (std.mem.startsWith(u8, text, "0x")) {
var result: BuildId = .{ .hexstring = undefined };
const slice = try std.fmt.hexToBytes(&result.hexstring.bytes, text[2..]);
result.hexstring.len = @as(u8, @intCast(slice.len));
return result;
}
return error.InvalidBuildIdStyle;
}
test parse {
try std.testing.expectEqual(BuildId.md5, try parse("md5"));
try std.testing.expectEqual(BuildId.none, try parse("none"));
try std.testing.expectEqual(BuildId.fast, try parse("fast"));
try std.testing.expectEqual(BuildId.uuid, try parse("uuid"));
try std.testing.expectEqual(BuildId.sha1, try parse("sha1"));
try std.testing.expectEqual(BuildId.sha1, try parse("tree"));
try std.testing.expect(BuildId.initHexString("").eql(try parse("0x")));
try std.testing.expect(BuildId.initHexString("\x12\x34\x56").eql(try parse("0x123456")));
try std.testing.expectError(error.InvalidLength, parse("0x12-34"));
try std.testing.expectError(error.InvalidCharacter, parse("0xfoobbb"));
try std.testing.expectError(error.InvalidBuildIdStyle, parse("yaddaxxx"));
}
};
pub const LtoMode = enum { none, full, thin };
/// Renders a `std.Target.Cpu` value into a textual representation that can be parsed
/// via the `-mcpu` flag passed to the Zig compiler.
/// Appends the result to `buffer`.
pub fn serializeCpu(buffer: *std.ArrayList(u8), cpu: std.Target.Cpu) Allocator.Error!void {
const all_features = cpu.arch.allFeaturesList();
var populated_cpu_features = cpu.model.features;
populated_cpu_features.populateDependencies(all_features);
try buffer.appendSlice(cpu.model.name);
if (populated_cpu_features.eql(cpu.features)) {
// The CPU name alone is sufficient.
return;
}
for (all_features, 0..) |feature, i_usize| {
const i: std.Target.Cpu.Feature.Set.Index = @intCast(i_usize);
const in_cpu_set = populated_cpu_features.isEnabled(i);
const in_actual_set = cpu.features.isEnabled(i);
try buffer.ensureUnusedCapacity(feature.name.len + 1);
if (in_cpu_set and !in_actual_set) {
buffer.appendAssumeCapacity('-');
buffer.appendSliceAssumeCapacity(feature.name);
} else if (!in_cpu_set and in_actual_set) {
buffer.appendAssumeCapacity('+');
buffer.appendSliceAssumeCapacity(feature.name);
}
}
}
pub fn serializeCpuAlloc(ally: Allocator, cpu: std.Target.Cpu) Allocator.Error![]u8 {
var buffer = std.ArrayList(u8).init(ally);
try serializeCpu(&buffer, cpu);
return buffer.toOwnedSlice();
}
const std = @import("std.zig");
const tokenizer = @import("zig/tokenizer.zig");
const assert = std.debug.assert;
const Allocator = std.mem.Allocator;
/// Return a Formatter for a Zig identifier, escaping it with `@""` syntax if needed.
///
/// - An empty `{}` format specifier escapes invalid identifiers, identifiers that shadow primitives
/// and the reserved `_` identifier.
/// - Add `p` to the specifier to render identifiers that shadow primitives unescaped.
/// - Add `_` to the specifier to render the reserved `_` identifier unescaped.
/// - `p` and `_` can be combined, e.g. `{p_}`.
///
pub fn fmtId(bytes: []const u8) std.fmt.Formatter(formatId) {
return .{ .data = bytes };
}
test fmtId {
const expectFmt = std.testing.expectFmt;
try expectFmt("@\"while\"", "{}", .{fmtId("while")});
try expectFmt("@\"while\"", "{p}", .{fmtId("while")});
try expectFmt("@\"while\"", "{_}", .{fmtId("while")});
try expectFmt("@\"while\"", "{p_}", .{fmtId("while")});
try expectFmt("@\"while\"", "{_p}", .{fmtId("while")});
try expectFmt("hello", "{}", .{fmtId("hello")});
try expectFmt("hello", "{p}", .{fmtId("hello")});
try expectFmt("hello", "{_}", .{fmtId("hello")});
try expectFmt("hello", "{p_}", .{fmtId("hello")});
try expectFmt("hello", "{_p}", .{fmtId("hello")});
try expectFmt("@\"type\"", "{}", .{fmtId("type")});
try expectFmt("type", "{p}", .{fmtId("type")});
try expectFmt("@\"type\"", "{_}", .{fmtId("type")});
try expectFmt("type", "{p_}", .{fmtId("type")});
try expectFmt("type", "{_p}", .{fmtId("type")});
try expectFmt("@\"_\"", "{}", .{fmtId("_")});
try expectFmt("@\"_\"", "{p}", .{fmtId("_")});
try expectFmt("_", "{_}", .{fmtId("_")});
try expectFmt("_", "{p_}", .{fmtId("_")});
try expectFmt("_", "{_p}", .{fmtId("_")});
try expectFmt("@\"i123\"", "{}", .{fmtId("i123")});
try expectFmt("i123", "{p}", .{fmtId("i123")});
try expectFmt("@\"4four\"", "{}", .{fmtId("4four")});
try expectFmt("_underscore", "{}", .{fmtId("_underscore")});
try expectFmt("@\"11\\\"23\"", "{}", .{fmtId("11\"23")});
try expectFmt("@\"11\\x0f23\"", "{}", .{fmtId("11\x0F23")});
// These are technically not currently legal in Zig.
try expectFmt("@\"\"", "{}", .{fmtId("")});
try expectFmt("@\"\\x00\"", "{}", .{fmtId("\x00")});
}
/// Print the string as a Zig identifier, escaping it with `@""` syntax if needed.
fn formatId(
bytes: []const u8,
comptime fmt: []const u8,
options: std.fmt.FormatOptions,
writer: anytype,
) !void {
const allow_primitive, const allow_underscore = comptime parse_fmt: {
var allow_primitive = false;
var allow_underscore = false;
for (fmt) |char| {
switch (char) {
'p' => if (!allow_primitive) {
allow_primitive = true;
continue;
},
'_' => if (!allow_underscore) {
allow_underscore = true;
continue;
},
else => {},
}
@compileError("expected {}, {p}, {_}, {p_} or {_p}, found {" ++ fmt ++ "}");
}
break :parse_fmt .{ allow_primitive, allow_underscore };
};
if (isValidId(bytes) and
(allow_primitive or !std.zig.isPrimitive(bytes)) and
(allow_underscore or !isUnderscore(bytes)))
{
return writer.writeAll(bytes);
}
try writer.writeAll("@\"");
try stringEscape(bytes, "", options, writer);
try writer.writeByte('"');
}
/// Return a Formatter for Zig Escapes of a double quoted string.
/// The format specifier must be one of:
/// * `{}` treats contents as a double-quoted string.
/// * `{'}` treats contents as a single-quoted string.
pub fn fmtEscapes(bytes: []const u8) std.fmt.Formatter(stringEscape) {
return .{ .data = bytes };
}
test fmtEscapes {
const expectFmt = std.testing.expectFmt;
try expectFmt("\\x0f", "{}", .{fmtEscapes("\x0f")});
try expectFmt(
\\" \\ hi \x07 \x11 " derp \'"
, "\"{'}\"", .{fmtEscapes(" \\ hi \x07 \x11 \" derp '")});
try expectFmt(
\\" \\ hi \x07 \x11 \" derp '"
, "\"{}\"", .{fmtEscapes(" \\ hi \x07 \x11 \" derp '")});
}
/// Print the string as escaped contents of a double quoted or single-quoted string.
/// Format `{}` treats contents as a double-quoted string.
/// Format `{'}` treats contents as a single-quoted string.
pub fn stringEscape(
bytes: []const u8,
comptime f: []const u8,
options: std.fmt.FormatOptions,
writer: anytype,
) !void {
_ = options;
for (bytes) |byte| switch (byte) {
'\n' => try writer.writeAll("\\n"),
'\r' => try writer.writeAll("\\r"),
'\t' => try writer.writeAll("\\t"),
'\\' => try writer.writeAll("\\\\"),
'"' => {
if (f.len == 1 and f[0] == '\'') {
try writer.writeByte('"');
} else if (f.len == 0) {
try writer.writeAll("\\\"");
} else {
@compileError("expected {} or {'}, found {" ++ f ++ "}");
}
},
'\'' => {
if (f.len == 1 and f[0] == '\'') {
try writer.writeAll("\\'");
} else if (f.len == 0) {
try writer.writeByte('\'');
} else {
@compileError("expected {} or {'}, found {" ++ f ++ "}");
}
},
' ', '!', '#'...'&', '('...'[', ']'...'~' => try writer.writeByte(byte),
// Use hex escapes for rest any unprintable characters.
else => {
try writer.writeAll("\\x");
try std.fmt.formatInt(byte, 16, .lower, .{ .width = 2, .fill = '0' }, writer);
},
};
}
pub fn isValidId(bytes: []const u8) bool {
if (bytes.len == 0) return false;
for (bytes, 0..) |c, i| {
switch (c) {
'_', 'a'...'z', 'A'...'Z' => {},
'0'...'9' => if (i == 0) return false,
else => return false,
}
}
return std.zig.Token.getKeyword(bytes) == null;
}
test isValidId {
try std.testing.expect(!isValidId(""));
try std.testing.expect(isValidId("foobar"));
try std.testing.expect(!isValidId("a b c"));
try std.testing.expect(!isValidId("3d"));
try std.testing.expect(!isValidId("enum"));
try std.testing.expect(isValidId("i386"));
}
pub fn isUnderscore(bytes: []const u8) bool {
return bytes.len == 1 and bytes[0] == '_';
}
test isUnderscore {
try std.testing.expect(isUnderscore("_"));
try std.testing.expect(!isUnderscore("__"));
try std.testing.expect(!isUnderscore("_foo"));
try std.testing.expect(isUnderscore("\x5f"));
try std.testing.expect(!isUnderscore("\\x5f"));
}
pub fn readSourceFileToEndAlloc(gpa: Allocator, input: std.fs.File, size_hint: ?usize) ![:0]u8 {
2024-02-26 22:26:19 -07:00
const source_code = input.readToEndAllocOptions(
gpa,
2024-02-26 22:26:19 -07:00
max_src_size,
size_hint,
.of(u8),
2024-02-26 22:26:19 -07:00
0,
) catch |err| switch (err) {
error.ConnectionResetByPeer => unreachable,
error.ConnectionTimedOut => unreachable,
error.NotOpenForReading => unreachable,
else => |e| return e,
};
errdefer gpa.free(source_code);
2024-02-26 22:26:19 -07:00
// Detect unsupported file types with their Byte Order Mark
const unsupported_boms = [_][]const u8{
"\xff\xfe\x00\x00", // UTF-32 little endian
"\xfe\xff\x00\x00", // UTF-32 big endian
"\xfe\xff", // UTF-16 big endian
};
for (unsupported_boms) |bom| {
if (std.mem.startsWith(u8, source_code, bom)) {
return error.UnsupportedEncoding;
}
}
// If the file starts with a UTF-16 little endian BOM, translate it to UTF-8
if (std.mem.startsWith(u8, source_code, "\xff\xfe")) {
if (source_code.len % 2 != 0) return error.InvalidEncoding;
// TODO: after wrangle-writer-buffering branch is merged,
// avoid this unnecessary allocation
const aligned_copy = try gpa.alloc(u16, source_code.len / 2);
defer gpa.free(aligned_copy);
@memcpy(std.mem.sliceAsBytes(aligned_copy), source_code);
const source_code_utf8 = std.unicode.utf16LeToUtf8AllocZ(gpa, aligned_copy) catch |err| switch (err) {
2024-02-26 22:26:19 -07:00
error.DanglingSurrogateHalf => error.UnsupportedEncoding,
error.ExpectedSecondSurrogateHalf => error.UnsupportedEncoding,
error.UnexpectedSecondSurrogateHalf => error.UnsupportedEncoding,
else => |e| return e,
};
gpa.free(source_code);
2024-02-26 22:26:19 -07:00
return source_code_utf8;
}
return source_code;
}
pub fn printAstErrorsToStderr(gpa: Allocator, tree: Ast, path: []const u8, color: Color) !void {
var wip_errors: std.zig.ErrorBundle.Wip = undefined;
try wip_errors.init(gpa);
defer wip_errors.deinit();
try putAstErrorsIntoBundle(gpa, tree, path, &wip_errors);
var error_bundle = try wip_errors.toOwnedBundle("");
defer error_bundle.deinit(gpa);
error_bundle.renderToStdErr(color.renderOptions());
}
pub fn putAstErrorsIntoBundle(
gpa: Allocator,
tree: Ast,
path: []const u8,
wip_errors: *std.zig.ErrorBundle.Wip,
) Allocator.Error!void {
var zir = try AstGen.generate(gpa, tree);
defer zir.deinit(gpa);
try wip_errors.addZirErrorMessages(zir, tree, tree.source, path);
}
pub fn resolveTargetQueryOrFatal(target_query: std.Target.Query) std.Target {
return std.zig.system.resolveTargetQuery(target_query) catch |err|
fatal("unable to resolve target: {s}", .{@errorName(err)});
}
pub fn parseTargetQueryOrReportFatalError(
allocator: Allocator,
opts: std.Target.Query.ParseOptions,
) std.Target.Query {
var opts_with_diags = opts;
var diags: std.Target.Query.ParseOptions.Diagnostics = .{};
if (opts_with_diags.diagnostics == null) {
opts_with_diags.diagnostics = &diags;
}
return std.Target.Query.parse(opts_with_diags) catch |err| switch (err) {
error.UnknownCpuModel => {
help: {
var help_text = std.ArrayList(u8).init(allocator);
defer help_text.deinit();
for (diags.arch.?.allCpuModels()) |cpu| {
help_text.writer().print(" {s}\n", .{cpu.name}) catch break :help;
}
std.log.info("available CPUs for architecture '{s}':\n{s}", .{
@tagName(diags.arch.?), help_text.items,
});
}
fatal("unknown CPU: '{s}'", .{diags.cpu_name.?});
},
error.UnknownCpuFeature => {
help: {
var help_text = std.ArrayList(u8).init(allocator);
defer help_text.deinit();
for (diags.arch.?.allFeaturesList()) |feature| {
help_text.writer().print(" {s}: {s}\n", .{ feature.name, feature.description }) catch break :help;
}
std.log.info("available CPU features for architecture '{s}':\n{s}", .{
@tagName(diags.arch.?), help_text.items,
});
}
fatal("unknown CPU feature: '{s}'", .{diags.unknown_feature_name.?});
},
error.UnknownObjectFormat => {
help: {
var help_text = std.ArrayList(u8).init(allocator);
defer help_text.deinit();
inline for (@typeInfo(std.Target.ObjectFormat).@"enum".fields) |field| {
help_text.writer().print(" {s}\n", .{field.name}) catch break :help;
}
std.log.info("available object formats:\n{s}", .{help_text.items});
}
fatal("unknown object format: '{s}'", .{opts.object_format.?});
},
error.UnknownArchitecture => {
help: {
var help_text = std.ArrayList(u8).init(allocator);
defer help_text.deinit();
inline for (@typeInfo(std.Target.Cpu.Arch).@"enum".fields) |field| {
help_text.writer().print(" {s}\n", .{field.name}) catch break :help;
}
std.log.info("available architectures:\n{s} native\n", .{help_text.items});
}
fatal("unknown architecture: '{s}'", .{diags.unknown_architecture_name.?});
},
else => |e| fatal("unable to parse target query '{s}': {s}", .{
opts.arch_os_abi, @errorName(e),
}),
};
}
/// Deprecated; see `std.process.fatal`.
pub const fatal = std.process.fatal;
/// Collects all the environment variables that Zig could possibly inspect, so
/// that we can do reflection on this and print them with `zig env`.
pub const EnvVar = enum {
ZIG_GLOBAL_CACHE_DIR,
ZIG_LOCAL_CACHE_DIR,
ZIG_LIB_DIR,
ZIG_LIBC,
ZIG_BUILD_RUNNER,
ZIG_VERBOSE_LINK,
ZIG_VERBOSE_CC,
ZIG_BTRFS_WORKAROUND,
ZIG_DEBUG_CMD,
CC,
NO_COLOR,
CLICOLOR_FORCE,
XDG_CACHE_HOME,
HOME,
pub fn isSet(comptime ev: EnvVar) bool {
return std.process.hasNonEmptyEnvVarConstant(@tagName(ev));
}
pub fn get(ev: EnvVar, arena: std.mem.Allocator) !?[]u8 {
if (std.process.getEnvVarOwned(arena, @tagName(ev))) |value| {
return value;
} else |err| switch (err) {
error.EnvironmentVariableNotFound => return null,
else => |e| return e,
}
}
pub fn getPosix(comptime ev: EnvVar) ?[:0]const u8 {
return std.posix.getenvZ(@tagName(ev));
}
};
pub const SimpleComptimeReason = enum(u32) {
// Evaluating at comptime because a builtin operand must be comptime-known.
// These messages all mention a specific builtin.
operand_Type,
operand_setEvalBranchQuota,
operand_setFloatMode,
operand_branchHint,
operand_setRuntimeSafety,
operand_embedFile,
operand_cImport,
operand_cDefine_macro_name,
operand_cDefine_macro_value,
operand_cInclude_file_name,
operand_cUndef_macro_name,
operand_shuffle_mask,
operand_atomicRmw_operation,
operand_reduce_operation,
// Evaluating at comptime because an operand must be comptime-known.
// These messages do not mention a specific builtin (and may not be about a builtin at all).
export_target,
export_options,
extern_options,
prefetch_options,
call_modifier,
compile_error_string,
inline_assembly_code,
atomic_order,
array_mul_factor,
slice_cat_operand,
inline_call_target,
generic_call_target,
wasm_memory_index,
work_group_dim_index,
// Evaluating at comptime because types must be comptime-known.
// Reasons other than `.type` are just more specific messages.
type,
array_sentinel,
pointer_sentinel,
slice_sentinel,
array_length,
vector_length,
error_set_contents,
struct_fields,
enum_fields,
union_fields,
function_ret_ty,
function_parameters,
// Evaluating at comptime because decl/field name must be comptime-known.
decl_name,
field_name,
struct_field_name,
enum_field_name,
union_field_name,
tuple_field_name,
tuple_field_index,
// Evaluating at comptime because it is an attribute of a global declaration.
container_var_init,
@"callconv",
@"align",
@"addrspace",
@"linksection",
// Miscellaneous reasons.
comptime_keyword,
comptime_call_modifier,
inline_loop_operand,
switch_item,
tuple_field_default_value,
struct_field_default_value,
enum_field_tag_value,
slice_single_item_ptr_bounds,
stored_to_comptime_field,
stored_to_comptime_var,
casted_to_comptime_enum,
casted_to_comptime_int,
casted_to_comptime_float,
panic_handler,
pub fn message(r: SimpleComptimeReason) []const u8 {
return switch (r) {
// zig fmt: off
.operand_Type => "operand to '@Type' must be comptime-known",
.operand_setEvalBranchQuota => "operand to '@setEvalBranchQuota' must be comptime-known",
.operand_setFloatMode => "operand to '@setFloatMode' must be comptime-known",
.operand_branchHint => "operand to '@branchHint' must be comptime-known",
.operand_setRuntimeSafety => "operand to '@setRuntimeSafety' must be comptime-known",
.operand_embedFile => "operand to '@embedFile' must be comptime-known",
.operand_cImport => "operand to '@cImport' is evaluated at comptime",
.operand_cDefine_macro_name => "'@cDefine' macro name must be comptime-known",
.operand_cDefine_macro_value => "'@cDefine' macro value must be comptime-known",
.operand_cInclude_file_name => "'@cInclude' file name must be comptime-known",
.operand_cUndef_macro_name => "'@cUndef' macro name must be comptime-known",
.operand_shuffle_mask => "'@shuffle' mask must be comptime-known",
.operand_atomicRmw_operation => "'@atomicRmw' operation must be comptime-known",
.operand_reduce_operation => "'@reduce' operation must be comptime-known",
.export_target => "export target must be comptime-known",
.export_options => "export options must be comptime-known",
.extern_options => "extern options must be comptime-known",
.prefetch_options => "prefetch options must be comptime-known",
.call_modifier => "call modifier must be comptime-known",
.compile_error_string => "compile error string must be comptime-known",
.inline_assembly_code => "inline assembly code must be comptime-known",
.atomic_order => "atomic order must be comptime-known",
.array_mul_factor => "array multiplication factor must be comptime-known",
.slice_cat_operand => "slice being concatenated must be comptime-known",
.inline_call_target => "function being called inline must be comptime-known",
.generic_call_target => "generic function being called must be comptime-known",
.wasm_memory_index => "wasm memory index must be comptime-known",
.work_group_dim_index => "work group dimension index must be comptime-known",
.type => "types must be comptime-known",
.array_sentinel => "array sentinel value must be comptime-known",
.pointer_sentinel => "pointer sentinel value must be comptime-known",
.slice_sentinel => "slice sentinel value must be comptime-known",
.array_length => "array length must be comptime-known",
.vector_length => "vector length must be comptime-known",
.error_set_contents => "error set contents must be comptime-known",
.struct_fields => "struct fields must be comptime-known",
.enum_fields => "enum fields must be comptime-known",
.union_fields => "union fields must be comptime-known",
.function_ret_ty => "function return type must be comptime-known",
.function_parameters => "function parameters must be comptime-known",
.decl_name => "declaration name must be comptime-known",
.field_name => "field name must be comptime-known",
.struct_field_name => "struct field name must be comptime-known",
.enum_field_name => "enum field name must be comptime-known",
.union_field_name => "union field name must be comptime-known",
.tuple_field_name => "tuple field name must be comptime-known",
.tuple_field_index => "tuple field index must be comptime-known",
.container_var_init => "initializer of container-level variable must be comptime-known",
.@"callconv" => "calling convention must be comptime-known",
.@"align" => "alignment must be comptime-known",
.@"addrspace" => "address space must be comptime-known",
.@"linksection" => "linksection must be comptime-known",
.comptime_keyword => "'comptime' keyword forces comptime evaluation",
.comptime_call_modifier => "'.compile_time' call modifier forces comptime evaluation",
.inline_loop_operand => "inline loop condition must be comptime-known",
.switch_item => "switch prong values must be comptime-known",
.tuple_field_default_value => "tuple field default value must be comptime-known",
.struct_field_default_value => "struct field default value must be comptime-known",
.enum_field_tag_value => "enum field tag value must be comptime-known",
.slice_single_item_ptr_bounds => "slice of single-item pointer must have comptime-known bounds",
.stored_to_comptime_field => "value stored to a comptime field must be comptime-known",
.stored_to_comptime_var => "value stored to a comptime variable must be comptime-known",
.casted_to_comptime_enum => "value casted to enum with 'comptime_int' tag type must be comptime-known",
.casted_to_comptime_int => "value casted to 'comptime_int' must be comptime-known",
.casted_to_comptime_float => "value casted to 'comptime_float' must be comptime-known",
.panic_handler => "panic handler must be comptime-known",
// zig fmt: on
};
}
};
test {
_ = Ast;
_ = AstRlAnnotate;
_ = BuiltinFn;
_ = Client;
_ = ErrorBundle;
_ = LibCDirs;
_ = LibCInstallation;
_ = Server;
_ = WindowsSdk;
_ = number_literal;
_ = primitives;
_ = string_literal;
_ = system;
_ = target;
_ = c_translation;
2019-03-02 16:46:04 -05:00
}