I was listening to “The Nvidia Way” recently, as recommended by “Oxide and Friends” Books in the Box episode. It mentions “The Innovator’s Dilemma” a LOT. So I picked up that audiobook too. I was surprised to find out that the forward (of this edition) of The Innovator’s Dilemma was written by Marc Benioff, CEO of Salesforce (which has owned and operated Heroku since 2011).
I’m still working through the book, but some of the terminology we’re now being thrust into seems to come from the book. A “sustaining” business. https://online.hbs.edu/blog/post/sustaining-vs-disruptive-innovation. In that context, “sustaining” isn’t a bad thing; it’s basically short for “predictable.” If you’re in the business of selling hammers, there are incremental improvements to materials or processes. You can find efficiency through forecasting and prior market knowledge. But it’s not a market where new upstarts are coming in with radically different things and taking over.
When I worked for GE as an intern in their Appliance Park (made refrigerators, etc.) there was an organizational strategy for delivering new products to market (such as French door refrigerators, or meeting new energy-star guidelines). It was called NPI (New Product Introduction), where those engineers got good at optimizing for speed of delivery to the market. And another org called PCTO (Product Cost Take Out), where they would take products already delivered and find ways to increase the margins on them (like using a cheaper compressor in exchange for more expensive insulation if that allowed you to still meet regulation and energy-star targets).
Even in such an old field as home appliances, there’s still work to be done. There are still competitive edges to be found and optimized. To me, that’s an example of a sustaining business.
No GE intern wanted to work in PCTO. It wasn’t “cool” or “glamorous.” But it’s the bulk of the work. It’s how the company stays competitive. It would have been “better” to “design it right the first time,” but that would lead to longer product introduction cycles, which not only means your competitors deliver before you, but it also means they’re learning what works and what doesn’t before you. The PCTO org brought value not just by having a cost-competitive product. It is also what enabled NPI to exist at all.
When I hear Heroku say it is moving to a “sustaining” engineering model, it doesn’t mean features stop. Heck, the first commercial fridge was introduced in 1913, and we’re still finding ways to add bells and whistles, like the water pitcher in the door and quad-door design of my most recent fridge. But those innovations aren’t disruptive; they’re iterative and relatively predictable. Those innovations are only possible because worse is better. i.e., GE figured out what mattered (shipping fast is more important than perfect), but it did it in a way that it didn’t stop there, once it’s shipped, it’s shipped again over and over until the kinks are worked out and the margins are competitive.
In “The Innovator’s Dilemma,” a “sustaining” innovation would be increasing the density of iron on a disk platter to achieve incrementally more storage density or going from one spindle to two. A “disruptive” technology would be the digital camera. It originally produced worse images than film and was very expensive. Kodak didn’t invest in it because it didn’t give their current customers what they needed. By the time digital cameras disrupted the film camera industry, Kodak was too late to make a difference.
In the context of Heroku, a transition to a “sustaining engineering model” means (to me) we’ve got to examine our sacred cows and focus on the most important pieces of the company. Some of this is a continuation of what we were already doing. There’s already been a push for toil reduction and increased automation. In my personal role, I want to move building Ruby binaries to be a completely automated process (right now it is semi-automated), with a balance between speed and automation, and security via checksum validation. This is an engineering investment in engineering. Spending hours now to save minutes for a thing that is predictable and recurring will not only free me up to work on other automations, but it will also reduce interruptions and toil. Previously, this was on the roadmap, but it’s been a lower priority. Other teams have other examples. For example, Next Generation Postgres is already in pilot and still moving ahead.
To me, “sustaining” means “focus.” Focus inward on what we’re doing today and what our customers need today. It might mean missing out on disruptive changes, but focusing on what the “next big thing” could be can cause you to miss out on the incremental progress improvements right in front of your nose.
More doesn’t always mean better. More processes, more ways to track work, and more inboxes to check all slow us down. Less doesn’t always mean worse. Sustaining isn’t just keeping lights on. It’s keeping your product and your customers nourished, sustained, not frozen in time or starved. When Heroku first came out, it was a mind-blowing, disruptive change from how people spun up servers before it. Sustaining means aligning on what not to do, beyond just what we would like to do. We can’t afford to be fat and lazy. We can’t afford to aim for 100% in 100% of everything we do. Frankly, that model wasn’t serving our customers very well.
]]>TLDR: Be clear in your communications what you support and “whose side are you on.” There’s a bullet point list of suggestions at the bottom.
Before connecting my thoughts back to tech and to “drama”, I’m going to start with a made-up example: Imagine you’re at school and a classmate comes to you and says:
“Hey idiot, tie your shoes.”
What do you do next? Do you say, “Hey, thanks! I appreciate you helping me not trip over myself,” or do you get defensive?
The schoolmate clearly led with an attack: “Hey, idiot.” So the rest of it: “tie your shoes” is likely to also be interpreted as an attack too. Maybe you guess they’ll say “made you look” and get the rest of your classmates to laugh too when you glance down at your shoes. Who knows. It’s not logical, it’s emotional.
Let’s break it down. In any “cause” or “drama,” there might be two sides, there’s at least three actors (bodies): Those involved, the speaker, and the listener.
In this case, the actors are:
Note: Here
||is a logical “or”. So read this as “You or the rest of the class”.
Every conflict has two sides. Here sides for/against are:
Now, instead, imagine they said, “Hey, I saw your shoes are untied. I don’t want you to trip over them, get hurt, and delay getting to lunch for everyone.” Do you think you would be more or less likely to check your shoes and take action?
The difference between the two approaches is: Calling out, versus calling in. In the second example, the classmate didn’t just raise awareness of the issue, but also worked to make it clear they were on the side of “not delaying lunch” (which falls on the side of “please tie your shoes”). They didn’t just point out a problem; they worked to ensure the listener could hear their message and take the desired action.
Recently, someone posted on Reddit, asking, “Why does tool X suck?” The actors involved are:
It was a genuine question, and kicked off a great discussion. But, it was made without awareness that the authors are active in the sub-reddit and would 100% for sure see the post. Whether the poster meant it this is the conflict inherent in the question:
If the intent of “Why does tool X suck?” was to say “I am mad at this tool and want to attack the authors,” then those would be good words to choose. If the intent is to understand the software, identify strengths and weaknesses, and possibly help the authors understand and make it better, the first step is not alienating them.
Now that you understand the for/against and involved/speaker/listener frame, it’s going to get complicated by getting recursive. As everyone involved can switch perspectives based on the flow of the conversation. Like the real “three body problem,” this switching of places of involved/speaker/listener makes real-world conflicts chaotic and unpredictable.
Take the prior case of “why does tool X suck?” After the post is made, someone sees it they comment something like “hey, that’s a really crappy thing to say…” as the original person “calling out” the tool, becomes called out. My suggested fix would be to recursively apply “calling in,” including “calling in” about “calling out.”
It also gets tricky, as a post on Mastodon (or other social media platforms) is read by individuals as if it were written “to” them, but “them” might be a random bystander or the leader of an organization. A statement, “I hate this tool,” might be true, but it might not be helpful if it’s missing the context and qualifiers needed to let people know what exactly you meant by making that statement.
Humans are really good at pattern matching and applying meaning, even when none exists. Others will read your communications and implicitly try to assign an “us” or a “them” side, whether it was meant that way or not.
For more info on this topic I like this PDF on “Calling In”, starting on page 2 for more examples.
All of that is well and good. Here’s some other rapid-fire suggestions for being able to both hear and be heard:
The above NVC is reworded as a bit of a checklist of questions, because if you formulaically apply it like an SAT essay, then it will sound robotic and condescending. Also, a big part of the real NVC practice is working to actively defuse perceived slights and perceived attacks from the speaker. NVC is not a magic bullet, and people can use it to harm. But I like thinking of it as structural de-composition. Every communication you send has all four parts in it. If something you say is missing a part, you’re asking the reader to do that work for you and fill in the blank. Not only is this a bit rude, but there’s also no guarantee they’ll do it correctly.
This section got longer than I was anticipating, but please don’t call me out on it. Instead, call me in.
]]>clippy.toml file to the root of a Rust project gives the ability to disallow a method or a type when running cargo clippy. This has been really useful. I want to share two quick ways that I’ve used it: Enhancing std::fs calls via fs_err and protecting CWD threadsafety in tests.
Update: you can also use this technique to disallow unwrap()! There’s also
unwrap_usedwhich you use by adding#![deny(clippy::unwrap_used)]to yourmain.rs.
I use the fs_err crate in my projects, which provides the same filesystem API as std::fs but with one crucial difference: error messages it produces have the name of the file you’re trying to modify. Recently, while I was skimming the issues, someone mentioned using clippy.toml to deny std::fs usage. I thought the idea was neat, so I tried it in my projects, and it worked like a charm. With this in the clippy.toml file:
disallowed-methods = [
# Use fs_err functions, so the filename is available in the error message
{ path = "std::fs::canonicalize", replacement = "fs_err::canonicalize" },
{ path = "std::fs::copy", replacement = "fs_err::copy" },
{ path = "std::fs::create_dir", replacement = "fs_err::create_dir" },
# ...
]
Someone running cargo clippy will get an error:
$ cargo clippy
Checking jruby_executable v0.0.0 (/Users/rschneeman/Documents/projects/work/docker-heroku-ruby-builder/jruby_executable)
Checking shared v0.0.0 (/Users/rschneeman/Documents/projects/work/docker-heroku-ruby-builder/shared)
warning: use of a disallowed method `std::fs::canonicalize`
--> ruby_executable/src/bin/ruby_build.rs:169:9
|
169 | std::fs::canonicalize(Path::new("."))?;
| ^^^^^^^^^^^^^^^^^^^^^ help: use: `fs_err::canonicalize`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.91.0/index.html#disallowed_methods
= note: `#[warn(clippy::disallowed_methods)]` on by default
Running cargo clippy --fix will now automatically update the code. Neat!
Why was I skimming issues in the first place? I suggested adding a feature to allow enhancing errors with debugging information, so instead of:
failed to open file `file.txt`: The system cannot find the file specified. (os error 2)
The message could contain a lot more info:
failed to open file `file.txt`: The system cannot find the file specified. (os error 2)
Path does not exist `file.txt`
- Absolute path `/path/to/dir/file.txt`
- Missing `file.txt` from parent directory:
`/path/to/dir`
└── `file.md`
└── `different.txt`
To implement that functionality, I wrote path_facts, a library that provides facts about your filesystem (for debugging purposes). And since the core value of the library is around producing good-looking output, I wanted snapshot tests that covered all my main branches. This includes content from both relative and absolute paths. A naive implementation might look like this:
let temp = tempfile::tempdir().unwrap();
std::env::set_current_dir(temp.path()).unwrap(); // <= Not thread safe
std::fs::write(Path::new("exists.txt"), "").unwrap();
insta::assert_snapshot!(
PathFacts::new(path)
.to_string()
.replace(&temp.path().canonicalize().unwrap().display().to_string(), "/path/to/directory"),
@r"
exists `exists.txt`
- Absolute: `/path/to/directory/exists.txt`
- `/path/to/directory`
└── `exists.txt` file [✅ read, ✅ write, ❌ execute]
")
In the above code, the test changes the current working directory to a temp dir where it is then free to make modifications on disk. But, since Rust uses a multi-threaded test runner and std::env::set_current_dir affects the whole process, this approach is not safe ☠️.
There are a lot of different ways to approach the fix, like using cargo-nextest, which executes all tests in their own process (where changing the CWD is safe). Though this doesn’t prevent someone from running cargo test accidentally. There are other crates that use macros to force non-concurrent test execution, but they require you to remember to tag the appropriate tests. I wanted something lightweight that was hard to mess up, so I turned to clippy.toml to fail if anyone used std::env::set_current_dir for any reason:
disallowed-methods = [
{
path = "std::env::set_current_dir",
reason = "Use `crate::test_help::SetCurrentDirTempSafe` to safely set the current directory for tests"
},
]
Then I wrote a custom type that used a mutex to guarantee that only one test body was executing at a time:
impl<'a> SetCurrentDirTempSafe<'a> {
pub(crate) fn new() -> Self {
// let global_lock = ...
// ...
#[allow(clippy::disallowed_methods)]
std::env::set_current_dir(tempdir.path()).unwrap();
You might call my end solution hacky (this hedge statement brought to you by too many years of being ONLINE), but it prevents anyone (including future-me) from writing an accidentally thread-unsafe test:
$ cargo clippy --all-targets --all-features -- --deny warnings
Checking path_facts v0.2.1 (/Users/rschneeman/Documents/projects/path_facts)
error: use of a disallowed method `std::env::set_current_dir`
--> src/path_facts.rs:395:9
|
395 | std::env::set_current_dir(temp.path()).unwrap();
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: Use `crate::test_help::SetCurrentDirTempSafe` to safely set the current directory for tests
= help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.91.0/index.html#disallowed_methods
= note: `-D clippy::disallowed-methods` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::disallowed_methods)]`
Those are only two quick examples showing how to use clippy.toml to enhance a common API, and how to safeguard against incorrect usage. There’s plenty more you can do with that file, including:
disallowed-macrosdisallowed-methodsdisallowed-namesdisallowed-typesYou wouldn’t want to use this technique of annotating your project with clippy.toml if the thing you’re trying to prevent would be actively malicious for the system if it executes, since clippy.toml rules won’t block your cargo build. You’ll also need to make sure to run cargo clippy --all-targets in your CI so some usage doesn’t accidentally slip through.
And that clippy lint work has paid off, my latest PR to fs_err was merged and deployed in version 3.2.0, and you can use it to speed up your development debugging by turning on the debug feature:
[dev-dependencies]
fs-err = { features = ["debug"] }
Clip cautiously, my friends.
]]>Keep reading in the Heroku blog post.
]]>Keep reading in the heroku blog post.
]]>Let’s set the scene: Our protagonist is burning the midnight oil. They’ve got a great idea for an open source RFC, but they want to do some due-diligence first with some co-workers so they open up their employee encouraged document sharing solution, Slack Canvas and write a beautiful bit of carefully annotated markdown. It looks … mostly right (why on earth do H3s look nearly identical to H2??) but it’s good enough. They got some great feedback, made some edits and now it’s time to go contribute to the great commons we call open source. They select all (CMD+a), they copy the text (CMD+c), they paste it into GitHub and hit enter. Slowly the blood drains from their face. It’s all gone. All the backticks, all the bolds, all the links. It kinda looks like the original document, but is quietly, subtly, horrifically mangled.
In desperation, they reach first for the Edit menu, then File. Flashes of the crappy https://quip.com/ markdown export feature flash through their memory. It had it’s own problems, but at least it was there. Silence covers the room. An almost imperceptible evil laugh of Google doc’s bastardized markdown support can be heard of the dawning realization: the app takes markdown synatax in. But it devours it. Destroys it. Vomiting rich text in in the place of where carefully currated backticks and astericks once shone brightly. The document is dead. Our hero weeps.
The whole point of markdown (as witnessed and confirmed in my headcannon as a developer since 2006) is that it provides a format that can be consumed as EITHER rendered text, OR as plain text. Markdown is not a format for styling a text document, it is a format that requires that the writer, can read the original text (not literally, you pedants, there’s not exactly a spec that says that, but perhaps there should be). Without this guarantee we get a sea of “almost markdown” formats. We are mark-drowning in them. They all behave slightly differently. For apps that do a decently good job of letting developers “read your write” there’s product-driven feature’s like Notion’s desire to embed rich objects such as spreadsheets and to add custom inter-notion document linking syntax that encourage users to subtly poison the portability of their original text.
I don’t begrudge them extending the syntax to meet their needs, but wish an export story was better thought out. Perhaps rich objects should be supported via iframes and exporting documents to markdown should support some kind of a “deep” export (i.e. it doesn’t matter if the markdown it exports is perfect, if it links to a bunch of gated and internal documents. Such thoughts probably get your cyber-security sense tingling (as they should) and likely bore investors and product managers to tears, therefore it’s a usecase that is sorely neglected given the millions/billions of dollors of “innovation” in the “markdown as a shared mangled RTF doc” apps that we’re collectively burried under.
So forget “hard mode” all I’m asking for is simple. If you take markdown in, you should allow me to take my original markdown out CHARACTER FOR CHARACTER. So far the best tools I’ve found for this are: Vim, GitHub, and Obsidian. (Yes, your favorite IDE too). These are all hacker tools, but why can’t we live in a world where ALL tools respect our input enough to let us read it back? Perhaps you’ve been toiling like our hero. These document inflicted wounds aren’t large, but they’re never-ending. They create toil, and burn trust in tools. Perhaps like me, you’ve not had the words to enumerate what exactly is missing. Hopefully, now you do. If a company asks for some feedback, please let them know that to “read my markdown write” is a fundamental and inalienable right.
Oh and while I’m ranting. I beg of you, please (please, please, please, please) add a newline after markdown headers. I.e. Don’t do this:
## I really don't like when
People do this. Hard to read.
Instead, please do this:
## Everyone who does this is amazing
They are the coolest people I know.
Why? Remember when I said markdown was supposed to optimize for rendering and plain text? When someone is reading your plain text, the first example without the vertical whitespace provides no visual pause. It’s notexactlythesamethingasreadingasentencewithnospaces. But you get how important white space can be when it comes to reading and comprehension speed. Yes, your favorite markdown (and markdown-ish) tools will render both of them the same, but to someone not viewing those docs in that same tool, they will appreciate you and star your github repos more and other vauge promises of things developers presumably want.
For some reason, I don’t entirely comprehend, the first style is much more popular on GitHub. To the point that the vast majority of LLM produced markdown does not have vertical whitespace after headers. We’re in a world now where “do what I do” (as opposed to do as I say) is fed back to us one token at a time. We might be past the point of no-return when it comes self-reinforcing behavior (as AI is now trained on the output that developers are committing as their own), but when skynet takes over and you’re looking for a shibboleth to prove you’re not a robot with an Austrian accent, you might remember this post and send a PR with some glorious, human, hand-crafted markdown.
]]>If I say to a friend, “I’m hungry, let’s go to McDonald’s” (or wherever), they’re not allowed to block me without making a counter-suggestion. They can’t just say “No,” they have to say something like “How about Arby’s” instead. This simple rule changes the dynamic of the suggester/blocker to one of the proposer/counter-proposer. If someone is simply refusing to be involved, they McBlocked me.
In practice, though, it’s hard to always have a suggestion you’re willing to run with, so a relaxed version of the rule is that the other person has to AT LEAST specify why not. Instead of “no” it must be “no, because”. For example, it could be “I had a burger for lunch” or “I’m banned for life after jumping on a table and demanding Szechuan dipping sauce.” This helps show that you’re not just blocking things, you understand the goal and want to move the conversation forward. It gives the other person something to work with. Easy for eats, but what about tech?
I work for Heroku, and recently, there was a stack EOL where customers were asked to migrate off of Ubuntu 20.04 (heroku-20). In this (many-month-long) deprecation process, I saw a lot of people make a lot of absolute statements. One of them was:
“You cannot run Rails 4 on heroku-22.”
Which, as you’ll guess, is only half the story. What they meant was:
“Rails 4.2 saw its last release in 2020 and is quite thoroughly EOL. That version cannot run on any Ruby version 3.1. x- 3.4. x, which are present on heroku-22 or above, due to library errors. Therefore, to run Rails 4 on heroku-22, you would have to fork it and patch the security vulnerabilities yourself and update it to run on a modern Ruby version.”
Which, to be fair, sounds a lot like “cannot be done,” but with more words. But, as you’ll also have likely guessed, once you know about the possible path forwards, however impractical, it might give you other ideas.
You might start asking questions like “if we have to fork and maintain it, anyone else would have to also, I wonder if someone else already did.” This could send you down a quick search where you might discover that Rails LTS is a thing and basically provides a managed fork of Rails 4.2 for a fee that runs with the latest Ruby versions.
I wrote about the existence of this service previously:
Now, that new thing could still be a bad idea, and you might still not end up doing it, but the key here is that you’re not saying “no,” you’re saying “here are the barriers I know about.” A good way to test if you’re just using more words to say “no” or not is if your statement is falsifiable or satisfiable in some way.
A “no, because” statement instead of a plain “no” moves the problem from a blocker into an opportunity. You can see this in a really good open source conversation. Instead of “this can’t be done,” someone can send a PR. Instead of “I won’t merge your PR” they can comment: “I agree/disagree with the problem/opportunity you’ve raised, I’m uncomfortable merging this because of <specific reason>.”
A quick story, and you can go. Before writing this post, I pitched the word-smithing of “McBlocker” to my wife at the dinner table (where you can tell we are very cool and fun people). My kids, age 7 and 9, were there. My 9 y/o asked me to take him to the library after dinner (did I mention how cool we are?), where I was talking to him about types of non-fiction that he might like. I was talking about biographies when he blurted out, “I don’t like biographies.” To which I responded, “Hey, don’t McBlock me,” and when I got a laugh of recognition in return, I figured the phrase was worth a blog post.
If you enjoyed this, you might enjoy my service for helping people contribute to open source (free) or my book How to Open Source (paid). Now, go McRepost this to your favorite federated social network!
]]>A “duplicate duck” is a type that implements a subset of traits of a popular type with the same results. In my case I wrote a type, MultiError, that I later realized was identically duck typed to syn::Error and that my struct added nothing. I deleted my type with no loss in functionality and the world was better for it.
I saved my code before throwing it away. The following is the story of my design process and eventual epiphany.
Quick
whoami: I write Rust for Heroku where I maintain the Ruby Cloud Native Buildpack. I also maintain a free service CodeTriage and wrote a book, How to Open Source, for turning coders into contributors.
I’ve been hacking on proc macros recently, you can read about a recent investigation “A Daft proc-macro trick: How to Emit Partial-Code + Errors”. I want proc macro authors to emit as many accumulated errors as possible (versus stopping on the first one), I’m also a fan of unit testing. I wanted to add a return type from my functions that said, “I return many accumulated errors,” and I wanted that return type to be unit-testable.
In my code, I’ve been accumulating errors with VecDeque<syn::Error>. This makes it easy to combine them into a single syn::Error :
if let Some(mut error) = errors.pop_front() {
for e in errors {
error.combine(e);
}
Some(error)
} else {
None
}
However, I don’t want to return a result of Result<T, VecDeque<syn::Error>> from my functions as the error state isn’t guaranteed to be non-empty. A good type should make invalid state impossible to represent.
To guarantee my type always had at least one error, I separated out the first error from the rest of the collection. Even if this container is empty, the type definition guarantees we can always turn this into a syn::Error
/// Guaranteed to hold at least one [`syn::Error`]
///
/// The [`syn::Error`] can hold multiple errors
/// through [`syn::Error::combine()`], however it
/// does not allow the receiver to distinguish
/// between the two cases, which makes testing
/// less precise. Using this type is a stronger
/// hint that the function accumulates errors.
///
#[derive(Debug, Clone)]
pub(crate) struct MultiError {
first: syn::Error,
rest: VecDeque<syn::Error>,
}
impl MultiError {
pub(crate) fn from(mut errors: VecDeque<syn::Error>) -> Option<Self> {
if let Some(first) = errors.pop_front() {
let rest = errors;
Some(Self { first, rest })
} else {
None
}
}
}
Warning: Just because the docs state something, doesn’t mean it’s true.
Note the visibility, by default I use
pub(crate)for the struct and associated functions but not for the fields (firstandrest). When I’m unsure of my design, it’s easier to change them later if all access goes through functions.
This type allowed me to introduce helper functions like this:
pub(crate) fn parse_attrs<T>(
attrs: &[syn::Attribute]
) -> Result<Vec<T>, MultiError>
where
T: syn::parse::Parse,
{
let mut errors = VecDeque::new();
// ...
if let Some(error) = MultiError::from(errors) { // <== HERE
Err(error)
} else {
Ok(
// ...
)
}
}
This code says “I take in any slice of syn::Attribute and then parse that attribute into a vector of T or return one or more syn errors”. So far, so good.
But my macro needs a syn::Error to generate error tokens and my function returns a MultiError. So I needed a way to convert my type into a syn::Error.
Based on the properties of the type, we know we can always convert into a syn::Error infallibly, so I can expose that via implementing Into<syn::Error>:
impl From<MultiError> for syn::Error {
fn from(value: MultiError) -> Self {
let MultiError { mut first, rest } = value;
for e in rest {
first.combine(e);
}
first
}
}
As a bonus, the try operator (?) will implicitly call into() which allows us to do things like this:
fn check_logic(...) -> Result<(), syn::Error> {
// ...
let result: Result<(), MultiError> = logic();
let _ = result?; // <=== Convert MultiError to syn::Error implicitly
// ...
}
With that added, I needed a way to test my logic to ensure I was capturing multiple errors.
To render the error on failure it needs to implement std::fmt::Display:
impl std::fmt::Display for MultiError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
Into::<syn::Error>::into(self.clone()).fmt(f)
}
}
It’s not pretty, but it worked and was easy. This code path is only ever called under test.
To expose multiple errors for testing, I chose to implement the IntoIterator trait:
impl IntoIterator for MultiError {
type Item = syn::Error;
type IntoIter = <VecDeque<syn::Error> as IntoIterator>::IntoIter;
fn into_iter(self) -> Self::IntoIter {
let MultiError { first, mut rest } = self;
rest.push_front(first);
rest.into_iter()
}
}
This code says that we can now convert our struct into something that produces a series of syn::Error-s. Since we’ve already got a VecDeque lying around, and I knew that it implemented the same trait, I piggybacked my logic on top. This allowed me to do things like this test:
#[test]
fn test_captures_many_field_errors() {
let field: syn::Field = syn::parse_quote! {
#[cache_diff(unknown)]
#[cache_diff(unknown)]
version: String,
};
let result: Result<Vec<ParseAttribute>, MultiError> =
crate::shared::parse_attrs::<ParseAttribute>(&field.attrs);
assert!(
result.is_err(),
"Expected {result:?} to be err but it is not"
);
let error = result.err().unwrap();
assert_eq!(2, error.into_iter().count()); // <== into_iter() HERE
}
enum ParseAttribute {
//...
}
impl syn::parse::Parse for ParseAttribute {
// ...
}
This code parses a single field with multiple syn::Attribute-s on it. In this case, cache_diff(unknown) is an invalid attribute, and I want to assert that it does not stop after the first one it sees. The code converts my result into an iterator and then asserts that there are two elements. Great!
While the above code example worked fine, I kept applying this pattern, bubbling up errors until I hit a failure in my code:
#[test]
fn test_captures_many_field_errors() {
let result = ParseContainer::from_derive_input(&syn::parse_quote! {
struct Metadata {
#[cache_diff(unknown)]
#[cache_diff(unknown)]
version: String,
#[cache_diff(unknown)]
#[cache_diff(unknown)]
name: String
}
});
assert!(
result.is_err(),
"Expected {result:?} to be err but it is not"
);
let error = result.err().unwrap();
assert_eq!(4, error.into_iter().count()); // <== FAILED here
}
The error said that I was returning only two errors instead of 4. Which was confusing. I moved the code into a trybuild integration test and saw 4 errors. At this point it dawned on me, that at some time I was storing multiple errors into a single syn::Error and then placing that combined error in my MultiError. Basically I had a multi MultiError.
If that was hard to follow, here’s some pseudo code:
let mut errors: VecDeque<syn::Error> = VecDeque::new();
match call_fun() { // Returns a MultiError
Ok(_) => todo!(),
// Combines it into a single `syn::Error`
Error(error) => errors.push_back(error.into())
}
// ...
if let Some(error) = MultiError::from(errors) {
Err(error)
} else {
Ok(
// ...
)
}
Essentially, my MultiError type allowed for what I thought was uninspectable-state. Each syn::Error could hold N errors.
As I went through the stages of grief for my beautiful type that had a fundamental flaw, I hit on the idea that perhaps I could upstream a change to expose the internal combined errors from syn::Error. I thought that the IntoIterator interface was a good candidate to add. But to my shock, when I opened the docs the impl IntoIterator for syn::Error was right there this whole time. I just missed it.
When I realized that syn::Error already implemented every trait that I needed, I was able to change every MultiError into syn::Error and replace every MultiError::from_error with a function that returns Option<syn::Error>. Then, with zero other logic changes, my code compiled. That confirmed my suspicions that I had written a duck-typed duplicate of a commonly available struct.
The only value my MultiError type brought was that it hinted that the function was written with error accumulation in mind, but could not guarantee that the accumulation logic was correct. It didn’t seem like this minor social hint was enough to justify the extra code. I could achieve similar goals with a type alias.
If a type doesn’t introduce new capabilities or constraints and can be replaced by an existing, stable type, it should probably be deleted in favor of the more common type.
Just because a type starts to smell a little foul (or rather “fowl”) does that mean you need to get rid of it? Producing a new type guarantees that there are no mix-up between your type and the common type. New typing could also allow you to restrict operations to a subset of the common type. Both of these things are about adding constraints.
A third reason to keep a duck around would be the stability of the interface. If you’re going to expose your type via a library and you’re worried it might change, then it could be helpful to wrap the type so your downstream user don’t have to change their code even if the underlying logic or implementation changes.
When in doubt, consider documenting your duck and explaining what constraints the new type adds over the original. After writing them down, search for an already existing type that has the same behaviors. Perhaps go so far as to document why those types don’t meet your needs. If you cannot enumerate those differences well, then perhaps it’s a sign you should ditch your duck.
In my case I had explicitly called out syn::Error and even went as far as implementing Into<syn::Error>. Those are two strong signs that I should have investigated my claims and looked for features provided by trait implementations.
One of the reasons I missed that syn::Error already met my needs that I didn’t stop to consider why certain traits were implemented on the struct or think about how they might be used to expose the data that I needed. Over time I’ve been better at internalizing and mentally mapping trait names to the behaviors they provide. Still, I’ve got some more work to do. Hopefully after this experience, with strong hints that I’m re-implementing an existing type as a duck, I won’t forget to check trait implementations for what I need.
Beyond “trying harder” and “writing a blog post as penitence so I don’t do it again,” I thought that it would be nice if this behavior was also shown via an example, so I sent a PR to syn to add some examples to syn::Error::combine. I don’t think we need to clutter all code with documenting every possible use case of every possible trait, but this very useful iteration functionality its in nicely in demonstrating how the combine behavior works. Hopefully, the addition of these docs will bre received well and not as an albatross
I would like to encourage everyone to pay attention to your types and the pain you’re feeling around them. If you find you’ve written a type that you later refactored away, consider pausing and capturing why it was written and why the world is better off without it. What other “bad type” patterns are out there and how can we make it easier for newcomers to spot and avoid them?
]]>Update (2025/04/02): The change I suggested below was merged in PR #64. It’s pretty neat I went from knowing nothing about this project to contributing to it in the span of a single blog post.
A recent Oxide and Friends podcast episode, “A crate is born,” detailed the creation of a proc macro for deriving “diffable” data structures with a trick I want to tell you about. To help rust-analyzer as much as possible, @rain explained that the macro should always emit as much valid source code as possible, even when an error is emitted. They didn’t go into detail, so I looked into the internals that made this code + error emitting behavior possible and wanted to share.
This post covers:
rust-analyzer ?Who am I? I write Rust code for Heroku, mainly on the Ruby Cloud Native Buildpack (CNB). CNBs are an alternative to Dockerfile for building OCI images. You can learn more by following a language-specific tutorial you can run locally. I also wrote a book on Open Source contribution (paid) and I maintain an Open Source contribution app - CodeTriage.com (free).
rust-analyzer?Skip this if you already understand the problem statement
The Rust compiler will stop when it hits code that cannot compile. However, rust-analyzer (the Language Server Protocol implementation that powers IDEs like vscode) tries to resume after an error because it can’t just stop rendering type hints.
Intuitively, it makes sense that if you have an invalid function in your code, it shouldn’t break syntax highlighting (or other features) in your valid code:
fn invalid_wrong_return() -> String {
()
}
fn valid() -> String {
"I am valid".to_string()
}
Daft (v0.1.2), emits trait implementations and sometimes generates new data structures. From the snapshot tests, an input of something like this:
#[derive(Debug, Eq, PartialEq, Diffable)]
struct Basic {
a: i32,
b: BTreeMap<Uuid, BTreeSet<usize>>,
}
Will generate code like this:
struct BasicDiff<'__daft> {
a: <i32 as ::daft::Diffable>::Diff<'__daft>,
b: <BTreeMap<Uuid, BTreeSet<usize>> as ::daft::Diffable>::Diff<'__daft>,
}
// ...
impl ::daft::Diffable for Basic {
type Diff<'__daft> = BasicDiff<'__daft> where Self: '__daft;
fn diff<'__daft>(&'__daft self, other: &'__daft Self) -> BasicDiff<'__daft> {
Self::Diff {
a: ::daft::Diffable::diff(&self.a, &other.a),
b: ::daft::Diffable::diff(&self.b, &other.b),
}
}
}
If the macro does not emit this information (possibly due to some hypothetical error not present in this example), then rust-analyzer wouldn’t know that the BasicDiff struct was expected to exist, what its fields were, or that Basic::diff() returned a BasicDiff struct. In short, the IDE would be generally less helpful.
Now that you understand the goal, how do we emit code when there’s an error?
The short version is that macros don’t output code or errors; they emit tokens. The daft crate collects errors and continues when possible. If it can generate code, it will emit that generated code as tokens before turning the errors into tokens and then emitting both. An earlier version of the code looked like this:
let errors = error_store
.into_inner()
.into_iter()
.map(|error| error.into_compile_error());
quote! {
#out
#(#errors)*
}
Where quote! produces tokens from both the code (#out) and the errors #(#errors)*.
But don’t take my word for it, read the source: The entry point for the Daft derive macro is internal::derive_diffable. This function returns a DeriveDiffableOutput. The DeriveDiffableOutput holds Option<TokenStream> for valid code that was generated and Vec<syn::Error> for errors
The DeriveDiffableOutput implements quote::ToTokens that emits the valid code followed by the errors (if they exist). code.
Put it all together, and you have a crate that emits partially generated code, even with errors. Neat.
Now that I knew how Daft implemented this feature, I wanted to understand when they chose to apply this pattern. I reviewed the snapshot tests and developed my own classifications.
There are three classes of failures in the snapshot tests:
First, proc-macros cannot emit warnings. If the coder entered slightly off information that wouldn’t affect compilation, the macro author must choose between letting it slide or raising an error. There’s no in-between.
When there’s a problem but daft can determine programmer intent, it will emit code and errors. I call this a “warning error.” The primary example in snapshot testing is when the #[daft(leaf)] attribute is used on an enum that is already a leaf by default (this is an internal concept to the crate).
Note that the daft crate does not use “warning error” as terminology. I am making that distinction based on my analysis of the snapshot test. Notes are here..
Second, the program can fail to compile even when code is emitted without error, for example, if a trait bound is not satisfied. The macro author cannot detect the problem because the reflection tools don’t expose the necessary information. They must rely on the compiler errors to guide their user.
Finally, there are situations where the author cannot safely emit code because an input is ambiguous or wrong. With these “plain” errors, if the macro author tried to guess and got it incorrect, they’re feeding rust-analyzer incorrect information, which might confuse the end user more. For example, if an attribute that doesn’t exist, such as #[daft(unknown)], is found, the macro author has no idea what was intended there and shouldn’t guess.
While working on this classification exercise I found two snapshot tests where code isn’t emitted but could be.
Obligatory: “It depends.”
If you find your rust-analyzer horribly broken due to a proc-macro problem, then this is a great trick to suggest. However, it isn’t a critical feature that every macro should have. Instead, libraries should focus on improving error accumulation (talked about later).
This code + error functionality requires a lot of plumbing, and ultimately, there’s only one code path (or two if they like my suggestion) that generates code + errors. Looking at how this code path came to be, it seems more that it was added because the plumbing already existed and the opportunity presented itself. From that lens, it’s easy to see why Daft goes this extra mile. The cost to implement was (comparatively) low:
While the lede: emitting code + errors is likely too much of an ask for most crates, every proc macro should accumulate errors. Let’s look at that now.
! for your $: Accumulate proc macro errorsWhat do I mean by accumulated errors? In a Python (or Ruby) program that raises an error, you have to fix that to find out if there’s another error lurking that also needs to be fixed. Rust programmers don’t like playing that game. They want as many errors upfront as possible:
error: #[daft(leaf)] specified multiple times
--> tests/fixtures/invalid/field-specified-multiple-times.rs:5:18
|
5 | #[daft(leaf, leaf, leaf)]
| ^^^^
error: #[daft(leaf)] specified multiple times
--> tests/fixtures/invalid/field-specified-multiple-times.rs:5:24
|
5 | #[daft(leaf, leaf, leaf)]
| ^^^^
error: #[daft(ignore)] specified multiple times
--> tests/fixtures/invalid/field-specified-multiple-times.rs:8:12
|
8 | #[daft(ignore)]
| ^^^^^^
error: #[daft(ignore)] specified multiple times
--> tests/fixtures/invalid/field-specified-multiple-times.rs:9:12
|
9 | #[daft(ignore)]
| ^^^^^^
This daft output says there are two errors on line 5. One error on lines 8 and 9. This code generated the following errors:
use daft::Diffable;
#[derive(Diffable)]
struct MyStruct {
#[daft(leaf, leaf, leaf)] // line 5
a: i32,
#[daft(ignore)]
#[daft(ignore)] // line 8
#[daft(ignore)] // line 9
b: String,
}
Fields and their attributes are parsed iteratively, so it’s common for a macro to stop iterating on the first problem (line 5) before returning. Instead, Daft stores the errors and continues parsing until it longer can.
I don’t think it’s the end of the world if a proc macro only emits a single error at a time, but it’s a requirement if you’re aiming for a “Michelin star proc-macro” experience.
Instead of using a Result<T, syn::Error> return, daft passes an accumulator that holds a Vec<syn::Error> to every fallible function. If there’s an error, it’s added to the accumulator.
This pattern also means that instead of having to choose between emitting T or syn::Error (via a Result<T, syn::Error>), the programmer can do both by returning a T while mutating the accumulator. That would indicate the problem is more of a “warning error” if the data structure can still be safely returned. Functions that return Option<T> indicate they’re likely holding one or more plain errors that would prevent code generation when a None is returned.
Beyond affording the ability to return code + errors, not using a Result means that the try operator (?) cannot be used accidentally for an early/eager return. This property encourages the macro author to capture as many errors as possible and emit them all instead of only emitting the first error. It’s a neat pattern, but it’s not the only way to accumulate errors.
syn::Error accumulationThe syn::Error struct has the capability of combining multiple errors without an accumulator by using syn::Error::combine. For example:
let mut errors = VecDeque::<syn::Error>::new();
// ...
if let Some(mut first) = errors.pop_front() {
for e in errors.into_iter() {
first.combine(e);
}
Err(first)
} else {
Ok(
// ...
)
}
This pattern is useful when the function signature isn’t changeable. For example, the syn::parse::Parse trait is commonly used by proc macros as a building block, and it has a fixed signature:
fn parse(input: syn::parse::ParseStream<'_>) -> Result<T, syn::Error>
With this pattern, the error behavior of the function is encoded in its return type:
Result<T, syn::Error>(T, Option<syn::Error>)Result<(T, Option<syn::Error>), syn::Error>.
Ok((T, None)) indicates no errorsOk((T, Some())) indicates an error that did not block code generation.Err() indicates that code could not be generated due to errorThat last one is verbose, but it prevents representing an invalid state when None code and None errors are returned simultaneously.
The downside of this technique is that nothing prevents an early return on error with try (?).
I was curious how this pattern would look implemented in place of the daft one, so I experimented with a draft (not daft) PR.
Note: The PR is to my own
mainbranch, not theirs. I don’t think any maintainer loves waking up to a giant PR with the “refactoring” in it.
We learned why rust-analyzer is sensitive to macro output. We explored the mechanics that Daft uses to emit code + errors, and accumulate errors. I introduced an alternative error accumulation method and I made some strong statements. Namely that emitting code + errors is a nice-to-have while accumulating and emitting all errors is an achievable best practice.
Coming from Ruby, proc macros are wonderful things that allow Rust developers to write powerful and expressive DSLs, and I love them. With the power to meta-program, there’s also the possibility to meta-confuse your end user or toolchain (like rust-analyzer). I love that the Daft maintainers put as much work and care into the failure modes as the rest of their logic in addition to accumulating and presenting as many errors as possible.
I hoped you enjoyed learning about these patterns as much as I did.
]]>FYI I’m working on a proc-macro tutorial and would love to hear from readers on Mastodon or Reddit about what real-world patterns you’ve seen around improving the end-user experience, especially around errors.
gem install sassc stopped working for me on my Mac (ARM). Initially, I thought this was because that gem was no longer maintained, and the last release was in 2020, but I was wrong. It’s 100% installable today. Read the rest to find out the real culprit and how to fix it.
FWIW some folks on lobste.rs suggested switching to sass-embedded for sass needs. This post still, works but into the future it might not.
In this post I’ll explain some things about native extensions libraries in Ruby and in the process tell you how to fix this error below if you’re getting it on your Mac:
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /Users/rschneeman/.gem/ruby/3.4.1/gems/sassc-2.4.0/ext
/Users/rschneeman/.rubies/ruby-3.4.1/bin/ruby extconf.rb
creating Makefile
current directory: /Users/rschneeman/.gem/ruby/3.4.1/gems/sassc-2.4.0/ext
make DESTDIR\= sitearchdir\=./.gem.20250314-33410-os7ibg sitelibdir\=./.gem.20250314-33410-os7ibg clean
current directory: /Users/rschneeman/.gem/ruby/3.4.1/gems/sassc-2.4.0/ext
make DESTDIR\= sitearchdir\=./.gem.20250314-33410-os7ibg sitelibdir\=./.gem.20250314-33410-os7ibg
compiling ./libsass/src/ast.cpp
compiling ./libsass/src/ast2c.cpp
make: *** [ast.o] Error 1
make: *** Waiting for unfinished jobs....
compiling ./libsass/src/ast_fwd_decl.cpp
make: *** [ast2c.o] Error 1
compiling ./libsass/src/ast_sel_super.cpp
make: *** [ast_fwd_decl.o] Error 1
compiling ./libsass/src/ast_sel_cmp.cpp
make: *** [ast_sel_super.o] Error 1
compiling ./libsass/src/ast_supports.cpp
make: *** [ast_sel_cmp.o] Error 1
compiling ./libsass/src/ast_sel_weave.cpp
make: *** [ast_supports.o] Error 1
compiling ./libsass/src/ast_values.cpp
compiling ./libsass/src/backtrace.cpp
make: *** [ast_sel_weave.o] Error 1
compiling ./libsass/src/ast_selectors.cpp
make: *** [ast_values.o] Error 1
compiling ./libsass/src/ast_sel_unify.cpp
make: *** [backtrace.o] Error 1
make: *** [ast_selectors.o] Error 1
make: *** [ast_sel_unify.o] Error 1
make failed, exit code 2
You can install the sassc on your Mac by:
sassc)$ xcode-select --version for me reports xcode-select version 2409)gem install sassc should workIf you want to know more about native compilation or my debugging process, read on!
There might be a simpler way to solve the problem (such as directly editing the rbconfig file), but I’m comfortable sharing the above steps because that’s what I’ve done. If you fixed this differently, post the solution on your own site or in the comments somewhere.
When I get an error, it makes sense to search for it and ask an LLM (if that’s your thing). I did both. GitHub copilot suggested that I make sure command-line tools are installed and that cmake is installed via homebrew. This was unhelpful, but it’s worth double-checking.
Searching libsass make: *** [ast2c.o] Error 1 brought me to https://github.com/sass/sassc-ruby/issues/248. This brought me to https://github.com/sass/sassc-ruby/issues/225#issuecomment-2391129846. Suggesting that the problem is related to RbConfig and native extensions. These have the fix in there, but don’t go into detail on the why the fix works. This post attempts to dig deeper using a debugging mindset.
Meta narrative: This article skips between explaining things I know to be true and debugging via doing. Skip any explanations you feel are tedious.
Skip if: You know what a native extension is
Most Ruby libraries are plain Ruby code. For an example look at https://github.com/zombocom/mini_histogram. When you gem install mini_histogram, it downloads the source code, and that’s all that’s needed to run it (well, that and a Ruby version installed on the machine). The term “native extensions” refers to libraries that use Ruby’s C API or FFI in some way. There are a few reasons why someone would want to do this:
psych gem uses libyaml, and the nokogiri gem uses libxml.For developers who haven’t used much C or C++, it’s useful to know that system-installed packages are how they (mostly) share code. There’s no rubygems.org for C packages. Things like apt for Ubuntu might be conflated as a “C package manager,” but it’s really like brew (for Mac), where it installs things globally. Then, when you compile a program in C, it can dynamically or statically link to other libraries to use them.
Back to native extensions: When a gem with a native extension is installed the source code is downloaded but then a secondary compilation process is invoked. Here’s a tutorial on creating a native extension. It utilizes a tool called rake-compiler. But under the hood it effectively boils down to when you gem install <native-extension> it will run compilation code such as $ make install on the system. This process generates compiled binaries, these binaries are compiled against a specific CPU architecture that is native to the machine you’re on, hence why they’re called native extensions. You’re using native (binary) code to extend Ruby’s capabilities.
Skip if: You understand why
libsassCPP files would be found in thesasscgem
Compiling code is hard. Or rather, dependency management is hard, and compiling code requires that the platform have certain dependencies installed; therefore, compiling code is hard. To make life easier, one common pattern that Ruby developers do is to vendor in dependencies into their native extension gem. Rather than assuming libsass is installed on the system in a location that is easy to find, it can instead bring that code along with it.
Here you can see that sassc from gem install sassc brings C++ source code from libsass:
$ ls /Users/rschneeman/.gem/ruby/3.4.2/gems/sassc-2.4.0/ext/libsass/src | head -n 3
MurmurHash2.hpp
ast.cpp
ast.hpp
In this case libsass may have dependencies that it hasn’t vendored and it expects to find on the system, but the key here is that when you gem install sassc it needs to make install not just its own bridge code (using Ruby’s C API), but it also needs to compile libsass as well. That is where the errors are coming from, it’s not able to compile these C++ files:
compiling ./libsass/src/ast2c.cpp
make: *** [ast.o] Error 1
make: *** Waiting for unfinished jobs....
For completeness: There’s another type of vendoring that native-extension gems can do. They can statically compile and vendor in a binary. This bypasses the need to make install and is much faster, but moves the burden to the gem maintainer. Here’s an example where Nokogiri 1.18.4 is precompiled to run on my ARM Mac. You don’t need to know this for debugging the sassc install problem, since that process isn’t being used here.
When debugging, I like to remove layers of abstraction when possible to boil the problem down to its core essence. You might think “I cannot run gem install sass “ is the problem, but really that’s the context; the real problem is that within that process, the make command fails. The output of the command isn’t terribly well structured, but there are hints that this is the core problem:
current directory: /Users/rschneeman/.gem/ruby/3.4.1/gems/sassc-2.4.0/ext
make DESTDIR\= sitearchdir\=./.gem.20250314-65761-9llhhv sitelibdir\=./.gem.20250314-65761-9llhhv
compiling ./libsass/src/ast.cpp
compiling ./libsass/src/ast2c.cpp
This is saying, “When I am in this directory” and “I run this command make <arguments>” then I get this output.
When someone is experiencing an exception on their Rails app, I encourage them to try copying that code into a rails console session to reproduce the problem without the overhead of the request/response cycle. This helps reduce the scope and removes a layer of abstraction.
Here removing abstraction will be manually go into that directory and run make. Doing this gave me the same error:
$ make clean && make
compiling ./libsass/src/ast.cpp
compiling ./libsass/src/ast2c.cpp
compiling ./libsass/src/ast_fwd_decl.cpp
make: *** [ast.o] Error 1
make: *** Waiting for unfinished jobs....
I was curious about how to get more information out of make and found a SO post suggesting that make -n will list out the commands. From man make or make --help I see this description:
-n, --just-print, --dry-run, --recon
Don't actually run any commands; just print them.
Running that gave me some output:
$ make -n
echo compiling ./libsass/src/ast.cpp
false -I. -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0/arm64-darwin24 -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0/ruby/backward -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0 -I. -I./libsass/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/libyaml/include -I/opt/homebrew/opt/gdbm/include -I/opt/X11/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -fno-common -fdeclspec -std=c++11 -DLIBSASS_VERSION='"3.6.4"' -arch arm64 -o ast.o -c ./libsass/src/ast.cpp
echo compiling ./libsass/src/ast2c.cpp
false -I. -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0/arm64-darwin24 -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0/ruby/backward -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0 -I. -I./libsass/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/libyaml/include -I/opt/homebrew/opt/gdbm/include -I/opt/X11/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -fno-common -fdeclspec -std=c++11 -DLIBSASS_VERSION='"3.6.4"' -arch arm64 -o ast2c.o -c ./libsass/src/ast2c.cpp
If you’re familiar with the output above you probably spotted the problem. If not, let’s detour and explain what this make tool even is.
Skip this if you know what make is and how to write a
Makefile
GNU make describes itself as:
GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program’s source files.
The library Rake is a similar concept implemented in Ruby. The name “Rake” is short for “Ruby (M)ake.”
In Rake, you can define a task and its prerequisites. The Rake tool will resolve those to ensure they’re run in the correct order without having to run them multiple times. This is commonly used for database migrations and generating assets for a web app, such as CSS and JS.
Technically, that’s all Make does as well, it allows you to define tasks in a reusable way, and it handles some of the logic of execution. In practice, make has become the go-to composition tool for compiling C programs. In that world there are projects that don’t even tell you how to build the binaries because they expect you to ./configure && make && make install in the same way some Ruby developers might forget instructions on adding a gem to the Gemfile in the README of their rubygem.
You can see a makefile in action following Ruby’s instructions on compilation
$ git clone https://github.com/ruby/ruby
$ cd ruby
$ ./autogen.sh
$ mkdir build && cd build
$ ../configure --prefix="${HOME}/.rubies/ruby-master"
$ cat Makefile | head -n 10
RUBY_RELEASE_YEAR = 2024
RUBY_RELEASE_MONTH = 06
RUBY_RELEASE_DAY = 06
# -*- mode: makefile-gmake; indent-tabs-mode: t -*-
SHELL = /bin/sh
NULLCMD = :
silence = no # yes/no
yes_silence = $(silence:no=)
no_silence = $(silence:yes=)
At the end of the day, make does very little. It’s almost more like its own language that happens to be useful for compiling code rather than a “compiling code” tool. The result is that the bulk of the logic comes from the contents of the Makefile and what the developer put in there rather than the Make tool itself. The output ends up being indistinguishable from a bunch of shell scripts in a trenchcoat.
Now we know that make does very little and we have its output we see two lines (I added a space for clarity):
$ make -n
echo compiling ./libsass/src/ast.cpp
false -I. -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0/arm64-darwin24 -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0/ruby/backward -I/Users/rschneeman/.rubies/ruby-3.4.1/include/ruby-3.4.0 -I. -I./libsass/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/opt/libyaml/include -I/opt/homebrew/opt/gdbm/include -I/opt/X11/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT -fno-common -fdeclspec -std=c++11 -DLIBSASS_VERSION='"3.6.4"' -arch arm64 -o ast.o -c ./libsass/src/ast.cpp
We could remove make from the equation by running them directly:
$ echo compiling ./libsass/src/ast.cpp
compiling ./libsass/src/ast.cpp
That worked as expected, what about the next line? It starts with false which, from the manual page
$ man false
...
DESCRIPTION
The false utility always returns with a non-zero exit code.
So no matter what comes after this command, it will simply exit non-zero. This command can never work. This seems odd, definetly not what the author of this makefile intended. If this is the bug, and I think it is, where is that false coming from? Is it dynamic from something in the environment (environment variables) or is it coming from shelling out to some other utility on disk or is it coming from some config file? Or is it static? Is it baked in already.
Re-running make -n with env vars (mentioned in the GitHub comments) such as CC="clang" CXX="clang++" has no effect. It’s the same output. This leads me to believe it’s something static.
Looking at the contents of the Makefile:
$ cat Makefile | grep false
CXX = false
Huh, that’s weird. Where is that used?
$ cat Makefile | grep CXX
CXX = false
CXXFLAGS = $(CCDLFLAGS) -fdeclspec -std=c++11 -DLIBSASS_VERSION='"3.6.4"' $(ARCH_FLAG)
LDSHAREDXX = $(CXX) -dynamic -bundle
$(Q) $(CXX) $(INCFLAGS) $(CPPFLAGS) $(CXXFLAGS) $(COUTFLAG)$@ -c $(CSRCFLAG)$<
That last line comes from this code in the Makefile:
.cc.o:
$(ECHO) compiling $(<)
$(Q) $(CXX) $(INCFLAGS) $(CPPFLAGS) $(CXXFLAGS) $(COUTFLAG)$@ -c $(CSRCFLAG)$<
Skip this if you know make syntax
To understand what this is doing, we can write a tiny make program:
$ cat Makefile
lol:
echo "hahaha"
The indentation under the
lol:should be a tab, but your editor or my blogging process might have converted it into a space.
Now when we run that:
$ make
echo "hahaha"
hahaha
It printed the command and then the output of that command. We’re not limited to static commands though. Modify the file:
$ cat Makefile
CMD=echo
lol:
$(CMD) "hahaha"
Here, we’ve extracted the command echo into a variable and are using that to produce the same effective command.
What that means is CXX=false tells make to replace $(CXX) with false which is not what we want. But where did CXX=false come from? I’m glad you asked. If you search the source code for that line, you won’t find it. That’s because this Makefile is generated.
When we looked at native extensions before, notice that I talked about rake-compiler and not about hand-rolling a Makefile. Even when we looked at ruby/ruby-s Makefile, it wasn’t hardcoded; it came to be after calling ./autogen.sh and ../configure. This Makefile is generated at install time.
false come from?When you compile Ruby ./configure && make && make install it needs to gather information about the system in order to know how to compile itself. Things like “what compiler are you using” (it could be gcc or clang, for example). Ruby isn’t the only program that needs to know this stuff; native extension code that compiles needs to know it, too.
When you compile Ruby it generates a rbconfig.rb file that contains information that Ruby users can access via RbConfig. From the docs:
The module storing Ruby interpreter configurations on building.
This file was created by mkconfig.rb when ruby was built. It contains build information for ruby which is used e.g. by mkmf to build compatible native extensions. Any changes made to this file will be lost the next time ruby is built.
So that info is what Ruby used at compile time. Where is it?
$ find /Users/rschneeman/.rubies -name rbconfig.rb
...
/Users/rschneeman/.rubies/ruby-3.4.1/lib/ruby/3.4.0/arm64-darwin24/rbconfig.rb
When I looked at that file I saw something alarming:
$ cat ../arm64-darwin24/rbconfig.rb | grep false
# frozen-string-literal: false
CONFIG["CXX"] = "false"
config[v] = false
When Ruby was compiled it came to the conclusion that it should use clang to compile C code:
CONFIG["CC"] = "clang"
But it mistakenly concluded that it should use the false command to compile C++ code (the meaning of these environment variables). It SHOULD be clang++ or something like clang++—std=gnu++11, but it’s not.
When the Makefile for the sassc gem is generated it hardcodes CXX=false into it by mistake because it is pulling that information from the RbConfig module generated by Ruby at compile time.
Why did it record false? Well, I don’t know. I assume it has something to do with the interplay between Ruby’s configuration script and Xcode developer tools. I didn’t debug down that pathway. Since we can fix the problem by re-installing the same version of Ruby with a newer version of the Xcode developer tools, it seems that the problem is in Xcode, but there might be a more complicated interaction involved (perhaps Ruby is doing something Xcode didn’t expect, for example).
Thankfully others came before me and came to the conclusion about where the problem was coming from and how to fix it. They suggested what I did above:
2409 worked for me)sassc to your heart’s contentAfter doing this you can inspect the RbConfig file:
$ cat /Users/rschneeman/.rubies/ruby-3.4.1/lib/ruby/3.4.0/arm64-darwin24/rbconfig.rb | grep CXX
CONFIG["LDSHAREDXX"] = "$(CXX) -dynamic -bundle"
CONFIG["CXXFLAGS"] = "-fdeclspec"
CONFIG["CXX"] = "clang++ -std=gnu++11"
Lookin good. It no longer reports false.
I mentioned above that it might be possible to manually edit these files to fix the problem. That would save the time and energy for re-compiling your Rubies. But you definitely want to upgrade your Xcode developer tools and ensure that future ruby installs have the right information. Going through the motions of this full process for at least one Ruby version (assuming you’re using a version switcher like chruby or asdf) is recommended. Personally, I uninstalled everything to decrease the chances that I have to re-learn about this problem and find this blog post X months/years in the future because I missed something in my process.
For those of you without this problem: Hopefully, this was educational. You might be wondering why I decided to blog about this specific topic (of all things). Well, I’ve got to do something while I’m recompiling all those rubies, and learning-via-teaching is a core pedagogy of mine.
If you enjoyed this post consider: