Jon Gjengset: Hi Ben, how amazing to see you here again.

Ben Striegel: It’s been quite a journey, Jon.

Jon: Yeah, I’m so excited this time, because we get to do arguably four releases today. Because there’s 1.56, there’s the Rust 2021 edition, there’s Rust 1.56.1, and there’s Rust 1.57.

Ben: It’s going to be quite a day, that’s for sure.

Jon: Yeah. Or night. Is it night? I don’t know.

Ben: It’s really relative, right, we can’t assume whoever’s listening to this right now is actually, you know, asleep. Maybe they’re actually— our soothing voices might lull them off, and that’s kind of their one ritual. They can only sleep every three months.

Jon: That— I mean, there are probably people like that, I think there are probably people using our very soft and soothing voices to fall asleep.

Ben: Oh, man. ASMR episode suddenly.

Jon: I know, right? That’s really what this channel is all about, you know. Putting people to sleep.

Ben: That’s what every podcast eventually evolves towards.

Jon: Yeah, exactly. I think we should just dig right into 1.56. What do you think?

Ben: I’m into it. Let’s jump in.

Jon: So obviously the big headline feature of 1.56 is the Rust 2021 edition, which was planned and announced, I think back in May, and has sort of gone through all the procedures since, of figuring out what exactly should and shouldn’t be in the edition. Do you want to talk a little bit about what an edition is, just to sort of recap?

Ben: Yeah, it’s useful to kind of just re-emphasize what an edition is. It’s kind of Rust’s solution for evolving a language, while being able to make quote- unquote breaking changes without actually breaking anybody. And so the idea is that users can opt-in to breaking changes. And also that the compiler can perform most changes automatically, so that you— actually, whenever you’re migrating, you don’t really need to. And also, whenever you get a new version of the compiler, if you create a new project using cargo new, it will automatically opt you in to the new edition, so that new projects over time tend to be on new editions. And so it’s a very, kind of like, gentle, gradual but inexorable push towards having a language that is quote-unquote modern in terms of correcting mistakes or making— giving itself wiggle room for future development, which we’ll see here shortly. And the important thing to note is that two crates on different editions can always interlink. You can always interoperate. There’s no split in the ecosystem. Underneath every Rust codebase or Rust compilation unit that you get from every library is all the same. They can all talk to each other on the same compiler version. So it’s incredibly useful, and it’s— once every three years or so, it seems to come, and this is our new gift that we can now present to you.

Jon: And I guess, explicitly, also part of it is that there’s no sort of release train, really. It’s not like, we need to rush this to get it into the next edition, or if we miss this edition, we have to wait another three years. My understanding at least is that it’s more, when we feel like we need to do an edition, and it’s sort of the right time to do an edition, then we do one.

Ben: Yeah, I think there’s kind of a general cadence for like, every three years, but that’s not like— there’s nothing— that’s not set in stone. It’s totally up to the people making it, all the volunteers can decide and— we’ll see this edition actually compared to the previous one is a bit less impactful. There’s still plenty of great stuff, but we can expect over time that as Rust gets more mature, the things that people regret will, you know, be reduced, in terms of the things that— we’re not making more mistakes, hopefully, as we develop the language, and that, you know, the pace slows down. So eventually. maybe there won’t be any more editions. Who knows? But the option is always there and this time there’s some great stuff in it. Let’s start with—

Jon: And it seems to be working pretty well, too. So far.

Ben: Yeah, I mean, like, people— I think we want to see more languages doing this. I think it’s going to be a big thing in the future. I don’t want to say Rust is the first one to do this. I think that back in the 90’s there was some Scheme variant that started doing this, in terms of like, you would opt into a new version and it would enable new things. And obviously you talked about, like use strict from Perl, and later JavaScript, it’s kind of the same thing. We’re opting into sort of a quote-unquote, better version of the language. But this is the most formalized it’s been, in terms of any kind of— is Rust mainstream? I want to say a mainstream language, but I’m not sure if Rust isn’t quite mainstream yet. What do you think?

Jon: I think Rust is going mainstream, I think it’s right on that, on the curb.

All right, so let’s dig into the very first part of the new edition, at least as mentioned in the release notes, which is disjoint captures in closures. Now, this is something that’s sort of been fielded a little bit up and down. It’s been available on nightly, I think for a while. And the very basic idea here is that if you have a closure that uses a field of some struct, like let’s say you have a struct x and you want to access field y inside of the closure. Then the closure will only now capture a borrow of y, or take y by value, whatever the closure needs, rather than bring all of x with it.

And there are a couple of cases where this comes up. For me, I think the most compelling one is if you have a— let’s say you have a method on a struct that takes a mut self, and inside of it you have a closure that maybe, I don’t know, filters an iterator or something, so it has to consume, probably in a read-only fashion, some field from the struct. But then you’re also in that same method trying to mutate a different field of self. In the past that wouldn’t really be doable, because the closure would end up capturing all of self in order to reference that one field. And therefore you couldn’t mutably access the field that you wanted to change. Or you could do it, but you would have to introduce some intermediate variable outside of the closure, that captures only that field by reference, and then move that reference into the closure, in order for the borrow checker to sort of understand what you meant. Whereas now with this partial capture, this will just work out of the box, which I think is a really nice change.

Ben: Speaking of methods, we should mention too that there’s a common frustration in Rust about the borrow checker, where if you have a method that takes self by reference, that might make it a bit more difficult to work with, because it will borrow the entire structure. It’ll borrow all of self, and not just any fields you want to work with. This does not fix that problem.

Jon: Yeah, so the take here is— I guess we should link this in the show notes. Niko Matsakis had this blog post on view types, which tries to get at this kind of idea, of being able to write a method that takes self but only needs to have access to certain fields, so that you could call it from another method that has self but is already borrowing a disjoint set of fields. Which is something that would be nice to get down the road, but as Ben says, this is not— we’re not quite there yet, even though this is a quality of life improvement, it’s tackling a slightly different problem.

Ben: Yeah, it is in the kind of vein of making the borrow checker more precise. And even though it’s not technically part of the borrow checker code base, any time that Rust creates an implicit reference, that’s still going to be— users will experience errors with the borrow checker if things don’t work out. So this is kind of like, you know, on the path of making the borrow checker a bit better in many cases. And previously you could work around this too, this disjoint capture issue, by manually taking a reference to whatever field you wanted in a new binding, and then closing over that binding in the closure. And this kind of just makes that pattern more implicit, or you know, not necessary any more. Less verbose. And that’s kind of the general pattern for, in C++, what you might call a capture clause. Where you can say, all the things I’m capturing and closing over in this closure I want to take this by reference and this by move, et cetera. Rust doesn’t have those; Rust just mostly infers everything that you want. And it mostly works out really good. Only in these kind of, like, edge cases does it not work out. So this just makes it a bit less necessary for Rust to even have capture clauses at all. But who knows if— what will happen in the future. There are still, again, like the view types kind of thing, are kind of like capture clauses, although not on closures, obviously on method fields, but we’ll see. Who knows? Someday in the future, maybe.

Jon: The next step is one that we’ve talked about before, which is the implementation of IntoIterator, the trait for arrays. And this is something that you can already do in the current edition. In this— as in— sorry, as in I guess the previous edition, so 2018. IntoIterator can be used on arrays, but it only works because of this little hack we talked about last time, and what’s going to happen in the 2021 edition is that that hack will be removed and there will just be a straight up implementation of IntoIterator for arrays, that will iterate over them by value and not by reference. So it’s really just sort of tidying up that, shall we call it, oddity from the previous edition, where this kind of worked and and also kind of didn’t.

Ben: Yeah, we spoke at length about this in the previous podcast, and we won’t go over it too much. I think it is worth pointing it out as kind of a feature of the editions where— so I think people might misunderstand the edition system, where you might think, oh, the old edition doesn’t get new things, it’s kind of like, you know, you might imagine it’s like being like an old version of the compiler, where you just stop getting new things. That’s not it at all, actually. So old editions still get all the new stuff that they can, which is pretty much everything. There’s only a few things that old editions can’t generally get. And so in this case around 52, sometime earlier this year, I think it was that version, somewhere around there. Every edition of Rust got the ability to use into_iter on arrays. In most contexts. There was only one context where you couldn’t, and it was a pretty important context, which is in the for loops.

And now the new edition has, uniformly everywhere across the board, the ability to use IntoIterator on arrays. And so, old editions are kind of— they’re featured, their full featured mostly, without— if a bit inconsistent, whereas the new edition is totally consistent in its ability to use this IntoIterator on arrays. And so it’s kind of an example of how you don’t need to lock people on old editions out of getting new things, you can still give them new things, even if nice isn’t— you know, the most nice or consistent form of those things.

Jon: The next one, I thought, is worth a little bit more discussion. So this is around or-patterns and macro_rules. So if you write a macro_rules defined macro, so a declarative macro in your code, and one of the arguments you try to take is using the fragment specifier type :pat for pattern. It used to be in past editions that this would not match or-patterns, so if you have like A | B. Whereas in this new edition, they will match A | B. And the reason for this was that in the old editions you could only use A | B at the top level of a pattern. So if you write match foo, in that first, like, the top level of the arm, you could say Some(1) | Some(2), but you couldn’t in the past write Some(1 | 2), where the 1 or 2 is inside of the Some. And therefore it wasn’t right for the macro pattern type to include ors, because it would mean that— the user might give you a pattern that contains an or, but the macro might put it in a place where it sort of ends up being a sub-pattern where or isn’t allowed. But now, because Rust, since I think 1.53, now supports or inside of nested patterns as well, it was realized in the 2021 edition, we should just make it so that the :pat fragment specifier type in macros just includes ors as well. So that all macros that that mention these patterns allow users to use the full power of pattern matching. And then there’s sort of a specific version of it called, I think “pat name”, is that what they used? Let me double-check— :pat_param, it’s called. That matches only the part without ors. And so this is one example where, if you move your crate from 2018 to 2021 you want to make sure that any macro rules you have that allow patterns as arguments, that you are actually okay with the patterns including ors. Because in the 2021 edition they will now do so by default. Which is almost always what you want.

Another change in the 2021 edition is that the Cargo feature resolver now defaults to version 2 of the resolver. We talked about version 2 of the resolver a while back, so this was in, I think Rust 1.51, that we talked about how Cargo now has this new feature resolver, and the new feature resolver tries to ensure that crates only get compiled with the features that they need, in the closure that they need.

Ben: The closure that they need?

Jon: Sorry, in the dependency closure that they need. So what I’m thinking of is something like dev-dependencies and build-dependencies for example, which are— in the past, Cargo would unify the features across dependency between the two, even though that’s not necessarily what you want. The new resolver will make sure to only use the features that are declared in the— in a given dependency block. So like, for your normal dependencies, your dev dependencies, your build dependencies. The feature sets will be kept different for dependencies, even though they happen to be used in multiple of them, which might mean that the dependency gets compiled more than once. But it also means that you only get the features that you actually asked for in that closure. Which has some implications for things like embedded workloads, where something might not even compile under a given target, but it would work fine as a build dependency. For example, if a feature was enabled. We talked about this at length back in the 1.51 episode, so I recommend going back and reading that if you’re— or listening to that, if you’re curious about it. But that’s at least now on by default, in Rust 2021.

Ben: Up next, there are some new additions to the prelude. So in Rust there is a thing called the prelude. It is essentially— imagine you’re writing a Rust program and ever wondered where, say, Vec comes from, or Option comes from. They’re available to you, you can write fn main, you know, let v = Vec::new(), or you can say, you know, let x = Some(...), et cetera, and it just works, you don’t need to have a use statement saying, you know, use std:: et cetera, Vec or Option. And that all is provided by something called the prelude. And it is just a few handpicked extremely essential items: traits, functions, macros, and types that want to be available by default. And adding things to the prelude, usually it doesn’t take much ceremony. There are already namespacing rules in Rust that make it so that if, say, someone adds a thing to the prelude, and it clashes with a thing that you already defined in your own crate, then your own crate just wins. There’s no real problem there.

The problem with adding traits specifically though, is that it might introduce a method on a trait that clashes with one of your own methods on a similar named trait. And then that might make it so that calling a method might become ambiguous, which might stop code from compiling. You wouldn’t get like a confusing method thing, it wouldn’t, you know, at runtime, it wouldn’t, you know, pick one at random, it would just say, hey, I can’t compile any more, I’m not sure what you mean here, please use the extremely verbose explicit syntax to tell me which trait method you want to call in this case. And so that means that to add a trait to the prelude, you do need to do it over an edition. And so, the specific traits that have been added in the new edition are TryInto, TryFrom, and FromIterator. Jon, do you want to talk about those first two?

Jon: Sure. So TryInto and TryFrom are, as their name implies, fallible versions of the Into and From trait. The idea here being that there are some types that can sometimes be converted into other types, but sometimes that conversion might fail. The obvious example here is parsing, right? So if you have a string, you can maybe turn it into a number, if it happens to contain a valid number, but it might also fail. You can think of TryInto and TryFrom as being the generic versions of something like the FromStr trait, which takes a string and— or takes any type that can be converted from a string fallibly. And TryFrom and TryInto were stabilized a long time ago, but it used to be that you explicitly bring them into scope in order to use them, which meant that people just didn’t use them that much, or didn’t even know that they existed. But now that they’re in the prelude, it’ll be a lot easier to just like type .try_into() or .try_from(), usually combined with a question mark operator, to do these conversions very straightforwardly.

Ben: As for the last one there, FromIterator, it is one half of a pair. So IntoIterator is the other side of FromIterator. IntoIterator is the trait that controls how for loops work. And so if your type implements IntoIterator, then you can use it with for loops natively. FromIterator is kind of the opposite, where mostly you would only encounter FromIterator as part of the collect method. So if you have an iterator chain of things, and then you call collect() on the end, under the hood that is using FromIterator to do all the magic. And collect (unintelligible— 18:51) very often and so, but you don’t need FromIterator in scope just to collect things. It’s because collect is already a method on iterators, which is already in scope, the Iterator trait. In the prelude, I mean.

And so, you might ask stuff like, why is it necessary? Well, I guess, you know, we could say it’s kind of just for neatness because IntoIterator is already in the prelude, although a bit more concrete use case is, if you’re mapping over something, and you want to say, you know, I want to map over this thing, that’s an iterator, you can just map over, say, like the first— using first class functions you can say, you know, Vec::from_iter() as opposed to having to say, you know, make a closure x and then call x.collect() on that closure. So that’s a bit nicer.

Jon: Yeah, that’s true. The next one is something that I know you’ve been a little bit involved with, and I think we should combine the next two, which is that panic!, macros now always expect format strings, just like println!. And that there is now reserved syntax for custom identifiers before the pound symbol. So think of something like raw strings like r# and then double quoted strings. Identifiers followed— or immediately preceding a string; think of something like byte strings. So if you type b and then double quotes what you get back is a u8 slice. And same thing for identifier preceding a character literal. So this would be something like b'x', which gives you a u8 instead of a character. Do you want to talk about these two and then how they interact?

Ben: Yeah, I’ll talk about this. So let’s back up real quick. There is something coming up in an upcoming version of Rust, which you can preview right now. It is called implicit formatting captures. So the idea is, currently, if you’re writing a println! macro and you want to, just like, you have a string "hello" and a string foo and you want to put them together, you might type println! and then you have to give your thing in quotes, your format specifier— your format string, really. And so it would be "{},{}", and then you after this— it’s hard to code in voice, actually.

Jon: Yeah, you’re right. You’re right.

Ben: The point is that in your println!, after you had your string, of how you want to print it, you would need to say your two variables, like maybe you have the string "hello", and a variable foo and the string "world" and variable bar you would have to end the println! invocation with , foo, bar and that tells the println! macro what to put in your brackets individually. And this is kind of a shorthand in fact. And so I think people are probably familiar with the brackets in the formatting strings syntax, where you can put, like, :, and you can put {:?} or you can put {:#?} or if you’re formatting integers or floats, there’s all kinds of various ways you can say that I want to, you know, add some white space here, I want you to pad this thing, I want you to round this thing to this many decimal points, et cetera.

So there’s a whole language, the format strings but people don’t realize that that colon that starts all of these implications actually is a separator. And before the colon, you can put exactly which variable you want to capture. You know, to format in that one bracket. And so— but, implicitly it just, you know, it’s an index counting from 0 to 1. And it will say, the first one goes in the first bracket you find, the second one goes in the second bracket that you find, and you could explicitly put 0 or 1, or you can put— you could print the same variable four times. Put 0, 0, 0, 0 inside the brackets, and that’s how it’ll work.

Alternatively, even more obscurely, in this syntax, you can give each individual variable to print its own little ID. So in my before-mentioned println! invocation, I could say x = foo, y = bar at the end of the println!, macro and then I could use x and y inside the braces to refer to the individual variables to be printed. And it’s a bit verbose, which is why it doesn’t really get used very often. If you’re doing really involved format strings, it could be useful.

But this explanation is useful to kind of— sorry, this explanation is useful to demonstrate what’s actually happening here under the hood now, with this new feature called implicit formatting captures. Which is nowadays, on current nightly, it’s got merged, like last week or so. If you just say println! and then like "{x}" and that’s the entire invocation, it will try to find a variable called x in your scope, and if it finds it, it’ll print that. And so nowadays, if you’re like, you know, used to format strings from other languages, the formatting facilities from, say, Python or JavaScript. This is how it normally works. So you don’t need to, you know, redundantly say the name of the variable you want to format. You just mention the name inside the format string and then the language will find it for you. and that’s how it will work in Rust in the future, where it will implicitly, if there’s no reference to, hey, I have no idea what x is, it will reach out in this running scope and find x, and then insert that. And so—

Jon: It’s going to be absolutely magical.

Ben: It’s going to be extremely magical. People have wanted this for a long time, kind of like, because Rust is like, it’s extremely explicit, but maybe a bit too much— Rust kind of always errs on the side of being explicit. And people are kind of like, well, this is kind of, maybe a bridge too far, and it is, kind of, naturally how macros work. But println! itself is not the most— not the least magical macro, let’s say that. It’s kind of involved. And so this is kind of, the minimum that you could do, to kind of like, approximate the format string facilities from— usually dynamic languages do this. So like, you know, JavaScript, Python, which many people are familiar with. And so, that will appeal to people used to that.

But there is a bit further yet you could go with this. And now I’m going to kind of get into the realm of speculation. Because this reserved syntax for this feature, now in Rust 2021, this would allow you to put any kind of arbitrarily— it would allow the language developers to put any arbitrary identifier in front of strings or character— string literals, character literals, et cetera. And this would allow them to consider adding, say, like an f specifier. Where as Jon was mentioning, you can put, today, you can put like an r for a raw string or a b for a byte string. Nowadays those are no longer special cased, and now every possible letter or combination of letters is reserved for future language development. And so you can imagine one day maybe there would be, say, like an f string, to kind of match Python 3’s f-strings syntax, where you can just say f and then quotation mark, and then you could say whatever and then close quotation mark, and it’ll just automagically create a formatted thing for you. And obviously this is Rust so it can’t really— you don’t necessarily want, say, to allocate, like many strings would do, it would be a bit more involved than that. It would probably create an arguments type, which is kind of an internal type from the format module in std. And so there’s a lot of details and kinks to work out before any of this could happen, but this reservation just makes it possible to consider someday having this feature.

Still, I would say there’s still a lot to consider, especially with regard to the actual syntax, because in Python, one of the things is that you can write any expression inside the format string, you could like— you could do calculations, you could call methods and functions, and anything you want. And whereas the newly stabilized implicit captures just lets you write identifiers. You can’t even do— there’s no field accesses, there’s no, like indexing allowed. It’s just identifiers. And so it remains to be seen whether or not people want to extend that to full expressions or some subset of expressions. These are all things that would need to be figured out before f-strings could even be considered. And so it’ll be a while before you, kind of get feedback from users on how they’re using it. Like we’ll see whether people are eager to move their println! statements over to format— the new format implicit captures, and their feedback on whether or not it’s sufficient for their use cases. So we’ll see. So this this syntax kind of just opens the door for this in the future. And the reason that it had to be an edition is because of, macros can kind of parse Rust these days, kind of like limited ways, and so theoretically this could break some macros. So that’s why it had to be an edition.

And then as for the panic! change, that kind of comes along with this. The idea of this new implicit formatting feature is that it should work on anything under the hood that uses the format_args! macro, which is what println! uses and assert_eq! underneath uses, all kinds of macros use this. And so if you have a macro today that uses internally this, then you will get this feature for free, whenever this stabilizes in like two or three releases. But panic! was kind of— this was an oversight back at 1.0. A single-argument panic! had a different kind of a code path than usual. It didn’t go through format_args machinery, it kind of just did a thing. And so it was a unique little snowflake. And so that has been melted, and now it has been made more consistent. But because that could break things, it’s only more consistent in the new edition.

Jon: Yeah, I’m so excited to see this, like making progress towards landing. I thought some of the other proposals for reserved syntax, what it could be used for, is nice too. Like you could imagine having, like, k#keyword allow you to create identifiers by the same name as a keyword in Rust. Stuff like that I think could be neat, too.

Ben: Yeah, that would be useful for— so I think in the previous edition, in the 2018 edition for example, one of the edition changes was to reserve various keywords, and many keywords were reserved in advance of them actually being used, kind of just like, you know, aspirationally, like async and kind of thing. So if you, say, had the ability to put k#, what you do now, with this new reserved syntax before an identifier, it would allow you to quote-unquote reserve identifiers, without the need for an edition. So it makes it easier to prototype, and to potentially get things in the hands of users, and then an edition would then be able to go through and you know, get rid of that k# on front of it. So kind of a— you can imagine it’s kind of a beta version of identifiers. So that’s kind of, one of the proposals. There’s plenty of things that we’re could talk about here. I wrote the RFC for this, so I could go on for for ages about the things that we could do here. But they’re all very provisional. I don’t want to get people’s hopes up, because really it’s just—

Jon: Well, you have my hopes up now, so—

Ben: Oh no!

Jon: So now you’re going to disappoint me, Ben.

Ben: Inevitably, like I always do.

Jon: The last one is— it’s not a big change, but it is one that’s worth noting as far as what editions are for, which is there are two warnings that have been promoted to errors in this edition. Bare trait objects, so this is something like, if you have a Box<MyTrait>, that is no longer legal. That now has to be Box<dyn MyTrait>. And there are lots of reasons why this became a warning in the first place, but the reason it’s becoming an error now— and the same is true for the other warning, ellipsis_inclusive_range_patterns, which is the ... syntax for inclusive ranges, which has been deprecated for a while and should be replaced with ..=. Both of these warnings have existed for a long time, and are really intended to— or when they initially landed, were intended to signal, this will be removed in the future. And we just want to give people, like, as long as we can, to sort of adapt to this change. And because it would be a breaking change to make them errors within an edition, we use the edition mechanism to also have these kind of deprecations actually take effect. Because the underlying machinery is the same, right? If you write Box<MyTrait>, it can compile to the same code as Box<dyn MyTrait>. So it’s only really the syntax that changes, and therefore we can turn it into an error at an edition boundary, because the edition boundary is explicitly opt-in anyway. And so this— I don’t think there are any warnings of this kind that are specifically being proposed right now for the 2021 edition, but it’s more that as we accumulate new warnings and deprecations and stuff, the edition boundary allows us to actually do that hard deprecation as part of that move.

Ben: Yeah, the policy towards warnings is interesting, in terms of— I think it’s evolved a bit since the previous edition. So the idea of this edition has not actually made anything— I don’t think it’s added any new warnings of the same type, where it’s— they will eventually become errors. Rust already has a facility for turning a warning into an error, that isn’t on an edition. And sometimes this is required for— because the Rust stability policy still allows breakage in the cases of unsoundness, or like, just obvious wrongness. And obviously, even though it’s allowed, that doesn’t mean people want to cause that kind of disruption. And so the idea is to minimize disruption. And sometimes people will say, you know, use an edition, sometimes people say, just do it. And just doing it means using a facility called the future incompatibility warnings. And this is kind of a newer thing, where— imagine you’re writing some Rust code using a dependency. Dependencies might have warnings in them, but by default, if you’re compiling with Cargo, it won’t show you— it’ll just kind of squelch all the warnings that dependency will give you. Because it figures that you don’t actually have control of it, in terms of like, you don’t— modifying the code. So like, don’t worry about it, it isn’t your problem. But if, say, some warning became an error in the future, that would cause your code to stop compiling when you upgraded your compiler, which is your problem. And so the idea is that this class of warnings called future incompatibility warnings will kind of pierce the veil, and will be shown to you if you’re using a crate that has them. And so this is kind of a newer thing in Cargo. I’m not sure if it’s, like, really being used yet.

Jon: Yeah, I think we mentioned this briefly last time, too. Because that’s when they mentioned that future compatibility lints would be added— would start to be added, or start to be used.

Ben: Yeah. So this obviously gets used sparingly because of the potential breakage. And so Rust has very good infrastructure for detecting breakage, in terms of running tests against the entire library of code on crates.io, which is an incredibly involved process. It takes like, days and days, but it’s extremely good at finding a secret breakage from any feature, and it gets run all the time on every Rust release. So it— ideally it would only be used for things that would have almost no impact, and that would, you know, have very big, like, security impacts, say like, you know, if there’s some kind of a unsoundness in something. But it’s kind of, I think subsumed the idea of doing these kind of warnings to errors in editions, unless they— obviously they could still do it if they wanted to. But I don’t recall that this edition has actually added any new warnings of this kind. And like I was saying before where, you know, over time, editions— we would expect them to become, you know, as the rough edges of Rust get smoothed over, fewer things need to be in those, maybe. It’s just that this time there was nothing that needed that kind of treatment.

Jon: Yeah, although I still feel like there’s a decent amount of good stuff that landed in this edition too. But I think you’re right that over time we should expect editions to get smaller in scope.

I want to mention one thing. So those are sort of the big changes in the edition. But I want to mention the cargo fix tool, which I think many people don’t really know about or don’t really know what it does. So cargo fix is a great way to have Cargo update your code, when Cargo knows how it needs to be updated. And in particular, it has a --edition flag that you can pass to make Cargo walk through your code base and make any changes that it can, like, do automatically to move you to the new edition. So this would be stuff like, if it realizes that you’re, like, manually including TryInto for example. I don’t know if that particular one is one that it supports, but it’s this kind of stuff where cargo fix --edition can just make any manual changes that you would be required to, if you were to adopt the new edition. It’s a great tool that’s worth knowing about, and looking into.

Ben: And that finishes off the edition-related things. You want to get into the new features on every edition, 1.56?

Jon: Always. I love these. So one big one is, Cargo now has a rust-version field in the package section of Cargo.toml, and this is something that’s sort of been on the table for a while, and there’s been a lot of sort of bikeshedding of what the name should be, but also some more fundamental questions about what the semantics should be. The basic idea here is that you can declare in your rust-version field, the minimum supported Rust version for that crate, and the reason why you want to do this is primarily to give your consumers better error messages. So currently, if someone takes a dependency on you, and you make a change that requires a newer version of the compiler, then what that dependent crate will see if someone compiles it, is there will just be some hard to understand compiler error. Like it might be some—

Ben: If they’re on an old version of the compiler.

Jon: If they use an old version of the compiler, exactly. Then they might see something like, this function doesn’t exist, or this edition doesn’t exist, or this feature isn’t stable, but has been stabilized in the version that you were using when you published that dependency. Whereas with— if you add this rust-version declaration in your Cargo.toml, then when you— or your crate gets built using an older version of the compiler, that compiler can just say, I’m not new enough to compile this crate, and therefore I need— you need to update the compiler, in order to compile it. And that’s a much more helpful user message because it actually tells them what went wrong and what version they have to update to.

Now it’s worth noting that there are some caveats to this feature. The first and most obvious one is that because support for rust-version was only added in Rust 1.56, there is no point setting rust-version to a value lower than 1.56, because a lower version wouldn’t know about the field in the first place.

The second caveat is that the rust-version field does not feed into Cargo’s dependency resolver. So if someone is using an old version of the compiler, and they take a dependency on your crate that has declared a rust-version, Cargo won’t pick the latest version that’s still compatible with their compiler version. It’ll just pick the latest version that matches the semantic versioning specifier. And then if that doesn’t compile with their version, it will give them an error. This may or may not change in the future. It’s a little unclear, because we do want to incentivize people to upgrade, and we don’t want them to sort of just transparently get an old version, and not know that they’re running on an old version. So there’s some subtlety there.

There’s another third caveat with this, which is that— remember that if you declare a rust-version in your Cargo.toml, but you have dependencies, then that Rust version might actually be— might actually be invalidated by a change to your dependencies. So imagine that you declare Rust version 1.56. and then one of your dependencies suddenly requires a feature from Rust 1.57, and upgrades their rust-version. Then now your crate lists rust-version = 1.56, but actually requires Rust 1.57, because of a dependency. There’s not really a good mechanism for enforcing something like this at the moment. Partially because most crates don’t really stick to a given minimum versioning sort of schedule. And this is a somewhat hard but also somewhat orthogonal question, that this doesn’t try to tackle, this is mainly trying to get at, can we give you better errors when a dependency requires a newer version of the compiler than you have. There’s still unresolved issues, like the ones we’ve talked about, but at least this is a step up in terms of the user experience here.

Ben: Yeah, I mean, so just as a notable example— so for example, the regex crate from Andrew Gallant (BurntSushi), that supports all Rust versions of 1.41 or greater currently. So if you are— that’s kind of, like, one of the more thoughtful versioning— so like, that crate kind of, like, sets the standard for, does regex support it. In terms of like, how old of a compiler, Rust compiler, does, like, a general well-developed, well-maintained crate support. So that was released, what, June of last year, about, so it’s— you would take in the future that kind of like, people have asked for, you know, long-term support releases of Rust, and it’s kind of been thinking about, what version should we support, and it always boils down to, like, what version shipped with Debian, et cetera, and in various package managers. So it’s a— this kind of, is one step towards figuring out, okay, like, what is our policy, what should a thoughtful crate do, for supporting old versions of Rust. Because it is kind of hard, because it means that you have to consciously not use features that are shiny and new. And for the sake of your users who don’t want to upgrade, or can’t upgrade, and kind of like, hold yourself back.

Jon: Yeah, it’s definitely, I think a part that the Rust ecosystem hasn’t really figured out yet. In part because it is challenging, once you take dependencies, because the dependencies are sort of out of your control. Unless you’re willing to, sort of, hold back your dependencies and say, I don’t depend on any 1.x version of my dependency, I depend on any version that’s, like, 1.x, but less than or equal to 1.5. Or whatever it might be. Right? You need to constrain them, because you don’t know whether future versions will conform to the same Rust version requirements that you promised to your consumers. So it’s a thorny situation, that I think we’re still navigating, but at least this provides a mechanism for declaring your intentions. Which is a step forward.

Ben: Let’s keep moving on. So an actual language feature this time. New bindings in binding @ pattern, where “at” is the @ sign. I think this is kind of one of the more obscure corners of the pattern syntax, and this has taken nothing from, just— it’s from ML, or Haskell, or some language that supports patterns, it has for a million years, it’s kind of just one of those things you don’t see very much in Rust. Maybe in those you see it more, but this allows you to bind an identifier to some pattern that you’ve already matched. And so this has been supported for a long time, but now you can have multiple identifiers bound in the same pattern. Or like, it’s kind of complicated but, just know that now— all of the restrictions have been removed in terms of, you know, letting you write this identifier in various places. There are still some things about, you know, the values must be Copy, I think. In terms of, to actually make it work, to avoid various ownership shenanigans, but it’s a pretty cool feature. If you’re into deep patterns, if you’re really into, like, match statements, learn what the @ symbol does, and see if you can use it in your code.

Jon: Yeah, I’ve certainly found the @ symbol handy. There are very limited cases where it comes in useful, but then it can really simplify your code structure, and it it makes me happy that now it’s— you can use @ in more cases to like, unlock that way of writing patterns.

Jon: I think we’re then onto stabilized APIs. So this time there weren’t too many sort of large ones that stuck out to me, we got a bunch of shrink_to methods. The idea here being that there’s already shrink_to_fit for many collection types, for example, that reduces the heap allocation used to store something to the minimal size that it needs to be in order to contain its elements. So remember how vectors for example, as you push to them when they grow they usually double in capacity. But that means that after you, if you know that you’re done pushing to them, you might then have all the spare capacity at the end and you want to reduce your memory use, so you can use shrink_to_fit. There’s now shrink_to, which takes an argument saying what capacity you want to shrink to. And that seems like a nice sort of quality of life improvement for those cases where you want finer control over how much memory you actually use. And this does— this doesn’t actually truncate the thing, like— it’s sort of bound by, or capped at the size— the length of the collection. But it is sort of a— now you have this option as well, of shrinking to something that’s not just the minimal fit.

We also got BufWriter::into_parts, which is really nice if you’re using a BufWriter for making your writes to your, sort of, I/O writes more efficient, then into_parts actually lets you take a BufWriter that you’ve written some stuff into, and turn it back into the original writer that you had, and any bytes that haven’t been flushed yet. Which is really nice if you’re writing sort of low level I/O code. I think that’s all I really wanted to mention there.

We got some constifications though. We got a constification of mem::transmute. Do you want to talk a little bit about that, Ben?

Ben: I want to mention it, but also I want to ask you if you know, it’s— okay. Obviously transmute is kind of one of the granddaddies of unsafe code, in terms of what it allows you to do, is just totally break type safety if you’re not careful. Which can itself trivially break memory safety. So you have to be extremely careful using this function. But now it works in const context. Or rather, it— before now, it worked only in const and static context. Now it works also in const fn. And so this means that it’s possible to, obviously, use it wrong, because it’s in unsafe code. But do you know what unsafe code does at compile time, if you use it wrong, in terms of, like, what does undefined behavior actually defined to do at compile time.

Jon: Yeah. So this is really interesting. Like, what does compile-time undefined behavior actually mean? And it is actually something that— we’re sort of still trying to figure out what the implications of that are. And I don’t think we have a great solution, or a great answer to that question yet. Apart from, if you have undefined behavior at compile time, it basically means your runtime code can end up being whatever. Like it’s not just that the runtime behavior can change from that point on, but you don’t even know that the code you end up with will actually be representative of the code that was written in the program. Like, you don’t know if the compilation will be correct.

Ben: Yeah, the code will still be wrong, obviously. Like, Rust doesn’t magically fix it for you, or make it not undefined behavior. Or not totally bananas. But I think that we should, you know, emphasize that it’s not going to, like, corrupt your compiler process. That’s like, one of the things that they do say about this, where it’s like, even though it’s undefined behavior, it’s not going to make the compiler go astray, it’s not going to like, you know, it’s not an attack vector on you yourself, if you’re compiling code, at const fn, that invokes some undefined behavior. And even in many cases, the expectation is that it will be able to catch undefined behavior, and then issue an error. Although it does not guarantee that. So the const engine named Miri will— does do its best to detect undefined behavior whenever it finds it. But obviously it can’t guarantee it. If we can guarantee it, we wouldn’t need to worry about unsafe blocks in the first place.

Jon: Yeah, it’s a really interesting, sort of, intermediate stage of undefined behavior, that I think currently you should think of as, if you invoke undefined behavior at compile time, what code actually ends up in your binary may not represent the code that’s in your code— in your source files. The two— just the moment you invoke undefined behavior, you’re sort of telling the compiler, you can optimize based on whatever assumptions you want, and I don’t care whether— or I don’t even now get the guarantee that the code is actually correct. Or accurate, maybe is a better representation of that.

Ben: One more library API thing to mention in this release is the implementation of From— so for all of the collections in std, so HashMap, etc., they now implement From an array of tuples, or From an array, and sometimes from an array of tuples for things that make sense like this. So essentially if you’ve ever written a Vec before, if you’ve ever made a Vec before, you’ll be familiar with the vec! macro. So like, just vec![1, 2, 3] makes a Vec of 1, 2, 3. And this is equivalent to Vec::from and then the argument to from could be a fixed-size array of like, [1, 2, 3]. So that— both that Vec macro and that From implementation have kind of been exclusive to Vec for a long time, even though Vec is only one of a few collections in std. And so people have asked for a long time for a nicer way to construct or initialize many collections. And so for example, let’s just use HashMaps. Like, everything I say for, you know, HashMaps will be true for everything else, too. Where if you want to make a HashMap these days, before this change you would say let mut x = HashMap::new() and then you would say, you know, x.insert, like 1, 2, 3, 4 etc. Actually, insert(1,2) and then the next line x.insert(3, 4), and so you need to have one insert, like insert command for every element that you wanted to have in this initialized HashMap. There was no real nice kind of like, one shot constructor for a HashMap before now, and people have, you know theorized, let’s just have like a hashmap!, macro and I’m sure you can find macros for this on crates.io. But there’s always the question of, what should the syntax be? Because every other language has their own syntax for, you know, hashmap constructors, and then people get bogged down in bikeshedding forever. And it’s kind of just like, let’s just, for now, instead of having to worry about syntax, let’s just give all these collections a From impl. And so the idea is, currently today you can already collect into these HashMaps, collect into a HashMap from an array of tuples. And so in this case it lets you just construct the HashMap outright from an array. So it’s kind of a nicer way— it’s kind of a half measure towards having a first class constructor for HashMaps and so now you would say, instead of having, you know, let mut x = HashMap::new() and then inserting various lines, you would say let x = HashMap::from([(1, 2), (3, 4), (5, 6)]). So it’s a fair bit nicer. It’s one of those nice quality of life things. And that applies for every collection these days, and doesn’t mean that there won’t be, maybe someday, some kind of macro for making it nicer. But honestly, ever since—

Jon: This gets pretty close.

Ben: It gets pretty close, and ever since const generics became powerful enough to have the From impl on Vec, like I mentioned before, I have honestly been preferring that, I don’t know, I kind of, I don’t have anything against macros, but I tend to gravitate towards functions whenever I don’t strictly need a macro.

Jon: Yeah, they also get formatted more nicely by rustfmt, in general.

Ben: Yeah and it’s also kind of just with IDs, it works a bit nicer to use methods instead of macros, in terms of like— easier for them to type check and present errors and that kind of thing, so—

Jon: That’s true. I had a couple of other things from the changelog too. So one of them— and if you read through, this might— you might read this and go, like, what does that even mean? Which is, there’s an entry there that says “Remove P: Unpin bound on impl Future for Pin”. And this is a PR that I filed, that I actually filed while writing Rust for Rustaceans, because there’s a section in there where I talk about, like, Pin, and why the various bounds are, what Unpin means. And then I got to this— the fact that there’s an implementation of the Future trait for the Pin type. And there’s a bound on there saying that P— like, this impl Future for Pin<P> had a bound that said, where P: Unpin and I just couldn’t articulate why that bound was necessary when writing the book. I just, from all I knew about this stuff, like that just didn’t make sense to me. And after thinking about it for a long time, I came to the conclusion that this must not be necessary. And so I filed the PR, and no one else could find that it was necessary either. And so now this bound has been removed. It’s not— I don’t think anyone cares about this implementation. It’s like, very much of a weird niche corner. But if you’re really curious about Pin and Unpin, it’s one that I recommend you go look at, and sort of look at the discussion and the documentation for why this is correct, because it is an interesting insight into how these all pieces, these weird pieces fit together.

Ben: On a personal note too, I do want to mention that that bullet point and the previous one, the impl From array for collections. Those were both hidden away in the detailed release notes. They didn’t make it to the actual blog post. And in fact, those two were written by yours truly, both— you wrote this one. I wrote the previous one. So I want to say, I think we’re being— I think there’s an agenda against us.

Jon: I think you’re right. I think that we’re—

Ben: They’re trying to silence us.

Jon: We’re not part of some kind of cabal that we should be part of.

Ben: The Rust illuminati.

Jon: Yeah, exactly. I want to be part of these backroom dealings for setting up the release notes.

There’s another really cool, like, low-level change that landed in 1.56, which is an improvement to Instant backsliding protection. And here we need to rewind, to give a little bit of context. So the std::time::Instant type in Rust guarantees that it is monotonic. That is, if you take an Instant, and then later on in your code you take another Instant, the duration between them is always going to be positive. So it’s always going— the value underlying the Instant is always going to go up. And normally you would sort of assume that this was guaranteed by whatever underlying mechanism the operating system or the platform provided. But of course, many of them have bugs, and do not actually guarantee this behavior. Like for example, on x86 systems on Windows, on I think AArch64 systems, and a couple of others, like particular versions of the Linux kernel on particular hardware. There’s, like, just a bunch of cases where this is just not true. You can take an Instant and then take another Instant, and the newer Instant has a lower value. And for this reason, the Rust standard library includes this thing called backsliding protection, which is basically: if it detects that it has an implementation that can sort of slide backwards in time, it forces the new value to be newer than the old one, so that code can rely on this guarantee. And the way it does that used to be through a mutex, so if it detected that the current platform has this kind of backsliding property, then it would take an Instant and then grab a mutex to check that the value was newer than the last value it gave out. Of course, mutexes experience contention, so this would cause some pretty serious performance degradations for any application that just like, takes a bunch of Instants and looks at the time. And so what landed in 1.56 was an improvement that moves this from being a mutex to being an AtomicU64, which has much less contention and doesn’t actually block at any time. It doesn’t have the same kind of contention experience. And so this is going to be potentially a pretty large performance improvement for performance-sensitive applications that do have to deal with time, which is basically all of them. Because they use it for things like profiling and logging, like it comes up so much that I’m very excited to see this land and I think there are probably production customers out there, that are like, this is going to significantly matter to our use case.

Ben: One last thing from the detailed release notes here, is that the LLVM 13 upgrade happened in 1.56. And that mostly brought a bunch of more like, you know, things that are in the weeds we won’t talk about in terms of improvements, I’m sure there’s plenty of performance improvements and compile-time improvements. One of the bigger things that came with LLVM 13, though, is called the new pass manager. And this has the potential for a lot of— a lot better compile times for various LLVM projects. And the— it’s actually not enabled right now. Some bugs were discovered a bit after enabling it— it had to do with recursively— mutually recursive functions getting inlined exponentially. So that’s currently still being worked on in the LLVM side, but once it finally gets enabled, it should have a pretty noticeable performance improvement. And the pass manager essentially— just LLVM as a compiler, as a back end. It does a bunch of little passes for simplifying code in various ways. And one pass might happen, the next pass happens after the previous pass. And the new pass manager essentially just has better caching and lazy behavior— and we’ll link to a blog post from LLVM talking about the details, because it’s a bit over my head, essentially.

Jon: I have one very last bit too, which is, if you care about, like, build systems or the configuration of Cargo. The Cargo configuration files— so whether that’s in /home/.cargo/.config or in .cargo/config. Now has an environment variable section, so you can have Cargo set environment variables for you, in the configuration. This might be helpful for, like, setting OPENSSLDIR or something, everywhere in your project. Or in any Cargo project that you build, if you have a particularly weird deployment setup. It’s just like a neat quality of life improvement for the people who need to configure Cargo and don’t want to have to do it all over the place, in lots of configuration files. That’s now supported directly by Cargo.

I think with that we can move to 1.56.1, and this is, this was a security release that came about from a CVE that spanned much beyond Rust. So this is CVE-2021-42574. And this CVE was all about Unicode code points that affect rendering of code. And it’s a problem that actually hit a lot of projects. Not just related to Rust code, like, this affected Python, Node.js, Go, like basically everywhere. And the basic vulnerability is that there are Unicode code points that allow bidirectional overrides for text. So this is something where you can put a sort of Unicode code point in a piece of, say, UTF-8 encoded text that reverses the direction of text. And then you could do it again. And what this allows is that if you have something that renders code, you can have the code contain a Unicode code point that sort of, makes the next text be written backwards over what was already written. And then you reverse direction again, and you write some different text over it. And there’s some good examples in the security advisory that we can put out, but this means that a malicious attacker can, like, make code that looks like it says one thing when you look at it in an editor, but the actual code when compiled and executed does something completely different. And of course this is not actually related to Rust. This is more about how code is rendered, and it might be rendered in a way where like, a security audit of the code, if someone reads the code it looks fine, but what actually gets compiled and runs is malicious somehow. But what the Rust project did was, they implemented a lint in the compiler that is deny by default, that basically yells at you if any source file contains any of these potentially problematic code points. Now, this doesn’t solve the problem, right? There’s still— you can still just opt out of this lint entirely, for example. But what it does do is at least give you sort of a warning sign if there is some code that contains this. That doesn’t help you if the code is, for example, on GitHub, and you’re just reading through it, which is why this CVE was so widespread in the community. Like, GitHub as well posted that they have addressed this vulnerability by adding a little warning flag at the top of all source files shown on GitHub that that contain these code points. So it’s just an interesting attack vector, where the Rust mitigation is really just giving you a sort of lint for this, but it’s not really a Rust problem. But at least the compiler can do what it— can do a little bit, to try to ease or mitigate the problem.

Ben: And to emphasize too, the big concern is like, comments mostly. Because you need to be in a context where you can put arbitrary Unicode characters which don’t— doesn’t happen in many places. So comments are the big deal here. Although Rust does now support Unicode identifiers. And I’m not sure if— I think some of these characters are allowed in Unicode identifiers. Rust does not support the full gamut of Unicode as identifiers, unlike, say, Swift, you can’t do an emoji function name, that kind of thing. But Rust, ever since the original release of Unicode identifiers, it has always linted very aggressively against extremely strange things like this. So any kind of confusables, or mixed scripts. It has a bunch— there’s maybe, like, four at least, lints individually checking for this kind of thing. So I know people are always worried, and you should be, but Rust takes it pretty seriously.

Jon: Yeah, and I think the Unicode standard actually has a, sort of— a section on identifiers, and how you should vet identifiers, and how you should— what subset of the Unicode symbol table you should allow in identifiers. And I think Rust follows that very closely. I know there’s a lot of discussion when Unicode identifiers were stabilized, on what rules exactly should we follow. But I think Rust ends up doing something very sensible there, and that probably reduced the impact of this particular CVE on Rust code too.

Ben: All right, and let’s move on to our final of four releases today, Rust 1.57.

Jon: It’s very exciting. It’s also odd, you know, because it’s—

Ben: It is odd. Yeah, it’s extremely odd. Although next time, probably less odd, the next release. Also, it’s much less impactful than 56. We won’t put an entire hour talking just about this one.

Jon: Yeah, that’s that’s probably true. Although there are some really cool changes in there. I’m excited to see this release too. So the first thing here is panic! in const contexts. Do you want to talk a little bit about this, Ben?

Ben: Yeah, sure. So the panic! macro is now usable in const fn. And so I might wonder what it’s useful for. Well, and— it’s actually really great for creating your own compiler errors. I’m pretty sure that’s— it’ll, like, just hook into the compiler error machinery, and if you panic in a const, you can kind of do a custom compiler error. I’m not sure if it’s— I’m not sure if that’s, like, the ideal way of doing that. There might be like a more, like, you know, first class way of doing it, but it’s a quick and easy compiler error. That’s the new hotness, and this also allows you to use the assert! macro in const context, which is pretty important in general, I’d say more important than panic! for writing code. Although we should mention it does not yet allow the assert_eq! macro because assert_eq! under the hood uses formatting code, and the formatting code has not yet been made const. And so— and in particular the panic! macro, that it works in const context, only allows certain forms that don’t include formatting.

Jon: Yeah, and this is also why unwrap and expect don’t work, right?

Ben: Yes. So working towards making const code be pretty and equivalent to non-const code. Now, you know, unwrap and inspect are pretty important for that, although we’re getting there.

Jon: Yeah, and I think maybe one way to frame this particular thing is that there are a bunch of things that internally use panic! that we would like to make const, and a sort of requirement for that is to make panic! itself usable in const. This would be things like assert!, which is now enabled, assert_eq!, which is not there yet. There’s also things like the todo! macro, and the unreachable! macro, which are also really just aliases for panic!.

Ben: I love todo! by the way, just like, one of my favorite macros.

Jon: Yeah, it’s pretty great.

Ben: What a good macro. I love it so much.

Jon: The next change in 1.57 is support for custom profiles in Cargo. So you might already know that Cargo has multiple compilation profiles. There are four that sort of come standard. There’s dev, release, test, and bench. That have slightly different configurations, like for example, dev includes debug info. release includes more optimizations. test includes the config test, among other things. And bench is like test, but with more optimizations. And now you can declare your own profile. So for— the example they give in the release notes is that you might have a production profile, that is like release but it also uses link-time optimization to like, really squeeze out every last inch of of performance from the binary. You can imagine having a profile that includes things like profile-guided optimization, or you might have a profile that is like release mode but with debug info, or test but with release optimizations. There’s all sorts of interesting profiles you can imagine coming up with here. And at least now you have the machinery for doing so rather than having to, like, modify the pre-existing profiles, and being able to give them better names.

Ben: Up next, we have fallible allocation. So these are library APIs which usually don’t get their own section in the release notes, but these ones are pretty important, because of what they kind of indicate for the broader direction, of Rust getting used, and that is Rust in the Linux kernel. And so the Linux kernel obviously, famously cares about not crashing. One of those things. And you might be wondering, well, what do these new APIs allow you to do? Why wouldn’t I use these all the time in my code? And they’re about memory allocation.

And so the idea is, very often when you write code, if you just say like— you make a Vec, say, if you’re, like, creating a Vec, And you want to push to it, that might involve reallocating the entire vector, which takes up more memory. And what happens when there is no more memory? For many applications, this question is entirely moot. The reason is, that on many OSes, Linux and macOS both, the OS just lies to you about how much memory you have. It is extremely common for the OS to say yes, I have memory, here you go, when in fact there is no memory backing it up. And it will kind of just like, lazily give you memory as you request it, or it’ll do shenanigans with like, trying to compress memory if there is contention. Or they have too much stuff being given out. And so generally if the OS lies to you, there’s really nothing you can do. And so many applications— the Rust standard library just doesn’t, just doesn’t care, really. And so it’s like we— it would be an extremely burdensome API requirement, to require every single thing that could possibly fail to allocate memory, to bubble that up to the user. And so if it happens, then we’ll just— we’ll panic and— actually it’ll abort, I believe, if that happens.

But, if you are in an environment where you do— can guarantee that the OS won’t lie to you, for example, if you’re writing an OS then it’s extremely useful to know that you are out of memory. And so this is kind of, it is not sufficient for Linux’s use case, I believe. I haven’t really read the proposal, but there was a long discussion. I’m not sure if it was a full RFC, but there have been many people involved with Linux, and contributing to the Linux kernel, have talked about how, okay, we would like to see various things from the standard library and Rust, to guarantee that we could use, even like, you know, libraries from crates.io, and you know, guarantee that they’re not going to allocate, you know, abort the process, you know, panic the kernel, just because they failed to allocate some memory. So this is kind of, maybe step one of that process. I’m sure there’s much more to go, but so essentially, Vec, String, HashMap, HashSet, and VecDeque all have try_reserve as a new function.

Jon: Yeah, and the big— one of the big missing parts here, I think, for the kernel, is that they want to sort of statically guarantee, so guarantee at compile time, that there’s nothing in there that could panic because of out-of- memory, which— this is a step in the right direction, but it doesn’t solve the problem, right? There is nothing that prevents someone from writing .push in the kernel, without first calling try_reserve. So there’s still a question of, how can we statically guarantee the things that the Linux kernel requires us to guarantee. And that’s still being figured out. But at least now we have the startings of the mechanisms that the kernel can use. And other use cases too, like embedded use cases, can use when they do want to know whether an allocation succeeded or failed.

Jon: We have a bunch of other stabilized APIs too, which are sort of less, monumentus than try_reserve. I think the first one I wanted to call out is, Command now— so this is std::process::Command, the thing you use if you want to spawn a process inside of a Rust program. So previously, it had, like— you do Command::new, you give a program name, and then you can add arguments, you can set the current directory, you can set environment variables. But you didn’t have a way to inspect a given Command before you execute it, and look at what arguments it is being passed, or what environment variables it is being passed. Whereas now, it’s gained a bunch of get_ functions, so there’s get_program, get_args, get_envs, and get_current_dir, which gives you the configuration of a Command and lets you introspect it before you choose to actually execute the command.

The other one that I think is pretty neat is that there is now a map_while method on Iterator. So the idea here is that, it’s sort of like filter_map on Option. Or I guess not, there’s a filter_map on Iterator too, where it will only— it will run each item through a closure, and the Iterator will only yield the items for which the closure returns Some and not the ones where it returns None. With map_while, it’s sort of similar, except that it will stop the first time it gets None. So it’s like “take until” (editor’s note: should be take_while), and then with a closure that when it returns None, it stops taking. And the closure also gets to map the item, so it’s like another nice utility on Iterator that makes it somewhat easier to write concise Iterator sequences, especially if you’re sort of trying to write your program in a somewhat functional style.

Ben: And constification continues unabated. This time with the unreachable_unchecked function. And so essentially, unreachable_unchecked— it’s an unsafe function. It’s a way of telling the Rust compiler, hey, here is a branch that can never be taken. And so for example if you have an if statement where it’s just literally if true, you could have an else branch with an unsafe { unreachable_unchecked() } in there, and then that would tell the compiler, hey, this branch can’t ever be reached. And that’s kind of a useless example, but there’s various examples where, for example, if Rust is requiring you to write a branch for something, like say you have— you know that some— that the value in an Option is a Some, you’ve already verified it to yourself, maybe you, like, made it yourself earlier up in the function. But Rust doesn’t know that. It just sees an Option. And so maybe it forces you to have a branch on there. If you want to use that branch and you wanted to put, you know, unsafe { unreachable_unchecked() } in there, that indicates to the optimizer that that is a totally unreachable branch, and it should be optimized away. And that means there won’t be any kind of branch there at all in the final binary. Obviously it’s unsafe, because you do have to manually ensure that you never get there. Because if you get there, undefined behavior.

Jon: Yeah, because unlike unreachable!, which panics, this does not include any panic machinery in that branch.

Ben: And so yeah, so there’s the unreachable! macro, which is just, you know, it’s great for indicating this, but also still being safe. And now this is a const function. And so nicely, it will give you a compiler error. So that’s actually, it won’t be undefined behavior at compile time. I’m pretty sure, right? Yeah, yeah. So Miri is smart enough that it will try to take the branch, and it will say, hey, it’s a compiler error. It’s— we’ve reached it. So it will give you a nice compiler error saying, hey, you messed up.

Jon: Yeah, assuming that it is being evaluated in const context, right? So if you have a const fn that includes this, then if that const fn is called from constant context, then hitting it will be a compiler error. But if you invoke it at runtime context, then it will be undefined behavior.

Here, too, we’ve done our usual thing of walking through the changelogs. And I think one that struck me as particularly entertaining was the fact that Vec::leak no longer allocates. Which— so you may be familiar with Box::leak. So Box::leak takes a Box, an owned Box, and returns to you a 'static mutable reference to the underlying memory. And the intention here is that once you leak the value, it will never be dropped, and therefore the heap memory is just valid forever. So it can give you a 'static reference to it. And the same thing exists for Vec, but with Vec::leak, what it would do is, it would first shrink the allocation to be actually the size of the length of the array, not just its capacity, and then give you back a reference to that slice. Of course this meant that it was a little weird to call Vec::leak, because it did an allocation and a copy of the entire vector, in order to move it into the smaller capacity. But now with this change, Vec::leak will do the sort of intuitive thing, of keeping the whole memory allocation and still just giving you a reference to the slice of just the contents. Which I thought was entertaining, that it didn’t do this in the past.

Ben: Similarly, the final item in my notes is that, as of the new release, Rust now has a new tier 3 target of the Nintendo 3DS.

Jon: It’s so great. I don’t know why— I don’t know why this was added, but it—

Ben: Somebody wanted to write Rust on the 3DS, and they were like, you know what, why not? Let’s do it. And so we should mention too, that the tier system— tier 1 are targets that get, like, you know, tested every single, like every single commit gets, like, the full test suite run on it. And so they’re very well tested. Tier 2 is— they are guaranteed to compile with every commit but they were— with every release, that is. But they won’t run the full test suite. And then tier 3 is anything goes. Like, we’ll add support for this if you really want us to, but it’s up to you to make sure it works. There’s no Nintendo 3DS test runner somewhere, in someone’s, like, kitchen, hooked up the internet, with, like, you know, running random Rust code every release to make sure it still works.

Jon: It’s very sad. We should definitely have a fleet of 3DSs.

Ben: We should.

Jon: I had a couple more things. So in Cargo— and this is actually a change that happened in 1.55 but but didn’t make the release notes. Which is that Cargo will no longer have RUSTFLAGS be set for build scripts. It used to be that if RUSTFLAGS was set in the environment, and then Cargo would just pass that through to build scripts. But it wasn’t entirely reliable, because if for example RUSTFLAGS was not set, but there were rustflags set in the Cargo configuration file through, like, build.rustflags for example. Those would not be set in RUSTFLAGS. So build scripts that relied on the RUSTFLAGS, environment variable would actually get a sort of misleading value. Now, since 1.55, and announced now in 1.56, Cargo will set a different environment variable called CARGO_ENCODED_RUSTFLAGS, which is guaranteed to hold the current set of RUSTFLAGS that Cargo is using. Encoded in a way where, like, whitespace splitting and such isn’t a problem, that build scripts can actually rely on.

Another change that I thought was kind of neat is that, there’s been a lot of additions of the must_use attribute to functions in the standard library. So I think we mentioned must_use back when it first stabilized. The basic idea is that you add it to a function definition, and anyone who calls that function is required to do something with the return value. The idea being that for example if you have a function that has no side effects, it’s a pure function, then calling it without looking at the return value is probably an error. An example of this might be saturating_add on integers, which returns the result of doing the saturating add. It doesn’t actually modify the value that you call saturating_add on. And therefore not using the return value means that you’re probably not doing what you think you’re doing. And so it has a must_use attribution. And a bunch of similar attributes were added in 1.56. Like, on the Stdin and Stdout and Stderr locks, on thread::Builder, on Rc::downgrade. There’s a whole host of them. And I think this is just a really good way to just eliminate errors that are hard to spot when you just like, look at the code briefly. And you really need to look at the signature. But here the compiler can actually help you realize that this was wrong. I think there are like 800 attributes that were added to the standard library in this— in 1.56, which is wild. And really cool.

I have two more that are both fairly minor. One of them is that File::read_to_end and File::read_to_string has got this really cool optimization now, where previously if you had, like a vector, and then you called— you open a File, you did read_to_end, read_to_end would just like, start streaming the file into the vector, and the vector would just like, dutifully double its size every time it fills. But that means that you spend, especially towards the end, you spend a lot of your time just reallocating the vector and copying over all the bytes that you’ve previously read. And so reading a file ends up being this, like, sort of exponential process of copying bytes, in edition to actually reading from disk. And what changed in 1.56 is that now, File will first look at, like inspect the file system, and look at the size of that file. And as long— if it can realize what the size of the file is, it will pre-allocate, or sort of resize the vector to be that size before it starts reading, which should potentially significantly speed up file reads, which is really neat.

It’s also an indication that there are still pretty significant improvements that can be made to the standard library. So like, feel free to dig into there and just like find something interesting and start poking at it, if something doesn’t perform to your satisfaction.

I think that’s actually all— the other one is a smaller, I’ll mention this as well, which is that if you have a macro invocation that uses curly braces, you can now use, like .method() or ? after that invocation. Which is nice for things like, if you’re using the quote! macro, from the quote crate, for doing, like, procedural macros and code generation, then you can now use quote! {, write some sort of code that’s going to be expanded into a token stream, }.into() to turn it into a proc_macro::TokenStream that Rust expects. Previously that .into() would fail to parse, for relatively silly reasons, and that will now just work. It’s a big quality of life improvement for just like, macros not being as weird.

Ben: That’s kind of an obscure feature that we could, like— as a gift for people who actually listen to the entire podcast, any macro invocation can use any bracket of, you know, parentheses, square brackets, curly brackets, and it’s just not a big deal. Like you can have a println! with square brackets if you want, or with curly brackets, or anything. Only convention stops us from having, you know, the vec! macro with curly brackets. You can do whatever you want. And so ideally, all of these would act the same. This just makes them, the curly brackets act the same as the square and the smooth brackets.

Jon: Yeah, it’s funny, right? Because with the square brackets, this already worked, right? If you write, like, quote![things].into(), it would have just worked. So this was just an oddity for— basically because curly brackets also need to be interpreted in other contexts, and there they have a different semantic meaning. So it’s just— it was really interesting, like, to look at the PR too, and see the reasoning for why this didn’t work and why it now works.

Ben: Yeah, just a parser quirk, essentially.

Jon: Yeah, exactly.

Ben: All right. And that’s it.

Jon: Yeah, I think we got through all four releases. Pretty good.

Ben: And that’s an entire year’s worth of Rust development we have finished again.

Jon: Oh yeah, that’s wild.

Ben: This is our last 2021 podcast.

Jon: That’s wild. We’ve been doing this for a while now, huh?

Ben: Yeah, we should do some kind of like, celebration at some point. I guess, I don’t know.

Jon: Well, I can do that right now. (organ music)

Ben: You’re just waiting the entire time, to use the sound board, weren’t you?

Jon: Exactly. That’s great. I’m excited to see what the new year will bring. And I believe that Rust cannot get any better. So I’m assuming that all following release notes will be empty.

Ben: Oh yeah, we’re done. That’s it.

Jon: Yep. We all know that 2021 was the year of Rust on the desktop, and now it is complete. There are no more bugs. There are only feature requests.

Ben: All languages have become Rust. They all submit to Rust. There’s only Rust.

Jon: Yeah, every other language is now transpiled to Rust.

Ben: All CPUs now run Rust code directly. No need for a compiler at all.

Jon: Yeah, microcode is also written in Rust now.

Ben: It’s Rust all the way down, every circuit on your CPU, no more logic gates. No more, like, transistors, or you know, neutrons, protons et cetera. It’s all Rust code.

Jon: Yeah, it’s Rust—

Both: all the way down.

Ben: Yeah, the fabric of the entire universe.

Jon: There’s a turtle at the very bottom.

Ben: Made of Rust.

Jon: Yeah, made of Rust. Yes, it’s a rusting turtle.

Ben: All right, well, I’ll see you next year then, Jon.

Jon: I will see you next year, and enjoy the holidays.

Ben: Happy 2021, as best you can survive it, to all of our listeners.

Jon: Bye, everyone.

Ben: Bye!