Jon Gjengset: Hello, Ben.

Ben Striegel: Hello, Jon. Happy New Year.

Jon: We are back yet again, believe it or not. I think we have to start off by giving a bit of an apology to our listeners, because we’re going to talk about Rust 1.40, and it came out December 19th. And instead of just delaying by a few weeks, as we usually do, we decided not to release the next episode until the next decade. It’s a very long time.

Ben: I think our listeners will forgive us, hopefully. But, I mean, it was a prudent choice, really.

Jon: That’s true. It was— we were just very busy. All right, Ben, let’s dig into the release notes for 1.40 and I guess at the top of that list is the non_exhaustive attribute.

Ben: Yeah.

Jon: So this attribute is really cool, especially if you’re a library author, because it lets you annotate a type with the non_exhaustive attribute. And what that means is that you intend to add things to this type later. So if that is a struct, that means that you might add fields to it later. Or if it’s an enum, you might add variants to it later. And this is really handy because it means that you can take— you can expose publicly a type that you know is going to change in the future, in such a way that it’s going to be backwards- compatible if you add things later.

So the trivial example of this is, you have an error type, like, an error enum, and you have a bunch of variants for the different errors, and you want the ability to add more error types later, without breaking people’s code. And what non_exhaustive does is one primary thing. It means that no one can match on your type, without including, sort of, the underscore pattern, right? They need— when they do the match, they always need to write the code under the assumption that more things might be added. And there are a lot of places where this makes sense. So one case is, let’s say you have a struct and some of the fields of that struct, you want to make public. Normally, this would be a bit of a problem, because if you did that, people couldn’t— and then people started matching on your struct, for example. Then there wasn’t really a way for you to add more public fields later without breaking people’s code if they relied on your type.

Ben: I want to kind of back up, though, because, like, in this sense, talk about matching, or any kind of pattern usage. Normally we think about enums, right? Like, it’s— if you ever use an enum in Rust, pretty much the way that you have to use it, one way or the other, is you have to match on it. And sometimes there are convenience methods on Option. That means you can just write is_some as opposed to have it going through a whole match expression. But, like, so you are probably familiar with, you know, the match, you say, first pattern, second pattern, then underscore, return this, as the default case, kind of analogous to a switch statement in other languages, where you have a default case at the very bottom, I think for structs, it’s a less well known fact that you can match on structs, or that you can use a struct in a pattern context. So why don’t you talk about, like, where structs are currently usable in pattern contexts and why you might want to do that.

Jon: Sure. So the most common case for this is, you have a struct of some type, and you want to look at just one field of that struct. And one way that you can do that is, just— you have a type you have a variable x of type foo, and you do something like x.field, right, and that is one way to get at that that field of that struct. But there are a bunch of other times where you might want to— let’s say, do a nested pattern match, right? So you want to look at a foo where it’s a is a bar where the bars b is something else, right? So you want to construct this nested pattern and you want to get, say, a reference to some field that’s deeper down in that pattern. And maybe there’s some enums or variants in between. And so it ends up being a pattern that’s not just a direct field access.

And in the past, as a library author, you sort of just have the choice of, either I make this field public, and then let people access it. But then if I add a field later, anyone who matches on it, that pattern is no longer going to compile, because it doesn’t mention this new field that I’ve added. So adding a public field would be a non-backwards-compatible change. Right? If I had a type foo that had only one public field a, then previously— or in the current version, people could match on foo and then just write, sort of, { a }, and then be able to access that field a. But if I add a public field b now, then that code that someone wrote with the current version of my library would no longer compile, because the compiler would say, this pattern doesn’t match foo because foo has two fields, a and b. And so the way that you do pattern matching for these non_exhaustive types, or in general, when you do pattern matching the way that you should probably do it, is to include, for structs at least, the sort of , .. pattern. So this is saying “I’m going to match on these fields, and there are others that I want to ignore.”

Ben: Is that actually— does that work today? The , .. pattern?

Jon: Yes, it does indeed. So this is a really common thing to use, even just internally in your code base, where— let’s say you want references to two of the fields of Foo and you don’t care about the others. You could write that as, like, Foo { field1, field2, and then match on all the other fields as well, and then, just map them to underscore something. But that means that if you later changed Foo in your code base, you would have to update all of the pieces where Foo is matched on, and that’s really annoying. And so instead, in the pattern you just write Foo { field1, field2, .. } and that means, there are other fields, ignore them. And so, that already works on stable today. What this change is letting you do is, mark your type as something where the compiler is going to enforce that people who match on your type must use the , .. pattern

Ben: Or for an enum, the _ => pattern.

Jon: Right, exactly. They must assume that there might be more things, in whatever pattern they write. And so this means that you can now add more variants, and know that you won’t— you won’t make users’ code not compile, just because you added a field or a variant.

Ben: Now I think it’s worth talking about here that there is a tradeoff, in that some users, in fact, do want their code to stop compiling when one of their dependencies adds a new field to some enum, and kind of the whole point of a match being exhaustive, in some ways, is that you can do this. And so the default behavior is still that that happens, obviously. But is it like, maybe, a thing to use with caution, this new attribute? Just because some users really enjoy the idea of, okay, I’ve covered every single case and nothing can go wrong. They don’t need a default case, or like, a fall-through backup case.

Jon: I think that’s definitely true, both for structs and for enums. But perhaps especially for enums, where this sort of— the compiler checking that you’re matching exhaustively is really handy. I think usually what people will end up doing, is have the underscore match map to, like, panic or something. But that just means that now you’re going to get runtime errors, which is something we want to avoid. And so in libraries, only use this if you really need it. If there’s a type where you expect that you will add more things, and you don’t expect that people are matching exhaustively on it. There are libraries today that try to hack around this by adding, like, a variant, that is called __non_exhaustive or something like that, and it works, but it’s not very pretty. And that does give users the option of exhaustively matching. But it does mean that you you might break semantic versioning accidentally, in some sense.

Ben: Yeah, I’ve seen, I think the url crate is very popular, obviously, has a version of this where it has, like, one variant, where it’s, you know, it’s there, you can use it, but it’s underscore underscore something un- guessable name, and then it’s marked #[doc(hidden)] so it doesn’t actually show up in documentation. So you have to actually go read the code to know that it’s there. It’s kind of like, hey, we want to be able to evolve this crate in a way that doesn’t necessarily break our users down the line. Because for a crate as fundamental as url, that’s a big deal. Like it has, may have— one breaking release in the past. And it was a huge problem for everyone involved. So in this case, I think really, the real guideline here is just, errors are a great place to use this attribute. Maybe not really anywhere else. Errors you might expect, hey, like, I want to be able to have my crate, you know, throw a new kind of error, you know, return a new kind of error, like maybe internally, your policy is that, you know, for every— each of the crates that your crate uses, its own dependencies, maybe you have, like, one kind of wrapped error for whatever it’s doing, and you don’t want to, like, oh, if I add a new crate, I can’t now— and it has its own error type, what am I going to do with my custom error type? Like, I can’t add a new variant to wrap that type, which is a very common pattern and how I really enjoy modeling my error handling in my Rust crates.

Jon: Yeah, I think that’s true. And especially for errors, it’s rare that you want to exhaustively match on them. Usually, what you want to do is, like, if it’s one of these two errors or three years, then I want to handle it, otherwise, I want to, like, just bubble it up with question mark or something.

Ben: Yeah, which is why I think that, like, yeah, it make sense here, in errors specifically.

Jon: Yeah, And I think even in the standard library, the std::io::ErrorKind works a little bit like this, where I believe that that either has, now, the non_exhaustive flags or there’s, like, a hidden variant that you can’t match on.

Ben: Yeah, it’s actually— it’s a pretty useful thing, and I think that I want to, like, talk about real briefly, just how there are some languages that really go the distance, in terms of really enabling libraries to evolve, like adding language features that then enable libraries to change, without having to necessarily impose breaking changes on their users. And I think the biggest language I’ve seen do this is Swift, which has a whole lot of features that are, like— it’s kind of— if you were to just read the feature list, you’d be like, when is this actually ever useful? And eventually at some point it clicks where, hey, they want to be able to change the underlying libraries, without forcing end users to have to make changes to their code. Because they want to be able to, like, shift a stable, swift runtime and library with all the Apple devices. And then have it just work, pretty much forever.

Jon: Yeah, I think in some sense this is, like, to do in the favor of building a robust ecosystem. Right? Where you want breaking changes to happen as rarely as they can.

Ben: And I think the poster child of this, kind of, idea of languages supporting features that enable libraries to change backwards compatibly, is probably something like, default arguments in functions because you can imagine like, oh, I have this function. And now people are using it. Oh, but I want to add a new parameter to it. And I can’t do it without making people change their things— oh, wait. If it has a default, then no one needs to change anything. So it’ll be a— it’s kind of a topic people have wanted for a long time in Rust, the idea of default function arguments, or keyword arguments. So who knows? Maybe we’ll eventually evolve in that direction.

Jon: Maybe one day. There have been a bunch of other changes in 1.40, and I think a large swath of them fall into the category of various macro improvements. And this is an area that we’ve touched on in the past, of how there have been changes to the Rust compiler up through the releases, that seem to be targeted primarily at macro authors. And it’s the same in 1.40, although here it seems like the focus is a little bit more on the use of macros. So the first of this, is that you can now call function-like procedural macros in type context.

Ben: Back up. What is a function-like procedural macro?

Jon: Yeah, there are a lot of words to unpack here. So, procedural macros, for those who aren’t aware of them, are basically Rust programs or functions that take as input Rust code, and produce as output Rust code. And this lets you write macros is in a somewhat convenient way, right? You get a token stream of input, which is just the Rust code that was provided to the macro. And then you produce Rust code as a token stream, which is going to be the code that actually gets compiled. And then there’s some— the function basically gets to juggle the incoming token stream around to produce the desired output code.

And a function-like macro is one that you call a little bit like a function. So these are the things that look like some name exclamation mark and then some open brackets and a bunch of arguments. There are other type of macros that are not function-like and these would be things like if you see a #[ ] in front of a function or a type or something, so this would be stuff like #[derive( )]. It might be things like— the various attributes you can add when you’re using serde serialization and deserialization. They can be other things, like non_exhaustive arguably is a macro attribute, although it isn’t actually implemented that way. And so there are a bunch of different types of macros. But the function-like ones are the ones you’re most familiar with that have the bang, and are the kind of things that are generated if you use, for example, macro_rules!. And what changed— the first thing here that changed is that now you can call those function-like procedural macros in a type context. So that is, you can write a function-like macro that produces a type signature. So, for example, the example they give in the release notes is, you say something like type Foo = and then a call to a macro that’s going to expand to some type. And this is something that you couldn’t do in the past.

Ben: I could also imagine, maybe a macro like this being used with, like, mem::size_of, perhaps. If you wanted to try to think about, like— because right now, you could still use, like, a macro_rules macro in this type context, trying to think of a thing that might be useful for a procedural macro specifically. And so say, if you had— there are some crates out there that use procedural macros to do really wild things, like, check certain database schemas at compile time, for you to verify that your SQL is correct. And you could imagine maybe something like that, creating a type, and then you might want to check the size of that, to see if it does certain things. So I think it’s mostly the use for— kind of like one flagship use for this.

Jon: Yeah, I think that would be one example. And the other is the recurring one that that we’ve mentioned before, which is this of having macros, in some sense, not be special. That anywhere where you can write Rust code, you can just swap in a macro and have it work. So in some sense, it might just be for the purposes of completeness, right? To remove rules around where you cannot use macros. Previously, you could couldn’t use a macro in that position, and now you can.

Ben: Although I think it’s worth talking about, what places can you still not use these sorts of procedural macros? So prior to this release, you could only use function-like procedural macros in what’s called item position, which is basically— item position is just like, look where you’d usually define your structs and your statics and your consts, and all these things in your code, it’s the very top level. So an item is something that exists, just like, not in expression position, inside like a function, say. That sort of thing. It’s a definition that you would see anywhere normally in your code. And now you can do it in type context, but, for example, you still can’t do it in expression context. And so for example, the println! macro, which is pretty much a function-like procedural macro— it’s a compiler built-in, but we’ll ignore that, for now— is usable, just in a normal expression. And so it is pretty much useless to actually do this, because the println! macro just returns nil.

Jon: Well, the format! macro might be a better example.

Ben: There you go. So it returns a String. And so the format! macro is not a thing you could write with macro_rules!. It is— actual interpretation of strings at compile time, inspects the interior, type-checks it, and in this case, then returns a String, and so you can use it as part of an expression. You can say let x = format!(something) and it’s incredibly useful. You can’t currently write your own procedural macros in this way, but there is a crate by the inimitable David Tolnay, author of serde, or I guess maintainer and author of serde, who— the crate is called proc_macro_hack, which does allow you to write procedural macros in these positions by effectively just defining for you a macro_rules! macro, which can be used currently, is both of those positions, and then having an internal crate— a separate extra internal crate, then it exposes different things. So, it shouldn’t be necessary, but right now it is possible to do, if you want it hard enough.

Jon: Yeah, I think it’s interesting too, to observe that macro_rules! is still fairly different from procedural macros. Right? A function-like macro that’s defined through macro_rules!, you can use in a bunch more places than you can function-like procedural macros, and I don’t really know why that is. But it’s interesting that it’s the case. Although I think they are trying to remove those differences as much as possible. And this is sort of a first step towards that.

Ben: Yeah, making things more uniform is certainly a nice simplification. We don’t want to have to explain, why does this work here? This doesn’t work here. Speaking of that, that’s a good segue into our next macro-related change. Now, all kinds of macros are usable within extern blocks, and so both macro_rules!, macros and procedural macros. And so, an extern block is, if you’ve ever used extern "C" fn before in Rust, as a way of defining the signature of a C function. extern blocks are a way of defining those kind of signatures, for many functions at once, as a convenient way, and for whatever reason, which I have— I don’t understand right now, I need to actually ask— macro_rules! macros wouldn’t expand within these blocks. Maybe it was just being a defensive kind of thing. But as of 1.40, macro_rules! macros now can be used inside of here. And you can also do things like put attributes on these extern fn definitions. So that’s, again, just another place in Rust code that is no longer special, things should just work as you expect. So that’s a nice change.

Jon: I think the next one falls in a very similar category too, of, previously, you couldn’t use macro_rules! to define a macro inside the output of a procedural macro, whereas now you can. Right? It’s just another instance of, like, this wasn’t possible before. And this is a place where macros had to be treated specialty. And now that is no longer the case.

This last one, though, I think, is interesting, because it’s a little bit odd when you first look at it. So when you use macro_rules! to define a macro that takes parameters, you can say what the type of those parameters should be, and the types here are a little bit different from normal, like, function parameter types, in that they are more types of expressions, rather than actual Rust types. So, for example, you can say that the first argument to a Rust macro should be a path to a type, or an expression, or an entire token tree, or a constant. So these are sort of, if you will, meta-level Rust constructs. You can say, I want this type of syntax, in this place, of a parameter for my macro, and one of those types is the meta type. The meta matcher, as it’s called. And the meta matcher matches anything that looks like a attribute-style macro. So these would be things like derives, for example, or the serde::Serialize attribute macros we talked about before, and the change that’s happened here is that now that matcher accepts basically arbitrary sequences of tokens. So that is, you can— if you use that matcher, it will now accept basically arbitrary Rust code.

And the reason for this change is a little interesting. Originally, the meta matcher was there because you wanted something that matched the syntax of these attribute-like macros, but over time as these types of macros have developed, the Rust developers have decided that they basically want to allow almost arbitrary syntax to be used in those attribute macros. Previously it would just like, #[path(attribute)]. Whereas now, they’re leaning more towards, you should allow any kind of Rust syntax in there. But that also means that the meta matcher becomes sort of useless, right? Because it has to match basically anything. And at that point, they had the option either of removing the meta matcher or just to make it support parsing arbitrary— or just accept arbitrary sequences of tokens. The problem is that it’s— for much of the standard library, they can deprecate methods and types just fine. But a matcher, it’s not entirely clear how you deprecate. There isn’t a mechanism in Rust to say that this value for a part of the syntax is now deprecated. It’s not really something that Rust has a story for, in terms of its backwards compatibility. It might be something that they could, like, remove the meta matcher in the next edition, maybe. But they don’t have a good way, at least for now, to even indicate that it’s been deprecated. And so the solution was to make the meta matcher just now support arbitrary Rust token streams. And so that’s what it does.

Ben: So those are all of our macro changes. And it is interesting to think about, where it’s something that— it’s become so useful in other ways, the interior of attribute macros that it’s now— this feature designed to accommodate them is now too broad, and isn’t really good for anything any more. But it’s a good thing.

Jon: Yeah, exactly.

Ben: It means that now attribute macro bodies are no longer special, they’re no longer unique. And in the future, maybe we won’t need to even have this feature of the language anymore.

Jon: Yeah, and I think if you’re— if people are curious about, sort of, some of the the internal discussion and process for making these decisions, looking into how that particular change ended up landing in the way that it did gives some useful insights, it’s worth reading up on.

Ben: So this next one here, borrow check migration warnings have become hard errors in Rust 2015, and I think we have spent, I think, maybe, two or three previous episodes talking in length about what this means for Rust 2018, and Rust 2015, and for the compiler, and for the borrow checker. And so I would recommend you just go listen to those episodes. I think 1.39 had some, I think 1.36, maybe 1.38, possibly. It’s just— we’ve been over this before. If you’re on Rust 2015 then you need to update your code to accommodate the new borrow checker that came last year. And I think it’s all we’ll say about that in this episode.

Jon: I think that’s reasonable.

Ben: We have spent a lot of time. Yeah. Oh, ongoing constification, making more standard library functions and methods const, usable in constant contexts. We really should have some kind of, like, dedicated musical interlude intro, because this happens every single release.

Jon: It’s true.

Ben: It’s a recurring segment. The is_power_of_two function in the standard library, which tells you whether or not an integer, is or is not a power of two, is now usable in constant contexts.

Jon: Is that what it does?

Ben: Yeah, yeah. It’s kind of subtle, because you couldn’t tell.

Jon: We should probably have an episode on that function.

Ben: Yeah, we could do maybe three or four. And in this case, something more interesting, perhaps, that I wanted to talk about is that, last episode we mentioned in passing how there was, at that moment, a PR to the Rust compiler to enable if and match expressions to work in constant contexts. And this is a big deal, because right now, that’s kind of the major thing that’s missing. Aside from, say, looping. But if if and match expressions were to work in a constant context, it would kind of unblock everything and, kind of like, unleash the floodgates for a whole lot more things to become const fns. And so, in this case, that PR has since landed in the past six weeks.

And, previously, in order to make certain things const fns in the standard library, we had to, kind of, make concessions to readability. And because a lot of what was supported previously in const fn was mostly bitwise operations, a lot of— in order to make things const, a lot of these const fns kind of became more unreadable as a result of taking out branches or match statements in order to make things work in constant context using only bitwise operations. And that was always all of— any time that happened, it was marked with a certain tag in the issue tracker called “const fn hack.” And so now that if and match, have become— not stable yet, but they are in nightly and usable— the “const fn hacks” have been removed. So if you are using nightly Rust, you are in fact using actual if and match const.

Jon: I think this is a good place to also plug your RustFest episode, or ours, I should say. RustFest episode where you talk to Oli.

Ben: Oh, yes. So, Oli— I forget his last name. “oli-obk” is how I know him. But it was the most recent episode, and I talked to him at RustFest 2019, last November. And it’s a great, just, talk on, hey, what is, like, how does the Rust compiler do constant evaluation? Like, what’s coming in the future? What is the underlying tool, called Miri, how cool is it? The answer is very. I don’t want to spoil it. But yeah, you should definitely go take a listen to that. It’s not super long. About half an hour. And we have more interviews on the way, so stay tuned for that.

Jon: I’m very excited. Another thing I’m very excited for— what a segue— is. The todo! macro, which, this has been— there’s been a lot of debate around this, and whether it’s, like, useful. And that is, in Rust 1.40, the Rust standard library gained a new macro called todo!. And todo!, much like unimplemented! and unreachable!, is really just a synonym for panic! with a custom message. And in the release notes, it’s mentioned as a shorter, more memorable, and convenient version of unimplemented!.

Ben: So, actually, I love this macro. I love unimplemented! in the first place. I use it all the time in Rust development. If you aren’t using this macro, like, you need to get on this, you need to get on the train. What I normally do is, if I’m, like, making a big change, or making a huge new addition, what you normally want to do, is you want to figure out— I’m not gonna impose, what I want to do normally, is I want to figure out all the type signatures first, I want to get everything into place and then start, like, implementing things one at a time, piecemeal. I want things to first start type- checking. Even if they panic at run time, just as kind of a basic case for knowing what I want to be doing. And so I’ll write a function signature. Maybe I’ll have, like, some new type that wants to implement a dozen traits. And all those traits have their own functions. And so, rather than having to go through and do all those function bodies upfront to get them to actually type-check, I will just write unimplemented! and it will just type-check. And this works because of the “never” type and diverging functions, and the bottom type, if you want to call it that. Let’s not get into it. The point is that any function that panics just type-checks, because panicking is always a valid, quote-unquote, return type, because it isn’t a return type.

And todo! is great. Just being shorter to type. If you’re using, like, VS Code, it doesn’t actually matter, because, type uni and then “tab” will automatically autocomplete it for you. But actually, I think the reason that this warrants inclusion is that unimplemented! kind of expresses something a bit different in the mind of a reader. And so we have the unreachable! macro, and the unreachable! macro says, hey, I can’t prove it to the compiler, but trust me, the author, the code’s never going to get to this point. And so, say, for example, you have let X = Some(42), like as an Option, and then later on, you have to then do some operation where it’s like, hey, you have an Option here. Do you know it’s a Some? And there’s no way to say to Rust today, hey, like, up here, if you just look, here is where I make it a Some, and it’s guaranteed to be a Some at this point in the code. So you just have to say unwrap or something. And if you were to instead use a match, you could say unreachable! on the None case, because you have proven to yourself, and to readers, hopefully with the aid of a comment, that hey, like, this can’t ever be taken, just look up here, like, there’s no actual point at which this branch ever be taken.

Jon: Right. In some sense unreachable! is like a piece of self-documenting code, so to speak.

Ben: And obviously, you could at some point get it wrong. Maybe in the future it changes. This is why it still has to panic at the end; Rust has to be conservative. There is, in fact, an unsafe version of unreachable in the standard library, if you know for sure that— you are more certain, and you don’t want some kind of performance hit, or potential branch that won’t ever be taken, to be generated in your code. But that’s that’s off-topic. And then unimplemented! is the version of that saying, hey, like, we haven’t implemented this, and then this kind of— in the past, this implicit assumption that hey, we will implement it in the future. But it turns out that it’s actually useful sometimes to be able to say, hey, like, I haven’t implemented this and I don’t intend to ever do that. And so, that’s kind of what in the future we’re moving forward with with having this split between todo! and unimplemented!, where you can use unimplemented! to say, hey, I don’t intend to ever implement this. If you want it, too bad. Do it yourself, fork my library. Whereas todo! macro is more for saying hey, like, I will do this in the future. Just that right now I haven’t gotten around to it.

And so, for example, in my work, when I work on Rust stuff, I might send a tentative draft PR, which just have, like, a billion todo!s, stubbed out. And the first time my boss— remember in the past, it was unimplemented. And first time my boss saw this, he was like, Do you actually want to, like, write this code? And I’m like, no, no, no, I will. I’m just trying to say like, hey, take a look at it. What do you think of this general structure? That kind of thing. And so this has actually been to me where people have looked at my unimplemented! and said, hey, do you— shouldn’t you be doing this? And just doesn’t really express the idea of “I will be doing this. I just haven’t yet.”” And so todo! for me is great for that reason. Or it just says, to any potential readers or reviewers, hey, like, this will be done. Just not right now.

Jon: Yeah, it’s interesting. I think people also use these in different ways, right? Like for me, unimplemented! is something that I know I have to fix before this code will be done. Whereas todo! is, I know this has to be fixed at some point, right? So people use it a little bit different. And I think part of the discussion that was happening when todo! was proposed, and then added was, should the standard library really have a macro for all of these different use cases? And some people say todo! is common enough and different enough that we should, and some people thought, well, sort of, this is— almost like a slippery slope argument, right? Like how many macros are we going to end up with? I think todo! is probably worthwhile to add, but there was a lot of discussion, and also discussion about “if we add todo!, should we deprecate unimplemented!?” I think ultimately the decision was to keep both, and I think that’s pretty reasonable. At least now that people know that both of them exist.

There are also been a bunch of other additions to the standard library, some small and some large. One of them is the slice::repeat function, which is really straightforward and pretty handy. So previously, we’ve talked about the repeat function, which lets you create a vector that just is a bunch of instances of one type. And now the slice module has also gotten a repeat function, and this lets you take a slice of type T and repeat it N times and then collect the result into a vector. And so this is just a handy way to, just sort of, repeat a sequence of things multiple times.

The other thing that’s under the standard library is mem::take. So this in the mem module, and the take function is really just a convenience wrapper around the mem::replace function. So mem::replace takes a mutable reference to a T, and a T and then it swaps those two, and then returns you the thing that the mutable reference was pointing to. So if you have some variable x that you have, you just have a mutable reference to. And you want to replace its entire value with a new value. So this might be, for example, you have a HashMap or you have a mutable reference to a HashMap, and you want to, sort of, take all of the values in that HashMap and leave an empty HashMap in its place. Then previously, you could do mem::replace, give it the mutable reference to the map, and then have the second argument be HashMap::new(). Or, if you will, if you have a type that implements Default, you could write HashMap::default() or Default::default(). And mem::take is just a handy shortcut for this, where you can say std::mem::take and then give a mutable reference to, say, a HashMap. And what that’s going to do is it’s going to stick a default hash map, so an empty one, in the place of where the mutable reference is pointing to, and then it’s gonna take the old map and give you an owned reference to that map, so it just swaps something with its default implementation value. This can be handy, especially if you’re writing sort of relatively performance-critical code, where you don’t want to, like clone all the elements and then call clear on the map, for example.

There’s another interesting function that some people might, sort of, squint at and go, why was this even added? Which is the get_key_value method on BTreeMap and HashMap. And previously there’s always been, sort of, the get function, which, you give a key, and then you get a reference to the value. But get_key_value seems a little odd at first glance, because if I give a key, why would I want a reference to the key given back to me? And this gets at a pretty subtle thing, that we’re not gonna spend too much time thinking about, or talking about. But it is interesting to know about, which is, if you look carefully at the method signature for the get function, you will see that it doesn’t take a key, that is, a reference to K, the key type of the map. It takes a reference to Q. It’s generic over Q and takes a reference to Q, where K can be borrowed as a Q. So that is, if you have a K, you can get a reference to a Q, and then the way it internally works, is you give it this Q type, and that Q type has to be comparable with the key type, if you borrow it.

This might seem relatively involved; I suggest you look at the method signature. But what this enables you to do is that you can have a key type that stores additional information beyond whatever the unique identifier for the element is. This could be something like, it also, stores the length of the string or the hash code of the key that’s actually there, or something like that. Some auxiliary information that isn’t really important for the purposes of using it as a key in the map. And you can still just use the unique identifier to do the lookup. So, for example, I might have a really complex key type that has all sorts of fields. But there’s, like, a string in there; that’s really what the key is. And as long as I can borrow that key type into a string, then I can just use a string as the key to do the look up in the map, and get_key_value now allows me to get back a reference to that more complex key type that’s actually stored in the map, which I don’t have just to do the look up. It’s a relatively involved explanation, and it’s a function that you will probably need relatively rarely. But if you need it, this is the only efficient way to get this functionality.

Do you want to talk about some of the new methods that Option has gotten?

Ben: Not really, like, I think it’s kind of just par for the course of adding in new things. So for example, if you even look at the release notes where the new functions are Option::as_deref, Option::as_deref_mut, and they’re, kind of, just analogous to the existing Option::as_ref and Option::as_mut. But they allow you to leverage the Deref and DerefMut traits. So it’s kind of just making Option more generally useful and more generally consistent with things that already exist, and more flexible, I suppose. So it’s, kind of, just par for the course for— we added in a, kind of, little convenience method. Sometimes it will be useful. Maybe not.

Do you want to talk about the— perhaps, the bigger thing for me is the floats to bytes functions that are now found in— the for floating point numbers?

Jon: Oh, yeah, this is pretty handy. So, we talked about this on one of the previous episodes, I believe, which is when these methods were added for integers. And so this is— imagine that you want to send a number over the wire, and so you need to serialize it into bytes that you’re actually going to send over TCP or whatnot. Previously, a bunch of methods on various types— various integer types were stabilized, that let you take a number and turn it into bytes, or take a sequence of bytes and turn it into a number, without having to resort to unsafe code. And now, in 1.40 similar methods exist for the floating point types. This is just sort of a handy extension of that. And hopefully will make you reach for the byteorder crate a little less often, even though it is a great crate.

I think another one that’s worth mentioning is— sorry go ahead.

Ben: I was going to, kind of, just talk about how— I really like the idea here, of when you’re adding functions to the standard library, to kind of look at what is popular out there in the ecosystem. Think about, sort of, what crates does everyone depend on, transitively or otherwise? How can we, sort of like, is there any functionality from those crates that it would make sense to have in std? That would, sort of, benefit a lot of use cases while also potentially reducing some dependency graphs. If you just need, say, in this case, the ability to send some floating point numbers over the wire, then now you can not use the byteorder crate, and which, kind of, I know people are kind of— in any case, you are sensitive to the idea of, in any of your applications written in Rust, the size of your dependency tree, and so beginning to reduce that, I think, is personally my theme for Rust 2020. I didn’t get around actually writing a blog post, but I think there’s plenty of things in the wire, that are of this sort where, we are looking, examining the most popular crates on crates.io, figuring out, hey, like, what are some that are, like, small and self-contained and ubiquitous, and easy to provide for people. And so my current favorite example is the lazy crate, or the once_cell, which provides a lazy type, which pretty much would just replace lazy_static for me entirely. And so it’s a macro-free way of doing what lazy_static currently does, and there’s also— I’m not sure if this has landed yet. But there is a crate out there called matches, which just provides a macro that tells you whether or not some pattern will match some value. And someone was doing some digging around, and just looking at hey, like, what is the single most, like, transitively depended- on crate on crates.io, and it was this crate. And so I think now, that it is actually— either already included or being proposed for inclusion in std in nightly.

Jon: That PR has actually already landed, and I think it’s even stabilized. So I think we’re getting that in 1.41.

Ben: Nice.

Jon: Yeah. I’ve been following that PR too, cause It’s really cool.

Ben: So it’s a very simple new addition. Doing things that, honestly, I personally haven’t needed. But apparently, a lot of people do. So it’s good to see, standard library, tentatively— like, we’re never going to have the world’s biggest standard library in Rust. Like, I wouldn’t expect to ever see, say, hyper in the standard library. But it’s good to see things moving towards very simple common dependencies being upstreamed into std.

Jon: Yeah, I agree. It’s a tricky balance, though, because you also don’t want std to get too large. And so it really, as you say, the process is pretty much, look at what does basically everyone depend on, at least transitively, and then find some common core that, like, if we just had this in the standard library, that would cover, like, 80% of use cases?

Ben: Yeah, I would love some day to write up a whole treatise on, hey what are the criteria that we use to gauge suitability for inclusion in std, where it’s like, maintenance burden, and like, standardization of a thing, like, broadly and, you know, precedents from other languages. And, will this thing change, given future language features? Or, how will the API potentially change? Is this stable? Is this widely used? There’s a whole lot to consider, and I think, you know, in the past it’s been extremely conservative, with, okay, standard library additions will mostly just add new convenience features for things that already exist, effectively making the standard library deeper but not broader in the sense of adding in new modules. And so, for example, Python is a very deep and broad standard library, where something like Go is not as deep but very broad. And Rust is not as broad, but very deep. So there are different approaches to these things.

Jon: Yeah, and it’s interesting because when you read through some of the PRs and RFCs that propose these features, very often the discussion ends up circling around like, is this something that is worthwhile to include in the standard library, because there’s sort of a commitment that comes with that, too.

Ben: Yeah. Again, maintenance burden, and you have to maintain it for all time is the question.

Jon: Exactly. That brings us to the end of the, sort of, official release notes. But there is, sort of at the bottom of each of these release notes, there’s this little section called “other changes” that links to the actual changelog in both Rust, Cargo and Clippy. And I always love digging into this, because I find some, like, nice little gems there. And there are two in particular I wanted to highlight this time, that may or may not affect you, but I thought they were interesting.

The first of these is that Cargo will now remember the warnings for your crate if it compiled without errors, and then if you run Cargo again, it will print those warnings again.

Ben: I love that.

Jon: This has been, like, a hill for me for so long where— you might have noticed this with Clippy. But even just with, like, cargo check or something, where you run it, it creates some error— it creates some warnings, but it compiled just fine. You run the command again, and it just says, like— it just completes immediately and says nothing. And it’s really annoying. And now that has been fixed. Now it will just, always display the same warnings, even if the crate has already been compiled.

The other thing is a change to the way that feature flags work in workspaces. We haven’t really talked too much about Cargo workspaces, but suffice to say, they’re a way of having multiple crates under one crate. And previously you could try to pass the --features flag inside the root of such a workspace, and that has now been turned into a hard error. That command will just error out saying that does not work. And this might come as a piece of— like, this might be frustrating to some people, because they’re like, ah, but I used to use this and now I can’t do it anymore. And what the release notes pointed out was that it actually used to do nothing. So if you used to do this, you were just compiling your crate twice with the same set of features. Passing this flag did absolutely nothing. And so that is why it is now a hard error. It is not as though the functionality was removed. It was a minor point, but it was an interesting one for those who might find this— might accidentally discover this change, and then get frustrated.

I think with that, those are all the things for 1.40.

Ben: Yeah, we had a bit more to talk about than we thought.

Jon: Yeah, it’s true. It’s a smaller release than 1.39, but at the same time, there are a lot of really interesting changes that, as we talked about last time, like, there’s been a lot of focus on async and await, and this is a release that has nothing about async/await in it. And yet there’s still a lot of meat to it. And I think that is something that we’re gonna expect to continue to see in the releases to come, that they’re just more and more of these features that are not focused on async/await now that that effort has sort of been unblocked from its major hurdle.

Ben: Well, speaking of continuing, now that we’ve established only releasing one episode every decade, I think now we have to wait until 2030 to do our next episode. So I think it’s gonna be about Rust 128 at that point. So I suppose I’ll see you in 10 years, Jon.

Jon: That’s great. I’m looking forward to it already, Ben.

Ben: I can’t wait to not see you for 10 years. That sounds great. Thanks a lot.

Jon: Exactly.

Ben: All right. Yeah. OK. See you, folks.

Jon: All right. Bye, everyone.