Strictly speaking, Rust doesn't support overloaded functions. Function overloading is when you define the same function but with different arguments, and the language selects the correct one based on the argument type. In this case, it's two different implementations of a trait for two different types. That said, it's close enough that this isn't really an issue, more of just a technical note, since this is a series trying to get into details.
> I can't find an explanation in the Rust documentation but I expect the reason is that someone could implement another trait that provides .into_iter() on whatever the x is in for y in x, thus resulting in a compiler error because there would be two candidate implementations.
Yep, I'm not sure that there is an official explanation anywhere else, but this is exactly what I would assume as well. This ensures that the correct implementation is called. This is also one of the reasons why adding a trait implementation isn't considered a breaking change, even if it could create a compiler error, because you can always expand it yourself to explicitly select the correct choice. Of course, these situations are usually treated more carefully then they have to be, because breakage isn't fun, even if it's technically allowed.
> But wait, you say, I'm doing exactly this in the first program, and indeed you are.
It's not the same, as the next paragraphs explain.
> We are able to examine the function and realize it's safe, but because the compiler wants to use local reasoning, it's not able to do so.
This is a super important point!
junon · 3h ago
> I can't find an explanation in the Rust documentation but I expect the reason is that someone could implement another trait that provides .into_iter() on whatever the x is in for y in x, thus resulting in a compiler error because there would be two candidate implementations.
Because nothing in Rust is identifier-based. Unlike python, all syntax magic (even the ? operator) relies on traits defined by `core`.
for x in y {
desugars to
let mut iter = IntoIterator::into_iter(y);
while let Some(x) = y.next() {
and
x?
desugars to
match x {
Ok(x) => x,
Err(e) => {
return Err(From::from(e));
}
}
and
x + y
desugars to
core::ops::Add::add(x, y)
etc.
All of those traits are expected to live explicitly in the core crate at well known paths. Otherwise you'd be writing methods with absolutely no idea how the language would interact with it. And if you had a Set type implement `add`, it'd have to accept exactly 2 arguments to be compatible with the language's `add` or something equally as unergonomic.
It's traits all the way down! There'd be no explanation needed because it'd be antithetical and contradictory to traits to begin with. Once one understands how traits are intended to be used, the explanation for why there aren't identifier based resolution semantics becomes obvious.
writebetterc · 37m ago
It's a bit annoying that there is a standard desugaring of all of this, but there's no actual way of implementing this sugaring yourself. I guess you could have a token macro (or whatever they're called) at the top scope and do it yourself?
Anyway, my conclusion is that in order to learn Rust you can't use any syntactic sugar the language provides, those are 'expert features'. There's no way I'd be able to figure out why this happens. Do rust-analyzer provide a way of desugaring these constructs?
norman784 · 54m ago
Interesting, I didn't knew about the for loop and the ops desugars to something in core, I though it was deal in the compiler, but makes sense since you can implement the ops::* traits for your custom types.
saagarjha · 1h ago
Swift has overloading and its own version of traits. None of what you said is actually required.
Animats · 5h ago
This is pretty good.
A useful way to think about this:
- All data in Rust has exactly one owner.
- If you need some kind of multiple ownership, you have to make the owner be a reference-counted cell, such as Rc or Arc.
- All data can be accessed by one reader/writer, or N readers, but not both at the same time.
- There is both compile time and run time machinery to strictly enforce this.
Once you get that, you can see what the borrow checker is trying to do for you.
diath · 1h ago
The first code snippet, which is as simple as it gets, perfectly illustrates why Rust is extremely annoying to work with. I understand why you need the into_iter bit and why the borrow checker complains about it, but the fact that even the simplest "for x in y" loop already makes you wrestle the compiler is just poor ergonomics.
0xdeafbeef · 1h ago
#include <vector>
#include <iostream>
int main() {
std::vector<int> vec = {1, 2, 3};
vec.reserve(3);
std::cout << "Initial capacity: " << vec.capacity() << std::endl;
std::cout << "Initial data address: " << (void*)vec.data() << std::endl;
int* ptr = vec.data();
std::cout << "Pointer before push_back: " << (void*)ptr << std::endl;
std::cout << "Value via pointer before push_back: " << *ptr << std::endl;
std::cout << "\nPushing back 4...\n" << std::endl;
vec.push_back(4);
std::cout << "New capacity: " << vec.capacity() << std::endl;
std::cout << "New data address: " << (void*)vec.data() << std::endl;
std::cout << "\nAttempting to access data via the old pointer..." << std::endl;
std::cout << "Old pointer value: " << (void*)ptr << std::endl;
int value = *ptr;
std::cout << "Read from dangling pointer (UB): " << value << std::endl;
return 0;
}
./a.out
Initial capacity: 3
Initial data address: 0x517d2b0
Pointer before push_back: 0x517d2b0
Value via pointer before push_back: 1
Pushing back 4...
New capacity: 6
New data address: 0x517d6e0
Attempting to access data via the old pointer...
Old pointer value: 0x517d2b0
Read from dangling pointer (UB): 20861
scoutt · 28m ago
You are not fighting the C++ compiler or showing why the C++ compiler might be annyoing. You are introducing a bug by poorly using a library (which has nothing to do with writing and compiling C++). Ergonomics I believe are fine?
I'm struggling hard trying to understand what or if your comment has anything to do with GP's comment. Perhaps you wanted to tell that the Rust compiler might have stopped you from producing a buggy program, but again, it has nothing to do with GP's comment.
masklinn · 41m ago
> I understand why you need the into_iter bit and why the borrow checker complains about it
The borrow checker is not really involved in the first snippet (in fact the solution involves borrowing). The compiler literally just prevents a UAF.
59nadir · 35m ago
The compiler isn't protecting them from anything in this particular example, we can all see this is valid code. It's provably valid code, just not provable by the Rust compiler.
masklinn · 7m ago
Literally nothing in your comment is correct. In the Rust snippet vector is moved into the loop, and thus freed.
There are situations where you hit the limits of the borrow checker, plenty of them. This is not one of them. Again, the borrow checker is not even involved in the original snippet.
baq · 1h ago
> wrestle the compiler
This is quite literally a skill issue, no offense.
'wrestle the compiler' implies you think you know better; this is usually not the case and the compiler is here to tell you about it. It's annoying to be bad at tracking ownership and guess what: most people are. The ones who aren't have decades of experience in C/C++ and employ much the same techniques that Rust guides you towards. If you really know better, there are ways to get around the compiler. They're verbose and marked unsafe to 1) discourage you from doing that and 2) warn others to be extra careful here.
If this is all unnecessary for you - and I want to underscore I agree it should be in most software development work - stick to GC languages. With some elbow grease they can be made as performant as low level languages if you write the critical parts in a way you'd have to do it in Rust and will be free to write the rest in a way which doesn't require years of experience tracking ownership manually. (Note it won't hurt to be tracking ownership anyway, it's just much less of an issue if you have to put a weakref somewhere once a couple years vs be aware at all times of what owns what.)
59nadir · 47m ago
No, GP can just not use Rust, they don't have to use GC languages to have something that makes sense and doesn't force you to always have a debate with the compiler about even simple things.
If they used Odin (or Zig) they could've looped through that dynamic array no problem, in fact:
package example
import "core:fmt"
main :: proc() {
xs: [dynamic]int
append(&xs, 1, 2)
for x in xs {
fmt.println(x)
}
fmt.println(len(xs))
}
It is ridiculous that Rust complains even about the simple for loop and to say that this somehow comes down to "Well, everyone would do it this way if they cared about memory safety" is just not really true or valuable input, it sounds like what someone would say if their only systems programming experience came from Rust and they post-rationalized everything they've seen in Rust as being how you have to do it.
My tips to people who maybe feel like Rust seems a bit overwrought:
Look for something else, check out Odin or Zig, they've got tons of ways of dealing with memory that simply sidestep everything that Rust is about (because inherently Rust and everything that uses RAII has a broken model of how resources should be managed).
I learned Odin just by reading its Overview page (https://odin-lang.org/docs/overview/) and trying stuff out (nowadays there are also good videos about Odin on YouTube), then found myself productively writing code after a weekend. Now I create 3D engines using just Odin (and we in fact use only a subset of what is on that Overview page). Things can be simple, straight forward and more about the thing you're solving than the language you're using.
dwattttt · 22m ago
I dunno; I've never tried Zig before, and it wasn't hard to check whether this kind of bug was easy to have:
const std = @import("std");
pub fn main() !void {
var gpa: std.heap.GeneralPurposeAllocator(.{})=.{};
const alloc=gpa.allocator();
var list = try std.ArrayList(u8).initCapacity(alloc, 1);
const a = try list.addOne();
a.* = 0;
std.debug.print("a={}\n", .{a.*});
const b = try list.addOne();
b.* = 0;
std.debug.print("a={}\n", .{a.*});
std.debug.print("b={}\n", .{b.*});
}
a=0
Segmentation fault at address 0x7f9f7b240000
59nadir · 19m ago
And why do you think that bug is relevant in the case of a loop that prints the elements of a container? We can all see and verify at a glance that the code is valid, it's just not provably valid by the Rust compiler.
I feel like these posts trying to show possible memory issues with re-allocated dynamic arrays are missing the point: There is no code changing the underlying array, there is no risk of any kind of use-after-free error. This is exactly the kind of case where all of this jumping through hoops shouldn't be needed.
mjburgess · 41m ago
Rust, in many ways, is a terrible first systems programming language.
To program a system is to engage with how the real devices of a computer work, and very little of their operation is exposed via Rust or even can be exposed. The space of all possible valid/safe Rust programs is tiny compare to the space of all useful machine behaviours.
The world of "safe Rust" is a very distorted image of the real machine.
masklinn · 3m ago
> Rust, in many ways, is a terrible first systems programming language.
Contrariwise, Rust in, in many way, an awesome first systems programming language. Because it tells you and forces you to consider all the issues upfront.
For instance in 59nadir's example, what if the vector is a vector of heap-allocated objects, and the loop frees them? In Rust this makes essentially no difference, because at iteration you tell the compiler whether the vector is borrowed or moved and the rest of the lifecycle falls out of that regardless of what's in the vector: with a borrowing iteration, you simply could not free the contents. The vector generally works and is used the same whether its contents are copiable or not.
mjburgess · 44s ago
None of these "issues" are systems issues, they're memory safety issues. If you think systems programming is about memory saftey, then you're demonstrating the problem.
Eg., some drivers cannot be memory safe, because memory is arranged outside of the driver to be picked up "at the right time, in the right place" and so on.
Statically-provable memory saftey is, ironically, quite a bad property to have for a systems programming language, as it prevents actually controlling the devices of the machine. This is, of course, why rust has "unsafe" and why anything actually systems-level is going to have a fair amount of it.
mjburgess · 45m ago
That systems languages have to establish (1) memory saftey, (2) statically; (3) via a highly specific kind of type system given in Rust; and (4) with limited inference -- suggests a lack of imagination.
The space of all possible robust systems languages is vastly larger than Rust.
It's specific choices force confronting the need to statically prove memory saftey via a cumbersome type system very early -- this is not a divine command upon language design.
kobebrookskC3 · 5m ago
sure, rust is not the final answer to eliminating memory safety bugs from systems programming. but what are the alternatives, that aren't even more onerous and/or limited in scope (ats, frama-c, proofs)?
shakna · 34m ago
> It's annoying to be bad at tracking ownership and guess what: most people are. The ones who aren't have decades of experience in C/C++ and employ much the same techniques that Rust guides you towards.
You wouldn't need to do that here, in SPARK, or Oberon, or just about any other memory safe language. This is unique to Rust, and their model - and it absolutely is not required for safety. It's not a skill issue. It's a language design problem.
diath · 46m ago
This is yet another issue with Rust, nowhere in my post have I mentioned C++, I made no effort comparing the two languages, I just pointed out poor developer ergonomics in Rust and you're instigating a language flame war as if you took valid criticism as a personal attack; you can do better than that.
rollcat · 34m ago
I'm starting to think Zig's strategy to memory management is generally friendlier to a developer. If a function needs to allocate memory, it must take an allocator as a parameter. If it needs scratch space to perform computation, it can use that allocator to create an arena for itself, then free it up before it returns (defer). If it returns a pointer, the caller should assume the object was allocated using that allocator, and becomes the owner. It may still be unclear what happens to a pointer if it's passed as a parameter to another function, but I'd normally consider that a borrow.
It's a lot of assumptions, and if you trip, Rust will yell at you much more often than Zig; and it will likely be right to do so. But in all seriousness, I'm tired of the yelling, and find Zig much more pleasant.
bsaul · 2h ago
"If we change .set_value() to take a &self instead of a &self"
guess it's "instead of a &mut self"
Surac · 5h ago
I really like the text. Giving more light to the memory management of rust will help me understand more of the language. I still think some concepts of rust are over verbose but I slowly understand the hype around rust. I myself use C or C++ but I will „borrow“ some of the rust ideas to make my code even more robust
ultimaweapon · 3h ago
I'm coming from C++ now I don't want to use C++ anymore. When C++ was still my primary language I always frustrated with some of its feature like non-destructive move, copy by default and dangling references then I found Rust fixed all of those problems. At the beginning I very frustrated with Rust because the borrow checker prevent me from doing what I usually do in C++ but I keep going.
cornholio · 4h ago
A 20 page document on how to use basic variables, function calls and methods. Except for the threading paragraph, which is hard in any language, this is all complexity and refactoring pain that Rust hoists onto every programmer every day, for relatively modest benefits, somewhat improved performance and memory usage vs the garbage collected/ref-counted version of the same code.
Essentially, you wouldn't and shouldn't make that tradeoff for anything other than system programming.
baq · 4h ago
> this is all complexity and refactoring pain that Rust hoists onto every programmer every day
This is what you should be doing when working with C/C++, except there is no compiler to call you names there if you don’t.
If you’re saying ‘use a GC language unless requirements are strict about it’, yeah hard to disagree.
bsaul · 2h ago
I see why you're saying that, and i almost entirely agree with you. However, i would say that if all you're doing is glueing calls to third party systems (like what most backend code is), then you won't fall into complex lifetime problems anyway, and the experience will remain quite pleasant.
Another point, is that the rust ecosystem is absolutely insanely good ( i've recently worked with uniffi and wasmbindgen, and those are 5 years ahead of anything else i've seen..)
sidcool · 6h ago
This is brilliantly written n
bnjms · 6h ago
Im sure it is. I don’t know how to program but read most of part 4 when it appeared here last.
> Function Overloads
Strictly speaking, Rust doesn't support overloaded functions. Function overloading is when you define the same function but with different arguments, and the language selects the correct one based on the argument type. In this case, it's two different implementations of a trait for two different types. That said, it's close enough that this isn't really an issue, more of just a technical note, since this is a series trying to get into details.
> I can't find an explanation in the Rust documentation but I expect the reason is that someone could implement another trait that provides .into_iter() on whatever the x is in for y in x, thus resulting in a compiler error because there would be two candidate implementations.
Yep, I'm not sure that there is an official explanation anywhere else, but this is exactly what I would assume as well. This ensures that the correct implementation is called. This is also one of the reasons why adding a trait implementation isn't considered a breaking change, even if it could create a compiler error, because you can always expand it yourself to explicitly select the correct choice. Of course, these situations are usually treated more carefully then they have to be, because breakage isn't fun, even if it's technically allowed.
> But wait, you say, I'm doing exactly this in the first program, and indeed you are.
It's not the same, as the next paragraphs explain.
> We are able to examine the function and realize it's safe, but because the compiler wants to use local reasoning, it's not able to do so.
This is a super important point!
Because nothing in Rust is identifier-based. Unlike python, all syntax magic (even the ? operator) relies on traits defined by `core`.
desugars to and desugars to and desugars to etc.All of those traits are expected to live explicitly in the core crate at well known paths. Otherwise you'd be writing methods with absolutely no idea how the language would interact with it. And if you had a Set type implement `add`, it'd have to accept exactly 2 arguments to be compatible with the language's `add` or something equally as unergonomic.
It's traits all the way down! There'd be no explanation needed because it'd be antithetical and contradictory to traits to begin with. Once one understands how traits are intended to be used, the explanation for why there aren't identifier based resolution semantics becomes obvious.
Anyway, my conclusion is that in order to learn Rust you can't use any syntactic sugar the language provides, those are 'expert features'. There's no way I'd be able to figure out why this happens. Do rust-analyzer provide a way of desugaring these constructs?
A useful way to think about this:
- All data in Rust has exactly one owner.
- If you need some kind of multiple ownership, you have to make the owner be a reference-counted cell, such as Rc or Arc.
- All data can be accessed by one reader/writer, or N readers, but not both at the same time.
- There is both compile time and run time machinery to strictly enforce this.
Once you get that, you can see what the borrow checker is trying to do for you.
Pushing back 4...
New capacity: 6 New data address: 0x517d6e0
Attempting to access data via the old pointer... Old pointer value: 0x517d2b0 Read from dangling pointer (UB): 20861
I'm struggling hard trying to understand what or if your comment has anything to do with GP's comment. Perhaps you wanted to tell that the Rust compiler might have stopped you from producing a buggy program, but again, it has nothing to do with GP's comment.
The borrow checker is not really involved in the first snippet (in fact the solution involves borrowing). The compiler literally just prevents a UAF.
There are situations where you hit the limits of the borrow checker, plenty of them. This is not one of them. Again, the borrow checker is not even involved in the original snippet.
This is quite literally a skill issue, no offense.
'wrestle the compiler' implies you think you know better; this is usually not the case and the compiler is here to tell you about it. It's annoying to be bad at tracking ownership and guess what: most people are. The ones who aren't have decades of experience in C/C++ and employ much the same techniques that Rust guides you towards. If you really know better, there are ways to get around the compiler. They're verbose and marked unsafe to 1) discourage you from doing that and 2) warn others to be extra careful here.
If this is all unnecessary for you - and I want to underscore I agree it should be in most software development work - stick to GC languages. With some elbow grease they can be made as performant as low level languages if you write the critical parts in a way you'd have to do it in Rust and will be free to write the rest in a way which doesn't require years of experience tracking ownership manually. (Note it won't hurt to be tracking ownership anyway, it's just much less of an issue if you have to put a weakref somewhere once a couple years vs be aware at all times of what owns what.)
If they used Odin (or Zig) they could've looped through that dynamic array no problem, in fact:
It is ridiculous that Rust complains even about the simple for loop and to say that this somehow comes down to "Well, everyone would do it this way if they cared about memory safety" is just not really true or valuable input, it sounds like what someone would say if their only systems programming experience came from Rust and they post-rationalized everything they've seen in Rust as being how you have to do it.My tips to people who maybe feel like Rust seems a bit overwrought:
Look for something else, check out Odin or Zig, they've got tons of ways of dealing with memory that simply sidestep everything that Rust is about (because inherently Rust and everything that uses RAII has a broken model of how resources should be managed).
I learned Odin just by reading its Overview page (https://odin-lang.org/docs/overview/) and trying stuff out (nowadays there are also good videos about Odin on YouTube), then found myself productively writing code after a weekend. Now I create 3D engines using just Odin (and we in fact use only a subset of what is on that Overview page). Things can be simple, straight forward and more about the thing you're solving than the language you're using.
I feel like these posts trying to show possible memory issues with re-allocated dynamic arrays are missing the point: There is no code changing the underlying array, there is no risk of any kind of use-after-free error. This is exactly the kind of case where all of this jumping through hoops shouldn't be needed.
To program a system is to engage with how the real devices of a computer work, and very little of their operation is exposed via Rust or even can be exposed. The space of all possible valid/safe Rust programs is tiny compare to the space of all useful machine behaviours.
The world of "safe Rust" is a very distorted image of the real machine.
Contrariwise, Rust in, in many way, an awesome first systems programming language. Because it tells you and forces you to consider all the issues upfront.
For instance in 59nadir's example, what if the vector is a vector of heap-allocated objects, and the loop frees them? In Rust this makes essentially no difference, because at iteration you tell the compiler whether the vector is borrowed or moved and the rest of the lifecycle falls out of that regardless of what's in the vector: with a borrowing iteration, you simply could not free the contents. The vector generally works and is used the same whether its contents are copiable or not.
Eg., some drivers cannot be memory safe, because memory is arranged outside of the driver to be picked up "at the right time, in the right place" and so on.
Statically-provable memory saftey is, ironically, quite a bad property to have for a systems programming language, as it prevents actually controlling the devices of the machine. This is, of course, why rust has "unsafe" and why anything actually systems-level is going to have a fair amount of it.
The space of all possible robust systems languages is vastly larger than Rust.
It's specific choices force confronting the need to statically prove memory saftey via a cumbersome type system very early -- this is not a divine command upon language design.
You wouldn't need to do that here, in SPARK, or Oberon, or just about any other memory safe language. This is unique to Rust, and their model - and it absolutely is not required for safety. It's not a skill issue. It's a language design problem.
It's a lot of assumptions, and if you trip, Rust will yell at you much more often than Zig; and it will likely be right to do so. But in all seriousness, I'm tired of the yelling, and find Zig much more pleasant.
guess it's "instead of a &mut self"
Essentially, you wouldn't and shouldn't make that tradeoff for anything other than system programming.
This is what you should be doing when working with C/C++, except there is no compiler to call you names there if you don’t.
If you’re saying ‘use a GC language unless requirements are strict about it’, yeah hard to disagree.
Another point, is that the rust ecosystem is absolutely insanely good ( i've recently worked with uniffi and wasmbindgen, and those are 5 years ahead of anything else i've seen..)